seaweedfs

History

Chris Lu 580940bf82 Merge accumulated changes related to message queue (#5098 ) * balance partitions on brokers * prepare topic partition first and then publish, move partition * purge unused APIs * clean up * adjust logs * add BalanceTopics() grpc API * configure topic * configure topic command * refactor * repair missing partitions * sequence of operations to ensure ordering * proto to close publishers and consumers * rename file * topic partition versioned by unixTimeNs * create local topic partition * close publishers * randomize the client name * wait until no publishers * logs * close stop publisher channel * send last ack * comments * comment * comments * support list of brokers * add cli options * Update .gitignore * logs * return io.eof directly * refactor * optionally create topic * refactoring * detect consumer disconnection * sub client wait for more messages * subscribe by time stamp * rename * rename to sub_balancer * rename * adjust comments * rename * fix compilation * rename * rename * SubscriberToSubCoordinator * sticky rebalance * go fmt * add tests * balance partitions on brokers * prepare topic partition first and then publish, move partition * purge unused APIs * clean up * adjust logs * add BalanceTopics() grpc API * configure topic * configure topic command * refactor * repair missing partitions * sequence of operations to ensure ordering * proto to close publishers and consumers * rename file * topic partition versioned by unixTimeNs * create local topic partition * close publishers * randomize the client name * wait until no publishers * logs * close stop publisher channel * send last ack * comments * comment * comments * support list of brokers * add cli options * Update .gitignore * logs * return io.eof directly * refactor * optionally create topic * refactoring * detect consumer disconnection * sub client wait for more messages * subscribe by time stamp * rename * rename to sub_balancer * rename * adjust comments * rename * fix compilation * rename * rename * SubscriberToSubCoordinator * sticky rebalance * go fmt * add tests * tracking topic=>broker * merge * comment		1 year ago
..
doc.go	go fmt	3 years ago
readme.md	add options to scaffold	3 years ago
ydb_queries.go	move to https://github.com/seaweedfs/seaweedfs	3 years ago
ydb_store.go	refactor filer_pb.Entry and filer.Entry to use GetChunks()	2 years ago
ydb_store_kv.go	move to https://github.com/seaweedfs/seaweedfs	3 years ago
ydb_store_test.go	move to https://github.com/seaweedfs/seaweedfs	3 years ago
ydb_types.go	fixes from comments	3 years ago

readme.md

YDB

database: https://github.com/ydb-platform/ydb

go driver: https://github.com/ydb-platform/ydb-go-sdk

options:

[ydb]
enabled=true
dsn=grpcs://ydb-ru.yandex.net:2135/?database=/ru/home/username/db
prefix="seaweedfs"
useBucketPrefix=true
poolSizeLimit=50
dialTimeOut = 10

Authenticate produced with one of next environment variables:

YDB_SERVICE_ACCOUNT_KEY_FILE_CREDENTIALS=<path/to/sa_key_file> — used service account key file by path
YDB_ANONYMOUS_CREDENTIALS="1" — used for authenticate with anonymous access. Anonymous access needs for connect to testing YDB installation
YDB_METADATA_CREDENTIALS="1" — used metadata service for authenticate to YDB from yandex cloud virtual machine or from yandex function
YDB_ACCESS_TOKEN_CREDENTIALS=<access_token> — used for authenticate to YDB with short-life access token. For example, access token may be IAM token
YDB_CONNECTION_STRING="grpcs://endpoint/?database=database"
i test using this dev database: make dev_ydb