seaweedfs

Commit Graph

Author	SHA1	Message	Date
chrislu	bb81894078	Update .gitignore	21 hours ago
Chris Lu	dde1cf63c2	S3 Object Lock: ensure x-amz-bucket-object-lock-enabled header (#6990 ) * ensure x-amz-bucket-object-lock-enabled header * fix tests * combine 2 metadata changes into one * address comments * Update s3api_bucket_handlers.go * Update weed/s3api/s3api_bucket_handlers.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update test/s3/retention/object_lock_reproduce_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update test/s3/retention/object_lock_validation_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update test/s3/retention/s3_bucket_object_lock_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_bucket_handlers.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_bucket_handlers.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update test/s3/retention/s3_bucket_object_lock_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_bucket_handlers.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * package name --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	23 hours ago
Chris Lu	4b040e8a87	adding cors support (#6987 ) * adding cors support * address some comments * optimize matchesWildcard * address comments * fix for tests * address comments * address comments * address comments * path building * refactor * Update weed/s3api/s3api_bucket_config.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * address comment Service-level responses need both Access-Control-Allow-Methods and Access-Control-Allow-Headers. After setting Access-Control-Allow-Origin and Access-Control-Expose-Headers, also set Access-Control-Allow-Methods: * and Access-Control-Allow-Headers: * so service endpoints satisfy CORS preflight requirements. * Update weed/s3api/s3api_bucket_config.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_object_handlers.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_object_handlers.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix * refactor * Update weed/s3api/s3api_bucket_config.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_object_handlers.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_server.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * simplify * add cors tests * fix tests * fix tests --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2 days ago
Chris Lu	1549ee2e15	implement PubObjectRetention and WORM (#6969 ) * implement PubObjectRetention and WORM * Update s3_worm_integration_test.go * avoid previous buckets * Update s3-versioning-tests.yml * address comments * address comments * rename to ExtObjectLockModeKey * only checkObjectLockPermissions if versioningEnabled * address comments * comments * Revert "comments" This reverts commit `6736434176`. * Update s3api_object_handlers_skip.go * Update s3api_object_retention_test.go * add version id to ObjectIdentifier * address comments * add comments * Add proper error logging for timestamp parsing failures * address comments * add version id to the error * Update weed/s3api/s3api_object_retention_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_object_retention.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * constants * fix comments * address comments * address comment * refactor out handleObjectLockAvailabilityCheck * errors.Is ErrBucketNotFound * better error checking * address comments --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	4 days ago
Chris Lu	d892538d32	More efficient copy object (#6665 ) * it compiles * refactored * reduce to 4 concurrent chunk upload * CopyObjectPartHandler * copy a range of the chunk data, fix offset size in copied chunks * Update s3api_object_handlers_copy.go What the PR Accomplishes: CopyObjectHandler - Now copies entire objects by copying chunks individually instead of downloading/uploading the entire file CopyObjectPartHandler - Handles copying parts of objects for multipart uploads by copying only the relevant chunk portions Efficient Chunk Copying - Uses direct chunk-to-chunk copying with proper volume assignment and concurrent processing (limited to 4 concurrent operations) Range Support - Properly handles range-based copying for partial object copies * fix compilation * fix part destination * handling small objects * use mkFile * copy to existing file or part * add testing tools * adjust tests * fix chunk lookup * refactoring * fix TestObjectCopyRetainingMetadata * ensure bucket name not conflicting * fix conditional copying tests * remove debug messages * add custom s3 copy tests	5 days ago
Chris Lu	51543bbb87	Admin UI: Add message queue to admin UI (#6958 ) * add a menu item "Message Queue" * add a menu item "Message Queue" * move the "brokers" link under it. * add "topics", "subscribers". Add pages for them. * refactor * show topic details * admin display publisher and subscriber info * remove publisher and subscribers from the topic row pull down * collecting more stats from publishers and subscribers * fix layout * fix publisher name * add local listeners for mq broker and agent * render consumer group offsets * remove subscribers from left menu * topic with retention * support editing topic retention * show retention when listing topics * create bucket * Update s3_buckets_templ.go * embed the static assets into the binary fix https://github.com/seaweedfs/seaweedfs/issues/6964	6 days ago
Chris Lu	02773a6107	Accumulated changes for message queue (#6600 ) * rename * set agent address * refactor * add agent sub * pub messages * grpc new client * can publish records via agent * send init message with session id * fmt * check cancelled request while waiting * use sessionId * handle possible nil stream * subscriber process messages * separate debug port * use atomic int64 * less logs * minor * skip io.EOF * rename * remove unused * use saved offsets * do not reuse session, since always session id is new after restart remove last active ts from SessionEntry * simplify printing * purge unused * just proxy the subscription, skipping the session step * adjust offset types * subscribe offset type and possible value * start after the known tsns * avoid wrongly set startPosition * move * remove * refactor * typo * fix * fix changed path	4 months ago
chrislu	f5f3b60a13	ignore	1 year ago
chrislu	928a4e8dff	rename	1 year ago
chrislu	dbdb7c8abe	Update .gitignore	1 year ago
Chris Lu	c471265837	build with pub sub clients for testing	1 year ago
Varun Upadhyay	77626666c5	Minor cleanup & gitignore update (#5144 )	2 years ago
Chris Lu	580940bf82	Merge accumulated changes related to message queue (#5098 ) * balance partitions on brokers * prepare topic partition first and then publish, move partition * purge unused APIs * clean up * adjust logs * add BalanceTopics() grpc API * configure topic * configure topic command * refactor * repair missing partitions * sequence of operations to ensure ordering * proto to close publishers and consumers * rename file * topic partition versioned by unixTimeNs * create local topic partition * close publishers * randomize the client name * wait until no publishers * logs * close stop publisher channel * send last ack * comments * comment * comments * support list of brokers * add cli options * Update .gitignore * logs * return io.eof directly * refactor * optionally create topic * refactoring * detect consumer disconnection * sub client wait for more messages * subscribe by time stamp * rename * rename to sub_balancer * rename * adjust comments * rename * fix compilation * rename * rename * SubscriberToSubCoordinator * sticky rebalance * go fmt * add tests * balance partitions on brokers * prepare topic partition first and then publish, move partition * purge unused APIs * clean up * adjust logs * add BalanceTopics() grpc API * configure topic * configure topic command * refactor * repair missing partitions * sequence of operations to ensure ordering * proto to close publishers and consumers * rename file * topic partition versioned by unixTimeNs * create local topic partition * close publishers * randomize the client name * wait until no publishers * logs * close stop publisher channel * send last ack * comments * comment * comments * support list of brokers * add cli options * Update .gitignore * logs * return io.eof directly * refactor * optionally create topic * refactoring * detect consumer disconnection * sub client wait for more messages * subscribe by time stamp * rename * rename to sub_balancer * rename * adjust comments * rename * fix compilation * rename * rename * SubscriberToSubCoordinator * sticky rebalance * go fmt * add tests * tracking topic=>broker * merge * comment	2 years ago
chrislu	89a1fd1751	Squashed commit of the following: commit `4827425146` Author: chrislu <chris.lu@gmail.com> Date: Sat Sep 16 15:05:38 2023 -0700 balancer works commit `3b50139f68` Author: chrislu <chris.lu@gmail.com> Date: Fri Sep 15 22:22:32 2023 -0700 comments commit `7f685ce7ba` Author: chrislu <chris.lu@gmail.com> Date: Fri Sep 15 22:20:05 2023 -0700 adjust APIs commit `436d99443b` Author: chrislu <chris.lu@gmail.com> Date: Thu Sep 14 23:49:05 2023 -0700 receive broker stats commit `b771fefa37` Merge: `0a851ec00` `890881037` Author: chrislu <chris.lu@gmail.com> Date: Wed Sep 13 00:03:47 2023 -0700 Merge branch 'master' into sub commit `0a851ec00b` Author: chrislu <chris.lu@gmail.com> Date: Sun Sep 10 22:01:25 2023 -0700 Create balancer.go commit `39941edc0b` Author: chrislu <chris.lu@gmail.com> Date: Thu Sep 7 23:55:19 2023 -0700 add publisher shutdown commit `875f562779` Author: chrislu <chris.lu@gmail.com> Date: Wed Sep 6 23:16:41 2023 -0700 server side send response at least once per second commit `984b6c54cf` Author: chrislu <chris.lu@gmail.com> Date: Wed Sep 6 23:15:29 2023 -0700 ack interval 128 commit `2492a45499` Author: chrislu <chris.lu@gmail.com> Date: Wed Sep 6 22:39:46 2023 -0700 ack interval commit `ba67e6ca29` Author: chrislu <chris.lu@gmail.com> Date: Mon Sep 4 21:43:50 2023 -0700 api for sub commit `9e4f985698` Author: chrislu <chris.lu@gmail.com> Date: Mon Sep 4 21:43:30 2023 -0700 publish, benchmark commit `cb470d44df` Author: chrislu <chris.lu@gmail.com> Date: Fri Sep 1 00:36:51 2023 -0700 can pub and sub commit `1eb2da46d5` Author: chrislu <chris.lu@gmail.com> Date: Mon Aug 28 09:02:12 2023 -0700 connect and publish commit `504ae8383a` Author: chrislu <chris.lu@gmail.com> Date: Mon Aug 28 09:01:25 2023 -0700 protoc version commit `dbcba75271` Author: chrislu <chris.lu@gmail.com> Date: Sun Aug 27 18:59:04 2023 -0700 rename to lookup commit `c9caf33119` Author: chrislu <chris.lu@gmail.com> Date: Sun Aug 27 18:33:46 2023 -0700 move functions commit `4d6c18d86f` Author: chrislu <chris.lu@gmail.com> Date: Sun Aug 27 17:50:59 2023 -0700 pub sub initial tests commit `4eb8e8624d` Author: chrislu <chris.lu@gmail.com> Date: Sun Aug 27 13:14:39 2023 -0700 rename commit `1990456670` Author: chrislu <chris.lu@gmail.com> Date: Sun Aug 27 13:13:14 2023 -0700 sub commit `905911853d` Author: chrislu <chris.lu@gmail.com> Date: Sat Aug 26 13:39:21 2023 -0700 adjust proto	2 years ago
chrislu	984b6c54cf	ack interval 128	2 years ago
a	7d981a1c0e	del	3 years ago
elee	921535001a	arangodb adapter	3 years ago
henry	e4ee25e6f0	remove sort	5 years ago
henry	247d5eed93	remove sort	5 years ago
henry	e92ce1d517	remove sort	5 years ago
henry	e5a3ad372f	gitignore	5 years ago
Lei Liu	1d9b75b536	weed.go: remove unused parameter Signed-off-by: Lei Liu <liul.stone@gmail.com>	6 years ago
Chris Lu	4119c61df8	HCFS can read files	7 years ago
Chris Lu	1cbd53c01c	WIP SeaweedFileSystem added mkdirs, getFileStatus, listStatus, delete	7 years ago
Chris Lu	acd8836d27	add ignore .class files	7 years ago
Chris Lu	2d13382c68	add releasing configs	9 years ago
Chris Lu	5ce6bbf076	directory structure change to work with glide glide has its own requirements. My previous workaround caused me some code checkin errors. Need to fix this.	9 years ago
Chris Lu	c24c1ffd1a	skip vendor folder	9 years ago
tnextday	662915e691	Delete all chunks when delete a ChunkManifest LoadChunkManifest can uncompress buffer move compress.go from storage to operation because of import cycle MakeFile add cross complete command	10 years ago
tnextday	a4f64c0116	edit git ignore	10 years ago
Chris Lu	2495ce6707	Adjust .gitignore	11 years ago
yanyiwu	7304c840e3	update .gitignore	11 years ago
wyy	d39c62bbed	add .gitignore	11 years ago

28 Commits (master)