6.6 KiB

Raw Blame History

SeaweedFS S3 Object Retention Tests

This directory contains comprehensive tests for SeaweedFS S3 Object Retention functionality, including Object Lock, Legal Hold, and WORM (Write Once Read Many) capabilities.

Overview

The test suite validates AWS S3-compatible object retention features including:

Object Retention: GOVERNANCE and COMPLIANCE modes with retain-until-date
Legal Hold: Independent protection that can be applied/removed
Object Lock Configuration: Bucket-level default retention policies
WORM Integration: Compatibility with legacy WORM functionality
Version-specific Retention: Different retention policies per object version
Enforcement: Protection against deletion and overwriting

Test Files

s3_retention_test.go - Core retention functionality tests
s3_worm_integration_test.go - WORM integration and advanced scenarios
test_config.json - Test configuration (endpoints, credentials)
Makefile - Comprehensive test automation
go.mod - Go module dependencies

Prerequisites

Go 1.21 or later
SeaweedFS binary built (make build-weed)
AWS SDK Go v2
Testify testing framework

Quick Start

1. Build and Start Server

# Build SeaweedFS and start test server
make start-server

2. Run Tests

# Run core retention tests
make test-retention-quick

# Run all retention tests
make test-retention

# Run WORM integration tests  
make test-retention-worm

# Run all tests with managed server
make test-with-server

3. Cleanup

make clean

Test Categories

Core Retention Tests

TestBasicRetentionWorkflow - Basic GOVERNANCE mode retention
TestRetentionModeCompliance - COMPLIANCE mode (immutable)
TestLegalHoldWorkflow - Legal hold on/off functionality
TestObjectLockConfiguration - Bucket object lock settings

Advanced Tests

TestRetentionWithVersions - Version-specific retention policies
TestRetentionAndLegalHoldCombination - Multiple protection types
TestExpiredRetention - Post-expiration behavior
TestRetentionErrorCases - Error handling and edge cases

WORM Integration Tests

TestWORMRetentionIntegration - New retention + legacy WORM
TestWORMLegacyCompatibility - Backward compatibility
TestRetentionOverwriteProtection - Prevent overwrites
TestRetentionBulkOperations - Bulk delete with retention
TestRetentionWithMultipartUpload - Multipart upload retention
TestRetentionExtendedAttributes - Extended attribute storage
TestRetentionBucketDefaults - Default retention application
TestRetentionConcurrentOperations - Concurrent operation safety

Individual Test Targets

Run specific test categories:

# Basic functionality
make test-basic-retention
make test-compliance-retention  
make test-legal-hold

# Advanced features
make test-retention-versions
make test-retention-combination
make test-expired-retention

# WORM integration
make test-worm-integration
make test-worm-legacy
make test-retention-bulk

Configuration

Server Configuration

The tests use these default settings:

S3 Port: 8333
Test timeout: 15 minutes
Volume directory: ./test-volume-data

Test Configuration (`test_config.json`)

{
  "endpoint": "http://localhost:8333",
  "access_key": "some_access_key1", 
  "secret_key": "some_secret_key1",
  "region": "us-east-1",
  "bucket_prefix": "test-retention-",
  "use_ssl": false,
  "skip_verify_ssl": true
}

Expected Behavior

GOVERNANCE Mode

Objects protected until retain-until-date
Can be bypassed with x-amz-bypass-governance-retention header
Supports time extension (not reduction)

COMPLIANCE Mode

Objects immutably protected until retain-until-date
Cannot be bypassed or shortened
Strictest protection level

Legal Hold

Independent ON/OFF protection
Can coexist with retention policies
Must be explicitly removed to allow deletion

Version Support

Each object version can have individual retention
Applies to both versioned and non-versioned buckets
Version-specific retention retrieval

Development

Running in Development Mode

# Start server for development
make dev-start

# Run quick test
make dev-test

Code Quality

# Format code
make fmt

# Run linter
make lint

# Generate coverage report
make coverage

Performance Testing

# Run benchmarks
make benchmark-retention

Troubleshooting

Server Won't Start

# Check if port is in use
netstat -tlnp | grep 8333

# View server logs
make logs

# Force cleanup
make clean

Test Failures

# Run with verbose output
go test -v -timeout=15m .

# Run specific test
go test -v -run TestBasicRetentionWorkflow .

# Check server health
make health-check

Dependencies

# Install/update dependencies
make install-deps

# Check dependency status
make check-deps

Integration with SeaweedFS

These tests validate the retention implementation in:

weed/s3api/s3api_object_retention.go - Core retention logic
weed/s3api/s3api_object_handlers_retention.go - HTTP handlers
weed/s3api/s3_constants/extend_key.go - Extended attribute keys
weed/s3api/s3err/s3api_errors.go - Error definitions
weed/s3api/s3api_object_handlers_delete.go - Deletion enforcement
weed/s3api/s3api_object_handlers_put.go - Upload enforcement

AWS CLI Compatibility

The retention implementation supports standard AWS CLI commands:

# Set object retention
aws s3api put-object-retention \
  --bucket mybucket \
  --key myobject \
  --retention Mode=GOVERNANCE,RetainUntilDate=2024-12-31T23:59:59Z

# Get object retention
aws s3api get-object-retention \
  --bucket mybucket \
  --key myobject

# Set legal hold
aws s3api put-object-legal-hold \
  --bucket mybucket \
  --key myobject \
  --legal-hold Status=ON

# Configure bucket object lock
aws s3api put-object-lock-configuration \
  --bucket mybucket \
  --object-lock-configuration ObjectLockEnabled=Enabled,Rule='{DefaultRetention={Mode=GOVERNANCE,Days=30}}'

Contributing

When adding new retention tests:

Follow existing test patterns
Use descriptive test names
Include both positive and negative test cases
Test error conditions
Update this README with new test descriptions
Add appropriate Makefile targets for new test categories

6.6 KiB Raw Blame History