chrislu
3 months ago
14 changed files with 26 additions and 617 deletions
-
321weed/admin/DESIGN.md
-
96weed/admin/NAVIGATION_TEST.md
-
12weed/admin/README.md
-
174weed/admin/S3_BUCKETS.md
-
4weed/admin/dash/admin_server.go
-
2weed/admin/dash/file_browser.go
-
4weed/admin/dash/handler_admin.go
-
8weed/admin/handlers/handlers.go
-
4weed/admin/view/app/admin.templ
-
2weed/admin/view/app/admin_templ.go
-
2weed/admin/view/app/file_browser.templ
-
2weed/admin/view/app/file_browser_templ.go
-
6weed/admin/view/app/s3_buckets.templ
-
6weed/admin/view/app/s3_buckets_templ.go
@ -1,321 +0,0 @@ |
|||
# SeaweedFS Admin Interface Web Component Design |
|||
|
|||
## Overview |
|||
|
|||
The SeaweedFS Admin Interface is a modern web-based administration interface for SeaweedFS clusters, following the **Gin + Templ + HTMX** architecture pattern. It provides comprehensive cluster management, monitoring, and maintenance capabilities through an intuitive web interface. |
|||
|
|||
## Architecture |
|||
|
|||
### Technology Stack |
|||
|
|||
- **Backend Framework**: Gin (Go HTTP web framework) |
|||
- **Template Engine**: Templ (Type-safe Go templates) |
|||
- **Frontend Enhancement**: HTMX (Dynamic interactions without JavaScript frameworks) |
|||
- **CSS Framework**: Bootstrap 5 (Modern responsive design) |
|||
- **Icons**: Font Awesome 6 (Comprehensive icon library) |
|||
- **Authentication**: Session-based with configurable credentials |
|||
|
|||
### Directory Structure |
|||
|
|||
``` |
|||
weed/admin/ |
|||
├── admin.go # Main entry point & router setup |
|||
├── dash/ # Core admin logic |
|||
│ ├── admin_server.go # Server struct & cluster operations |
|||
│ ├── handler_auth.go # Authentication handlers |
|||
│ ├── handler_admin.go # Main admin handlers |
|||
│ ├── middleware.go # Authentication middleware |
|||
│ └── ... # Additional handlers |
|||
├── view/ # Template components |
|||
│ ├── layout/ |
|||
│ │ └── layout.templ # Base layout & login form |
|||
│ └── app/ |
|||
│ ├── admin.templ # Admin page template |
|||
│ └── template_helpers.go # Formatting utilities |
|||
├── static/ # Static assets |
|||
│ ├── css/ |
|||
│ │ └── admin.css # Custom styles |
|||
│ └── js/ |
|||
│ └── admin.js # Interactive functionality |
|||
└── templates/ # Embedded templates |
|||
``` |
|||
|
|||
## Core Features |
|||
|
|||
### 1. **Cluster Management** |
|||
|
|||
#### Topology Visualization |
|||
- **Data Center/Rack/Node Hierarchy**: Visual representation of cluster topology |
|||
- **Real-time Status Monitoring**: Live status updates for all cluster components |
|||
- **Capacity Planning**: Volume utilization and capacity tracking |
|||
- **Health Assessment**: Automated health scoring and alerts |
|||
|
|||
#### Master Node Management |
|||
- **Leader/Follower Status**: Clear indication of Raft leadership |
|||
- **Master Configuration**: View and modify master settings |
|||
- **Cluster Membership**: Add/remove master nodes |
|||
- **Heartbeat Monitoring**: Track master node availability |
|||
|
|||
#### Volume Server Operations |
|||
- **Server Registration**: Automatic detection of new volume servers |
|||
- **Disk Usage Monitoring**: Real-time disk space and volume tracking |
|||
- **Performance Metrics**: I/O statistics and throughput monitoring |
|||
- **Maintenance Mode**: Graceful server shutdown and maintenance |
|||
|
|||
### 2. **Volume Management** |
|||
|
|||
#### Volume Operations |
|||
- **Volume Creation**: Create new volumes with replication settings |
|||
- **Volume Listing**: Comprehensive volume inventory with search/filter |
|||
- **Volume Details**: Detailed information per volume (files, size, replicas) |
|||
- **Volume Migration**: Move volumes between servers |
|||
- **Volume Deletion**: Safe volume removal with confirmation |
|||
|
|||
#### Storage Operations |
|||
- **Volume Growing**: Automatic volume expansion based on policies |
|||
- **Vacuum Operations**: Reclaim deleted file space |
|||
- **Compaction**: Optimize volume storage efficiency |
|||
- **Rebalancing**: Distribute volumes evenly across servers |
|||
|
|||
### 3. **File Management** |
|||
|
|||
#### File Browser |
|||
- **Directory Navigation**: Browse filer directories with breadcrumbs |
|||
- **File Operations**: Upload, download, delete, rename files |
|||
- **Batch Operations**: Multi-file operations with progress tracking |
|||
- **Metadata Display**: File attributes, timestamps, permissions |
|||
- **Search Functionality**: Find files by name, type, or content |
|||
|
|||
#### Storage Analytics |
|||
- **Usage Statistics**: File count, size distribution, growth trends |
|||
- **Access Patterns**: Popular files and access frequency |
|||
- **Storage Efficiency**: Compression ratios and duplicate detection |
|||
|
|||
### 4. **Monitoring & Metrics** |
|||
|
|||
#### Real-time Dashboards |
|||
- **System Overview**: Cluster health at a glance |
|||
- **Performance Metrics**: Throughput, latency, and error rates |
|||
- **Resource Utilization**: CPU, memory, disk, and network usage |
|||
- **Historical Trends**: Long-term performance analysis |
|||
|
|||
#### Alerting System |
|||
- **Threshold Monitoring**: Configurable alerts for key metrics |
|||
- **Health Checks**: Automated health assessment and scoring |
|||
- **Notification Channels**: Email, webhook, and dashboard notifications |
|||
|
|||
### 5. **Configuration Management** |
|||
|
|||
#### Cluster Configuration |
|||
- **Master Settings**: Replication, security, and operational parameters |
|||
- **Volume Server Config**: Storage paths, limits, and performance settings |
|||
- **Filer Configuration**: Metadata storage and caching options |
|||
- **Security Settings**: Authentication, authorization, and encryption |
|||
|
|||
#### Backup & Restore |
|||
- **Configuration Backup**: Export cluster configuration |
|||
- **Configuration Restore**: Import and apply saved configurations |
|||
- **Version Control**: Track configuration changes over time |
|||
|
|||
### 6. **System Maintenance** |
|||
|
|||
#### Maintenance Operations |
|||
- **Garbage Collection**: Clean up orphaned files and metadata |
|||
- **Volume Repair**: Fix corrupted or inconsistent volumes |
|||
- **Cluster Validation**: Verify cluster integrity and consistency |
|||
- **Performance Tuning**: Optimize cluster performance parameters |
|||
|
|||
#### Log Management |
|||
- **Log Aggregation**: Centralized logging from all cluster components |
|||
- **Log Analysis**: Search, filter, and analyze system logs |
|||
- **Error Tracking**: Identify and track system errors and warnings |
|||
- **Log Export**: Download logs for external analysis |
|||
|
|||
## User Interface Design |
|||
|
|||
### Layout Components |
|||
|
|||
#### Header Navigation |
|||
- **Cluster Status Indicator**: Quick health overview |
|||
- **User Information**: Current user and session details |
|||
- **Quick Actions**: Frequently used operations |
|||
- **Search Bar**: Global search across cluster resources |
|||
|
|||
#### Sidebar Navigation |
|||
- **Cluster Section**: Topology, status, and management |
|||
- **Management Section**: Files, volumes, and operations |
|||
- **System Section**: Configuration, logs, and maintenance |
|||
- **Contextual Actions**: Dynamic actions based on current view |
|||
|
|||
#### Main Content Area |
|||
- **Dashboard Cards**: Key metrics and status summaries |
|||
- **Data Tables**: Sortable, filterable resource listings |
|||
- **Interactive Charts**: Real-time metrics visualization |
|||
- **Action Panels**: Operation forms and bulk actions |
|||
|
|||
### Responsive Design |
|||
- **Mobile Responsive**: Optimized for tablets and mobile devices |
|||
- **Progressive Enhancement**: Works with JavaScript disabled |
|||
- **Accessibility**: WCAG 2.1 compliant interface |
|||
- **Theme Support**: Light/dark mode support |
|||
|
|||
## Security Features |
|||
|
|||
### Authentication & Authorization |
|||
- **Configurable Authentication**: Optional password protection |
|||
- **Session Management**: Secure session handling with timeouts |
|||
- **Role-based Access**: Different permission levels for users |
|||
- **Audit Logging**: Track all administrative actions |
|||
|
|||
### Security Hardening |
|||
- **HTTPS Support**: TLS encryption for all communications |
|||
- **CSRF Protection**: Cross-site request forgery prevention |
|||
- **Input Validation**: Comprehensive input sanitization |
|||
- **Rate Limiting**: Prevent abuse and DoS attacks |
|||
|
|||
## API Design |
|||
|
|||
### RESTful Endpoints |
|||
```go |
|||
// Public endpoints |
|||
GET /health # Health check |
|||
GET /login # Login form |
|||
POST /login # Process login |
|||
GET /logout # Logout |
|||
|
|||
// Protected endpoints |
|||
GET /admin # Main admin interface |
|||
GET /overview # Cluster overview API |
|||
|
|||
// Cluster management |
|||
GET /cluster # Cluster topology view |
|||
GET /cluster/topology # Topology data API |
|||
GET /cluster/status # Cluster status API |
|||
POST /cluster/grow # Grow volumes |
|||
POST /cluster/vacuum # Vacuum operation |
|||
POST /cluster/rebalance # Rebalance cluster |
|||
|
|||
// Volume management |
|||
GET /volumes # Volumes list page |
|||
GET /volumes/list # Volumes data API |
|||
GET /volumes/details/:id # Volume details |
|||
POST /volumes/create # Create volume |
|||
DELETE /volumes/delete/:id # Delete volume |
|||
|
|||
// File management |
|||
GET /filer # File browser page |
|||
GET /filer/browser # File browser interface |
|||
GET /filer/browser/api/* # File operations API |
|||
POST /filer/upload # File upload |
|||
DELETE /filer/delete # File deletion |
|||
|
|||
// Monitoring |
|||
GET /metrics # Metrics dashboard |
|||
GET /metrics/data # Metrics data API |
|||
GET /metrics/realtime # Real-time metrics |
|||
GET /logs # Logs viewer |
|||
GET /logs/download/:type # Download logs |
|||
|
|||
// Configuration |
|||
GET /config # Configuration page |
|||
GET /config/current # Current configuration |
|||
POST /config/update # Update configuration |
|||
GET /config/backup # Backup configuration |
|||
|
|||
// Maintenance |
|||
GET /maintenance # Maintenance page |
|||
POST /maintenance/gc # Garbage collection |
|||
POST /maintenance/compact # Volume compaction |
|||
GET /maintenance/status # Maintenance status |
|||
``` |
|||
|
|||
## Development Guidelines |
|||
|
|||
### Code Organization |
|||
- **Handler Separation**: Separate files for different functional areas |
|||
- **Type Safety**: Use strongly typed structures for all data |
|||
- **Error Handling**: Comprehensive error handling and user feedback |
|||
- **Testing**: Unit and integration tests for all components |
|||
|
|||
### Performance Considerations |
|||
- **Caching Strategy**: Intelligent caching of cluster topology and metrics |
|||
- **Lazy Loading**: Load data on demand to improve responsiveness |
|||
- **Batch Operations**: Efficient bulk operations for large datasets |
|||
- **Compression**: Gzip compression for API responses |
|||
|
|||
### Monitoring Integration |
|||
- **Metrics Export**: Prometheus-compatible metrics endpoint |
|||
- **Health Checks**: Kubernetes-style health and readiness probes |
|||
- **Distributed Tracing**: OpenTelemetry integration for request tracing |
|||
- **Structured Logging**: JSON logging for better observability |
|||
|
|||
## Deployment Options |
|||
|
|||
### Standalone Deployment |
|||
```bash |
|||
# Start dashboard server |
|||
./weed dashboard -port=9999 \ |
|||
-masters="master1:9333,master2:9333" \ |
|||
-filer="filer:8888" \ |
|||
-adminUser="admin" \ |
|||
-adminPassword="secretpassword" |
|||
``` |
|||
|
|||
### Docker Deployment |
|||
```yaml |
|||
# docker-compose.yml |
|||
version: '3.8' |
|||
services: |
|||
dashboard: |
|||
image: seaweedfs:latest |
|||
command: dashboard -port=9999 -masters=master:9333 -filer=filer:8888 |
|||
ports: |
|||
- "9999:9999" |
|||
environment: |
|||
- ADMIN_USER=admin |
|||
- ADMIN_PASSWORD=secretpassword |
|||
``` |
|||
|
|||
### Kubernetes Deployment |
|||
```yaml |
|||
apiVersion: apps/v1 |
|||
kind: Deployment |
|||
metadata: |
|||
name: seaweedfs-dashboard |
|||
spec: |
|||
replicas: 1 |
|||
selector: |
|||
matchLabels: |
|||
app: seaweedfs-dashboard |
|||
template: |
|||
metadata: |
|||
labels: |
|||
app: seaweedfs-dashboard |
|||
spec: |
|||
containers: |
|||
- name: dashboard |
|||
image: seaweedfs:latest |
|||
command: ["weed", "dashboard"] |
|||
args: |
|||
- "-port=9999" |
|||
- "-masters=seaweedfs-master:9333" |
|||
- "-filer=seaweedfs-filer:8888" |
|||
ports: |
|||
- containerPort: 9999 |
|||
``` |
|||
|
|||
## Future Enhancements |
|||
|
|||
### Advanced Features |
|||
- **Multi-cluster Management**: Manage multiple SeaweedFS clusters |
|||
- **Advanced Analytics**: Machine learning-powered insights |
|||
- **Custom Dashboards**: User-configurable dashboard layouts |
|||
- **API Integration**: Webhook integration with external systems |
|||
|
|||
### Enterprise Features |
|||
- **SSO Integration**: LDAP, OAuth, and SAML authentication |
|||
- **Advanced RBAC**: Fine-grained permission system |
|||
- **Audit Compliance**: SOX, HIPAA, and PCI compliance features |
|||
- **High Availability**: Multi-instance dashboard deployment |
|||
|
|||
This design provides a comprehensive, modern, and scalable web interface for SeaweedFS administration, following industry best practices and providing an excellent user experience for cluster operators and administrators. |
|||
@ -1,96 +0,0 @@ |
|||
# Navigation Menu Test |
|||
|
|||
## Quick Test Guide |
|||
|
|||
To verify that the S3 Buckets link appears in the navigation menu: |
|||
|
|||
### 1. Start the Admin Server |
|||
```bash |
|||
# Start with minimal setup |
|||
weed admin -port=23646 -masters=localhost:9333 -filer=localhost:8888 |
|||
|
|||
# Or with dummy values for testing UI only |
|||
weed admin -port=23646 -masters=dummy:9333 -filer=dummy:8888 |
|||
``` |
|||
|
|||
### 2. Open Browser |
|||
Navigate to: `http://localhost:23646` |
|||
|
|||
### 3. Check Navigation Menu |
|||
Look for the sidebar navigation on the left side. You should see: |
|||
|
|||
**CLUSTER Section:** |
|||
- Admin |
|||
- Cluster |
|||
- Volumes |
|||
|
|||
**MANAGEMENT Section:** |
|||
- **S3 Buckets** ← This should be visible! |
|||
- File Browser |
|||
- Metrics |
|||
- Logs |
|||
|
|||
**SYSTEM Section:** |
|||
- Configuration |
|||
- Maintenance |
|||
|
|||
### 4. Test S3 Buckets Link |
|||
- Click on "S3 Buckets" in the sidebar |
|||
- Should navigate to `/s3/buckets` |
|||
- Should show the S3 bucket management page |
|||
- The "S3 Buckets" menu item should be highlighted as active |
|||
|
|||
### 5. Expected Behavior |
|||
- Menu item has cube icon: `📦 S3 Buckets` |
|||
- Link points to `/s3/buckets` |
|||
- Active state highlighting works |
|||
- Page loads S3 bucket management interface |
|||
|
|||
## Troubleshooting |
|||
|
|||
If the S3 Buckets link is not visible: |
|||
|
|||
1. **Check Template Generation:** |
|||
```bash |
|||
cd weed/admin |
|||
templ generate |
|||
``` |
|||
|
|||
2. **Rebuild Binary:** |
|||
```bash |
|||
cd ../.. |
|||
go build -o weed weed/weed.go |
|||
``` |
|||
|
|||
3. **Check Browser Console:** |
|||
- Open Developer Tools (F12) |
|||
- Look for any JavaScript errors |
|||
- Check Network tab for failed requests |
|||
|
|||
4. **Verify File Structure:** |
|||
```bash |
|||
ls -la weed/admin/view/layout/layout_templ.go |
|||
``` |
|||
|
|||
5. **Check Server Logs:** |
|||
- Look for any error messages when starting admin server |
|||
- Check for template compilation errors |
|||
|
|||
## Files Modified |
|||
|
|||
- `weed/admin/view/layout/layout.templ` - Added S3 Buckets menu item |
|||
- `weed/admin/static/js/admin.js` - Updated navigation highlighting |
|||
- `weed/command/admin.go` - Added S3 routes |
|||
|
|||
## Expected Navigation Structure |
|||
|
|||
```html |
|||
<ul class="nav flex-column"> |
|||
<li class="nav-item"> |
|||
<a class="nav-link" href="/s3/buckets"> |
|||
<i class="fas fa-cube me-2"></i>S3 Buckets |
|||
</a> |
|||
</li> |
|||
<!-- ... other menu items ... --> |
|||
</ul> |
|||
``` |
|||
@ -1,174 +0,0 @@ |
|||
# S3 Bucket Management |
|||
|
|||
The SeaweedFS Admin Interface now includes comprehensive S3 bucket management capabilities. |
|||
|
|||
## Features |
|||
|
|||
### Bucket Overview |
|||
- **Dashboard View**: List all S3 buckets with summary statistics |
|||
- **Bucket Statistics**: Total buckets, storage usage, object counts |
|||
- **Status Monitoring**: Real-time bucket status and health indicators |
|||
|
|||
### Bucket Operations |
|||
- **Create Buckets**: Create new S3 buckets |
|||
- **Delete Buckets**: Remove buckets and all their contents (with confirmation) |
|||
- **View Details**: Browse bucket contents and object listings |
|||
- **Export Data**: Export bucket lists to CSV format |
|||
|
|||
### Bucket Information |
|||
Each bucket displays: |
|||
- **Name**: Bucket identifier |
|||
- **Created Date**: When the bucket was created |
|||
- **Object Count**: Number of objects stored |
|||
- **Total Size**: Storage space used (formatted in KB/MB/GB/TB) |
|||
- **Region**: Configured AWS region |
|||
- **Status**: Current operational status |
|||
|
|||
## Usage |
|||
|
|||
### Accessing S3 Bucket Management |
|||
|
|||
1. Start the admin server: |
|||
```bash |
|||
weed admin -port=23646 -masters=localhost:9333 -filer=localhost:8888 |
|||
``` |
|||
|
|||
2. Open your browser to: `http://localhost:23646` |
|||
|
|||
3. Click the "S3 Buckets" button in the dashboard toolbar |
|||
|
|||
4. Or navigate directly to: `http://localhost:23646/s3/buckets` |
|||
|
|||
### Creating a New Bucket |
|||
|
|||
1. Click the "Create Bucket" button |
|||
2. Enter a valid bucket name (3-63 characters, lowercase letters, numbers, dots, hyphens) |
|||
3. Select a region (defaults to us-east-1) |
|||
4. Click "Create Bucket" |
|||
|
|||
### Deleting a Bucket |
|||
|
|||
1. Click the trash icon next to the bucket name |
|||
2. Confirm the deletion in the modal dialog |
|||
3. **Warning**: This permanently deletes the bucket and all its contents |
|||
|
|||
### Viewing Bucket Details |
|||
|
|||
1. Click on a bucket name to view detailed information |
|||
2. See all objects within the bucket |
|||
3. View object metadata (size, last modified, etc.) |
|||
|
|||
## API Endpoints |
|||
|
|||
The S3 bucket management feature exposes REST API endpoints: |
|||
|
|||
### List Buckets |
|||
``` |
|||
GET /api/s3/buckets |
|||
``` |
|||
Returns JSON array of all buckets with metadata. |
|||
|
|||
### Create Bucket |
|||
``` |
|||
POST /api/s3/buckets |
|||
Content-Type: application/json |
|||
|
|||
{ |
|||
"name": "my-bucket-name", |
|||
"region": "us-east-1" |
|||
} |
|||
``` |
|||
|
|||
### Delete Bucket |
|||
``` |
|||
DELETE /api/s3/buckets/{bucket-name} |
|||
``` |
|||
Permanently deletes the bucket and all contents. |
|||
|
|||
### Get Bucket Details |
|||
``` |
|||
GET /api/s3/buckets/{bucket-name} |
|||
``` |
|||
Returns detailed bucket information including object listings. |
|||
|
|||
## Technical Implementation |
|||
|
|||
### Backend Integration |
|||
- **Filer Integration**: Uses SeaweedFS filer for bucket storage at `/buckets/` |
|||
- **Streaming API**: Efficiently handles large bucket listings |
|||
- **Error Handling**: Comprehensive error reporting and recovery |
|||
|
|||
### Frontend Features |
|||
- **Bootstrap UI**: Modern, responsive web interface |
|||
- **Real-time Updates**: Automatic refresh after operations |
|||
- **Form Validation**: Client-side bucket name validation |
|||
- **Modal Dialogs**: User-friendly create/delete workflows |
|||
|
|||
### Security Considerations |
|||
- **Confirmation Dialogs**: Prevent accidental deletions |
|||
- **Input Validation**: Prevent invalid bucket names |
|||
- **Error Messages**: Clear feedback for failed operations |
|||
|
|||
## Bucket Naming Rules |
|||
|
|||
S3 bucket names must follow these rules: |
|||
- 3-63 characters in length |
|||
- Contain only lowercase letters, numbers, dots (.), and hyphens (-) |
|||
- Start and end with a lowercase letter or number |
|||
- Cannot contain spaces or special characters |
|||
- Cannot be formatted as an IP address |
|||
|
|||
## Storage Structure |
|||
|
|||
Buckets are stored in the SeaweedFS filer at: |
|||
``` |
|||
/buckets/{bucket-name}/ |
|||
``` |
|||
|
|||
Each bucket directory contains: |
|||
- Object files with their original names |
|||
- Nested directories for object key prefixes |
|||
- Metadata preserved from S3 operations |
|||
|
|||
## Performance Notes |
|||
|
|||
- **Lazy Loading**: Bucket sizes and object counts are calculated on-demand |
|||
- **Streaming**: Large bucket listings use streaming responses |
|||
- **Caching**: Cluster topology data is cached for performance |
|||
- **Pagination**: Large object lists are handled efficiently |
|||
|
|||
## Troubleshooting |
|||
|
|||
### Common Issues |
|||
|
|||
1. **Bucket Creation Fails** |
|||
- Check bucket name follows S3 naming rules |
|||
- Ensure filer is accessible and running |
|||
- Verify sufficient storage space |
|||
|
|||
2. **Bucket Deletion Fails** |
|||
- Ensure bucket exists and is accessible |
|||
- Check for permission issues |
|||
- Verify filer connectivity |
|||
|
|||
3. **Bucket List Empty** |
|||
- Verify filer has `/buckets/` directory |
|||
- Check filer connectivity |
|||
- Ensure buckets were created through S3 API |
|||
|
|||
### Debug Steps |
|||
|
|||
1. Check admin server logs for error messages |
|||
2. Verify filer is running and accessible |
|||
3. Test filer connectivity: `curl http://localhost:8888/` |
|||
4. Check browser console for JavaScript errors |
|||
|
|||
## Future Enhancements |
|||
|
|||
- **Bucket Policies**: Manage access control policies |
|||
- **Lifecycle Rules**: Configure object lifecycle management |
|||
- **Versioning**: Enable/disable bucket versioning |
|||
- **Replication**: Configure cross-region replication |
|||
- **Metrics**: Detailed usage and performance metrics |
|||
- **Notifications**: Bucket event notifications |
|||
- **Search**: Search and filter bucket contents |
|||
Write
Preview
Loading…
Cancel
Save
Reference in new issue