chrislu
|
fe41380d51
|
feat: Phase 2 - Add DDL operations and real MQ broker integration
Implements comprehensive DDL support for MQ topic management:
New Components:
- Real MQ broker connectivity via BrokerClient
- CREATE TABLE → ConfigureTopic gRPC calls
- DROP TABLE → DeleteTopic operations
- DESCRIBE table → Schema introspection
- SQL type mapping (SQL ↔ MQ schema types)
Enhanced Features:
- Live topic discovery from MQ broker
- Fallback to cached/sample data when broker unavailable
- MySQL-compatible DESCRIBE output
- Schema validation and error handling
- CREATE TABLE with column definitions
Key Infrastructure:
- broker_client.go: gRPC communication with MQ broker
- sql_types.go: Bidirectional SQL/MQ type conversion
- describe.go: Table schema introspection
- Enhanced engine.go: Full DDL routing and execution
Supported SQL Operations:
✅ SHOW DATABASES, SHOW TABLES (live + fallback)
✅ CREATE TABLE table_name (col1 INT, col2 VARCHAR(50), ...)
✅ DROP TABLE table_name
✅ DESCRIBE table_name / SHOW COLUMNS FROM table_name
Known Limitations:
- SQL parser issues with reserved keywords (e.g., 'timestamp')
- Requires running MQ broker for full functionality
- ALTER TABLE not yet implemented
- DeleteTopic method needs broker-side implementation
Architecture Decisions:
- Broker discovery via filer lock mechanism (same as shell commands)
- Graceful fallback when broker unavailable
- ConfigureTopic for CREATE TABLE with 6 default partitions
- Schema versioning ready for ALTER TABLE support
Testing:
- Unit tests updated with filer address parameter
- Integration tests for DDL operations
- Error handling for connection failures
Next Phase: SELECT query execution with Parquet scanning
|
2 months ago |
