Added comprehensive logging to identify why Parquet files fail with
'EOFException: Still have: 78 bytes left'.
Key additions:
1. SeaweedHadoopOutputStream constructor logging with 🔧 marker
- Shows when output streams are created
- Logs path, position, bufferSize, replication
2. totalBytesWritten counter in SeaweedOutputStream
- Tracks cumulative bytes written via write() calls
- Helps identify if Parquet wrote 762 bytes but only 684 reached chunks
3. Enhanced close() logging with 🔒 and ✅ markers
- Shows totalBytesWritten vs position vs buffer.position()
- If totalBytesWritten=762 but position=684, write submission failed
- If buffer.position()=78 at close, buffer wasn't flushed
Expected scenarios in next run:
A) Stream never created → No 🔧 log for .parquet files
B) Write failed → totalBytesWritten=762 but position=684
C) Buffer not flushed → buffer.position()=78 at close
D) All correct → totalBytesWritten=position=684, but Parquet expects 762
This will pinpoint whether the issue is in:
- Stream creation/lifecycle
- Write submission
- Buffer flushing
- Or Parquet's internal state
The persistent EOFException shows Parquet expects 78 more bytes than exist.
This suggests a mismatch between what was written vs what's in chunks.
Added logging to track:
1. Buffer state at close (position before flush)
2. Stream position when flushing metadata
3. Chunk count vs file size in attributes
4. Explicit fileSize setting from stream position
Key hypothesis:
- Parquet writes N bytes total (e.g., 762)
- Stream.position tracks all writes
- But only (N-78) bytes end up in chunks
- This causes Parquet read to fail with 'Still have: 78 bytes left'
If buffer.position() = 78 at close, the buffer wasn't flushed.
If position != chunk total, write submission failed.
If attr.fileSize != position, metadata is inconsistent.
Next run will show which scenario is happening.
ROOT CAUSE: Maven was downloading seaweedfs-client:3.80 from Maven Central
instead of using the locally built version in CI!
Changes:
- Changed all versions from 3.80 to 3.80.1-SNAPSHOT
- other/java/client/pom.xml: 3.80 → 3.80.1-SNAPSHOT
- other/java/hdfs2/pom.xml: property 3.80 → 3.80.1-SNAPSHOT
- other/java/hdfs3/pom.xml: property 3.80 → 3.80.1-SNAPSHOT
- test/java/spark/pom.xml: property 3.80 → 3.80.1-SNAPSHOT
Maven behavior:
- Release versions (3.80): Downloaded from remote repos if available
- SNAPSHOT versions: Prefer local builds, can be updated
This ensures the CI uses the locally built JARs with our debug logging!
Also added unique [DEBUG-2024] markers to verify in logs.
chrislu
dependabot[bot]
orthoxerox
chrislu
Chris Lu
limd