chrislu
a5bccca443
debug: add critical diagnostics for EOFException (78 bytes missing)
The persistent EOFException shows Parquet expects 78 more bytes than exist.
This suggests a mismatch between what was written vs what's in chunks.
Added logging to track:
1. Buffer state at close (position before flush)
2. Stream position when flushing metadata
3. Chunk count vs file size in attributes
4. Explicit fileSize setting from stream position
Key hypothesis:
- Parquet writes N bytes total (e.g., 762)
- Stream.position tracks all writes
- But only (N-78) bytes end up in chunks
- This causes Parquet read to fail with 'Still have: 78 bytes left'
If buffer.position() = 78 at close, the buffer wasn't flushed.
If position != chunk total, write submission failed.
If attr.fileSize != position, metadata is inconsistent.
Next run will show which scenario is happening.
1 week ago
dependabot[bot]
c14e513964
chore(deps): bump org.apache.hadoop:hadoop-common from 3.2.4 to 3.4.0 in /other/java/hdfs3 ( #7512 )
* chore(deps): bump org.apache.hadoop:hadoop-common in /other/java/hdfs3
Bumps org.apache.hadoop:hadoop-common from 3.2.4 to 3.4.0.
---
updated-dependencies:
- dependency-name: org.apache.hadoop:hadoop-common
dependency-version: 3.4.0
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
* add java client unit tests
* Update dependency-reduced-pom.xml
* add java integration tests
* fix
* fix buffer
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
2 weeks ago
chrislu
2caa0e3741
java 3.80
1 year ago
chrislu
915f9f5054
update java client to 3.71, also adjust the groupId
1 year ago
chrislu
83fe2bfc36
java 3.71
1 year ago
chrislu
cd01a2346a
Java 3.59
fix https://github.com/seaweedfs/seaweedfs/issues/5001
2 years ago
chrislu
710e88f713
Java: upgrade to 3.55
2 years ago
chrislu
ea2637734a
refactor filer proto chunk variable from mtime to modified_ts_ns
3 years ago
chrislu
707abd5b2d
3.30 java
3 years ago
chrislu
d003bb0166
java 3.13
3 years ago
chrislu
19555385f7
2.85
4 years ago
chrislu
5799a20f71
2.84
4 years ago
chrislu
5ea9715721
2.81
also sync java client version to SeaweedFS version
4 years ago
Chris Lu
6fb6480a3b
Java: 1.7.0
update org.apache.httpcomponents to 4.5.13
update grpc API to use cacheRemoteObjectToLocalCluster
4 years ago
Chris Lu
8cd7a0365b
1.6.9
4 years ago
Chris Lu
20ac710ceb
2.68
4 years ago
Chris Lu
e9128e75d0
Java: 1.6.7 Support Mounted Remote Storage
4 years ago
Chris Lu
2d85ffe7c5
java 1.6.6
5 years ago
Chris Lu
6c1c72b1f4
java client 1.6.5
5 years ago
Chris Lu
be25bc6766
Java client 1.6.4
5 years ago
Chris Lu
b5e10bf511
Java client 1.6.3
5 years ago
Chris Lu
c276117fef
Java: 1.6.2
5 years ago
Chris Lu
5b1def9080
Java: 1.6.1 refacoring API
5 years ago
Chris Lu
5138d3954f
Java 1.6.0 fix filerProxy mode
5 years ago
Chris Lu
9c1efdf11b
HCFS: 1.5.9
5 years ago
Chris Lu
6a2a9b67e8
Java: 1.5.8
5 years ago
Chris Lu
87d1bfa862
Hadoop Compatible FS: 1.5.7
5 years ago
Chris Lu
da04bb3d1b
1.5.6
5 years ago
Chris Lu
7d24980840
HCFS: 1.5.4
5 years ago
Chris Lu
4f676aa7d3
2.09
5 years ago
Chris Lu
596d476e3d
HCFS: 1.5.2
5 years ago
Chris Lu
b7c224cf61
HCFS: 1.5.1
5 years ago
Chris Lu
253323ce6d
Hadoop 1.5.0
5 years ago
Chris Lu
8e2f3b510e
Hadoop client: 1.4.9
5 years ago
Chris Lu
436ab7d811
HCFS: 1.4.8
5 years ago
Chris Lu
5eee4983f3
1.4.7 hdfs configurable fs.seaweed.buffer.size
5 years ago
Chris Lu
48ed7f8a32
generated files
5 years ago
Chris Lu
f5d6b8b777
Hadoop HCFS 1.4.6 random access file
5 years ago
Chris Lu
4f195a54ca
1.4.5
change the scope for hadoop depencies to provided
5 years ago
Chris Lu
e1f070a9a1
Hadoop: 1.4.4
5 years ago
Chris Lu
6b41c5250b
Hadoop file system: 1.4.3
added buffered fs input stream
5 years ago
Chris Lu
6f058b3791
HCFS: 1.4.1
5 years ago
Chris Lu
54f9f9138e
HCFS: 1.4.0
5 years ago
Chris Lu
f90d2c93c9
1.3.9 remove logs
5 years ago
Chris Lu
3abd74b1d7
1.3.8
5 years ago
Chris Lu
54445b207d
Hadoop: 1.3.6
5 years ago
Chris Lu
7bca72deed
reuse bytebuffer
5 years ago
Chris Lu
86c8137546
1.3.4
5 years ago
Chris Lu
bc3be0bb37
Hadoop: 1.3.3
improve memory efficiency
5 years ago
Chris Lu
fc9af962ec
1.3.2
5 years ago