Commit Graph

  • a1fa949221 feat: extract chunk IDs from write log and download from volume chrislu 2025-11-23 21:32:02 -0800
  • c774b807e1 fix: search temporary directories for Parquet files chrislu 2025-11-23 21:23:33 -0800
  • 7b9b04cd59 feat: add explicit logging when employees Parquet file is written chrislu 2025-11-23 21:14:25 -0800
  • 09b0a2505c fix: poll for files to appear instead of fixed sleep chrislu 2025-11-23 21:12:52 -0800
  • 64357e73bf feat: proactive download - grab files BEFORE Spark deletes them chrislu 2025-11-23 20:43:32 -0800
  • 8e0635b8ba fix: search for filename in 'Encountered error' message chrislu 2025-11-23 20:32:57 -0800
  • c5c29bc820 fix: search for failing file in read context (SeaweedInputStream) chrislu 2025-11-23 20:13:30 -0800
  • e76107c22e fix: extract chunk ID for the EXACT file causing EOF error chrislu 2025-11-23 20:03:28 -0800
  • 0afe330b4e feat: add detailed offset analysis for 78-byte discrepancy chrislu 2025-11-23 19:58:34 -0800
  • 72b4bf9098 fix: extract correct chunk ID (not source_file_id) chrislu 2025-11-23 19:37:28 -0800
  • 4ec6fbcdc7 fix: download Parquet data directly from volume server chrislu 2025-11-23 19:30:55 -0800
  • 4224fcf4f8 chore: trigger new workflow run with real-time monitoring chrislu 2025-11-23 19:28:39 -0800
  • a4af6d880d fix: download Parquet file in real-time when EOF error occurs chrislu 2025-11-23 19:18:07 -0800
  • 09384e41e3 fix: add comprehensive diagnostics for file location chrislu 2025-11-23 19:09:04 -0800
  • 8ea2646084 fix: keep containers running during file download chrislu 2025-11-23 18:54:37 -0800
  • f2a20aec8b fix: download Parquet file immediately after test failure chrislu 2025-11-23 18:15:14 -0800
  • 2548ad91f7 debug: add comprehensive volume and container diagnostics chrislu 2025-11-23 18:06:21 -0800
  • 911eb60946 debug: add directory structure inspection before file download chrislu 2025-11-23 14:58:31 -0800
  • 55b5f7f0aa fix: replace heredoc with echo pipe to fix YAML syntax chrislu 2025-11-23 14:33:15 -0800
  • 0dc95c0669 fix: run Spark integration tests on all branches chrislu 2025-11-23 14:30:21 -0800
  • ac9fbeefac refactor: remove emojis from logging and workflow messages chrislu 2025-11-23 14:28:07 -0800
  • 588e29ae57 debug: improve file download with better diagnostics and fallbacks chrislu 2025-11-23 14:14:45 -0800
  • fae232075f fix: restart SeaweedFS services before downloading files on test failure chrislu 2025-11-23 14:04:52 -0800
  • 8c22780091 fix: restart SeaweedFS services before downloading files on test failure chrislu 2025-11-23 14:04:52 -0800
  • af7ee4bfb6 docs: push summary for Parquet diagnostics chrislu 2025-11-23 13:56:35 -0800
  • afce69db1e Revert "docs: comprehensive analysis of persistent 78-byte Parquet issue" chrislu 2025-11-23 13:44:15 -0800
  • b767825ba0 test: add Parquet file download and inspection on failure chrislu 2025-11-23 13:42:39 -0800
  • 8e5f1d60ee docs: comprehensive analysis of persistent 78-byte Parquet issue chrislu 2025-11-23 13:41:12 -0800
  • 1ca6d7f441 debug parquet footer writing chrislu 2025-11-23 13:29:15 -0800
  • 65c3ead62f debug: enhance logging to capture footer writes and getPos calls chrislu 2025-11-23 13:28:38 -0800
  • 9e774d8d75 docs: add Parquet 1.16.0 upgrade summary and testing guide chrislu 2025-11-23 13:23:54 -0800
  • 12504dc1a6 feat: upgrade Apache Parquet to 1.16.0 to fix EOFException chrislu 2025-11-23 13:23:15 -0800
  • 3dd14ad2df fmt chrislu 2025-11-23 12:47:20 -0800
  • 885354bb19 fix: reduce write() logging verbosity, add summary stats chrislu 2025-11-23 12:46:36 -0800
  • 221252d34e fmt chrislu 2025-11-23 12:43:08 -0800
  • b13c13187b
    Merge 09d1b0922d into c89f394aba #4975 zehweh 2025-11-23 17:25:41 -0300
  • 48a2ddf6f8 debug: track ALL writes to Parquet files chrislu 2025-11-23 12:24:52 -0800
  • 86ae3da174 fmt chrislu 2025-11-23 12:15:46 -0800
  • 10ff54a661 Revert "docs: comprehensive analysis of 78-byte EOFException" chrislu 2025-11-23 12:15:35 -0800
  • 94ab173eb0 docs: comprehensive analysis of 78-byte EOFException chrislu 2025-11-23 12:09:13 -0800
  • 7b067d2e59 debug: add getPos() method to track position queries chrislu 2025-11-23 12:08:23 -0800
  • a3cf4eb843 debug: track stream lifecycle and total bytes written chrislu 2025-11-23 11:54:01 -0800
  • a5bccca443 debug: add critical diagnostics for EOFException (78 bytes missing) chrislu 2025-11-23 11:39:47 -0800
  • d9ab0721b9 refactor: merge workflow jobs into single job chrislu 2025-11-23 11:29:33 -0800
  • 81867f0dd9 debug: add detailed verification for Maven artifact upload chrislu 2025-11-23 11:14:45 -0800
  • f52d2902b2 fix: copy Maven artifacts into workspace instead of mounting $HOME/.m2 chrislu 2025-11-23 11:07:05 -0800
  • 052365a627 fix: use explicit $HOME path for Maven mount and add verification chrislu 2025-11-23 10:41:34 -0800
  • 966b053ed3 fix: use SNAPSHOT version to force Maven to use locally built JARs chrislu 2025-11-23 10:17:12 -0800
  • f20bad97e4 fix: force Maven update and verify JAR contains updated code chrislu 2025-11-23 10:15:29 -0800
  • 2fbc2432cb fix: force Maven clean build to pick up updated Java client JARs chrislu 2025-11-23 09:59:57 -0800
  • 6a73f03000 debug: track position and buffer state at close time chrislu 2025-11-23 00:03:23 -0800
  • bdf8d89fbb debug: add detailed buffer tracking to identify lost 78 bytes chrislu 2025-11-23 00:02:29 -0800
  • d7f0579c99 fix: replace deprecated slf4j-log4j12 with slf4j-reload4j chrislu 2025-11-22 23:54:12 -0800
  • 551883694b debug: add detailed chunk size logging to diagnose EOF issue chrislu 2025-11-22 23:52:34 -0800
  • 65d9aacceb debug: enable detailed logging for SeaweedFS client file operations chrislu 2025-11-22 23:51:10 -0800
  • 94615996ed workaround: increase Spark task retries for eventual consistency chrislu 2025-11-22 23:48:22 -0800
  • 53daabf07c fix: remove ping command not available in Maven container chrislu 2025-11-22 23:38:43 -0800
  • 780a1fd059 fix: add file sync and cache settings to prevent EOF on read chrislu 2025-11-22 23:32:53 -0800
  • 90f5a2371e debug: add DNS verification and disable Java DNS caching chrislu 2025-11-22 23:26:30 -0800
  • a481a345ac refactor: run Spark tests fully in Docker with bridge network chrislu 2025-11-22 23:21:12 -0800
  • 150d084b3b fix: use localhost publicUrl and -max=100 for host-based Spark tests chrislu 2025-11-22 23:18:35 -0800
  • ce40e2fd58 fix: use container hostname for volume server to enable automatic volume creation chrislu 2025-11-22 23:09:27 -0800
  • 3586f6786e fix: force volume creation before tests to prevent 'No writable volumes' error chrislu 2025-11-22 23:05:36 -0800
  • 6683a9941b ci: add volume.list diagnostic for troubleshooting 'No writable volumes' chrislu 2025-11-22 22:42:54 -0800
  • e253030d2c ci: add volume cleanup and verification steps chrislu 2025-11-22 22:39:09 -0800
  • 7e0d8315bc security: upgrade nimbus-jose-jwt to 10.0.2 to fix GHSA-xwmg-2g98-w7v9 chrislu 2025-11-22 22:16:19 -0800
  • 470c05af97 Update pom.xml chrislu 2025-11-22 22:14:04 -0800
  • 9078ea64f1 security: upgrade nimbus-jose-jwt to 9.37.4 (patched version) chrislu 2025-11-22 22:10:38 -0800
  • e2e89b52b7 security: add dependency overrides for vulnerable transitive deps chrislu 2025-11-22 22:06:29 -0800
  • 2ca03582da fix: restore Jetty dependency management with version 12.0.12 chrislu 2025-11-22 22:05:04 -0800
  • 1296fed511 4.1.125.Final chrislu 2025-11-22 21:51:41 -0800
  • b2186b3f8f fix: remove Jetty dependency management due to unavailable versions chrislu 2025-11-22 21:47:48 -0800
  • 342705c99e fmt chrislu 2025-11-22 21:38:12 -0800
  • fd51091abd fix: add persistent volume data directory for volume server chrislu 2025-11-22 21:32:37 -0800
  • e48bf9a791 security: upgrade Jetty from 9.4.53 to 12.0.16 chrislu 2025-11-22 20:49:37 -0800
  • a1a14259c3 fix: add -max=0 to volume server for unlimited volumes chrislu 2025-11-22 20:42:03 -0800
  • c49abc0c2f security: upgrade Apache ZooKeeper to 3.9.4 chrislu 2025-11-22 17:55:17 -0800
  • a051452fe6 security: upgrade Apache ZooKeeper to 3.9.3 chrislu 2025-11-22 17:55:03 -0800
  • 150deefdc0 fix: aggressively suppress Parquet DEBUG logging chrislu 2025-11-22 17:52:02 -0800
  • f71e3448b4 ci: skip central-publishing plugin during build chrislu 2025-11-22 17:50:14 -0800
  • b018588c14 security: upgrade Netty to 4.1.124.Final (patched version) chrislu 2025-11-22 14:22:57 -0800
  • 4dd55783da security: upgrade Netty to 4.1.118.Final chrislu 2025-11-22 14:22:41 -0800
  • fab383dc10 fix: use 127.0.0.1 for volume server IP registration chrislu 2025-11-22 14:21:02 -0800
  • 707e7732a7 fix: suppress verbose Parquet DEBUG logging chrislu 2025-11-22 14:20:11 -0800
  • 8c13794a49 security: upgrade Netty to 4.1.115.Final to fix CVE chrislu 2025-11-22 14:09:54 -0800
  • abaf933971 fix: add publicUrl to volume server for host network access chrislu 2025-11-22 14:08:20 -0800
  • 01e20a350c refactor: extract surefire JVM args to property chrislu 2025-11-22 14:07:01 -0800
  • 3074b1ee2f refactor: externalize seaweedfs-hadoop3-client version to property chrislu 2025-11-22 14:06:21 -0800
  • 7b548be48e security: add dependencyManagement to fix vulnerable transitives chrislu 2025-11-22 14:05:24 -0800
  • 9e8b8276ea fix: build statically linked binary for Alpine Linux chrislu 2025-11-22 13:57:40 -0800
  • a61af2989c ci: add comprehensive failure diagnostics chrislu 2025-11-22 13:52:27 -0800
  • 0a7917704e ci: add debugging and force rebuild of Docker images chrislu 2025-11-22 13:51:42 -0800
  • e29163dfa4 fix: remove invalid shell operators from Dockerfile COPY chrislu 2025-11-22 13:46:32 -0800
  • 459ff0bd38 fix: improve binary copy and chmod in Dockerfile chrislu 2025-11-22 13:40:46 -0800
  • ec08fadf85 fix: align maven-compiler-plugin with compiler properties chrislu 2025-11-22 13:39:22 -0800
  • becb250ab8 refactor: eliminate code duplication in channel creation chrislu 2025-11-22 13:38:57 -0800
  • c7e31c5ddb refactor: remove unused imports in FilerGrpcClient chrislu 2025-11-22 13:38:11 -0800
  • 45b45c4a8d fix: ensure weed binary is executable in Docker image chrislu 2025-11-22 13:36:46 -0800
  • e8e9df2680 test: improve docker-compose config for Spark tests chrislu 2025-11-22 13:35:52 -0800
  • dd9c1c6190 fix: add -peers=none to master command for standalone mode chrislu 2025-11-22 13:34:01 -0800