Browse Source

filer replication: compare content changes directly

Fix https://github.com/seaweedfs/seaweedfs/issues/3714

The destination chunks may be empty. For example, the file is updated and the volume is vacuumed. In this case, the sync would miss the old chunks. This is fine. However, the entry would have correct metadata but missing chunks.

For this case, the simple metadata comparison would be wrongly skipping data changes, and the file will stay empty unless file content md5 is changed.
pull/3733/head
chrislu 2 years ago
parent
commit
b463ca1a2f
  1. 4
      weed/replication/sink/filersink/filer_sink.go

4
weed/replication/sink/filersink/filer_sink.go

@ -187,10 +187,6 @@ func (fs *FilerSink) UpdateEntry(key string, oldEntry *filer_pb.Entry, newParent
// skip if already changed // skip if already changed
// this usually happens when the messages are not ordered // this usually happens when the messages are not ordered
glog.V(2).Infof("late updates %s", key) glog.V(2).Infof("late updates %s", key)
} else if filer.ETag(newEntry) == filer.ETag(existingEntry) {
// skip if no change
// this usually happens when retrying the replication
glog.V(3).Infof("already replicated %s", key)
} else { } else {
// find out what changed // find out what changed
deletedChunks, newChunks, err := compareChunks(filer.LookupFn(fs), oldEntry, newEntry) deletedChunks, newChunks, err := compareChunks(filer.LookupFn(fs), oldEntry, newEntry)

Loading…
Cancel
Save