Skip to content

Commit c4055c5

Browse files
merge a bunch so the result is always the same (doesn't change based on what partition it decides to merge)
1 parent 03b9c9e commit c4055c5

File tree

1 file changed

+7
-0
lines changed

1 file changed

+7
-0
lines changed

examples/custom-merge-replacing.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,6 +179,13 @@ def part_func(row: dict) -> str:
179179
print("============= merging =============")
180180
# here we reduce the rows to a single latest row
181181
merged_log, new_file, partition, merged_files, meta = ice.merge()
182+
merged_log, new_file, partition, merged_files, meta = ice.merge()
183+
# redundant merges do nothing
184+
merged_log, new_file, partition, merged_files, meta = ice.merge()
185+
merged_log, new_file, partition, merged_files, meta = ice.merge()
186+
merged_log, new_file, partition, merged_files, meta = ice.merge()
187+
merged_log, new_file, partition, merged_files, meta = ice.merge()
188+
merged_log, new_file, partition, merged_files, meta = ice.merge()
182189
print(f"merged {len(merged_files)} data files from partition {partition}")
183190

184191
print("============= check number of raw rows in data =============")

0 commit comments

Comments
 (0)