UniversalPageStorageServiceCheckpointTest.DumpAndRead fails in tsan

[xzhangxian1008]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

run tsan tests

2. What did you expect to see? (Required)

no error

3. What did you see instead (Required)

libc++abi: terminating due to uncaught exception of type Poco::FileNotFoundException: File not found

4. What is your TiFlash version? (Required)

nightly(e84ee46)

[JaySon-Huang]

[2025-11-12T18:38:03.736Z] ^[[0;33mNote: Google Test filter = UniversalPageStorageServiceCheckpointTest.DumpAndRead
[2025-11-12T18:38:03.736Z] ^[[m^[[0;32m[==========] ^[[mRunning 1 test from 1 test case.
[2025-11-12T18:38:03.736Z] ^[[0;32m[----------] ^[[mGlobal test environment set-up.
[2025-11-12T18:38:03.736Z] ^[[0;32m[----------] ^[[m1 test from UniversalPageStorageServiceCheckpointTest
[2025-11-12T18:38:03.736Z] ^[[0;32m[ RUN      ] ^[[mUniversalPageStorageServiceCheckpointTest.DumpAndRead
[2025-11-12T18:38:03.736Z] [2025/11/12 18:16:23.717 +00:00] [WARN] [Context.cpp:1661] ["GlobalStoragePool has already been initialized."] [source=Context] [thread_id=1]
[2025-11-12T18:38:03.736Z] [2025/11/12 18:16:23.717 +00:00] [INFO] [PageStorageImpl.cpp:54] ["PageStorageImpl start. Config{ PageStorageConfig {blob_file_limit_size: 268435456, blob_spacemap_type: 2, blob_heavy_gc_valid_rate: 0.500, blob_heavy_gc_valid_rate_raft_data: 0.050, blob_block_alignment_bytes: 0, wal_roll_size: 2097152, wal_max_persisted_log_files: 4}
 }"] [source=__global__.log] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.717 +00:00] [INFO] [PageStorageImpl.cpp:54] ["PageStorageImpl start. Config{ PageStorageConfig {blob_file_limit_size: 268435456, blob_spacemap_type: 2, blob_heavy_gc_valid_rate: 0.500, blob_heavy_gc_valid_rate_raft_data: 0.050, blob_block_alignment_bytes: 0, wal_roll_size: 2097152, wal_max_persisted_log_files: 4}
 }"] [source=__global__.data] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.717 +00:00] [INFO] [PageStorageImpl.cpp:54] ["PageStorageImpl start. Config{ PageStorageConfig {blob_file_limit_size: 268435456, blob_spacemap_type: 2, blob_heavy_gc_valid_rate: 0.500, blob_heavy_gc_valid_rate_raft_data: 0.050, blob_block_alignment_bytes: 0, wal_roll_size: 2097152, wal_max_persisted_log_files: 4}
 }"] [source=__global__.meta] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.718 +00:00] [INFO] [PageDirectoryFactory.cpp:68] ["PageDirectory restored, max_page_id=0 max_applied_ver=0"] [source=__global__.log] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.718 +00:00] [INFO] [PageDirectoryFactory.cpp:68] ["PageDirectory restored, max_page_id=0 max_applied_ver=0"] [source=__global__.data] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.718 +00:00] [INFO] [PageDirectoryFactory.cpp:68] ["PageDirectory restored, max_page_id=0 max_applied_ver=0"] [source=__global__.meta] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.718 +00:00] [INFO] [StorageConfigParser.cpp:640] ["enable with StorageS3Config{endpoint= bucket=mockbucket root=tiflash_ut/ max_connections=4096 max_redirections=10 connection_timeout_ms=1000 request_timeout_ms=30000 access_key_id_size=0 secret_access_key_size=0 enable_http_pool=false enable_poco_client=true}"] [
thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.718 +00:00] [INFO] [BlobStore.cpp:130] ["Ignore not blob file, dir=/tiflash/tmp/UniversalPageStorageServiceCheckpointTest.DumpAndRead file=page err_msg="] [source=test.t] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.719 +00:00] [INFO] [PageDirectoryFactory.cpp:68] ["PageDirectory restored, max_page_id=0 max_applied_ver=0"] [source=test.t] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.719 +00:00] [DEBUG] [TiFlashTestEnv.cpp:288] ["Created bucket mockbucket"] [source="bucket=mockbucket root=tiflash_ut/"] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.719 +00:00] [INFO] [BlobStat.cpp:147] ["Created a new BlobStat [blob_id=10] [capacity=268435456]"] [source=test.t] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.719 +00:00] [INFO] [WALStore.cpp:143] ["Creating log file for writing, fullname=/tiflash/tmp/UniversalPageStorageServiceCheckpointTest.DumpAndRead/wal/log_1_0"] [source=test.t] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.720 +00:00] [DEBUG] [S3Common.cpp:1068] ["listPrefix prefix=s2/manifest/, total_keys=0, cost=0.00s"] [source="bucket=mockbucket root=tiflash_ut/"] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.720 +00:00] [INFO] [S3LockLocalManager.cpp:114] ["restore from S3 finish, elapsed=0.000s last_upload_sequence=0 num_s3_entries=0 num_copied_entries=0 last_prefix={None}"] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.720 +00:00] [INFO] [PageDirectory.cpp:2366] ["Dumped snapshot to edits, sequence=6 edit_size=3"] [source=test.t] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.721 +00:00] [DEBUG] [PageDirectory.cpp:2327] ["After MVCC gc in memory, general_lowest_seq=none delta_tree_only_seq=5. clean invalid_snapshot_nums=0 invalid_page_nums=0 invalid_raft_pages_nums=0 total_deref_counter=0 all_del_entries=0. Still exist, snapshot_nums=0 page_nums=3. Longest alive snapshot: {alive_time=
0.000 seq=0 type=General tracing_id=} stale_snapshot_nums=0"] [source=test.t_1] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.721 +00:00] [INFO] [PageDirectory.cpp:2366] ["Dumped snapshot to edits, sequence=5 edit_size=3"] [source=test.t_1] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.725 +00:00] [DEBUG] [CPDataFileStat.cpp:191] ["CheckpointData stats {compaction=[] unchanged=[]} RemoteGCThreshold{min_age=3600 min_file_threshold=131072 valid_rate=0.50%}"] [source=UniPSService] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.725 +00:00] [DEBUG] [CPFilesWriter.cpp:105] ["Prepare to dump 3 records, sequence=1, manifest_file_id=s2/manifest/mf_1"] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.725 +00:00] [DEBUG] [CPFilesWriter.cpp:241] ["compact stats: {}"] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.727 +00:00] [DEBUG] [S3Common.cpp:689] ["uploadFile local_fname=/tiflash/tmp/checkpoint_upload_1/dat_1_0, key=s2/data/dat_1_0, write_bytes=431 cost=0.001s"] [source="bucket=mockbucket root=tiflash_ut/"] [thread_id=6]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.727 +00:00] [DEBUG] [S3Common.cpp:601] ["uploadEmptyFile key=lock/s2/dat_1_0.lock_s2_1, cost=0.00s"] [source="bucket=mockbucket root=tiflash_ut/"] [thread_id=6]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.728 +00:00] [DEBUG] [S3Common.cpp:689] ["uploadFile local_fname=/tiflash/tmp/checkpoint_upload_1/mf_1, key=s2/manifest/mf_1, write_bytes=338 cost=0.000s"] [source="bucket=mockbucket root=tiflash_ut/"] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:23.728 +00:00] [INFO] [UniversalPageStorage.cpp:660] ["Checkpoint result: files=[/tiflash/tmp/checkpoint_upload_1/dat_1_0] dump_snapshot=0.000s dump_data=0.005s upload=0.001s copy_checkpoint_info=0.000s total=0.007s sequence=1 CPDataDumpStats{incremental_data_bytes=48 compact_data_bytes=0 n_records{total=3 pages_unc
hanged=0 pages_compact=0 pages_incremental=2 ext_pages=0 ref_pages=1 delete=0 other=0} types[{type=Unknown keys=3 bytes=48} {type=Log keys=0 bytes=0} {type=Data keys=0 bytes=0} {type=Meta keys=0 bytes=0} {type=KVStore keys=0 bytes=0} {type=RaftEngine keys=0 bytes=0} {type=KVEngine keys=0 bytes=0} {type=LocalKV keys=0 bytes=0}]}"] [source=test.t] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:24.968 +00:00] [INFO] [UniversalPageStorageService.cpp:303] ["Upload checkpoint success, upload_sequence=1 incremental_bytes=48 compact_bytes=0"] [source=UniPSService] [thread_id=1]
[2025-11-12T18:38:03.737Z] [2025/11/12 18:16:24.968 +00:00] [INFO] [SegmentReadTaskScheduler.cpp:311] ["schedule sched_round=1 erased_pool_count=0 sched_null_count=0 sched_succ_count=0 reap=0ms cost=1006ms"] [thread_id=7]
[2025-11-12T18:38:03.737Z] libc++abi: terminating due to uncaught exception of type Poco::FileNotFoundException: File not found