[QST] how dose the num_worker in DataLoader work in dynamicemb's example.py

[jiashuy]

What is your question?

---

By submitting this issue, you agree to follow our code of conduct and our contributing guidelines.

[jiashuy]

After upgdade to torch 2.11, the torch.utils.data.DataLoader will throw a runtime error: initialization error
when build DataLoader with num_worker != 0, to reproduce it:

    test_loader = DataLoader(
        test_dataset,
        batch_size=args.batch_size,
        shuffle=False,
        collate_fn=collate_fn,
        num_workers=4,
        sampler=test_sampler,
    )

export NGPU=2
torchrun --standalone --nproc_per_node=${NGPU} example.py --load --dump "$@" 

change the num_workers to 0 can fix the issue.

As AI prompts, DataLoader will fork multiple processes internally and copy CUDA context, but this is illegal in CUDA and has not reported errors in the past. However, Torch 2.11 checks this issue more thoroughly, but it needs to be verified.

[shijieliu]

close as resolved.