Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
3c4d1ae
Add initial MoE benchmark
Aug 13, 2021
1ef1612
Update README.moe.md with reference speeds
Sep 6, 2021
9ea39b2
Add Tutel boost for Fairseq MoE acceleration (#3873)
ghostplant Nov 16, 2021
ebea007
Add instructions for using eval-lm with MoE models
Dec 18, 2021
f0e9c9b
support infinibatch
shumingma Nov 16, 2022
bf7c38d
fx
shumingma Nov 16, 2022
5605b44
fx
shumingma Nov 16, 2022
978a6ce
megatron checkpoint
shumingma Apr 7, 2023
5b52116
bf16
shumingma Apr 11, 2023
a4ee205
test push
njb-ms Apr 21, 2023
f2c15d9
refactor by removing function inside function
njb-ms Apr 21, 2023
5140937
factor our buf and params
njb-ms Apr 21, 2023
d31dff0
factor out pgs
njb-ms Apr 21, 2023
82b808d
harmonize grad division
njb-ms Apr 21, 2023
8079646
reduce moe by size 1 pg
njb-ms Apr 21, 2023
2d41d29
always div by largest world size
njb-ms Apr 21, 2023
5c0b09f
set moe pg from torchscale
njb-ms Apr 22, 2023
c69ff2a
num experts optional
njb-ms Apr 24, 2023
01cce8e
Merge pull request #1 from njb-ms/moe_more_gpus
shumingma Apr 26, 2023
ba85672
fx import torchscale error
shumingma Apr 30, 2023
96aad9d
Update search.py
buaahsh May 4, 2023
51d2c06
Merge branch 'shumingma:moe' into moe
buaahsh Jun 27, 2023
a1c2c69
Update trainer.py, remove sample length assert
buaahsh Jun 27, 2023
291a26a
Update utils.py
buaahsh Aug 23, 2023
b169faa
support torchrun
buaahsh Oct 23, 2023
c3e77b6
dataloader: resume job with more #GPUs
donglixp Dec 2, 2023
2905f8c
Update checkpoint_utils.py
donglixp Dec 2, 2023
42bbc51
Merge pull request #2 from donglixp/patch-5
buaahsh Dec 4, 2023
821ee69
Merge pull request #3 from donglixp/patch-6
buaahsh Dec 4, 2023
960a0bf
fx megatron trainer load ckpt
buaahsh Dec 5, 2023
0372449
Update fairseq_criterion.py
yushuiwx May 29, 2024
c0173dc
Update legacy_distributed_data_parallel.py
yushuiwx May 30, 2024
67b73ea
Update legacy_distributed_data_parallel.py
yushuiwx May 30, 2024
6dc3b97
Update legacy_distributed_data_parallel.py
yushuiwx May 30, 2024
8f63e49
Update legacy_distributed_data_parallel.py
yushuiwx May 30, 2024
3ba1de2
Update legacy_distributed_data_parallel.py
yushuiwx May 31, 2024
e9ada9c
Update legacy_distributed_data_parallel.py
yushuiwx May 31, 2024
f788465
Update legacy_distributed_data_parallel.py
yushuiwx May 31, 2024
ab5a1cd
Update fairseq_criterion.py
yushuiwx May 31, 2024
a7e65fa
Update fairseq_criterion.py
yushuiwx May 31, 2024
7666f2b
Update fairseq_criterion.py
yushuiwx May 31, 2024
d056ae5
Update fairseq_criterion.py
yushuiwx May 31, 2024
793a5c4
Update fairseq_criterion.py
yushuiwx May 31, 2024
a25d55e
Update legacy_distributed_data_parallel.py
yushuiwx Jun 15, 2024
758d101
Update legacy_distributed_data_parallel.py
yushuiwx Jun 15, 2024
9a74df7
Update legacy_distributed_data_parallel.py
yushuiwx Jun 15, 2024
c944693
Update legacy_distributed_data_parallel.py
yushuiwx Jun 15, 2024
bbe6b56
Update legacy_distributed_data_parallel.py
yushuiwx Jun 15, 2024
3d9507e
Update legacy_distributed_data_parallel.py
yushuiwx Jun 15, 2024
14533ba
Update legacy_distributed_data_parallel.py
yushuiwx Jun 15, 2024
f3ee1f5
Update legacy_distributed_data_parallel.py
yushuiwx Jun 15, 2024
cabf010
Update legacy_distributed_data_parallel.py
yushuiwx Jun 15, 2024
7063aaa
Update legacy_distributed_data_parallel.py
yushuiwx Jun 15, 2024
12bdff7
Update legacy_distributed_data_parallel.py
yushuiwx Jun 15, 2024
b44bd89
Update legacy_distributed_data_parallel.py
yushuiwx Jun 15, 2024
1ac4823
Update legacy_distributed_data_parallel.py
yushuiwx Jun 15, 2024
6359b7d
Update legacy_distributed_data_parallel.py
yushuiwx Jun 15, 2024
58912a2
Update legacy_distributed_data_parallel.py
yushuiwx Jun 16, 2024
ee270ed
Update legacy_distributed_data_parallel.py
yushuiwx Jun 16, 2024
f921d44
Update legacy_distributed_data_parallel.py
yushuiwx Jun 16, 2024
9a78abe
Update legacy_distributed_data_parallel.py
yushuiwx Jun 16, 2024
802d6e9
Update legacy_distributed_data_parallel.py
yushuiwx Jun 16, 2024
ddfc38b
Update legacy_distributed_data_parallel.py
yushuiwx Jun 17, 2024
89e778f
Update legacy_distributed_data_parallel.py
yushuiwx Jun 17, 2024
495cd07
Update legacy_distributed_data_parallel.py
yushuiwx Jun 18, 2024
f513b25
Update legacy_distributed_data_parallel.py
yushuiwx Jun 18, 2024
ba4d469
Update legacy_distributed_data_parallel.py
yushuiwx Jun 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 0 additions & 77 deletions CODE_OF_CONDUCT.md

This file was deleted.

216 changes: 0 additions & 216 deletions README.md

This file was deleted.

1 change: 1 addition & 0 deletions README.md
Loading