First, find the veScale repo.
This will compile and install a patched version of PyTorch.
This will compile and install a patched version of TorchdistX (based on its master).
This will install veScale and its dependencies.
Make sure it is in the veScale directory.
It may take a while to build the image.
Once the building process is finished, you can docker run
with the id.
Nano GPT: <repo>/examples/nanogpt_4D_finetune
Open LLAMA: <repo>/examples/open_llama_4D_benchmark
Mixtral: <repo>/examples/mixtral_4D_benchmark