Reconstruct distributed LLM training traces

Show HN: Reconstruct distributed LLM training traces

Reconstruct distributed LLM training traces is a business in show hn launch.

Profile

When we train large language models, there are a lot of systems challenges and different sharding schemes one can use. While there are many great resources on scaling LLMs out there (https://huggingface.co/spaces/nanotron/ultrascale-playbook or https://jax-ml.github.io/scaling-book/), I felt like there was still a gap when it comes to visualising different forms of parallelism and building intuition around overlaps and execution order for a distributed training runThe idea is to make it easier to visualise FSDP/Tensor Parallel/Expert Parallel/Context parallel and reason about it - you can drag and drop compute kernels and collectives to create DDP/TP/FSDP/EP/CP traces based on real torchtita

Within the show hn launch space, the company focuses on its core offering rather than a broad product line.

Directory data places it at the Show HN launch stage.

Website: trace.vladsavinov.com