| 1 | """ |
| 2 | This is the script to test 2D Parallel which combines Tensor/Sequence |
| 3 | parallel with Fully Sharded Data Parallel (TP/SP + FSDP) on a example |
| 4 | Llama2 model. We show an E2E working flow from forward, backward |
nothing calls this directly
no test coverage detected