OpenAI’s Sora Diffusion Transformer Architecture

OpenAI’s SORA is a diffusion transformer (DiT) from the paper (Peebles & Xie, 2023). Their researchers replaced the U-net in a diffusion model with a MultiHeadAttention transformer.

Here's the annotated model in Pytorch.