Architectures / Generative
🎞️ Latent Video Diffusion Transformer
The shape every 2024-2025 video generator shares: a causal 3D VAE compresses the clip, the latent is patchified into space-time tokens, a DiT stack denoises them.
Every number on this page is computed from the graph by the same functions the app runs, not written by hand.
When to pick it
Structure
15 layers. Output shapes are propagated from the input shape, batch dimension excluded.
| Layer | Type | Parameters | Output shape | |
|---|---|---|---|---|
| 1 | Video clip (16 frames, 256x256) | Input | shape=[3, 16, 256, 256] | 3 × 16 × 256 × 256 |
| 2 | Causal VAE Down 128 | Causal Conv3D | outChannels=128, inChannels=3, kernelSize=3 | 128 × 16 × 128 × 128 |
| 3 | Causal VAE Down 256 | Causal Conv3D | outChannels=256, inChannels=128, kernelSize=3 | 256 × 8 × 64 × 64 |
| 4 | Causal VAE to latent 16 | Causal Conv3D | outChannels=16, inChannels=256, kernelSize=3 | 16 × 4 × 32 × 32 |
| 5 | Patchify 1x2x2 | Tubelet Embedding (3D Patch) | embedDim=1152 | 1024 × 1152 |
| 6 | Pos Encoding | Positional Encoding | maxLen=1024 | 1024 × 1152 |
| 7 | Diffusion timestep | Input | shape=[1] | 1 |
| 8 | Timestep Embed | Time Embedding | 1152 | |
| 9 | DiT Block 1 (AdaLN-Zero) | DiT Block (AdaLN-Zero) | numHeads=16 | 1024 × 1152 |
| 10 | DiT Block 2 (AdaLN-Zero) | DiT Block (AdaLN-Zero) | numHeads=16 | 1024 × 1152 |
| 11 | DiT Block 3 (AdaLN-Zero) | DiT Block (AdaLN-Zero) | numHeads=16 | 1024 × 1152 |
| 12 | DiT Block 4 (AdaLN-Zero) | DiT Block (AdaLN-Zero) | numHeads=16 | 1024 × 1152 |
| 13 | Final LayerNorm | LayerNorm | normalizedShape=1152 | 1024 × 1152 |
| 14 | Unpatchify to latent | Linear | outFeatures=64, inFeatures=1152 | 1024 × 64 |
| 15 | predicted latent velocity | Output | 1024 × 64 |
What the verifier says
The same 43 structural checks that run on every edit in the app, on this graph.
No finding. Shapes propagate end to end, every divisibility condition holds, and no advisory rule fires. See the checks.
The PyTorch it exports
Generated from the graph above. First 46 lines; the app exports the whole file, plus the training loop, the data contract and a deploy bundle.
# Architecture designed with Neurarch: https://neurarch.com
# PyTorch: compatible with Python 3.8+ and torch>=1.12
# Colab: pip install torch torchvision (usually pre-installed)
import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import Tuple
class TimestepEmbedding(nn.Module):
"""Sinusoidal diffusion-timestep features, then an MLP. Takes a scalar per
sample and returns one conditioning vector per sample."""
def __init__(self, dim: int = 256):
super().__init__()
self.dim = dim
self.mlp = nn.Sequential(nn.Linear(dim, dim), nn.SiLU(), nn.Linear(dim, dim))
def forward(self, t: torch.Tensor) -> torch.Tensor:
t = t.reshape(t.size(0), -1)[:, 0].float()
half = self.dim // 2
freqs = torch.exp(
-torch.arange(half, device=t.device, dtype=t.dtype) * (9.2103403719762 / max(1, half - 1)))
ang = t[:, None] * freqs[None]
emb = torch.cat([ang.cos(), ang.sin()], dim=-1)
if emb.size(-1) < self.dim:
emb = F.pad(emb, (0, self.dim - emb.size(-1)))
return self.mlp(emb)
class DiTBlock(nn.Module):
"""DiT block with adaLN-Zero: the conditioning vector produces the scale,
shift and gate for both sublayers, and the gates start at zero so a fresh
block is the identity."""
def __init__(self, hidden_dim: int = 1152, num_heads: int = 16, cond_dim: int = 1152):
super().__init__()
self.norm1 = nn.LayerNorm(hidden_dim, elementwise_affine=False)
self.attn = nn.MultiheadAttention(hidden_dim, num_heads, batch_first=True)
self.norm2 = nn.LayerNorm(hidden_dim, elementwise_affine=False)
self.mlp = nn.Sequential(
nn.Linear(hidden_dim, 4 * hidden_dim), nn.GELU(), nn.Linear(4 * hidden_dim, hidden_dim))
self.ada = nn.Sequential(nn.SiLU(), nn.Linear(cond_dim, 6 * hidden_dim))
nn.init.zeros_(self.ada[1].weight)
nn.init.zeros_(self.ada[1].bias)For agents
This architecture is machine-readable end to end. An agent can list the set, fetch this graph, edit it, and have the edit verified before any GPU time is spent.