Comparisons / Whisper Small vs T5 Small
Whisper Small vs T5 Small
Speech-to-text against text-to-text, both encoder-decoder.
T5 Small has 9.7M more parameters than Whisper Small: 14 layers added, 9 removed, 8 changed.
Whisper Small
- Layers
- 15
- Parameters
- 47M
- Input
- 80 × 3000
- Output
- 1 × 448 × 51865
- Forward-passes
- yes
- Est. train cost
- $0.705
T5 Small
- Layers
- 20
- Parameters
- 57M
- Input
- 1 × 512
- Output
- 1 × 128 × 32128
- Forward-passes
- yes
- Est. train cost
- $0.207
The deltas
Every number is T5 Small relative to Whisper Small.
Which GPUs each one fits
Each side is measured at its own declared input (80 × 3000 against 1 × 512). Both columns are right about their own model; the difference between them is not a fact about the designs.
| GPU | Whisper Small | T5 Small |
|---|---|---|
| T4 16GB | fits | fits |
| A100 40GB | fits | fits |
| H100 80GB | fits | fits |
Layer by layer
Aligned in topological order. 1 of 32 rows are the same layer with the same parameters.
Hide all 32 rows
| Whisper Small | T5 Small | ||||||
|---|---|---|---|---|---|---|---|
| Layer | Params | Output | Layer | Params | Output | ||
| 1 | changed shape | mel_features Input | 80 × 3000 | encoder_ids Input | 1 × 512 | ||
| 2 | changed shape | decoder_tokens Input | 1 × 448 | decoder_ids Input | 1 × 128 | ||
| 3 | removed | conv1 Audio Conv | 1.5K | 384 × 3000 | — | ||
| 4 | changed numEmbeddings, embeddingDim | token_embed Embedding | 20M | 1 × 448 × 384 | shared_embed Embedding | 16M | 1 × 512 × 512 |
| 5 | removed | gelu_1 Gelu | 384 × 3000 | — | |||
| 6 | removed | dec_pos_emb Positional Encoding | 1 × 448 × 384 | — | |||
| 7 | removed | conv2 Audio Conv | 1.5K | 384 × 1500 | — | ||
| 8 | removed | gelu_2 Gelu | 384 × 1500 | — | |||
| 9 | removed | to_tokens Permute | 1500 × 384 | — | |||
| 10 | removed | enc_pos_emb Positional Encoding | 1500 × 384 | — | |||
| 11 | removed | enc_block_1 Transformer Block | 1.8M | 1500 × 384 | — | ||
| 12 | added | — | dec_embed Embedding | 16M | 1 × 128 × 512 | ||
| 13 | added | — | enc_norm Rms Norm | 512 | 1 × 512 × 512 | ||
| 14 | added | — | dec_sa_norm Rms Norm | 512 | 1 × 128 × 512 | ||
| 15 | changed type, embedDim, numHeads, ffDim | enc_block_2 Transformer Block | 1.8M | 1500 × 384 | enc_self_attn Multi Head Attention | 1.1M | 1 × 512 × 512 |
| 16 | added | — | dec_self_attn Causal Attention | 1.0M | 1 × 128 × 512 | ||
| 17 | added | — | enc_residual Add | 1 × 512 × 512 | |||
| 18 | added | — | dec_sa_residual Add | 1 × 128 × 512 | |||
| 19 | added | — | enc_ffn_norm Rms Norm | 512 | 1 × 512 × 512 | ||
| 20 | added | — | dec_ca_norm Rms Norm | 512 | 1 × 128 × 512 | ||
| 21 | added | — | enc_ffn Feed Forward | 2.1M | 1 × 512 × 512 | ||
| 22 | added | — | enc_ffn_residual Add | 1 × 512 × 512 | |||
| 23 | changed normalizedShape | enc_norm Layer Norm | 768 | 1500 × 384 | enc_out_norm Layer Norm | 1.0K | 1 × 512 × 512 |
| 24 | removed | dec_block_1 Transformer Block | 1.8M | 1 × 448 × 384 | — | ||
| 25 | changed type, embedDim, numHeads, ffDim | dec_block_2 Transformer Block | 1.8M | 1 × 448 × 384 | cross_attn Multi Head Attention | 1.1M | 1 × 128 × 512 |
| 26 | added | — | dec_ca_residual Add | 1 × 128 × 512 | |||
| 27 | added | — | dec_ffn_norm Rms Norm | 512 | 1 × 128 × 512 | ||
| 28 | added | — | dec_ffn Feed Forward | 2.1M | 1 × 128 × 512 | ||
| 29 | added | — | dec_ffn_residual Add | 1 × 128 × 512 | |||
| 30 | changed normalizedShape | dec_norm Layer Norm | 768 | 1 × 448 × 384 | dec_out_norm Layer Norm | 1.0K | 1 × 128 × 512 |
| 31 | changed outFeatures | lm_head Linear | 1 × 448 × 51865 | lm_head Linear | 1 × 128 × 32128 | ||
| 32 | same | token_logits Output | 1 × 448 × 51865 | logits Output | 1 × 128 × 32128 | ||
What this is not
- The two are priced at different declared inputs (80 × 3000 against 1 × 512), so memory, cost and GPU fit are each right about their own model and are not a comparison between them. The layer and parameter deltas are unaffected.
- Parameter counts are derived from the graph, not read from a checkpoint. They are exact for a graph that is fully specified and approximate for one that is not.
- Cost and GPU fit are estimates from the graph under one set of assumptions, not measurements of a run.
Take it further
Open either graph in the editor, change it, and check it again: Whisper Small · T5 Small
Compare any two models of your own, including anything on Hugging Face: the comparison tool.
Machine-readable: this page as markdown ·
the pair index ·
POST https://www.neurarch.com/api/v1/plan for a graph of your own.