Architectures / Audio

🎙️ Whisper Small

Whisper speech encoder-decoder — conv1d audio stem + transformer encoder/decoder (384D)

Layers

Parameters

46.99M

Input

1 × 80 × 3000

Output

1 × 448 × 51865

Verifier

2 advisories

Every number on this page is computed from the graph by the same functions the app runs, not written by hand.

Open Whisper Small on the canvas Free, no account needed

When to pick it

Pick for ASR or as a pretrained audio encoder — drop the decoder + add a head for audio classification (UrbanSound, ESC-50).

Structure

17 layers. Output shapes are propagated from the input shape, batch dimension excluded.

	Layer	Type	Parameters	Output shape
1	mel_features	Input	shape=[1, 80, 3000]	1 × 80 × 3000
2	conv1	Audio Conv	outChannels=384, kernelSize=3, stride=1	1 × 384 × 3000
3	gelu_1	GELU		1 × 384 × 3000
4	conv2	Audio Conv	outChannels=384, kernelSize=3, stride=2	1 × 384 × 1500
5	gelu_2	GELU		1 × 384 × 1500
6	enc_pos_emb	Positional Encoding	embedDim=384, maxLen=1500	1 × 384 × 1500
7	enc_block_1	Transformer Block	embedDim=384, numHeads=6, ffDim=1536	1 × 384 × 1500
8	enc_block_2	Transformer Block	embedDim=384, numHeads=6, ffDim=1536	1 × 384 × 1500
9	enc_norm	LayerNorm	normalizedShape=384	1 × 384 × 1500
10	decoder_tokens	Input	shape=[1, 448]	1 × 448
11	token_embed	Embedding		1 × 448 × 384
12	dec_pos_emb	Positional Encoding	embedDim=384, maxLen=448	1 × 448 × 384
13	dec_block_1	Transformer Block	embedDim=384, numHeads=6, ffDim=1536	1 × 448 × 384
14	dec_block_2	Transformer Block	embedDim=384, numHeads=6, ffDim=1536	1 × 448 × 384
15	dec_norm	LayerNorm	normalizedShape=384	1 × 448 × 384
16	lm_head	Linear	outFeatures=51865	1 × 448 × 51865
17	token_logits	Output		1 × 448 × 51865

What the verifier says

The same 41 structural checks that run on every edit in the app, on this graph.

warntransformerBlock 'enc_block_1': embedDim=384 but upstream last dim is 1500; attention expects them equal (project the input or fix embedDim) (enc_block_1)
attention-in-mismatch

warntransformerBlock 'enc_block_2': embedDim=384 but upstream last dim is 1500; attention expects them equal (project the input or fix embedDim) (enc_block_2)
attention-in-mismatch

The PyTorch it exports

Generated from the graph above. First 46 lines; the app exports the whole file, plus the training loop, the data contract and a deploy bundle.

# Architecture designed with Neurarch: https://neurarch.com
# PyTorch: compatible with Python 3.8+ and torch>=1.12
# Colab: pip install torch torchvision  (usually pre-installed)
# Audio: pip install torchaudio

import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import Tuple
import torchaudio

class WhisperSmall(nn.Module):
    def __init__(self):
        super().__init__()

        self.audioConv_1 = nn.Conv1d(1, 384, kernel_size=3, stride=1, padding=1)
        self.gelu_1 = nn.GELU()
        self.audioConv_2 = nn.Conv1d(1, 384, kernel_size=3, stride=2, padding=1)
        self.gelu_2 = nn.GELU()
        self.transformerBlock_1 = nn.TransformerEncoderLayer(d_model=384, nhead=6, dim_feedforward=1536, batch_first=True)
        self.transformerBlock_2 = nn.TransformerEncoderLayer(d_model=384, nhead=6, dim_feedforward=1536, batch_first=True)
        self.layerNorm_1 = nn.LayerNorm(1500)
        self.embedding_1 = nn.Embedding(51865, 384)
        self.transformerBlock_3 = nn.TransformerDecoderLayer(d_model=384, nhead=6, dim_feedforward=1536, batch_first=True)
        self.transformerBlock_4 = nn.TransformerDecoderLayer(d_model=384, nhead=6, dim_feedforward=1536, batch_first=True)
        self.layerNorm_2 = nn.LayerNorm(384)
        self.linear_1 = nn.Linear(384, 51865)

    def forward(self, src, tgt=None):
        # mel_features shape: [1,80,3000]
        # decoder_tokens shape: [1,448]
        audio_conv_Conv_1 = self.audioConv_1(src)
        gelu_gelu_1 = self.gelu_1(audio_conv_Conv_1)
        audio_conv_Conv_2 = self.audioConv_2(gelu_gelu_1)
        gelu_gelu_2 = self.gelu_2(audio_conv_Conv_2)
        # positionalEncoding: add positional encoding externally (e.g. sinusoidal or learned PE)
        transformer_block__enc_1 = self.transformerBlock_1(gelu_gelu_2)
        transformer_block__enc_2 = self.transformerBlock_2(transformer_block__enc_1)
        layer_norm_rm_enc = self.layerNorm_1(transformer_block__enc_2)
        embedding_ng_dec = self.embedding_1(tgt)
        # positionalEncoding: add positional encoding externally (e.g. sinusoidal or learned PE)
        transformer_block__dec_1 = self.transformerBlock_3(embedding_ng_dec, layer_norm_rm_enc)
        transformer_block__dec_2 = self.transformerBlock_4(transformer_block__dec_1, layer_norm_rm_enc)
        layer_norm_rm_dec = self.layerNorm_2(transformer_block__dec_2)
        linear_ar_out = self.linear_1(layer_norm_rm_dec)
        # Output

For agents

This architecture is machine-readable end to end. An agent can list the set, fetch this graph, edit it, and have the edit verified before any GPU time is spent.

Markdown twin Index JSON This graph as JSON MCP server POST /api/v1/check