📐 Wide & Deep

Memorization (wide linear) + generalization (deep MLP) joint trained — Cheng et al. 2016

From Cheng et al. (2016). Wide & Deep Learning for Recommender Systems. DLRS 2016. This page is the graph, not the PDF: open it, edit it, verify it, export it.

Layers

Parameters

3.65M

Input

10000

Output

Verifier

Clean

Every number on this page is computed from the graph by the same functions the app runs, not written by hand.

Open Wide & Deep on the canvas Free, no account needed

When to pick it

Pick as a simpler, more interpretable alternative to DLRM for CTR. Linear part captures memorized rules, deep part generalizes.

Structure

13 layers. Output shapes are propagated from the input shape, batch dimension excluded.

	Layer	Type	Parameters	Output shape
1	Wide Input (cross feats)	Input	shape=[10000]	10000
2	Wide Linear	Linear	outFeatures=1, inFeatures=10000	1
3	Deep Input (sparse cat)	Input	shape=[50]	50
4	Embeddings	Embedding	vocabSize=100000	50 × 32
5	Flatten	Flatten		1600
6	Deep FC 1	Linear	outFeatures=256, inFeatures=1600	256
7	ReLU 1	ReLU		256
8	Deep FC 2	Linear	outFeatures=128, inFeatures=256	128
9	ReLU 2	ReLU		128
10	Deep Out	Linear	outFeatures=1, inFeatures=128	1
11	Wide + Deep	Add		1
12	Sigmoid CTR	Sigmoid		1
13	P(click)	Output		1

What the verifier says

The same 43 single-graph checks that run on every edit in the app, on this graph.

info"Sigmoid CTR" feeds directly into Output. PyTorch's nn.CrossEntropyLoss already applies log-softmax internally, an explicit Softmax causes double-application and degrades training stability. Fix: Remove Softmax/Sigmoid for training. Restore it in a separate inference wrapper or ONNX export. (Sigmoid CTR)
output-activation

infoSigmoid saturates to [0,1] / [-1,1], and its gradient approaches zero for large inputs. In networks deeper than 5 layers, this halts learning in early layers. Fix: Use ReLU, GELU, or SiLU for hidden layers. Keep Sigmoid only at binary classification outputs; Tanh in specific contexts (GAN generators, LSTM gates). (Sigmoid CTR)
vanishing-gradient

info10 layers with no BatchNorm, LayerNorm, or GroupNorm. Without normalization, activations can explode or vanish across layers, causing slow or unstable training. Fix: Add BatchNorm after Conv2d (CV tasks), LayerNorm after attention/FFN (NLP/LLM), or GroupNorm for small batch sizes.
deep-no-norm

The PyTorch it exports

Generated from the graph above. First 46 lines; the app exports the whole file, plus the training loop, the data contract and a deploy bundle.

# Architecture designed with Neurarch: https://neurarch.com
# PyTorch: compatible with Python 3.8+ and torch>=1.12
# Colab: pip install torch torchvision  (usually pre-installed)

import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import Tuple

class WideDeep(nn.Module):
    def __init__(self):
        super().__init__()

        self.linear_1 = nn.Linear(10000, 1)
        self.embedding_1 = nn.Embedding(100000, 32)
        self.linear_2 = nn.Linear(1600, 256)
        self.linear_3 = nn.Linear(256, 128)
        self.linear_4 = nn.Linear(128, 1)

    def forward(self, src, tgt=None):
        # Wide Input (cross feats) shape: [10000]
        # Deep Input (sparse cat) shape: [50]
        linear_ide_fc = self.linear_1(src)
        embedding_ep_emb = self.embedding_1(tgt)
        flatten_p_flat = torch.flatten(embedding_ep_emb, 1)
        linear_ep_fc1 = self.linear_2(flatten_p_flat)
        relu__relu1 = F.relu(linear_ep_fc1)
        linear_ep_fc2 = self.linear_3(relu__relu1)
        relu__relu2 = F.relu(linear_ep_fc2)
        linear_ep_out = self.linear_4(relu__relu2)
        add_merge = linear_ide_fc + linear_ep_out
        sigmoid_igmoid = torch.sigmoid(add_merge)
        # Output
        return sigmoid_igmoid


if __name__ == '__main__':
    model = WideDeep()
    model.eval()

    src = torch.randn(1, 10000)  # Wide Input (cross feats)
    tgt = torch.randint(0, 100000, (1, 50))  # Deep Input (sparse cat)
    with torch.no_grad():
        output = model(src, tgt)

    print(f'src    shape : {tuple(src.shape)}')

For agents

This architecture is machine-readable end to end. An agent can list the set, fetch this graph, edit it, and have the edit verified before any GPU time is spent.

Markdown twin Index JSON This graph as JSON MCP server POST /api/v1/check

Also in Recommendation

🗼 Two-Tower

User+Item dual encoder for retrieval — embeddings → MLP per side → dot product score

15 layers · 70.44M

🛒 DLRM

Meta's Deep Learning Recommendation Model — bottom MLP for dense, embedding for sparse, feature interaction, top MLP

14 layers · 64.35M

🤝 NeuMF (Fused GMF + MLP)

He et al

16 layers · 105.61M

🕸 GraphSAGE Recommender

Inductive node embeddings via neighbor sampling + aggregation — for graph-based recommenders

10 layers · 263.0K