Architectures / Recommendation

🕒 SLi-Rec

Yu et al. 2019 — Short and Long-term Interest Recommender. Time-LSTM + ASVD attention fused via gate

Layers

Parameters

64.05M

Input

Output

Verifier

1 advisory

Every number on this page is computed from the graph by the same functions the app runs, not written by hand.

Open SLi-Rec on the canvas Free, no account needed

When to pick it

Pick when both short-session intent and long-term preferences matter and you want each modelled separately, then fused via a learned gate.

Structure

20 layers. Output shapes are propagated from the input shape, batch dimension excluded.

	Layer	Type	Parameters	Output shape
1	User History (T items)	Input	shape=[50]	50
2	Item Embed	Embedding	vocabSize=1000000	50 × 32
3	Time-LSTM (short)	LSTM	inFeatures=32, hiddenSize=64, numLayers=1	64
4	ASVD Attn (long)	Self-Attention	numHeads=4	50 × 32
5	to_channels	Permute		32 × 50
6	Pool Long	GlobalAvgPool1D		32
7	Long Proj	Linear	outFeatures=64, inFeatures=32	64
8	Target Item	Input	shape=[1]	1
9	Target Embed	Embedding	vocabSize=1000000	1 × 32
10	Flatten	Flatten		32
11	Target Proj	Linear	outFeatures=64, inFeatures=32	64
12	Fusion Gate σ(W·[s;l;t])	Linear	outFeatures=64, inFeatures=192	64
13	Gate σ	Sigmoid		64
14	α·short + (1−α)·long	Add		64
15	Concat [user, target]	Concatenate		128
16	MLP 1	Linear	outFeatures=64, inFeatures=128	64
17	PReLU	PReLU		64
18	Score Head	Linear	outFeatures=1, inFeatures=64	1
19	Sigmoid	Sigmoid		1
20	P(click)	Output		1

What the verifier says

The same 41 structural checks that run on every edit in the app, on this graph.

info"Sigmoid" feeds directly into Output. PyTorch's nn.CrossEntropyLoss already applies log-softmax internally, an explicit Softmax causes double-application and degrades training stability. Fix: Remove Softmax/Sigmoid for training. Restore it in a separate inference wrapper or ONNX export. (Sigmoid)
output-activation

warn1 attention layer(s) present but no positional encoding found. Attention is permutation-invariant, without position information the model cannot distinguish token order. Fix: Add a PositionalEncoding (sinusoidal) or RoPE layer before the first attention layer. (ASVD Attn (long))
attention-no-pe

infoSigmoid saturates to [0,1] / [-1,1], and its gradient approaches zero for large inputs. In networks deeper than 5 layers, this halts learning in early layers. Fix: Use ReLU, GELU, or SiLU for hidden layers. Keep Sigmoid only at binary classification outputs; Tanh in specific contexts (GAN generators, LSTM gates). (Gate σ)
vanishing-gradient

info17 layers with no BatchNorm, LayerNorm, or GroupNorm. Without normalization, activations can explode or vanish across layers, causing slow or unstable training. Fix: Add BatchNorm after Conv2d (CV tasks), LayerNorm after attention/FFN (NLP/LLM), or GroupNorm for small batch sizes.
deep-no-norm

info"Long Proj" feeds directly into "Fusion Gate σ(W·[s;l;t])" with no activation between them. Two stacked linear maps collapse into one (W₂·W₁), so the extra layer costs parameters but adds no representational power. Fix: Add a non-linearity (ReLU/GELU) between them. If this is a deliberate low-rank / factorized projection (down-proj → up-proj), this hint is safe to ignore. (Long Proj)
consecutive-linear-no-activation

info"Target Proj" feeds directly into "Fusion Gate σ(W·[s;l;t])" with no activation between them. Two stacked linear maps collapse into one (W₂·W₁), so the extra layer costs parameters but adds no representational power. Fix: Add a non-linearity (ReLU/GELU) between them. If this is a deliberate low-rank / factorized projection (down-proj → up-proj), this hint is safe to ignore. (Target Proj)
consecutive-linear-no-activation

infoPyTorch initializes Linear/Conv with Kaiming (He) init, which is derived for ReLU-family activations. Feeding a saturating activation (sigmoid/tanh) from a He-initialized layer starts training in the saturated tails, shrinking early gradients. Fix: Initialize these layers with Xavier instead: nn.init.xavier_uniform_(w, gain=nn.init.calculate_gain("sigmoid"|"tanh")), or switch the activation to a ReLU-family one. (Fusion Gate σ(W·[s;l;t]))
init-activation-mismatch

The PyTorch it exports

Generated from the graph above. First 46 lines; the app exports the whole file, plus the training loop, the data contract and a deploy bundle.

# Architecture designed with Neurarch: https://neurarch.com
# PyTorch: compatible with Python 3.8+ and torch>=1.12
# Colab: pip install torch torchvision  (usually pre-installed)
#
# WARNING: 2 layer(s) below are not yet supported by the PyTorch
# exporter and pass their input through UNCHANGED in forward():
#   - to_channels (permute)
#   - Pool Long (globalAvgPool1d)

import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import Tuple

class SLi_Rec(nn.Module):
    def __init__(self):
        super().__init__()

        self.embedding_1 = nn.Embedding(1000000, 32)
        self.lstm_1 = nn.LSTM(32, 64, num_layers=1, batch_first=True)
        self.selfAttention_1 = nn.MultiheadAttention(embed_dim=128, num_heads=4, batch_first=True)
        self.linear_1 = nn.Linear(32, 64)
        self.embedding_2 = nn.Embedding(1000000, 32)
        self.linear_2 = nn.Linear(32, 64)
        self.linear_3 = nn.Linear(192, 64)
        self.linear_4 = nn.Linear(128, 64)
        self.prelu_1 = nn.PReLU(num_parameters=1)
        self.linear_5 = nn.Linear(64, 1)

    def forward(self, src, tgt=None):
        # User History (T items) shape: [50]
        # Target Item shape: [1]
        embedding_st_emb = self.embedding_1(src)
        lstm_lstm = self.lstm_1(embedding_st_emb)[0][:, -1, :]
        self_attention_g_attn = self.selfAttention_1(embedding_st_emb, embedding_st_emb, embedding_st_emb)[0]
        # TODO: layer 'to_channels' (permute) is not yet supported by the exporter; passing through unchanged
        # TODO: layer 'Pool Long' (globalAvgPool1d) is not yet supported by the exporter; passing through unchanged
        linear_ong_fc = self.linear_1(self_attention_g_attn)
        embedding_gt_emb = self.embedding_2(tgt)
        flatten_t_flat = torch.flatten(embedding_gt_emb, 1)
        linear_tgt_fc = self.linear_2(flatten_t_flat)
        linear_gate = self.linear_3(lstm_lstm)
        sigmoid_te_sig = torch.sigmoid(linear_gate)
        add_fused = sigmoid_te_sig + lstm_lstm + linear_ong_fc
        concatenate__final = torch.cat([add_fused, linear_tgt_fc], dim=-1)
        linear_fc1 = self.linear_4(concatenate__final)

For agents

This architecture is machine-readable end to end. An agent can list the set, fetch this graph, edit it, and have the edit verified before any GPU time is spent.

Markdown twin Index JSON This graph as JSON MCP server POST /api/v1/check

Also in Recommendation

🗼 Two-Tower

User+Item dual encoder for retrieval — embeddings → MLP per side → dot product score

Meta's Deep Learning Recommendation Model — bottom MLP for dense, embedding for sparse, feature interaction, top MLP

14 layers · 32.35M

🤝 Neural Collaborative Filtering

He et al

16 layers · 105.61M