Comparisons / Behavior Sequence Transformer vs SLi-Rec
Behavior Sequence Transformer vs SLi-Rec
Two sequential recommenders with different notions of time.
SLi-Rec has 65M fewer parameters than Behavior Sequence Transformer: 6 layers added, 5 removed, 10 changed.
Behavior Sequence Transformer
- Layers
- 16
- Parameters
- 129M
- Input
- 50
- Output
- 1
- Forward-passes
- yes
- Est. train cost
- $0.083
The deltas
Every number is SLi-Rec relative to Behavior Sequence Transformer.
Which GPUs each one fits
Memory for the graph at the input shape both declare. A highlighted row is a card one of them fits and the other does not, which is the difference that decides a purchase.
| GPU | Behavior Sequence Transformer | SLi-Rec |
|---|---|---|
| T4 16GB | fits | fits |
| A100 40GB | fits | fits |
| H100 80GB | fits | fits |
Layer by layer
Aligned in topological order. 5 of 26 rows are the same layer with the same parameters.
Hide all 26 rows
| Behavior Sequence Transformer | SLi-Rec | ||||||
|---|---|---|---|---|---|---|---|
| Layer | Params | Output | Layer | Params | Output | ||
| 1 | removed | Behavior Seq (T items) Input | 50 | — | |||
| 2 | changed shape | Target Item Input | 1 | User History (T items) Input | 50 | ||
| 3 | changed shape | User Profile Input | 16 | Target Item Input | 1 | ||
| 4 | changed embeddingDim | Item Embed Embedding | 64M | 50 × 64 | Item Embed Embedding | 32M | 50 × 32 |
| 5 | changed embeddingDim | Target Embed Embedding | 64M | 1 × 64 | Target Embed Embedding | 32M | 1 × 32 |
| 6 | removed | User Tower Linear | 544 | 32 | — | ||
| 7 | removed | Pos Encoding Positional Encoding | 50 × 64 | — | |||
| 8 | removed | [seq; tgt] Concatenate | 51 × 64 | — | |||
| 9 | added | — | Time-LSTM (short) Lstm | 17K | 64 | ||
| 10 | changed type, dModel, numHeads, dFf, numLayers, causal | Transformer Encoder Transformer Block | 51 × 64 | ASVD Attn (long) Self Attention | 50 × 32 | ||
| 11 | added | — | Flatten Flatten | 32 | |||
| 12 | same | to_channels Permute | 64 × 51 | to_channels Permute | 32 × 50 | ||
| 13 | added | — | Target Proj Linear | 2.1K | 64 | ||
| 14 | same | Mean Pool Global Avg Pool1d | 64 | Pool Long Global Avg Pool1d | 32 | ||
| 15 | added | — | Long Proj Linear | 2.1K | 64 | ||
| 16 | changed numInputs | Concat all Concatenate | 96 | Concat [s; l; t] Concatenate | 192 | ||
| 17 | changed inFeatures, outFeatures | MLP 1024 Linear | 99K | 1024 | Fusion Gate σ(W·[s;l;t]) Linear | 12K | 64 |
| 18 | changed type | PReLU Prelu | 1 | 1024 | Gate σ Sigmoid | 64 | |
| 19 | added | — | α·short + (1−α)·long Add | 64 | |||
| 20 | added | — | Concat [user, target] Concatenate | 128 | |||
| 21 | changed inFeatures, outFeatures | MLP 512 Linear | 525K | 512 | MLP 1 Linear | 8.3K | 64 |
| 22 | same | PReLU Prelu | 1 | 512 | PReLU Prelu | 1 | 64 |
| 23 | removed | MLP 256 Linear | 131K | 256 | — | ||
| 24 | changed inFeatures | CTR Head Linear | 257 | 1 | Score Head Linear | 65 | 1 |
| 25 | same | Sigmoid Sigmoid | 1 | Sigmoid Sigmoid | 1 | ||
| 26 | same | P(click) Output | 1 | P(click) Output | 1 | ||
What this is not
- Parameter counts are derived from the graph, not read from a checkpoint. They are exact for a graph that is fully specified and approximate for one that is not.
- Cost and GPU fit are estimates from the graph under one set of assumptions, not measurements of a run.
Take it further
Open either graph in the editor, change it, and check it again: Behavior Sequence Transformer · SLi-Rec
Compare any two models of your own, including anything on Hugging Face: the comparison tool.
Machine-readable: this page as markdown ·
the pair index ·
POST https://www.neurarch.com/api/v1/plan for a graph of your own.