Architectures / Recommendation

🧭 Multi-Interest Network (MIND)

Alibaba 2019 - dynamic routing turns one behaviour sequence into K interest vectors, so a user is K points in item space rather than one.

From Li et al. (2019). Multi-Interest Network with Dynamic Routing for Recommendation at Tmall. CIKM 2019. This page is the graph, not the PDF: open it, edit it, verify it, export it.

Every number on this page is computed from the graph by the same functions the app runs, not written by hand.

When to pick it

Structure

12 layers. Output shapes are propagated from the input shape, batch dimension excluded.

What the verifier says

The same 43 structural checks that run on every edit in the app, on this graph.

The PyTorch it exports

Generated from the graph above. First 46 lines; the app exports the whole file, plus the training loop, the data contract and a deploy bundle.

# Architecture designed with Neurarch: https://neurarch.com
# PyTorch: compatible with Python 3.8+ and torch>=1.12
# Colab: pip install torch torchvision  (usually pre-installed)

import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import Tuple

class MultiInterest(nn.Module):
    """MIND behaviour-to-interest dynamic routing. One user becomes K vectors,
    not one, so every downstream layer sees a rank it has to agree with."""

    def __init__(self, embed_dim: int, num_interests: int = 4, num_iterations: int = 3):
        super().__init__()
        self.k = num_interests
        self.iters = max(1, num_iterations)
        self.bilinear = nn.Linear(embed_dim, embed_dim, bias=False)

    @staticmethod
    def _squash(x: torch.Tensor) -> torch.Tensor:
        n2 = x.pow(2).sum(-1, keepdim=True)
        return (n2 / (1.0 + n2)) * x / (n2.sqrt() + 1e-8)

    def forward(self, seq: torch.Tensor) -> torch.Tensor:
        u = self.bilinear(seq)
        b = seq.new_zeros(seq.size(0), self.k, seq.size(-2))
        v = self._squash(torch.bmm(b.softmax(dim=1), u))
        for _ in range(self.iters - 1):
            b = b + torch.bmm(v, u.transpose(1, 2))
            v = self._squash(torch.bmm(b.softmax(dim=1), u))
        return v


class Multi_InterestNetworkMIND(nn.Module):
    def __init__(self):
        super().__init__()

        self.embedding_1 = nn.Embedding(1000000, 64)
        self.multiInterest_1 = MultiInterest(embed_dim=64, num_interests=4, num_iterations=3)
        self.linear_1 = nn.Linear(64, 64)
        self.embedding_2 = nn.Embedding(1000000, 64)

    def forward(self, src, tgt=None):
        # Behavior Seq (50 items) shape: [50]
        # Candidate Item shape: [1]

For agents

Also in Recommendation