Comparisons / EEGNet vs EEG Conformer
EEGNet vs EEG Conformer
A compact EEG convnet against a convolution-plus-transformer one.
EEG Conformer has 76K more parameters than EEGNet: 12 layers added, 5 removed, 6 changed.
EEGNet
- Layers
- 14
- Parameters
- 2.7K
- Input
- 1 × 22 × 1000
- Output
- 4
- Forward-passes
- yes
- Est. train cost
- $0.044
EEG Conformer
- Layers
- 21
- Parameters
- 79K
- Input
- 1 × 22 × 1000
- Output
- 4
- Forward-passes
- yes
- Est. train cost
- $0.045
The deltas
Every number is EEG Conformer relative to EEGNet.
Which GPUs each one fits
Memory for the graph at the input shape both declare. A highlighted row is a card one of them fits and the other does not, which is the difference that decides a purchase.
| GPU | EEGNet | EEG Conformer |
|---|---|---|
| T4 16GB | fits | fits |
| A100 40GB | fits | fits |
| H100 80GB | fits | fits |
Layer by layer
Aligned in topological order. 5 of 28 rows are the same layer with the same parameters.
Hide all 28 rows
| EEGNet | EEG Conformer | ||||||
|---|---|---|---|---|---|---|---|
| Layer | Params | Output | Layer | Params | Output | ||
| 1 | same | eeg_window Input | 1 × 22 × 1000 | eeg_window Input | 1 × 22 × 1000 | ||
| 2 | changed outChannels, kernelSize, padding | temporal_conv Conv2d | 80 | 8 × 22 × 1001 | temporal_conv Conv2d | 400 | 40 × 22 × 976 |
| 3 | removed | norm Batch Norm | 16 | 8 × 22 × 1001 | — | ||
| 4 | changed type, outChannels, depthMultiplier | spatial_depthwise Depthwise Conv2d | 20 | 16 × 1 × 1001 | spatial_conv Conv2d | 400 | 40 × 1 × 976 |
| 5 | changed normalizedShape | norm Batch Norm | 32 | 16 × 1 × 1001 | norm Batch Norm | 80 | 40 × 1 × 976 |
| 6 | same | act Elu | 16 × 1 × 1001 | act Elu | 40 × 1 × 976 | ||
| 7 | changed kernelSize, stride | pool Avgpool2d | 16 × 1 × 250 | pool Avgpool2d | 40 × 1 × 61 | ||
| 8 | removed | drop Dropout | 16 × 1 × 250 | — | |||
| 9 | changed type, outChannels, kernelSize, padding | separable_conv Separable Conv2d | 41 | 16 × 1 × 251 | patch_proj Conv2d | 400 | 40 × 1 × 61 |
| 10 | added | — | to_tokens Reshape | 61 × 40 | |||
| 11 | changed type, normalizedShape | norm Batch Norm | 32 | 16 × 1 × 251 | norm Layer Norm | 80 | 61 × 40 |
| 12 | removed | act Elu | 16 × 1 × 251 | — | |||
| 13 | removed | pool Avgpool2d | 16 × 1 × 31 | — | |||
| 14 | removed | drop Dropout | 16 × 1 × 31 | — | |||
| 15 | added | — | self_attn Multi Head Attention | 6.6K | 61 × 40 | ||
| 16 | added | — | residual Add | 61 × 40 | |||
| 17 | added | — | norm Layer Norm | 80 | 61 × 40 | ||
| 18 | added | — | dense Feed Forward | 13K | 61 × 40 | ||
| 19 | added | — | residual Add | 61 × 40 | |||
| 20 | added | — | norm Layer Norm | 80 | 61 × 40 | ||
| 21 | added | — | self_attn Multi Head Attention | 6.6K | 61 × 40 | ||
| 22 | added | — | residual Add | 61 × 40 | |||
| 23 | added | — | norm Layer Norm | 80 | 61 × 40 | ||
| 24 | added | — | dense Feed Forward | 13K | 61 × 40 | ||
| 25 | added | — | residual Add | 61 × 40 | |||
| 26–28 | same | 3 unchanged layers | |||||
What this is not
- Parameter counts are derived from the graph, not read from a checkpoint. They are exact for a graph that is fully specified and approximate for one that is not.
- Cost and GPU fit are estimates from the graph under one set of assumptions, not measurements of a run.
Take it further
Open either graph in the editor, change it, and check it again: EEGNet · EEG Conformer
Compare any two models of your own, including anything on Hugging Face: the comparison tool.
Machine-readable: this page as markdown ·
the pair index ·
POST https://www.neurarch.com/api/v1/plan for a graph of your own.