Comparisons / LLaVA-1.5 vs CLIP ViT-B/32
LLaVA-1.5 vs CLIP ViT-B/32
A vision-language model against the encoder it is built on.
CLIP ViT-B/32 has 6.9B fewer parameters than LLaVA-1.5: 7 layers added, 196 removed, 32 changed.
LLaVA-1.5
- Layers
- 226
- Parameters
- 7.1B
- Input
- 3 × 336 × 336
- Output
- 1 × 2624 × 32000
- Forward-passes
- yes
- Est. train cost
- $690.06
CLIP ViT-B/32
- Layers
- 37
- Parameters
- 151M
- Input
- 3 × 224 × 224
- Output
- 512
- Forward-passes
- yes
- Est. train cost
- $0.313
The deltas
Every number is CLIP ViT-B/32 relative to LLaVA-1.5.
Which GPUs each one fits
Each side is measured at its own declared input (3 × 336 × 336 against 3 × 224 × 224). Both columns are right about their own model; the difference between them is not a fact about the designs.
| GPU | LLaVA-1.5 | CLIP ViT-B/32 |
|---|---|---|
| T4 16GB | no | fits |
| A100 40GB | no | fits |
| H100 80GB | no | fits |
Layer by layer
Aligned in topological order. 1 of 236 rows are the same layer with the same parameters.
Show all 236 rows
| LLaVA-1.5 | CLIP ViT-B/32 | ||||||
|---|---|---|---|---|---|---|---|
| Layer | Params | Output | Layer | Params | Output | ||
| 1 | changed shape | image Input | 3 × 336 × 336 | image Input | 3 × 224 × 224 | ||
| 2 | changed shape | text_tokens Input | 1 × 2048 | tokens Input | 77 | ||
| 3 | changed imgSize, patchSize, embedDim | clip_patch_14 Patch Embed | 603K | 576 × 1024 | patch_embed Patch Embed | 2.4M | 49 × 768 |
| 4 | changed numEmbeddings, embeddingDim | token_embed Embedding | 131M | 1 × 2048 × 4096 | tok_embed Embedding | 25M | 77 × 512 |
| 5 | added | — | vis_pos Positional Encoding | 49 × 768 | |||
| 6 | changed maxLen, embedDim | vis_pos Positional Encoding | 576 × 1024 | txt_pos Positional Encoding | 77 × 512 | ||
| 7 | changed embedDim, numHeads, ffDim | clip_block_1 Transformer Block | 13M | 576 × 1024 | vis_block_1 Transformer Block | 7.1M | 49 × 768 |
| 8 | changed embedDim, numHeads, ffDim | clip_block_2 Transformer Block | 13M | 576 × 1024 | txt_block_1 Transformer Block | 3.2M | 77 × 512 |
| 9 | changed embedDim, numHeads, ffDim | clip_block_3 Transformer Block | 13M | 576 × 1024 | vis_block_2 Transformer Block | 7.1M | 49 × 768 |
| 10 | changed embedDim, numHeads, ffDim | clip_block_4 Transformer Block | 13M | 576 × 1024 | txt_block_2 Transformer Block | 3.2M | 77 × 512 |
| 11 | changed embedDim, numHeads, ffDim | clip_block_5 Transformer Block | 13M | 576 × 1024 | vis_block_3 Transformer Block | 7.1M | 49 × 768 |
| 12 | changed embedDim, numHeads, ffDim | clip_block_6 Transformer Block | 13M | 576 × 1024 | txt_block_3 Transformer Block | 3.2M | 77 × 512 |
| 13 | changed embedDim, numHeads, ffDim | clip_block_7 Transformer Block | 13M | 576 × 1024 | vis_block_4 Transformer Block | 7.1M | 49 × 768 |
| 14 | changed embedDim, numHeads, ffDim | clip_block_8 Transformer Block | 13M | 576 × 1024 | txt_block_4 Transformer Block | 3.2M | 77 × 512 |
| 15 | changed embedDim, numHeads, ffDim | clip_block_9 Transformer Block | 13M | 576 × 1024 | vis_block_5 Transformer Block | 7.1M | 49 × 768 |
| 16 | changed embedDim, numHeads, ffDim | clip_block_10 Transformer Block | 13M | 576 × 1024 | txt_block_5 Transformer Block | 3.2M | 77 × 512 |
| 17 | changed embedDim, numHeads, ffDim | clip_block_11 Transformer Block | 13M | 576 × 1024 | vis_block_6 Transformer Block | 7.1M | 49 × 768 |
| 18 | changed embedDim, numHeads, ffDim | clip_block_12 Transformer Block | 13M | 576 × 1024 | txt_block_6 Transformer Block | 3.2M | 77 × 512 |
| 19 | changed embedDim, numHeads, ffDim | clip_block_13 Transformer Block | 13M | 576 × 1024 | vis_block_7 Transformer Block | 7.1M | 49 × 768 |
| 20 | changed embedDim, numHeads, ffDim | clip_block_14 Transformer Block | 13M | 576 × 1024 | txt_block_7 Transformer Block | 3.2M | 77 × 512 |
| 21 | changed embedDim, numHeads, ffDim | clip_block_15 Transformer Block | 13M | 576 × 1024 | vis_block_8 Transformer Block | 7.1M | 49 × 768 |
| 22 | changed embedDim, numHeads, ffDim | clip_block_16 Transformer Block | 13M | 576 × 1024 | txt_block_8 Transformer Block | 3.2M | 77 × 512 |
| 23 | changed embedDim, numHeads, ffDim | clip_block_17 Transformer Block | 13M | 576 × 1024 | vis_block_9 Transformer Block | 7.1M | 49 × 768 |
| 24 | changed embedDim, numHeads, ffDim | clip_block_18 Transformer Block | 13M | 576 × 1024 | txt_block_9 Transformer Block | 3.2M | 77 × 512 |
| 25 | changed embedDim, numHeads, ffDim | clip_block_19 Transformer Block | 13M | 576 × 1024 | vis_block_10 Transformer Block | 7.1M | 49 × 768 |
| 26 | changed embedDim, numHeads, ffDim | clip_block_20 Transformer Block | 13M | 576 × 1024 | txt_block_10 Transformer Block | 3.2M | 77 × 512 |
| 27 | changed embedDim, numHeads, ffDim | clip_block_21 Transformer Block | 13M | 576 × 1024 | vis_block_11 Transformer Block | 7.1M | 49 × 768 |
| 28 | changed embedDim, numHeads, ffDim | clip_block_22 Transformer Block | 13M | 576 × 1024 | txt_block_11 Transformer Block | 3.2M | 77 × 512 |
| 29 | changed embedDim, numHeads, ffDim | clip_block_23 Transformer Block | 13M | 576 × 1024 | vis_block_12 Transformer Block | 7.1M | 49 × 768 |
| 30 | changed embedDim, numHeads, ffDim | clip_block_24 Transformer Block | 13M | 576 × 1024 | txt_block_12 Transformer Block | 3.2M | 77 × 512 |
| 31 | removed | projector_1 Linear | 4.2M | 576 × 4096 | — | ||
| 32 | removed | projector_gelu Gelu | 576 × 4096 | — | |||
| 33 | removed | projector_2 Linear | 17M | 576 × 4096 | — | ||
| 34 | removed | vision_tokens Reshape | 1 × 576 × 4096 | — | |||
| 35 | removed | image+text_tokens Concatenate | 1 × 2624 × 4096 | — | |||
| 36 | removed | llm_attn_norm_1 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 37 | removed | llm_attn_1 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 38 | removed | llm_res1_1 Add | 1 × 2624 × 4096 | — | |||
| 39 | removed | llm_ffn_norm_1 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 40 | removed | llm_ffn_1 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 41 | removed | llm_res2_1 Add | 1 × 2624 × 4096 | — | |||
| 42 | removed | llm_attn_norm_2 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 43 | removed | llm_attn_2 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 44 | removed | llm_res1_2 Add | 1 × 2624 × 4096 | — | |||
| 45 | removed | llm_ffn_norm_2 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 46 | removed | llm_ffn_2 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 47 | removed | llm_res2_2 Add | 1 × 2624 × 4096 | — | |||
| 48 | removed | llm_attn_norm_3 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 49 | removed | llm_attn_3 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 50 | removed | llm_res1_3 Add | 1 × 2624 × 4096 | — | |||
| 51 | removed | llm_ffn_norm_3 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 52 | removed | llm_ffn_3 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 53 | removed | llm_res2_3 Add | 1 × 2624 × 4096 | — | |||
| 54 | removed | llm_attn_norm_4 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 55 | removed | llm_attn_4 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 56 | removed | llm_res1_4 Add | 1 × 2624 × 4096 | — | |||
| 57 | removed | llm_ffn_norm_4 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 58 | removed | llm_ffn_4 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 59 | removed | llm_res2_4 Add | 1 × 2624 × 4096 | — | |||
| 60 | removed | llm_attn_norm_5 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 61 | removed | llm_attn_5 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 62 | removed | llm_res1_5 Add | 1 × 2624 × 4096 | — | |||
| 63 | removed | llm_ffn_norm_5 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 64 | removed | llm_ffn_5 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 65 | removed | llm_res2_5 Add | 1 × 2624 × 4096 | — | |||
| 66 | removed | llm_attn_norm_6 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 67 | removed | llm_attn_6 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 68 | removed | llm_res1_6 Add | 1 × 2624 × 4096 | — | |||
| 69 | removed | llm_ffn_norm_6 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 70 | removed | llm_ffn_6 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 71 | removed | llm_res2_6 Add | 1 × 2624 × 4096 | — | |||
| 72 | removed | llm_attn_norm_7 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 73 | removed | llm_attn_7 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 74 | removed | llm_res1_7 Add | 1 × 2624 × 4096 | — | |||
| 75 | removed | llm_ffn_norm_7 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 76 | removed | llm_ffn_7 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 77 | removed | llm_res2_7 Add | 1 × 2624 × 4096 | — | |||
| 78 | removed | llm_attn_norm_8 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 79 | removed | llm_attn_8 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 80 | removed | llm_res1_8 Add | 1 × 2624 × 4096 | — | |||
| 81 | removed | llm_ffn_norm_8 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 82 | removed | llm_ffn_8 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 83 | removed | llm_res2_8 Add | 1 × 2624 × 4096 | — | |||
| 84 | removed | llm_attn_norm_9 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 85 | removed | llm_attn_9 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 86 | removed | llm_res1_9 Add | 1 × 2624 × 4096 | — | |||
| 87 | removed | llm_ffn_norm_9 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 88 | removed | llm_ffn_9 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 89 | removed | llm_res2_9 Add | 1 × 2624 × 4096 | — | |||
| 90 | removed | llm_attn_norm_10 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 91 | removed | llm_attn_10 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 92 | removed | llm_res1_10 Add | 1 × 2624 × 4096 | — | |||
| 93 | removed | llm_ffn_norm_10 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 94 | removed | llm_ffn_10 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 95 | removed | llm_res2_10 Add | 1 × 2624 × 4096 | — | |||
| 96 | removed | llm_attn_norm_11 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 97 | removed | llm_attn_11 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 98 | removed | llm_res1_11 Add | 1 × 2624 × 4096 | — | |||
| 99 | removed | llm_ffn_norm_11 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 100 | removed | llm_ffn_11 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 101 | removed | llm_res2_11 Add | 1 × 2624 × 4096 | — | |||
| 102 | removed | llm_attn_norm_12 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 103 | removed | llm_attn_12 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 104 | removed | llm_res1_12 Add | 1 × 2624 × 4096 | — | |||
| 105 | removed | llm_ffn_norm_12 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 106 | removed | llm_ffn_12 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 107 | removed | llm_res2_12 Add | 1 × 2624 × 4096 | — | |||
| 108 | removed | llm_attn_norm_13 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 109 | removed | llm_attn_13 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 110 | removed | llm_res1_13 Add | 1 × 2624 × 4096 | — | |||
| 111 | removed | llm_ffn_norm_13 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 112 | removed | llm_ffn_13 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 113 | removed | llm_res2_13 Add | 1 × 2624 × 4096 | — | |||
| 114 | removed | llm_attn_norm_14 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 115 | removed | llm_attn_14 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 116 | removed | llm_res1_14 Add | 1 × 2624 × 4096 | — | |||
| 117 | removed | llm_ffn_norm_14 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 118 | removed | llm_ffn_14 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 119 | removed | llm_res2_14 Add | 1 × 2624 × 4096 | — | |||
| 120 | removed | llm_attn_norm_15 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 121 | removed | llm_attn_15 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 122 | removed | llm_res1_15 Add | 1 × 2624 × 4096 | — | |||
| 123 | removed | llm_ffn_norm_15 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 124 | removed | llm_ffn_15 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 125 | removed | llm_res2_15 Add | 1 × 2624 × 4096 | — | |||
| 126 | removed | llm_attn_norm_16 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 127 | removed | llm_attn_16 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 128 | removed | llm_res1_16 Add | 1 × 2624 × 4096 | — | |||
| 129 | removed | llm_ffn_norm_16 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 130 | removed | llm_ffn_16 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 131 | removed | llm_res2_16 Add | 1 × 2624 × 4096 | — | |||
| 132 | removed | llm_attn_norm_17 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 133 | removed | llm_attn_17 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 134 | removed | llm_res1_17 Add | 1 × 2624 × 4096 | — | |||
| 135 | removed | llm_ffn_norm_17 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 136 | removed | llm_ffn_17 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 137 | removed | llm_res2_17 Add | 1 × 2624 × 4096 | — | |||
| 138 | removed | llm_attn_norm_18 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 139 | removed | llm_attn_18 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 140 | removed | llm_res1_18 Add | 1 × 2624 × 4096 | — | |||
| 141 | removed | llm_ffn_norm_18 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 142 | removed | llm_ffn_18 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 143 | removed | llm_res2_18 Add | 1 × 2624 × 4096 | — | |||
| 144 | removed | llm_attn_norm_19 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 145 | removed | llm_attn_19 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 146 | removed | llm_res1_19 Add | 1 × 2624 × 4096 | — | |||
| 147 | removed | llm_ffn_norm_19 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 148 | removed | llm_ffn_19 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 149 | removed | llm_res2_19 Add | 1 × 2624 × 4096 | — | |||
| 150 | removed | llm_attn_norm_20 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 151 | removed | llm_attn_20 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 152 | removed | llm_res1_20 Add | 1 × 2624 × 4096 | — | |||
| 153 | removed | llm_ffn_norm_20 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 154 | removed | llm_ffn_20 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 155 | removed | llm_res2_20 Add | 1 × 2624 × 4096 | — | |||
| 156 | removed | llm_attn_norm_21 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 157 | removed | llm_attn_21 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 158 | removed | llm_res1_21 Add | 1 × 2624 × 4096 | — | |||
| 159 | removed | llm_ffn_norm_21 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 160 | removed | llm_ffn_21 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 161 | removed | llm_res2_21 Add | 1 × 2624 × 4096 | — | |||
| 162 | removed | llm_attn_norm_22 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 163 | removed | llm_attn_22 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 164 | removed | llm_res1_22 Add | 1 × 2624 × 4096 | — | |||
| 165 | removed | llm_ffn_norm_22 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 166 | removed | llm_ffn_22 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 167 | removed | llm_res2_22 Add | 1 × 2624 × 4096 | — | |||
| 168 | removed | llm_attn_norm_23 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 169 | removed | llm_attn_23 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 170 | removed | llm_res1_23 Add | 1 × 2624 × 4096 | — | |||
| 171 | removed | llm_ffn_norm_23 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 172 | removed | llm_ffn_23 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 173 | removed | llm_res2_23 Add | 1 × 2624 × 4096 | — | |||
| 174 | removed | llm_attn_norm_24 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 175 | removed | llm_attn_24 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 176 | removed | llm_res1_24 Add | 1 × 2624 × 4096 | — | |||
| 177 | removed | llm_ffn_norm_24 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 178 | removed | llm_ffn_24 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 179 | removed | llm_res2_24 Add | 1 × 2624 × 4096 | — | |||
| 180 | removed | llm_attn_norm_25 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 181 | removed | llm_attn_25 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 182 | removed | llm_res1_25 Add | 1 × 2624 × 4096 | — | |||
| 183 | removed | llm_ffn_norm_25 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 184 | removed | llm_ffn_25 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 185 | removed | llm_res2_25 Add | 1 × 2624 × 4096 | — | |||
| 186 | removed | llm_attn_norm_26 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 187 | removed | llm_attn_26 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 188 | removed | llm_res1_26 Add | 1 × 2624 × 4096 | — | |||
| 189 | removed | llm_ffn_norm_26 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 190 | removed | llm_ffn_26 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 191 | removed | llm_res2_26 Add | 1 × 2624 × 4096 | — | |||
| 192 | removed | llm_attn_norm_27 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 193 | removed | llm_attn_27 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 194 | removed | llm_res1_27 Add | 1 × 2624 × 4096 | — | |||
| 195 | removed | llm_ffn_norm_27 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 196 | removed | llm_ffn_27 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 197 | removed | llm_res2_27 Add | 1 × 2624 × 4096 | — | |||
| 198 | removed | llm_attn_norm_28 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 199 | removed | llm_attn_28 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 200 | removed | llm_res1_28 Add | 1 × 2624 × 4096 | — | |||
| 201 | removed | llm_ffn_norm_28 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 202 | removed | llm_ffn_28 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 203 | removed | llm_res2_28 Add | 1 × 2624 × 4096 | — | |||
| 204 | removed | llm_attn_norm_29 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 205 | removed | llm_attn_29 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 206 | removed | llm_res1_29 Add | 1 × 2624 × 4096 | — | |||
| 207 | removed | llm_ffn_norm_29 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 208 | removed | llm_ffn_29 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 209 | removed | llm_res2_29 Add | 1 × 2624 × 4096 | — | |||
| 210 | removed | llm_attn_norm_30 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 211 | removed | llm_attn_30 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 212 | removed | llm_res1_30 Add | 1 × 2624 × 4096 | — | |||
| 213 | removed | llm_ffn_norm_30 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 214 | removed | llm_ffn_30 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 215 | removed | llm_res2_30 Add | 1 × 2624 × 4096 | — | |||
| 216 | removed | llm_attn_norm_31 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 217 | removed | llm_attn_31 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 218 | removed | llm_res1_31 Add | 1 × 2624 × 4096 | — | |||
| 219 | removed | llm_ffn_norm_31 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 220 | removed | llm_ffn_31 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 221 | removed | llm_res2_31 Add | 1 × 2624 × 4096 | — | |||
| 222 | removed | llm_attn_norm_32 Rms Norm | 4.1K | 1 × 2624 × 4096 | — | ||
| 223 | removed | llm_attn_32 Grouped Query Attention | 67M | 1 × 2624 × 4096 | — | ||
| 224 | removed | llm_res1_32 Add | 1 × 2624 × 4096 | — | |||
| 225 | changed type, normalizedShape | llm_ffn_norm_32 Rms Norm | 4.1K | 1 × 2624 × 4096 | vis_norm Layer Norm | 1.5K | 49 × 768 |
| 226 | removed | llm_ffn_32 Swiglu | 135M | 1 × 2624 × 4096 | — | ||
| 227 | removed | llm_res2_32 Add | 1 × 2624 × 4096 | — | |||
| 228 | changed type, normalizedShape | final_norm Rms Norm | 4.1K | 1 × 2624 × 4096 | txt_norm Layer Norm | 1.0K | 77 × 512 |
| 229 | added | — | to_channels_vis Permute | 768 × 49 | |||
| 230 | added | — | to_channels_txt Permute | 512 × 77 | |||
| 231 | added | — | vis_pool Global Avg Pool1d | 768 | |||
| 232 | added | — | txt_pool Global Avg Pool1d | 512 | |||
| 233 | added | — | img_proj Linear | 394K | 512 | ||
| 234 | changed inFeatures, outFeatures, bias | lm_head Linear | 131M | 1 × 2624 × 32000 | txt_proj Linear | 263K | 512 |
| 235 | added | — | similarity Matmul | 512 | |||
| 236 | same | logits Output | 1 × 2624 × 32000 | logits Output | 512 | ||
What this is not
- The two are priced at different declared inputs (3 × 336 × 336 against 3 × 224 × 224), so memory, cost and GPU fit are each right about their own model and are not a comparison between them. The layer and parameter deltas are unaffected.
- Parameter counts are derived from the graph, not read from a checkpoint. They are exact for a graph that is fully specified and approximate for one that is not.
- Cost and GPU fit are estimates from the graph under one set of assumptions, not measurements of a run.
Take it further
Open either graph in the editor, change it, and check it again: LLaVA-1.5 · CLIP ViT-B/32
Compare any two models of your own, including anything on Hugging Face: the comparison tool.
Machine-readable: this page as markdown ·
the pair index ·
POST https://www.neurarch.com/api/v1/plan for a graph of your own.