Key Takeaways
Llama 3.1 405B Instruct wins:
- Higher intelligence benchmark
- Better at coding
- Better at math
Llama 3.2 11B Vision Instruct wins:
- Cheaper input tokens
- Cheaper output tokens
- Larger context window
- Faster response time
- Supports vision
Price Advantage
Llama 3.2 11B Vision Instruct
Benchmark Advantage
Llama 3.1 405B Instruct
Context Window
Llama 3.2 11B Vision Instruct
Speed
Llama 3.2 11B Vision Instruct
Pricing Comparison
Price Comparison
| Metric | Llama 3.1 405B Instruct | Llama 3.2 11B Vision Instruct | Winner |
|---|---|---|---|
| Input (per 1M tokens) | $4.00 | $0.05 | Llama 3.2 11B Vision Instruct |
| Output (per 1M tokens) | $4.00 | $0.05 | Llama 3.2 11B Vision Instruct |
Using a 3:1 input/output ratio, Llama 3.2 11B Vision Instruct is 99% cheaper overall.
Llama 3.1 405B Instruct Providers
Hyperbolic $4.00 (Cheapest)
Google $5.00
Llama 3.2 11B Vision Instruct Providers
Cloudflare $0.05 (Cheapest)
DeepInfra $0.05 (Cheapest)
Novita $0.06
Together $0.18
Benchmark Comparison
8
Benchmarks Compared
7
Llama 3.1 405B Instruct Wins
0
Llama 3.2 11B Vision Instruct Wins
Benchmark Scores
| Benchmark | Llama 3.1 405B Instruct | Llama 3.2 11B Vision Instruct | Winner |
|---|---|---|---|
Intelligence Index Overall intelligence score | 14.2 | 10.9 | |
Coding Index Code generation & understanding | 14.5 | 4.3 | |
Math Index Mathematical reasoning | 3.0 | 1.7 | |
MMLU Pro Academic knowledge | 73.2 | 46.4 | |
GPQA Graduate-level science | 51.5 | 22.1 | |
LiveCodeBench Competitive programming | 30.5 | 11.0 | |
Aider Real-world code editing | 66.2 | - | - |
AIME Competition math | 21.3 | 9.3 |
Llama 3.1 405B Instruct significantly outperforms in coding benchmarks.
Cost vs Quality
X-axis:
Y-axis:
Loading chart...
Llama 3.1 405B Instruct
Other models
Context & Performance
Context Window
Llama 3.1 405B Instruct
131,000
tokens
Llama 3.2 11B Vision Instruct
131,072
tokens
Max output: 16,384 tokens
Llama 3.2 11B Vision Instruct has a 0% larger context window.
Speed Performance
| Metric | Llama 3.1 405B Instruct | Llama 3.2 11B Vision Instruct | Winner |
|---|---|---|---|
| Tokens/second | 25.2 tok/s | 69.7 tok/s | |
| Time to First Token | 0.79s | 0.41s |
Llama 3.2 11B Vision Instruct responds 177% faster on average.
Capabilities
Feature Comparison
| Feature | Llama 3.1 405B Instruct | Llama 3.2 11B Vision Instruct |
|---|---|---|
| Vision (Image Input) | ||
| Tool/Function Calls | ||
| Reasoning Mode | ||
| Audio Input | ||
| Audio Output | ||
| PDF Input | ||
| Prompt Caching | ||
| Web Search |
License & Release
| Property | Llama 3.1 405B Instruct | Llama 3.2 11B Vision Instruct |
|---|---|---|
| License | Open Source | Open Source |
| Author | Meta-llama | Meta-llama |
| Released | Jul 2024 | Sep 2024 |
Llama 3.1 405B Instruct Modalities
Input
text
Output
text
Llama 3.2 11B Vision Instruct Modalities
Input
textimage
Output
text