Key Takeaways
Phi-3 Medium 128K Instruct wins:
- Higher intelligence benchmark
Llama 3.1 Nemotron 70B Instruct wins:
- Cheaper input tokens
- Cheaper output tokens
- Larger context window
- Faster response time
- Better at coding
- Better at math
Price Advantage
Llama 3.1 Nemotron 70B Instruct
Benchmark Advantage
Llama 3.1 Nemotron 70B Instruct
Context Window
Llama 3.1 Nemotron 70B Instruct
Speed
Llama 3.1 Nemotron 70B Instruct
Pricing Comparison
Price Comparison
| Metric | Phi-3 Medium 128K Instruct | Llama 3.1 Nemotron 70B Instruct | Winner |
|---|---|---|---|
| Input (per 1M tokens) | $1.00 | $0.90 | Llama 3.1 Nemotron 70B Instruct |
| Output (per 1M tokens) | $1.00 | $0.90 | Llama 3.1 Nemotron 70B Instruct |
| Cache Read (per 1M) | N/A | $0.45 | Llama 3.1 Nemotron 70B Instruct |
Using a 3:1 input/output ratio, Llama 3.1 Nemotron 70B Instruct is 10% cheaper overall.
Phi-3 Medium 128K Instruct Providers
No provider data available
Llama 3.1 Nemotron 70B Instruct Providers
No provider data available
Benchmark Comparison
9
Benchmarks Compared
1
Phi-3 Medium 128K Instruct Wins
2
Llama 3.1 Nemotron 70B Instruct Wins
Benchmark Scores
| Benchmark | Phi-3 Medium 128K Instruct | Llama 3.1 Nemotron 70B Instruct | Winner |
|---|---|---|---|
Intelligence Index Overall intelligence score | 32.0 | 13.4 | |
Coding Index Code generation & understanding | - | 10.8 | - |
Math Index Mathematical reasoning | - | 11.0 | - |
MMLU Pro Academic knowledge | 41.2 | 69.0 | |
GPQA Graduate-level science | 11.5 | 46.5 | |
LiveCodeBench Competitive programming | - | 16.9 | - |
Aider Real-world code editing | - | 54.9 | - |
AIME Competition math | - | 24.7 | - |
BBH Big-Bench Hard | 48.5 | - | - |
Llama 3.1 Nemotron 70B Instruct significantly outperforms in coding benchmarks.
Cost vs Quality
X-axis:
Y-axis:
Loading chart...
Phi-3 Medium 128K Instruct
Other models
Context & Performance
Context Window
Phi-3 Medium 128K Instruct
128,000
tokens
Llama 3.1 Nemotron 70B Instruct
131,072
tokens
Llama 3.1 Nemotron 70B Instruct has a 2% larger context window.
Speed Performance
| Metric | Phi-3 Medium 128K Instruct | Llama 3.1 Nemotron 70B Instruct | Winner |
|---|---|---|---|
| Tokens/second | N/A | 35.5 tok/s | |
| Time to First Token | N/A | 0.51s |
Capabilities
Feature Comparison
| Feature | Phi-3 Medium 128K Instruct | Llama 3.1 Nemotron 70B Instruct |
|---|---|---|
| Vision (Image Input) | ||
| Tool/Function Calls | ||
| Reasoning Mode | ||
| Audio Input | ||
| Audio Output | ||
| PDF Input | ||
| Prompt Caching | ||
| Web Search |
License & Release
| Property | Phi-3 Medium 128K Instruct | Llama 3.1 Nemotron 70B Instruct |
|---|---|---|
| License | Open Source | Proprietary |
| Author | Microsoft | Nvidia |
| Released | Unknown | Oct 2024 |
Phi-3 Medium 128K Instruct Modalities
Input
Output
Llama 3.1 Nemotron 70B Instruct Modalities
Input
text
Output
text