LLM Speed & Latency News - Inference Performance Updates
AI speed and performance news. Inference optimization, latency improvements, throughput benchmarks, and model efficiency.
News Feed
KernelEvolve: How Meta’s Ranking Engineer Agent Optimizes AI Infrastructure
This is the second post in the Ranking Engineer Agent blog series exploring the autonomous AI capabilities accelerating Meta’s Ads Ranking innovation. The previous post introduced Ranking Engineer Agent’s ML exploration capability, which autonomously designs, executes, and analyzes ranking model experiments. This post covers how to optimize the low-level infrastructure that makes those models run [...] Read More... The post KernelEvolve: How Meta’s Ranking Engineer Agent Optimizes AI Infrastructure appeared first on Engineering at Meta .
New ways to balance cost and reliability in the Gemini API
Gemini API Dials
Scaling seismic foundation models on AWS: Distributed training with Amazon SageMaker HyperPod and expanding context windows
This post describes how TGS achieved near-linear scaling for distributed training and expanded context windows for their Vision Transformer-based SFM using Amazon SageMaker HyperPod. This joint solution cut training time from 6 months to just 5 days while enabling analysis of seismic volumes larger than previously possible.
xAI: Grok 4.20 (x-ai/grok-4.20)
Grok 4.20 is xAI's newest flagship model with industry-leading speed and agentic tool calling capabilities. It combines the lowest hallucination rate on the market with strict prompt adherance, delivering consistently precise and truthful responses. Reasoning can be enabled/disabled using the `reasoning` `enabled` parameter in the API. Learn more in our docs
Meta Adaptive Ranking Model: Bending the Inference Scaling Curve to Serve LLM-Scale Models for Ads
Meta continues to lead the industry in utilizing groundbreaking AI Recommendation Systems (RecSys) to deliver better experiences for people, and better results for advertisers. To reach the next frontier of performance, we are scaling Meta’s Ads Recommender runtime models to LLM-scale & complexity to further a deeper understanding of people’s interests and intent. This increase [...] Read More... The post Meta Adaptive Ranking Model: Bending the Inference Scaling Curve to Serve LLM-Scale Models for Ads appeared first on Engineering at Meta .
[AINews] Everything is CLI
a quiet day lets us reflect on the growing trend of CLIs for ~everything~ agents
Run Generative AI inference with Amazon Bedrock in Asia Pacific (New Zealand)
Today, we’re excited to announce that Amazon Bedrock is now available in the Asia Pacific (New Zealand) Region (ap-southeast-6). Customers in New Zealand can now access Anthropic Claude models (Claude Opus 4.5, Opus 4.6, Sonnet 4.5, Sonnet 4.6, and Haiku 4.5) and Amazon (Nova 2 Lite) models directly in the Auckland Region with cross region inference. In this post, we explore how cross-Region inference works from the New Zealand Region, the models available through geographic and global routing, and how to get started with your first API call. We
Introducing Amazon Polly Bidirectional Streaming: Real-time speech synthesis for conversational AI
Today, we’re excited to announce the new Bidirectional Streaming API for Amazon Polly, enabling streamlined real-time text-to-speech (TTS) synthesis where you can start sending text and receiving audio simultaneously. This new API is built for conversational AI applications that generate text or audio incrementally, like responses from large language models (LLMs), where users must begin synthesizing audio before the full text is available.
Gemini 3.1 Flash Live: Making audio AI more natural and reliable
Our latest voice model has improved precision and lower latency to make voice interactions more fluid, natural and precise.
AI system learns to prevent warehouse robot traffic jams, boosting throughput 25%
Inside a giant autonomous warehouse, hundreds of robots dart down aisles as they collect and distribute items to fulfill a steady stream of customer orders. In this busy environment, even small traffic jams or minor collisions can snowball into massive slowdowns. To avoid such an avalanche of inefficiencies, researchers from MIT and the tech firm Symbotic developed a new method that automatically keeps a fleet of robots moving smoothly.
Google unveils TurboQuant, a new AI memory compression algorithm — and yes, the internet is calling it ‘Pied Piper’
Google’s TurboQuant has the internet joking about Pied Piper from HBO's "Silicon Valley." The compression algorithm promises to shrink AI’s “working memory” by up to 6x, but it’s still just a lab experiment for now.
Get our weekly newsletter on pricing changes, new releases, and tools.
Deploy OpenClaw in Under 1 Minute— We handle hosting, scaling, and maintenance
Built by @aellman
Tools
Directories
Models & Pricing
Endpoints
Rankings
- All Rankings
- All Benchmarks
- Best LLM for Coding
- Best LLM for Math
- Best LLM for Writing
- Best LLM for RAG
- Best Local LLM
- Best LLM for OpenClaw
- Best LLM for Cursor
- Best LLM for Windsurf
- Best LLM for Cline
- Best LLM for Aider
- Best LLM for GitHub Copilot
- Best LLM for Bolt
- Best LLM for Continue.dev
- MMLU-Pro
- GPQA
- LiveCodeBench
- Aider
- AIME
- MATH (Hard)
- Big-Bench Hard
2026 68 Ventures, LLC. All rights reserved.