AI Pulse · AI Trends Pulse

Nvidia says its Groq 3 LPX inference chip is now in full production. It comes from Nvidia’s $20 billion Groq acquisition and is designed for the part of AI work that happens after a model has been trained, when it is generating responses token by token. Nvidia reports 500MB of on-chip memory, 256 chips per rack on its Vera Rubin platform, and a benchmark result of 3,400 output tokens per second on a Gemma 4 31B test with a 100,000-token context window. Nebius is the first cloud provider deploying it, according to Nvidia’s announcement.

For a business owner, the key point is not the chip specification. It’s that AI providers are building separate infrastructure for AI agents that need to read, reason and respond quickly over long workflows. GPUs remain essential, particularly for training and broad AI workloads. But Nvidia is making a clear structural bet that high-volume agent responses, often called decoding, need dedicated hardware too.

That matters because faster and more predictable inference can make AI practical in customer service, internal operations and research workflows where waiting several seconds for each step breaks the experience. It may also help providers manage their own compute bottlenecks as demand rises. This is the kind of thing we build into an AI command centre, tracking the infrastructure shifts that will affect the cost, speed and reliability of AI inside your company.