Scaling AI Inference with Nutanix Enterprise AI: Sharding, Instances, KV Cache and Agent Gateway

General

When discussing AI inference infrastructure, the word “scaling” can mean several very different things. We might need more GPUs because a model is too large for a single GPU, more copies of a model because concurrency is increasing, or better routing between those copies because repeated context is consuming unnecessary compute. Eventually, we may also […]

Read More
Simon Todd in No Comments

Neocloud Enters Its Consolidation Era: What’s Driving It and What It Means for AI Buyers

BlockchainFinanceFinancial
More
FutureGeneralHealthcareHuman ResourcesInformationalInfrastructureRecruitmentSecuritySocietyTechnology

Over the last two years, a new class of GPU-first Neocloud providers such as CoreWeave, Lambda, Voltage Park, Crusoe, and others has moved from niche to necessary. They stand out for their cutting-edge accelerators, near bare-metal performance, faster time to capacity, and flexible terms for AI workloads including shorter commitments, lower egress fees, and container-native […]

Read More
Simon Todd in No Comments

Why ARM May Struggle to Dominate AI Servers and How the Intel + NVIDIA NVLink Deal Reinforces x86

BlockchainFinanceFinancial
More
FutureGeneralHealthcareHuman ResourcesInformationalInfrastructureRecruitmentSecuritySocietyTechnology

The big question in AI infrastructure Every year, new GPUs and super chips dominate headlines. But behind every GPU cluster sits a crucial decision: which CPU architecture should host it, ARM or x86? ARM has surged in visibility thanks to NVIDIA Grace, AWS Graviton, and AmpereOne. But despite rapid growth, ARM still isn’t the dominant […]

Read More
Simon Todd in No Comments

Intel Gaudi 3 PCIe: A Game Changer for AI Workloads Beyond OAM

FutureGeneralInformational
More
InfrastructureTechnology

Intel’s Gaudi 3 AI accelerator has been a significant advancement in AI hardware, previously available primarily in the OAM (Open Accelerator Module) form factor. The introduction of the PCIe version marks a pivotal shift, enabling broader adoption and integration into existing enterprise infrastructures. What Is Intel Gaudi 3 PCIe? The Intel Gaudi 3 PCIe (HL-338) […]

Read More
Simon Todd in No Comments