Scaling AI Inference with Nutanix Enterprise AI: Sharding, Instances, KV Cache and Agent Gateway

General

When discussing AI inference infrastructure, the word “scaling” can mean several very different things. We might need more GPUs because a model is too large for a single GPU, more copies of a model because concurrency is increasing, or better routing between those copies because repeated context is consuming unnecessary compute. Eventually, we may also […]

Read More
Simon Todd in No Comments

Why Liquid Cooling Won’t Take Off Until the Industry Fixes One Big Problem

FutureGeneralInformational
More
InfrastructureTechnology

A Silent Aisle, A Loud Truth The data centre aisle was unnervingly quiet. High-density AI servers hummed with an intensity that pushed the limits of what traditional airflow could sustain. In the middle of this familiar landscape sat a single liquid-cooled node: compact, efficient and completely alone. It looked like a glimpse of the future […]

Read More
Simon Todd in No Comments

LLMs vs SLMs: Why Smaller Models Could Power the Future of Agentic AI

FutureGeneralInformational
More
InfrastructureTechnology

The AI landscape has been dominated by Large Language Models (LLMs)—massive neural networks trained on trillions of tokens, spanning hundreds of billions of parameters. These models, such as GPT-4 or Claude, have shown remarkable general-purpose intelligence, but they come with steep costs: enormous compute requirements, GPU dependency, and operational overheads that make them inaccessible for […]

Read More
Simon Todd in No Comments

The Most Popular LLMs, VLLMs, and SLMs in Enterprise AI Today

BlockchainFinanceFinancial
More
FutureGeneralHealthcareHuman ResourcesInformationalInfrastructureRecruitmentSecuritySocietyTechnology

As enterprises rapidly adopt AI to improve efficiency, customer experience, and innovation, the choice of model architecture has become a critical factor. Whether it’s deploying a massive Large Language Model (LLM), an efficient Very Large Language Model (VLLM), or a compute-friendly Small Language Model (SLM), organisations are increasingly strategic about balancing performance, cost, and accuracy. […]

Read More
Simon Todd in No Comments