精选

Rapidly scaling online storage to serve over 1 billion ChatGPT users

Learn how OpenAI evolved Habitat from a Python library into a globally distributed storage platform serving 1 billion ChatGPT users and 22M requests per second.

proto-v0.1.0

vllm-proto 0.1.0

v0.29.0

v0.29.0 Highlights This release features 594 commits from 277 contributors (91 new)! Model Runner V2 is now the default for all models ( #53183 ), completing the rollout that began with pooling models ( #48290 ). MRV2 also gained CUDA graph memory profiling for KV cache auto-sizing ( #53306 ), batch-sharded sampling that cuts per-step logits memory by 1/TP ( #50465 ), prompt embeds ( #42963 ), extract_hidden_states speculation ( #49811 ), padded FULL cudagraph dispatch for uniform decode under spec decode ( #53407 ), and DP-sync skipping before EAGLE/MTP draft prefill ( #53694 ). MRV1 remains in use for a few ROCm models and features MRV2 doe

Kubernetes v1.37: Scheduler Preemption for In-Place Pod Resize (Alpha)

In Kubernetes, resource allocation has historically been a static decision made during a Pod's initial scheduling and placement. With the graduation of the core in-Place Pod resize feature to General Availability in v1.35, application developers and cluster operators gained the powerful ability to dynamically adjust CPU and memory allocations of running containers without incurring disruptive restarts or application downtime. However, in-place resizing introduced a unique resource scheduling gap: if a running Pod requested a resource scale-up that exceeded the host node's allocatable headroom, the Kubelet was forced to mark the request as Def

High-Throughput Structure Prediction with BioNeMo Inference Runtime

Biomolecular structure prediction is now often run at proteome scale, where the goal is to move an entire worklist through the pipeline efficiently. NVIDIA...

Where AI Inference Needs to Run

Every enterprise AI strategy eventually runs into the same question: not just which models to use, but where to run them. That question is…

Energy in hands

The post Energy in hands appeared first on Schneider Electric Blog .

From Wafer-Out to First Token: Codifying Supply Chain Expertise with Nemotron and Palantir Foundry

NVIDIA has one of the largest and most complex supply chains in the world, and its performance is measured from wafer-out to first token. The interval is in two...

Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL

Validating multi-Region DR for Terraform Enterprise with AWS FIS

Learn how AWS, HashiCorp, and Athenahealth designed and chaos-tested a multi-Region disaster recovery strategy for Terraform Enterprise on AWS. This post walks through three-phase AWS Fault Injection Service experiments across Amazon EC2, Aurora, and Amazon S3, the 12-14 minute recovery times achieved, and the state file dependency pitfall to avoid.

CUDA Toolkit 13.4 Adds Windows on Arm Support and Greater Control over Shared GPUs

Every NVIDIA CUDA Toolkit release adds functionality and performance improvements that help developers get more from NVIDIA GPUs and the broader NVIDIA software...

Tiered KV Cache Offloading in vLLM

A host-centric framework for scaling KV cache across host memory, filesystems, object stores, and remote peers — reducing recomputation and increasing serving c

Our blueprint for responsible clean energy growth in Finland

Animation showing Google’s approach to responsible energy growth in Finland

Introducing CUDA Rust: Two Tracks for Writing GPU Kernels

In September 2026, NVIDIA announced it is leaning into native GPU programming in Rust. CUDA C++ and CUDA Python are mature, enterprise-grade toolchains, and...

You Need an Easier Way to Find and Connect to Ecosystem Partners

The rise of distributed AI workloads is changing the way networks function. Enterprises need connections to more endpoints in more places, and they need…

v0.29.0rc6

[Bugfix][Core] Apply dense prefix cache default to hybrid models ( #55 …

GLM 5.3 Optimizations, Part 1: Hybrid HiSparse Offloading in vLLM

vLLM integrates HiSparse as a pressure-driven memory tier that composes with the Hybrid Memory Allocator and KV offloading, letting GLM 5.3 requests keep decodi

vLLM x AgentX: Optimizing for Real-World Agentic Serving

How vLLM optimizes KV cache management, parallelism, scheduling, and P/D disaggregation for agentic workloads, validated on SemiAnalysis AgentX with up to 130K

Serving LLMs on Tenstorrent Hardware: Inside the vLLM TT Plugin

Tenstorrent accelerators join vLLM as an out-of-tree platform plugin, driven by mesh-architecture choices: phase-based scheduling, single-process data paralleli

Equinix Horizon 2026: The Recap

Equinix has been in business for 28 years and just held our first ever customer event this week. Equinix Horizon changed that, bringing together…

The Modern CUDA Toolbox in Practice: A Step-by-Step Optimization Walkthrough

NVIDIA CUDA remains the foundation of GPU-accelerated computing, powering everything from scientific simulations to large-scale AI training. But writing...

Our new partnership brings long-duration energy storage to West Virginia.

Google is collaborating with MN8 Energy and Eos Energy Enterprises on a new clean energy project on the PJM grid.

The Coordination Economy: How Enterprises Really Build AI Value

Most companies are pursuing AI. Many are putting significant funding behind it. But EY’s Work Reimagined Survey found that although 88% of respondents were…

Building an Adaptive Agentic Cybersecurity System with NVIDIA Nemotron

AI is changing the pace of cybersecurity. Agentic systems can coordinate work and pursue complex objectives over long horizons. Security teams are beginning to...

Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI

MiniMax H3 on vLLM-Omni: From System-Wide Optimization to Real-Time Serving with FastVideo’s FastH3

How vLLM-Omni optimizes and scales the complete MiniMax H3 stack, then integrates FastVideo’s four-step FastH3 for generation faster than playback.

Kubernetes v1.37: Metrics API graduates to stable

Kubernetes v1.37 promotes the metrics.k8s.io API to stable ( v1 ). This API provides CPU and memory usage for nodes and Pods, and is the API behind commands such as kubectl top and resource-metrics-based autoscaling. For cluster operators and application developers, this graduation means that the API now has the stability guarantees associated with a Kubernetes stable API. The v1 API has the same resource types and fields as v1beta1 ; this is an API-version graduation, not a change to the metrics that are collected or returned. A long-lived API reaches stable The resource Metrics API was introduced as alpha in Kubernetes v1.6 and became beta

Previewing the Model Hardware Standard

Gallup scales real-time coaching for thousands with Amazon Bedrock

Gallup transformed 90 years of workplace science into Gallup AI, a generative AI assistant powered by Amazon Bedrock that delivers real-time, personalized coaching to leaders directly within the Gallup Access application.

Part 1: Managing Large-Scale LLM Training with AWS ParallelCluster

This post references AWS ParallelCluster. Check out AWS Parallel Computing Service (AWS PCS), our new managed Slurm service for running HPC and AI workloads on AWS. Introduction The Korean Government announced a national AI initiative to provide high-performance GPU infrastructure for Korea’s national AI research teams. AWS was selected as a supplier of GPU resources […]