Google Cloud Adds Day-0 Support for Reflection Beam Model

Google Cloud has announced Day-0 support and ready-to-run deployment recipes for Reflection’s newly released Beam model across its AI Hypercomputer infrastructure. Beam is an open-weight sparse Mixture-of-Experts (MoE) model with 501 billion total parameters, 23 billion active parameters per token, and an approximately 1-million-token context window designed specifically for autonomous software engineering. By co-designing the serving stack with the open-source llm-d framework on vLLM and NVIDIA Dynamo on SGLang, Google Cloud reports inference speedups reaching up to 5x higher throughput and 8x lower latency on A4X Max instances compared to baseline Kubernetes deployments. The release directly targets multi-turn bottlenecks, including KV-cache thrashing and stateless prompt recomputation.

Reflection Beam Architecture and Open-Weight Release

Reflection released Beam under an Apache 2.0 license, publishing weights in BF16, FP8, and Blackwell NVFP4 formats on its Hugging Face repository. Designed for frontier software engineering and long-horizon agentic workflows, the model routes each token through 23 billion active parameters out of 501 billion total parameters. The model integrates natively with existing development harnesses, including Reflection’s Mirror CLI, Claude Code, Hermes, and OpenCode, allowing agents to ingest repository contexts, process diffs, and execute command-line tooling.

According to Reflection’s engineering team, Beam was trained using reinforcement learning penalties structured to discourage verbose reasoning chains. The company claims this approach yields 3x to 4x higher reasoning compute efficiency compared to peer open-weight models by arriving at correct code implementations in fewer generated tokens. However, while model-level token efficiency reduces per-turn generation requirements, multi-turn agent sessions introduce secondary bottlenecks at the cluster serving layer that parameter sparsity alone cannot resolve.

Solving Multi-Turn Agentic Bottlenecks in Inference

In standard interactive coding workflows, agents call the model dozens of times in a single session. Each turn appends compiler outputs, terminal logs, and code diffs to the session history, quickly expanding the context to hundreds of thousands of tokens. In a conventional deployment, two primary infrastructure bottlenecks emerge:

• Prefill compute bottlenecks: Standard Kubernetes load balancers route incoming requests using stateless round-robin algorithms. Subsequent turns from the same coding session often route to different replicas, forcing the recipient GPU to recompute the entire historical prompt from scratch, wasting compute cycles and extending Time-to-First-Token (TTFT) latency.
• KV-cache memory exhaustion: Storing multi-turn attention states in GPU High-Bandwidth Memory (HBM) for multiple concurrent users quickly exhausts available capacity. Once HBM fills, serving engines evict older cache blocks, degrading cache hit rates and stalling token generation.

To mitigate these issues, Google Cloud co-designed an integrated architecture on AI Hypercomputer that treats inference pods as state-aware distributed nodes rather than stateless endpoints:

• Prefix- and load-aware routing: The GKE ingress layer pairs with llm-d’s Sticky Endpoint Picker (EPP) or the NVIDIA Dynamo router to inspect prefix hashes and queue depth. Follow-up turns are steered to worker replicas that already hold warm KV prefixes, while turn-priority flow control prevents queue saturation.
• Disaggregated Prefill and Decode (P/D): Separating prefill workers from decode workers isolates generation loops from heavy prompt injection bursts, preventing incoming context spikes from interrupting active token streaming.
• Tiered KV-cache hierarchy: Active KV blocks remain in GPU HBM (Tier 1). When memory limits are reached, evicted blocks spill into host CPU DRAM and local NVMe solid-state drives (Tier 2) via vLLM’s OffloadingConnector or SGLang’s HiCache, allowing historical conversation states to be retrieved in milliseconds rather than recomputed.
• Accelerated generation: Deployments leverage Blackwell native NVFP4 precision, Expert Parallelism (EP) across Beam’s 23B active experts, Data-Parallel Attention, and DFlash2 speculative decoding to maximize token throughput.

Evaluating Benchmarks and Hardware Requirements

Google Cloud published internal customer benchmark results detailing performance gains across two distinct infrastructure setups. The reported improvements compare optimized serving against a stateless baseline on the identical hardware configuration rather than comparing the two tiers directly:

Platform teams evaluating these figures should note that performance gains scale non-linearly with cluster size. Larger replica pools give the router more warm KV targets to hit and provide greater headroom to decouple prefill and decode pools. Furthermore, while 23 billion active parameters per token lower execution compute relative to dense 500B models, hosting the full 501B parameter footprint and managing a 1-million-token KV context requires substantial hardware commitment. Infrastructure teams must account for node-to-node interconnect bandwidth and local storage performance when configuring host offloading pipelines.

What happens next

Google Cloud has released its deployment templates, Kubernetes manifests, and benchmarking configurations in the open-source AI-Hypercomputer/gpu-recipes repository. The manifests provide turnkey setups for both the llm-d stack on G4 VMs and the NVIDIA Dynamo stack on A4X Max infrastructure. Organizations running within Google Cloud can also access Beam directly through Model Garden on the Gemini platform.

Google Cloud and Reflection plan to demonstrate these multi-turn routing and disaggregation architectures at upcoming industry events. Technical sessions covering KV-cache-aware routing and P/D disaggregation are scheduled for the PyTorch Conference North America in San Jose, followed by deployment demonstrations at NVIDIA GTC Berlin. Engineering teams planning to deploy Beam should evaluate their existing agent trajectory datasets, measure baseline cache hit rates, and verify DRAM-to-GPU bus transfer speeds before provisioning high-concurrency production clusters.