IN Brief:
- Leo X-Series attaches dedicated memory directly to AI scale-up fabrics for KV-cache and agent-context workloads.
- Leo 2 E- and P-Series support PCIe 6, CXL 3.2, DDR4, and DDR5 while doubling bandwidth and capacity over the previous generation.
- Hardware test, memory-health, RAS, and repair functions support reuse of existing DDR4 DIMMs in newer server fleets.
Astera Labs has expanded its Leo smart memory-controller family with three architectures covering accelerator-attached memory, conventional CPU expansion, and pooled rack-scale capacity, as AI inference places increasing pressure on both memory bandwidth and the amount of DRAM available to individual computing systems.
The new Leo X-Series is designed to attach a dedicated memory tier directly to AI scale-up fabrics, while the second-generation Leo E-Series and P-Series address CPU-connected CXL expansion and pooled or shared capacity. Across the three products, Astera is treating DRAM as a more flexible infrastructure resource rather than leaving available memory determined entirely by the channels and DIMM slots attached to each processor.
Leo X is the most substantial departure from conventional server memory expansion. Used alongside Astera’s Scorpio X-Series fabric switches, it allows memory to connect to the same scale-up environment as AI accelerators, creating a dedicated tier for workloads such as key-value cache and retained agent context.
Those requirements grow as inference systems support longer context windows and more persistent conversations. KV caches store information generated while a model processes previous tokens, reducing the amount of work required when producing subsequent output. The approach improves inference efficiency but can consume substantial memory as context length, user count, and model scale increase.
Attaching additional memory closer to the accelerator fabric provides an alternative to reaching capacity through a host CPU or using NVMe storage as a slower tier. Astera reports up to a 62% improvement in time to first token and up to 22% more tokens per second under its published test configuration. Those figures are vendor measurements and will vary with accelerator, model, software, topology, memory placement, and workload.
The Leo 2 E-Series addresses a more conventional CXL use case. It provides CPU-attached expansion through a PCIe 6 x16 host connection and supports CXL 3.2, allowing additional DDR4 or DDR5 capacity to sit beyond the processor’s native memory channels. Four memory controllers double both bandwidth and supported capacity compared with Astera’s previous generation.
Leo 2 P-Series uses dual PCIe 6 x8 interfaces for pooled and shared memory. Rather than assigning all installed DRAM permanently to one server, a rack-level architecture can make capacity available across multiple hosts and alter allocation as workload demand changes. The objective is to reduce stranded capacity when one machine has unused memory while another is constrained.
CXL memory is already moving beyond small laboratory expansion systems. Recent rack-scale CXL deployments have demonstrated memory pools exceeding 100TB, shifting controllers and switches from peripheral components into infrastructure that can influence application placement, resiliency, and rack economics.
Astera is also using the second-generation Leo family to address memory reuse. Both DDR4 and DDR5 are supported, allowing operators to redeploy DIMMs removed from older servers through CXL-connected expansion rather than discarding otherwise usable capacity simply because a new host platform has moved to a different local-memory generation.
The economics are attractive, but reused memory has to remain dependable. Astera includes hardware test engines, memory-health management, error reporting, event logging, scrubbing, automated repair functions, thermal management, and resilient firmware-update mechanisms. At hyperscale, the ability to identify deteriorating DIMMs before repeated workload disruption becomes part of the viability of memory reuse.
PCIe 6 also helps address one of the basic limitations of expanded memory: capacity is useful only if the link to it can deliver sufficient throughput for the workload. CXL-attached DRAM will still have different latency and bandwidth characteristics from local processor memory or accelerator HBM, so applications and operating software have to understand which tier they are using.
That creates a more complicated hierarchy. A rack can contain local DDR5, accelerator HBM, reused DDR4, CXL expansion, pooled capacity, fabric-attached memory, and storage tiers, each with a different combination of latency, bandwidth, capacity, power, reliability, and cost. The infrastructure problem becomes one of placing data on the appropriate tier rather than merely installing more DRAM.
Astera’s COSMOS software provides telemetry and management around its connectivity devices, giving operators data that can support those placement and health decisions. Hardware alone cannot determine whether a KV cache belongs in accelerator-adjacent memory or whether a general-purpose workload can tolerate a higher-latency CXL pool; that depends on the application and scheduler.
The three Leo variants therefore address different parts of the same memory constraint. E-Series adds capacity to individual CPU systems, P-Series turns installed memory into a resource shareable between hosts, and X-Series moves another tier towards accelerators running memory-intensive inference.
The products are sampling with hyperscaler customers. Their practical value will depend less on maximum installed capacity than on how predictably software can exploit the resulting hierarchy without losing the theoretical benefit to additional latency, fabric contention, or management complexity. If that can be controlled, memory controllers will increasingly determine how effectively AI infrastructure uses the DRAM already installed around it rather than simply how much more can be purchased.