Memory Design for Physical AI

Memory Design for Physical AI
Tuesday, November 17, 2026
  • •SK hynix’s fifth AI Ecosystem installment features Professor Namseung Kim on memory design for physical AI.
  • •Robots, vehicles, and wearables face power, heat, and latency constraints that make local data processing important.
  • •The column recommends combining HBM, PIM, CXL, HBF, and other memory tiers to match each workload.

In the fifth installment of SK hynix’s AI Ecosystem series, Namseung Kim, an endowed professor at the University of Illinois Urbana-Champaign, says memory and system design must change as AI expands beyond data centers to robots, vehicles, industrial drones, smart glasses, and wearable devices. The column is by an outside expert and may not reflect SK hynix’s official position.

Physical AI, which interacts with the real world, is constrained by power use, heat, response time, and device size. Smart glasses must quickly interpret information in the wearer’s field of view, while robots need to perceive their surroundings and adjust movement in near real time. Vehicles and industrial drones also need to make decisions when connections drop or responses are delayed, so they cannot rely on data centers for all processing. Processing and decisions close to where data is generated are a better fit.

Hyperscale data centers can use large GPU clusters, ample board space, substantial power, and large cooling systems. Edge and mobile devices, by contrast, are constrained by battery capacity, heat, package size, and wiring, and cannot readily use large GPU clusters or expandable standard memory modules. Their available energy and heat-dissipation space are limited. Long network round trips or unstable connections can affect decisions and system reliability, making memory capacity, data paths, and latency key design considerations.

These constraints also shape algorithm choices such as model compression and quantization, a technique for representing data with fewer bits. Quantization represents model parameters with fewer bits to reduce storage and data-transfer demands. TurboQuant focuses on compressing the KV cache during inference to use less memory while maintaining response quality. Engram stores frequently used patterns and information in a conditional memory lookup table, then retrieves relevant information based on the input, replacing some repeated computation through neural network layers with memory lookups. This can reduce computation, but the larger volume of stored information and data to read makes memory capacity and data supply limits on speed and efficiency. The column says the two techniques balance computation and memory in different ways, and software optimization alone has limits in raising real-world performance.

Memory-centered design includes custom HBM, 3D-stacked DRAM, custom DRAM, PIM, CXL, HBF, and SSDs. Custom HBM and custom DRAM adjust capacity, bandwidth, power, and package characteristics to suit workloads and system requirements. HBM stacks multiple DRAM dies vertically and connects them through a wide interface to provide high bandwidth in the same footprint. 3D-stacked DRAM places DRAM above a logic chip to shorten data travel and widen pathways. PIM handles some tasks with frequent data access inside or near memory, reducing energy use and latency from data movement.

When device memory alone cannot provide enough capacity, CXL connects processors, accelerators, and memory at high speed to expand and share memory resources. Large data, such as robot sensor logs and autonomous-driving map data, can be stored on 3D NAND-based SSDs and loaded into DRAM or HBM when needed. Large model weights frequently used during inference require higher bandwidth. HBF is presented as a memory tier that uses 3D NAND dies and a wide interface to combine NAND’s large capacity with high transfer speeds. It is being discussed as a tier between HBM and SSD, with a logic process on the base die aimed at delivering much greater capacity than HBM, similar bandwidth, and cost efficiency.

The column says HBM, 3D-stacked DRAM, and PIM provide memory close to computation; CXL enables flexible connections; and SSDs and HBF provide large-capacity storage and data supply. AI systems need high bandwidth and capacity for large-scale training and inference, while small on-device products need energy efficiency and low latency. Memory tiers therefore need to be combined to fit each workload and product form. SK hynix says it aims to offer full-stack AI memory solutions spanning HBM, DRAM, 3D-stacked DRAM, custom memory, SSD, HBF, CXL, and PIM, along with memory libraries tailored to customer architectures and workloads. It also describes a role as an architecture partner that co-designs memory combinations, beyond supplying products to customers.

Subscribe to the AIB Newsletter

The fastest, most reliable AI news, straight to your inbox