Samsung Touts 10-Fold AI Speed Gain With Stacked zHBM Memory
Samsung Electronics and SK hynix presented next-generation memory technologies at the AI Infra Summit in Santa Clara. Samsung showcased zHBM, a stacked memory directly on AI accelerators. SK hynix introduced processing-in-memory (PIM), which computes directly inside the memory chip. These innovations aim to accelerate AI performance significantly. Samsung projects a tenfold increase in conversational AI response speed, from 100 to 1,000 tokens per second.
Beyond faster memory, a fundamental shift in AI architecture is underway. Samsung's zHBM and SK hynix's PIM both tackle the GPU-HBM bottleneck by moving computation closer to or inside memory. Samsung's target of 1,000 tokens per second for agentic AI is ambitious. It suggests a future where memory design dictates AI system capabilities, not just GPU power.
This competition between Samsung and SK hynix directly benefits Korea's semiconductor dominance. Both companies are pushing memory innovation beyond incremental HBM gains. SK hynix claims PIM offers 300 times the capacity of SRAM in the same area. This could give Korean firms a critical edge in supplying advanced memory for the next generation of AI accelerators globally.
The thing to watch is adoption by major AI chip designers. NVIDIA, AMD, and Intel must integrate these new memory solutions into their next-gen platforms. If zHBM or PIM become standard, it cements Korea's position in the AI supply chain. The test for both companies is demonstrating real-world performance gains beyond their summit presentations.
Related reading
6 stories
Huawei brings near-packaged optics to AI with Atlas 960E superpod
Huawei's Atlas 960E superpod also tackles the GPU-HBM bottleneck, showing this is a global architectural shift.
Will AI adoption shrink office space? Asia-Pacific firms are on the fence in survey
Chip foundries better insulated in an AI slowdown than Asia-Pacific tech peers, S&P says
OCBC Extends Bank of Ningbo Partnership as 20 Percent Shareholder
China to see major shift to Huawei for AI model training in 2027: rotating chair
