Tim Messegee, Rambus
DesignCon is an industry event I always look forward to attending. This year marked its 30-year anniversary, which added some celebratory energy.
It also took place amid a flurry of AI news, including the release of DeepSeek’s R1 model, sending shockwaves through the industry, and the stock market. This, paired with conversations led by fellow industry colleagues from NVIDIA, Amazon, Cisco, Broadcom, and more, set the stage, sparking a slew of timely, much-needed conversations about how we can innovate to keep pace.
It certainly left me and the entire Rambus team who attended with some high-level takeaways as to what may be next for chip designers as we work to bring forward new solutions that can enable AI’s next chapter.
Demand for AI-specific chips continues
With the continued, explosive demand for AI-specific chips, the semiconductor industry is being faced with increasingly high expectations and challenges. It’s also speeding up design cycles across the board.
John Linford from NVIDIA spoke about this in his DesignCon keynote, along with the progress their team has made to integrate AI as a way to improve internal processes. He explained that AI’s continued acceleration is driving demand across the hardware landscape and forcing us to operate within increasingly expedited timelines.
He also pointed out that we’re being faced with a new industrial revolution, because we’re no longer building solutions for the same problems. He shared more about how NVIDIA is leveraging AI to create digital twins and simulations, expediting their own design processes by 15,000x. This has helped improve their overall creative freedom, energy efficiency, and has lowered the computational cost of chip designs.
We know that AI is being used to extract meaning and value, thus it requires immense amounts of memory capacity as well as memory and interconnect bandwidth. As applications grow, AI-focused customers are often requesting solutions that don’t even yet exist. It’s a trend that is not expected to change in the foreseeable future. And while this demand puts pressure on the entire semiconductor community, it’s key that we keep innovating at a rapid pace.
Opportunities to innovate in memory and power
We know that server costs continue to scale in service of AI and memory is a large contributor to this. It’s another reason why DeepSeek caught the industry’s attention as they were able to make innovative use of system resources including memory.
Thankfully, because DeepSeek’s model is open source, it’s allowed us to learn from these methods and understand the technology at a deeper level. Something that DeepSeek did well with a process called Multi-Head Latent Attention (MLA), meaning it can process large amounts of data during inference, slashing memory usage in half. As an industry, we know there’s a lot of opportunity to innovate within memory and power, which was a recurring topic of conversation at DesignCon.
During the show, NXP Semiconductors’ Dan Beeker spoke about the opportunities with memory stacking to improve the board design and streamline how data moves, with noteworthy insights on signal integrity. My colleague, Dr. Steven Woo, is also a fan of pursuing memory stacking, but recognizes there could be challenges with power depending on the approach.
Beyond memory stacking, another potential solution and something that my teammate Lou Ternullo had the opportunity to discuss with ZeroPoint Technologies, is the use of low latency data compression to make better use of available memory resources.
When broken down, memory accounts for up to 50% of general purpose server costs, and stranded memory can be up to 40% of installed memory. If we can eliminate stranded memory, it could save ~20%, reducing the total cost of ownership. Computer Express Link (CXL™) can step in and support efficient sharing of memory resources, which also improves the overall power budget.
Data drives everything
As more users and companies engage with AI at all levels, and capabilities grow, data needs to move faster. We spoke at length about what’s needed to ensure GPUs are prepared for the next set of data, along with critically identifying bottlenecks and where they affect compute and data movement.
While DeepSeek’s R1 model grabbed the industry’s attention, democratizing data and providing open source information has once again shifted the conversation to the next stage of AI, and one with efficiency and collaboration in mind. DeepSeek’s open-source model is a net positive for the chip design industry. The expected costs and demand for memory and power will still rise, especially with current LLMs charging forward with exponential amounts of information to process.
I left Santa Clara looking forward to the possibilities, knowing it’s a critical time to bring forward new solutions within memory, processing and power. It’s a great time to be in the semiconductor space — great promise lies ahead.
