Sagence AI today emerged from stealth unveiling a groundbreaking advanced analog in-memory compute architecture that directly addresses the untenable power/performance/price and environmental sustainability conundrum facing AI inferencing. Driven by its industry-first architectural innovations using analog technology, Sagence AI makes possible multiple orders of magnitude improvement in energy efficiency and cost reductions, while sustaining performance equivalent to high performance GPU/CPU based systems.
Compared to industry’s leading volume GPU processing the Llama2-70B large language model with performance normalized to 666K tokens/sec, Sagence technology performs with 10X lower power, 20X lower price, and 20X smaller rack space. Using a modular chiplet architecture for maximum integration, Sagence technology makes possible a highly efficient inference machine that scales from data center generative AI to edge computer visions applications across multiple industries. This previously unimaginable balance of high performance and low power at affordable cost addresses the growing ROI problem for generative AI applications at scale, as AI compute in the data center shifts from training models to deployment of models to inference tasks.
“A fundamental advancement in AI inference hardware is vital to the future of AI. Use of large language models (LLMs) and Generative AI drives demand for rapid and massive change at the nucleus of computing, requiring an unprecedented combination of highest performance at lowest power and economics that match costs to the value created,” said Vishal Sarin, CEO & Founder, Sagence AI. “The legacy computing devices today that are capable of extreme high performance AI inference cost too much to be economically viable and consume too much energy to be environmentally sustainable. Our mission is to break those performance and economic limitations in an environmentally responsible way.”
“The demands of the new generation of AI models have resulted in accelerators with massive on-package memory and consequently extremely high-power consumption. Between 2018 and today, the most powerful GPUs have gone from 300W to 1200W, while top-tier server CPUs have caught up to the power consumption levels of NVIDIA’s A100 GPU from 2020,” said Alexander Harrowell, Principal Analyst, Advanced Computing, Omdia. “This has knock-on effects for data center cooling, electrical distribution, AI applications’ unit economics, and much else. One way out of the bind is to rediscover analog computing, which offers much lower power consumption, very low latency, and permits working with mature process nodes.”
On the Frontier of Analog In-memory Compute
Sagence AI leads the industry on the frontier of in-memory compute innovation. Sagence technology is the first to do deep subthreshold compute inside multi-level memory cells, an unprecedented combination that opens doors to the orders of magnitude improvements necessary to deliver inference at scale. As digital technology reaches limits in ability to scale power and cost, Sagence innovated a new analog path forward leveraging the inherent benefits of analog in energy efficiency and costs to make possible mass adoption of AI that is both economically viable and environmentally sustainable.
In-memory Computing Aligned to AI Inference
In-memory computing aligns closely with the essential elements of efficiency in AI inference applications. Merging storage and compute inside memory cells eliminates single-purpose memory storage and complex scheduled multiply-accumulate circuits that run the vector-matrix multiplication integral to AI computing. The resulting chips and systems are much simpler, lower cost, lower power and with vastly more compute capability.
Sagence views the AI inference challenge not as a general-purpose computing problem, but a mathematically intensive data processing problem. Managing the massive amount of arithmetic processing needed to “run” a neural network on CPU/GPU digital machines requires extremely complicated hardware reuse and hardware scheduling. The natural hardware solution is not a general-purpose computing machine, rather an architecture that more closely mirrors how biological neural networks operate.