Startup FuriosaAI debuts RNGD chip for LLM and multimodal AI inference

Source: SiliconANGLE

FuriosaAI's new chip is called RNGD, pronounced “Renegade,” and it was unveiled at the Hot Chips 2024 conference in Stanford University today. It’s sampling to early access customers now, with broader availability slated for next year.

According to Furiosa, the RNGD chip is an extremely efficient data center accelerator that’s designed to support high-performance LLMs and multimodal model inference. The company is positioning it as an alternative to Nvidia Corp.’s graphics processing units.

RNGD is based on a Tensor Contraction Processor or TCP architecture, which the company says provides the perfect balance between efficiency, programmability and performance. It boasts some formidable specifications, with a Thermal Design Power of 150-watts, compared to more than 1,000 watts for some of the leading GPUs on the market today. Furiosa also claims extremely high performance, with the chip packing 48 gigabytes of high-bandwidth memory. That makes it possible to run open-source LLMs such as Meta Platforms Inc.’s Llama 3.1 8B efficiently on a single card.

Furiosa co-founder and Chief Executive June Paik revealed RNGD is the result of years of innovation by the startup. “RNGD is a sustainable and accessible AI computing solution that meets the industry’s real-world needs for inference,” he said. “With our hardware now running LLMs at full speed, we’re entering an exciting phase of continuous advancement.”

Read the full article here.