Deploying AI Models at the Edge

Quadric Chimera GPNPU IP core now supports Llama 2 model

Categories Edge Computing News  |  Hardware
Quadric Chimera GPNPU IP core now supports Llama 2 model

Quadric, a provider of edge AI chips, recently revealed that its Chimera general-purpose neural processing unit IP core will now be compatible with the Llama 2 model.

Achieving compatibility through a software update, Quadric says the Chimera GPNPU can integrate the Llama 2 model without requiring hardware modifications.

Following the advent of generative AI and large language models suitable for edge devices such as smartphones and laptops, various chip and IP providers started investing in enabling on-device implementations of these models.

Qualcomm, for instance, has outlined its intention to incorporate Llama 2 into its chips by 2024, a process anticipated to span more than six months for the adaptation. In contrast, Quadric says it stands out among the chip provider landscape due to its current capability to operate Llama 2.

According to Steve Roddy, the chief marketing officer at Quadric, there is no need for time-intensive expensive re-spins of silicon to run new machine learning code.

“SoCs with Quadric’s Chimera GPNPU are ready to run Llama2 today,” adds Roddy.

Quadric clarifies that, in theory, the existing CPUs found in consumer devices can execute virtually any machine-learning model, including Llama 2. Nevertheless, utilizing CPUs for running complex ML models poses substantial challenges, particularly in performance and power efficiency. While GPUs are considered an alternative, they often demand significant power and may not be well-suited for compact, battery-powered devices, Quadric executives say.

The company emphasizes that its GPNPU holds a distinct advantage, combining the programmability characteristic of CPUs and GPUs with a specialized design tailored to deliver the precise balance of performance and power efficiency necessary for portable devices.

Additionally, Quadric adds that its capability to implement any machine learning model without requiring physical hardware modifications makes it an appealing choice for manufacturers.

Earlier this year, Quadric partnered with Silicon Catalyst, an incubator focused on semiconductor solutions, to bring edge computing AI chips to a broader audience. After being part of the Silicon Catalyst In-Kind Partner (IKP) ecosystem, Quadric’s intellectual property building blocks will be available to the 97 companies in the incubator’s fold.

Article Topics

 |   |   | 

Comments

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Featured Edge Computing Company

Edge Ecosystem Videos

Automating the Edge

“Automating

Deploying AI Models at the Edge

“Deploying

Latest News