• Hybrid Von Neuman + 2D SIMD matrix architecture
• 64b Instruction word, single instruction issue per clock
• 7-stage, in-order pipeline
• Scalar / vector / matrix instructions modelessly intermixed with granular predication
• Deterministic, non-speculative execution delivers predictable performance levels
• Configurable AXI Interfaces to system memory (independent data and instruction access)
• Configurable Instruction cache (64/128/256K)
• Distributed tightly coupled local register memories (LRM) with data broadcast networks within the matrix array allows overlapped compute and data movement to maximize performance
• Local L2 data memory (multi-bank, configurable 1MB to 16MB) minimizes off-chip DDR access, lowering power dissipation
• Optimized for INT8 machine learning inference (with optional FP16 floating point MAC support, and A8W4 MAC option) plus 32b ALU DSP ops for full C++ compiler support
• Compiler-driven, fine-grained clock gating delivers power savings

ML / AI inference solutions are most often performance- and power-dissipation-limited by memory system bandwidth utilization. With most state-of-the-art AI models having millions or billions of parameters, fitting an entire model into on-chip memory within an advanced System-on-a-Chip (SoC) is generally not possible. Therefore, smart management of available on-chip data storage of both weights and activations is a prerequisite to achieving high efficiency. To further complicate the design of SoCs the rate of change of AI models – both operator types and model topologies – far outpaces the design and deployment lifecycles of modern SoC designs. System architects must pick IP today to run models of unknown complexity in the coming years.

Many second-generation NPU accelerators in systems today are hardwired finite state machines (FSMs) that offload several performance intensive building-block AI operators such as convolution and pooling. These FSM solutions deliver high efficiency only if the ultimate network to be run on the SoC does not waver from the limited scope of the operator parameters that have been hard-coded into the silicon. This hard-coded behavior extends to the supported memory management strategies deployed for those FSM accelerators. A FSM solution does not allow for future fine-tuning of memory management strategies as network workloads evolve. The Chimera processor family solves this limitation by being fully programmable and powered by compiler-driven DMA management.

A SoC design with a Chimera GPNPU has four levels of data storage. Off-chip DDR offers a vast lake of storage for even the largest new AI models. But accessing DDR is expensive both in terms of power dissipation and cycle count. Therefore, the Chimera GPNPU contains a configurable private buffer SRAM on chip - the L2 memory – which is managed by compiler-driven code. L2 size is determined by the SoC architect at chip design time and can range from 1 MB to 16 MB. L2MEM holds data that is temporally beneficial in speeding up algorithm execution, such as model weights and activations for neural networks and a variety of data and coefficients for DSP algorithms. The partitioning of the usage of L2MEM is software and compiler driven, offering long-term flexibility to adapt to changing inference workloads.

NPUs – neural processing units – are a type of hardware block used in semiconductor chip design to accelerate the performance of machine learning / artificial intelligence workloads. NPUs are generally referred to as accelerators or offload engines and are paired with pre-existing legacy programmable processors such as CPUs, GPUs and DSPs.

The NPU concept first was conceived circa 2015 as silicon designers realized that emerging new AI/ML algorithms would not run at sufficiently high speeds on legacy CPU, DSP or GPU processors, hence the idea of “offloading” portions of the machine learning graph onto an accelerator.

The concept is simple: identify the most performance intensive elements of the AI/ML workload - the matrix multiplication common in convolutional neural networks - and carve that out of the graph to run on a specialized hardware engine, leaving the remainder of the algorithm to run on the original CPU or DSP.

Virtually all commercially licensable NPUs follow this approach, often sold alongside the original legacy processor offered by the IP vendor. Under this concept, both the legacy CPU or DSP host and the NPU are dedicated in tandem to running the AI/ML workloads. The NPU is a fixed-function block with a set “vocabulary” of AI functions – or Operators – that it can execute, and the CPU/DSP carries the rest of the workload. If newer algorithms are invented with different operators, more of that future workload defaults to running on the legacy CPU/DSP, potentially negatively impacting the overall throughput performance of the device.

GPNPUs – general purpose neural processing units – are a type of fully C++ programmable processor IP used in semiconductor chip design. GPNPUs run entire AI/ML workloads fully self-contained without the need for a companion CPU, DSP or GPU.

GPNPUs blend the high matrix performance of an NPU with the flexibility and programmability of traditional processors. GPNPUs – such as Quadric’s Chimera GPNPU - are future-proof because new operators can be easily written as new C++ kernels running at high-speed on the GPNPU itself. And because GPNPUs are code-driven, a compilation toolchain automates the targeting of new algorithms to the GPNPU with hundreds of different AI models compiled automatically to the platform.