Can their approach deliver solid performance?
SK hynix, TetraMem, and researchers from the University of Southern California have developed a memristor-based in-memory computing (IMC) system-on-chip (SoC) for AI edge devices. The device is designed to accelerate neural network inference in lightweight AI models while consuming a fraction of the power that higher-end GPUs or NPUs would. To a large degree, the SoC is a proof-of-concept chip, as its performance would peak at around 2.54 TOPS in a theoretical best-case scenario, which is 16X below Microsoft's Copilot+ requirements.
A DWC-optimized IMC architecture
Memristor-based in-memory computing (IMC) accelerates neural networks by performing analog computations directly inside memory arrays, which reduces data movement and power consumption. However, depthwise convolution (DWC) — a core operation in lightweight networks such as MobileNet — performs independent per-channel filtering with limited data reuse and therefore maps poorly onto conventional crossbar arrays. To address this limitation, researchers from SK hynix, TetraMem, and USC developed an SoC that features both conventional IMC crossbars and a memristor-based IMC architecture specifically optimized for DWC.
The jointly developed SoC is based on an embedded RISC-V processor that schedules workloads and features 10 neural processing units (NPUs). One NPU out of 10 is dedicated to depthwise convolution, while the remaining nine execute pointwise and dense operations. Nine out of 10 NPU include a 256 × 256 memristor crossbar that performs the analog vector-matrix multiplication (VMM), 256 8-bit DACs that convert digital activations into analog voltages, 256 8-bit ADCs that convert the analog outputs back into digital values, and additional peripheral circuitry for reading, writing, programming, and controlling the crossbar.
The DWC-optimized NPU replaces its conventional array with eight specialized 252 × 28 zig-zag crossbar blocks, but retains DACs and ADCs. SK hynix developed and fabricated the memristor devices and integrated the resistive switching cells on top of the 65 nm CMOS circuitry using its back-end process.
That DWC-optimized NPU is the key feature of the whole SoC. To accelerate depthwise convolution, TetraMem replaced the straight selection lines used in conventional 1T1R crossbars with a zig-zag topology. As a result, the NPU contains eight 252 × 28 crossbar blocks whose diagonal selection lines activate 252 memory cells across 28 columns, which enables 28 independent 3 × 3 convolutions to run in parallel while using 100% of the array for weight storage. The remaining nine NPUs retain conventional 1T1R crossbars for 1×1 pointwise and dense layers and preserve the throughput and energy efficiency of traditional in-memory computing.
Great efficiency, low performance overall
To demonstrate the architecture, the researchers deployed a customized MobileNetV1Small neural network for the Visual Wake Words benchmark. The network contains approximately 36,000 parameters; all depthwise layers were mapped to the dedicated NPU, and pointwise layers were mapped to the remaining NPUs.
Because the memristor-based IMC hardware natively performs unsigned analog vector-matrix multiplication, inputs and weights are quantized to unsigned 8-bit values before execution. Since each memristor device can be programmed with only slightly more than 2 bits of effective precision, the design uses a two-subarray compensation technique that boosts effective weight precision to roughly 4 bits.
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
Conceptually, the approach is somewhat analogous to Nvidia's NVFP4 philosophy, in that both seek to achieve higher effective precision from low-precision hardware. However, the implementations are fundamentally different: NVFP4 relies on a digital floating-point representation and scaling factors, whereas the memristor SoC improves precision by compensating for analog programming errors using two programmed subarrays.
When it comes to accuracy, the SoC achieved an end-to-end inference accuracy of 80.36%, which matches the corresponding 4-bit software model. As for performance, the SoC delivers a peak throughput of 0.254 TOPS per NPU and reaches an energy efficiency of 21.3 TOPS/W at 100 MHz and 11.9 TOPS/W at 400 MHz. According to the authors, this compares favorably with published SRAM-based compute-in-memory accelerators despite being manufactured on an older 65 nm process. The SoC also exceeds Nvidia's A100 INT8 energy efficiency by an order of magnitude, the joint paper claims. Yet, these claims are largely unsubstantiated.
First up, the MobileNet demonstration does not even use all 10 NPUs. It uses one dedicated DWC NPU, five standard NPUs for pointwise layers, and leaves four standard NPUs idle. The demonstration thereby does not reveal total SoC throughput (TOPS), sustained throughput running a real network, and throughput with all 10 NPUs simultaneously saturated. In fact, the paper does not even reveal whether all 10 NPUs can be used at the same time. To that end, the 2.54 TOPS figure we mentioned earlier in the story is highly theoretical.
Validated approach
SK hynix, TetraMem, and researchers from the University of Southern California have developed a memristor-based IMC SoC featuring a novel depthwise convolution accelerator that improves crossbar utilization for lightweight AI workloads. The partners have managed to fabricate it using an outdated 65nm process technology and make it work, achieving a 21.3 TOPS/W energy efficiency and inference accuracy comparable to a 4-bit software model despite the fact that memristors can be programmed with a circa 2-bit accuracy. While the architecture validates that the approach works, the paper does not disclose the full performance of the SoC, and it is not clear whether the chip's 10 NPUs can be saturated at all.
Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.
Anton Shilov is a contributing writer at Tom’s Hardware. Over the past couple of decades, he has covered everything from CPUs and GPUs to supercomputers and from modern process technologies and latest fab tools to high-tech industry trends.
Facts Only
* SK hynix, TetraMem, and University of Southern California researchers developed a memristor-based in-memory computing (IMC) system-on-chip for AI edge devices.
* The device accelerates neural network inference in lightweight AI models using low power consumption.
* The architecture uses an embedded RISC-V processor scheduling workloads across ten NPUs.
* One NPU is dedicated to depthwise convolution (DWC); the remaining nine execute pointwise and dense operations.
* Nine of the ten NPUs include a 256 × 256 memristor crossbar for analog vector-matrix multiplication (VMM).
* The DWC-optimized NPU replaces conventional crossbars with eight specialized 252 × 28 zig-zag crossbar blocks.
* The system uses unsigned 8-bit quantization, employing a two-subarray compensation technique to achieve effective weight precision of roughly four bits.
* The demonstrated MobileNetV1Small network achieved an end-to-end inference accuracy of 80.36%, matching the 4-bit software model.
* Peak throughput is stated as 0.254 TOPS per NPU, with energy efficiencies of 21.3 TOPS/W at 100 MHz and 11.9 TOPS/W at 400 MHz.
* The chip was fabricated on a 65 nm CMOS process by SK hynix.
Executive Summary
SK hynix, TetraMem, and researchers from the University of Southern California developed a memristor-based in-memory computing system-on-chip for AI edge devices. This system is designed to accelerate neural network inference in lightweight models while reducing power consumption compared to high-end GPUs or NPUs. The core innovation involves a Depthwise Convolution (DWC)-optimized architecture utilizing both conventional and memristor-based in-memory computing structures to handle operations efficiently. The chip features an embedded RISC-V processor with ten Neural Processing Units (NPUs), one dedicated to depthwise convolution, while the other nine handle pointwise and dense operations using a combination of crossbars and digital-to-analog converters.
The system demonstrated an end-to-end inference accuracy of 80.36%, matching a corresponding 4-bit software model. Performance metrics show a peak throughput of 0.254 TOPS per NPU and energy efficiencies reaching 21.3 TOPS/W at 100 MHz and 11.9 TOPS/W at 400 MHz, which the authors claim favorably compare to other compute-in-memory accelerators. However, the demonstration used only one DWC NPU and did not reveal the total SoC throughput or simultaneous saturation of all ten NPUs; the reported peak performance figures are noted as theoretical best-case scenarios.
Full Take
The narrative balances high-level claims of efficiency and novelty with significant caveats regarding performance validation. The comparison to Nvidia's NVFP4 highlights an attempt to redefine precision through analog hardware, but the mechanism for achieving higher effective precision—analog error compensation versus digital scaling factors—represents a fundamental divergence in architectural philosophy. The reported energy efficiency metrics are presented as competitive against existing accelerators, yet the text explicitly qualifies these claims by noting they are largely unsubstantiated and that the demonstration failed to test saturation scenarios or total system throughput across all NPUs. This suggests a pattern where technical feasibility is emphasized over comprehensive performance validation, which is a common feature in early-stage research dissemination.
The shift from conventional crossbars to zig-zag topologies for DWC demonstrates an engineering focus on mapping specific layer requirements (DWC) onto physical constraints (crossbar structure), suggesting that optimization around known network structures can yield tangible architectural gains, even when using bleeding-edge but immature components like memristors in a 65nm process. The challenge lies in separating the validated proof-of-concept—that the approach functions and offers energy gains relative to benchmarks—from the unvalidated scalability of the entire system under realistic load conditions. What this implies is that while novel material science and architectural restructuring can create theoretically superior pathways, the transition to commercially relevant performance requires moving beyond theoretical peak scenarios to demonstrate sustained, multi-unit throughput reliability.
Bridge Questions: If the demonstrated accuracy holds across a wider range of network architectures or larger models, does the limitation in demonstrating total SoC saturation become a secondary concern? How should the field weigh validated energy efficiency gains from novel memory structures against the need for fully characterized, high-throughput operational performance? Does the current focus on specialized accelerators risk fragmenting the development of generalized, scalable AI compute paradigms?
Sentinel — Human
The text blends specific technical details with critical evaluation, suggesting human journalism framing a research breakthrough by focusing on performance claims and methodology limitations.