The Landscape of Compute-near-memory and Compute-in-memory: A Research and Commercial Overview

As depicted in Figure 1a, the interaction between memory and the processor in the Von Neumann architecture is facilitated through address and data buses. However, because CPU performance significantly outpaces memory performance, the Von Neumann models are often bottlenecked by the memory. To address this challenge and mitigate the impact of larger memory access latencies on the CPU, modern processors incorporate a tiered hierarchy of caches. Caches are smaller memory units that, while being much smaller compared to main memory and storage, are notably faster. The first-level cache (L1) is typically integrated onto the CPU chip and operates nearly at CPU speed, enabling single-cycle access. L2 cache is usually shared by multiple cores and can vary in location depending on the design goals. Some systems even include an L3 cache, usually larger and situated off-chip.

Note that Von Neumann architectures are characterized by the sequential execution of instructions. However, in multi-core CPU systems, parallel execution of instructions at various levels of granularity, including instruction-level, data-level, and thread-level parallelisms, is supported. To enhance performance and energy efficiency in resource-constrained systems, specialized accelerators are often developed and integrated on the same chip. For instance, in application domains such as embedded systems, digital signal processing, and networking, multiprocessor system-on-chip (MPSoC) architectures are employed. These integrate multiple processor cores, memory, input/output interfaces, and potentially specialized hardware accelerators, enabling parallel execution of different tasks to meet specific constraints.

Although these designs may significantly enhance performance when compared to conventional CPU-only systems, the underlying design principle remains CPU-centric and follows the Von Neumann model of execution. Consequently, the performance improvements, largely resulting from concurrent execution, heavily rely on the nature of the application. In cases where an application is memory-bound, i.e., most of the execution time is spent on the memory accesses and not on the actual compute operations, the shared data bus is fully occupied and becomes a bottleneck. Even for compute-bound applications, where these architectures can yield substantial gains in execution time, power consumption remains largely unaffected and might even increase due to the complex structure of these systems.

Unlike conventional computing systems where CPU has a central role and is responsible for all computations, most computations in the memory-centric designs are performed within or near memory. As depicted in Figure 1b, the core concept revolves around minimizing data transfer on the bus by relocating a substantial share of computations closer to the data (memory). The CPU’s primary role becomes issuing commands and handling computations that cannot be effectively executed in close proximity to the memory.

The concept of memory-centric computing is not a novel one, but it has experienced a significant surge in recent years. Consequently, various terms have emerged, often referring to the same idea, and more detailed classifications have been introduced in architectural designs. This section aims to clarify the terminology surrounding these approaches.

The memory subsystem typically consists of a four-level hierarchy, each with its own characteristics and access time. The fastest and smallest level is the CPU registers, while the slowest one in the hierarchy is storage devices (HDD/SSD). The CNM/CIM concepts have been proposed at different levels in the memory hierarchy, e.g., in-cache computing ²⁴, in-DRAM ²⁵, in-storage computing ²⁶. However, CNM/CIM at the main memory level have rightly gained more attention than others.

The main memory is also typically organized hierarchically, as shown in Figure 4. The main components of the hierarchy are memory cells, subarrays, banks, and ranks. A cell is the fundamental working unit and serves as a building block to form arrays. An array is a 2D grid of cells connected via word lines and bitlines. A word line is a horizontal line that connects all the memory cells in a row, while bitlines, on the other hand, connect all cells in a column. Memory cells are placed at the intersection of word lines and bitlines. The combination of a specific word line and a bit line uniquely identifies each memory cell. When a particular memory cell needs to be accessed, the corresponding word line and bit line are activated, allowing data to be read from or written into that cell.

dram is the most mature and widely used memory technology today. A DRAM cell is composed of a transistor and a capacitor. When the capacitor is fully charged, it represents the logical value 1, while a discharged capacitor represents the logical value 0. To access data from the DRAM array, the memory controller brings a particular row into the row buffer by sending an activate command. A read command is then issued to read specific column(s) from the row buffer and put them on the bus.

DRAM has scaled nicely for decades and has been used across application domains and systems ranging from HPC to portable devices. However, it is presently facing several challenges to remain the dominant technology. The increasing demand for higher capacity has put tremendous pressure on the DRAM capacitor size to shrink which makes it susceptible to errors. Also, the increase in capacity is significantly increasing the refresh power budget.

To address the escalating demands for higher bandwidth in modern applications, 3D stacked DRAM architectures, such as high bandwidth memory (HBM) (see Section 4.1.3), have been proposed. These architectures consist of stacked DRAM dies atop a logic layer, interconnected through through-silicon vias (TSVs), resulting in remarkably higher bandwidth. These structures are also employed in a series of CNM solutions, where the logic layer is used to implement custom logic and perform computations in closer proximity to the data ²⁷, ²⁸.

From the CIM perspective, the majority of in-DRAM implementations rely on charge sharing, wherein multiple rows are activated simultaneously. The shared charge is then utilized in a controlled manner to carry out various logic and data copy operations ²⁵. Moreover, cleverly manipulating the memory timing parameters, deviating from standard timings, has also been employed to implement different logic operations ²⁹.

SRAM is another mature memory technology that provides fast and efficient memory accesses. It is commonly used in caches, register files, and other high-speed memory applications where speed and low access latency are critical. An SRAM cell consists of multiple transistors arranged in a specific configuration to hold one bit of data. The most common configuration of an SRAM cell is a pair of cross-coupled inverters that are connected in a feedback loop, forming a latch.

To read and store data on the cell, the bitline terminals, bitline (BL) and bitline bar (BLB) (see Fig. LABEL:fig:sram-cell), are precharged and discharged, and the wordline( word line (WL)) is activated or deactivated depending on the values reading/writing from/to the cell.

There have been proposals for in-SRAM computing, especially at the last-level cache, which can be considerably slower compared to the L1 cache (e.g., by an order of magnitude). Similar to DRAM, most in-SRAM computing architectures also leverage charge sharing in the bitlines. Specifically, precharging the bitlines in a controlled manner and activating multiple rows simultaneously enables performing logic operations ²⁴. For bitwise multiplication in SRAM, research has demonstrated that the amplitude of the input voltage at the WL directly influences the discharge rate of the BLB. The voltage discharge on BLB, achieved within a specific timeframe, effectively represents a one-bit multiplication of the data stored in the SRAM cell.

PCM is resistive memory technology that employs reversible phase changes in materials to store data. The earliest demonstration of a 256-bit PCM prototype dates back to 1970 ³³. Today, PCM stands as one of the most extensively researched NVM technologies. A PCM device comprises a phase-changing material sandwiched between two electrodes (very similar to Fig. LABEL:fig:reram), which transitions between crystalline (low resistance state) and amorphous (high resistance state) phases. These two resistance states represent binary logic states, i.e., 1 and 0.

Typically, PCM requires a relatively high programming current (>$200\mu A$), but this can be mitigated to less than $10\mu A$ by scaling down the device size ¹⁰, ³⁴. As PCM stores data based on resistance, it can be programmed to encompass more than two resistance states, allowing for multi-level cells to represent more than a single bit of information. Nevertheless, relying on multiple resistance states for prolonged periods poses challenges, as the device resistance tends to drift over time, making it difficult to discern between resistance states.

RRAM is another class of resistive memory technologies that utilizes the resistive switching phenomenon in metal oxide materials to store data ¹¹. As shown in Fig. LABEL:fig:reram, a typical RRAM cell comprises a top and a bottom electrode with a thin oxide layer sandwiched in between. To achieve resistive switching, a high electric field is applied to the RRAM cell, leading to the creation of oxygen vacancies within the metal oxide layer. This process results in the formation of conductive filaments, causing the device state to transition from a high resistance to a low resistance (set) state. To revert to the high resistance (reset) state, the device is subjected to $V_{RESET}$, which breaks the conductive filament, allowing the oxygen ions to migrate back to the bulk. Compared to PCM, RRAM exhibits several advantages, including higher write endurance (>$10^{10}$), faster write operations, larger resistance on-off ratios, and improved scalability prospects ¹¹. However, ReRAM does suffer from inconsistent electrical characteristics, meaning it exhibits larger variations in resistance across different devices ¹⁰.

mram store data in nano-scale ferromagnetic elements via magnetic orientation ³⁵. An MRAM cell is a magnetic tunnel junction (MTJ) device composed of two ferromagnetic layers, namely a fixed reference layer and a free layer, separated by an insulating layer. The free layer holds the data bit, and reading it involves passing an electric current and measuring its resistance. For data writing into an MRAM cell, various techniques can be used. The most common method is the spin-transfer-torque (STT), which utilizes spin-polarized electric current to change the free layer’s magnetic orientation. Spin-orbit-torque (SOT)-MRAM, on the other hand, uses an in-plane current through the heavy metal layer to generate a spin current that exerts a torque on the magnetization of the free layer.The relative orientations of the free and fixed layers result in different resistance states. MRAM exhibits virtually unlimited endurance and acceptable access latency. However, it is faced with challenges such as a larger cell size and a smaller on/off resistance ratio, limiting an MRAM cell to store only one bit of data ³⁶.

Since the discovery of ferroelectricity in hafnium oxide, FeFETs have received considerable attention. FeFETs are non-volatile three-terminal devices, offering high $I_{on}/I_{off}$ ratios and low read voltage. Unlike metal-oxide-semiconductor (MOS)-FETs, FeFETs incorporate a ferroelectric oxide layer in the gate stack, as shown in Figure LABEL:fig:fefet. The nonvolatility arises from hysteresis due to the coupling between the ferroelectric and CMOS capacitances (C${}_{FE}$ and C${}_{CMOS}$). The three-terminal structure of FeFETs enables separate read and write paths. Reading involves sensing the drain-source current, while writing involves switching the ferroelectric polarization with an appropriate V${}_{gs}$ voltage. Unlike two-terminal devices with variable resistance, FeFETs do not require a drain-source current during the writing process, leading to low writing energy consumption ³⁷. There are various CIM architectures exploiting different properties of FeFETs. For instance, for boolean operations, ³⁷ proposes precharging the bitlines followed by simultaneous activation of the target rows, and using differential sense amplifiers to discern the output.

Table 1 presents a comparison between mainstream and emerging memory devices, discussed in the preceding sections, with respect to performance, reliability, and energy consumption, among others. This analysis gives insights into their suitability for different application domains. It is clear that no single memory device can optimize all metrics. Nonetheless, recent investigations into machine learning use cases show that different phases of machine learning tasks demand different memory device properties, potentially offering the opportunity to employ various devices for various application domains (or tasks within a domain) and achieve the best results ³⁸.

PCM, RRAM, MRAM and FeFET fall under the category of memristive technologies, where devices can exhibit multiple resistance states. This characteristic has been effectively leveraged to perform MVM, as depicted in Figure LABEL:fig:cim-crossbar. Although the analog computation may not be entirely precise, some loss in accuracy is acceptable in many application domains, particularly for machine learning applications. Numerous CIM architectures have been proposed using this technique to accelerate neural network models (see Section 4.2).

Figure 6 shows the importance of various device properties for neural network training and inference. For training, frequent weight updates within the memory are crucial, making memory technologies like PCM and RRAM, with limited endurance and expensive write operations, poorly suitable for training acceleration. However, in inference, where operations are predominantly read-based with minimal writes to the crossbar array, the same technology could outperform others by orders of magnitude. Similarly, retention for training is the least important but is critical for inference.

The arguments around Figure 6 generally hold true for other metrics and other application domains. Although machine learning constitutes a significant area, it is not the only domain benefiting from the CIM paradigm. Many other data-intensive application domains have also effectively exploited the CIM paradigm. Figure 7 shows a landscape of CIM applications, emphasizing precision considerations, computational complexity and memory access requirements. These applications are classified into three categories based on their precision demands. This data pertains to 2020. Over the past two years, additional application domains, such as databases, bioinformatics, and solving more complex algebraic tasks, have gained significant attention as well.

The core principle of compute-near-memory is to perform computations in the memory proximity by placing processing units on/near the memory chip. The first CNM architecture dates back to the 1990s that aimed at integrating compute units with embedded DRAM on the same chip to achieve higher bandwidth. However, due to technological limitations and costly fabrication processes, even the promising initial CNM proposals like IRAM ⁴⁰, DIVA ⁴¹, and FlexRAM ⁴² never commercialized.

In recent years, due to the advancements in integration and die-stacking technologies, CNM has regained interest within both industry and academia. PUs are being integrated at different locations within memory devices, including within the memory chip as well as outside the memory chip on the module level, i.e., dual in-line memory module (DIMM). A DIMM typically consists of multiple memory chips, each consisting of multiple ranks, banks, and subarrays. It is worth noting that some researchers also classify PUs integrated at the memory controller level as CNM. However, following the classification and terminology adopted in this report, we categorize it under the COM class.

In the following, we explain some of the common CNM architectures. We start by examining the planar 2D DRAM-based CNM designs, then transition to discussing the 2.5D and 3D DRAM-based CNM systems, and ultimately conclude on the NVM-based CNM architectures.

Much like CNM, the concept of CIM systems is not entirely novel; however, it has gained significant momentum due to breakthroughs in various NVM devices over the past decade. Figure 14 shows a partial landscape of CIM systems, along with a corresponding timeline. Most of the depicted CIM accelerators originate from academia and are not taped out. However, in recent years, several semiconductor industry giants, including Intel, Samsung, TSMC, GlobalFoundries, and IBM, have invested in developing their own CIM prototypes, mostly focused on the machine learning case. IBM, in particular, stands out among others when it comes to the development of CIM systems for different use cases.

In this section, we overview some of the prominent CIM designs from academia and industry. However, before going into the details of individual CIM designs, we first introduce circuits that are typically used as basic CIM primitives in these architectures.

The surge in CIM and CNM systems is largely attributed to the revolution in data-intensive applications. According to recent research by TSMC, traditional SRAM and DRAM technologies have effectively scaled to meet capacity and bandwidth demands in the past decades ⁸², but their future scalability is uncertain due to reaching inherent technological limits. This underscores the pivotal role that NVM will play in the future of computing.

Especially in edge scenarios such as automotive, augmented reality, and AI, where energy efficiency is paramount, NVM technologies are poised to play a pivotal role. As energy efficiency increases through specialized hardware, domain-specific architectures harnessing these NVMs for CIM and CNM solutions are anticipated to experience an unprecedented surge in the coming years. A recent article from Intel ⁷⁹ compares the performance of conventional digital accelerators with the emerging analog and digital CIM and CNM accelerators. Conventional accelerators still achieve higher throughput because CIM systems are relatively less optimized, array sizes are small, and the peripheral circuitry overhead is non-negligible. Yet, they are orders of magnitude better in terms of power consumption. As of the time of writing, the most recent comparison depicted in  ⁸² shows similar trends.

Table 2 presents a summary and comparison of the architectures discussed in this section. For brevity, we only compare important parameters, such as the underlying memory technology, available function (boolean logic, arithmetic, etc.), evaluation technique (simulation, prototype, analytic), programming model, application domain, and technology node.

Axelera ⁸⁴ is one of the notable Semiconductor startups in Europe. Founded in 2021 and backed by tech giants like Bitfury and IMEC, it had already taped out its first CIM chip, Thetis, in December 2021 (just four months after its founding). Today, it offers a fully integrated system-on-chip (SoC) powered by its Metis AI processing units (AIPU).

About the AI core, as per the company’s website: “Axelera AI has fundamentally changed the architecture of “compute-in-place” by introducing an SRAM-based digital in-memory computing (D-IMC) engine. In contrast to analog in-memory computing approaches, Axelera’s D-IMC design is immune to noise and memory non-idealities that affect the precision of the analog matrix-vector operations as well as the deterministic nature and repeatability of the matrix-vector multiplication results. Our D-IMC supports INT8 activations and weights, but the accumulation maintains full precision at INT32, which enables state-of-the-art FP32 iso-accuracy for a wide range of applications without the need for retraining”.

Axelera’s latest SoC consists of 4 cores and a RISC-V based control core. For programming these systems, Axelera provides an end-to-end integrated framework for application development. The high-level framework takes users along the development processes without needing to understand the underlying architecture or even the machine learning concepts.

d-Matrix is at the forefront of driving the transformation in data center architecture toward digital in-memory computing (DIMC) ⁸⁵. Founded in 2019, the company has received substantial support from prominent investors and strategic partners, including Playground Global, M12 (Microsoft Venture Fund), SK Hynix, Nautilus Venture Partners, Marvell Technology, and Entrada Ventures.

Leveraging their in-SRAM digital computing techniques, a chipset-based design, high-bandwidth BoW interconnects, and a full stack of machine learning and large language model tools and software, d-Matrix pioneers best-performing solutions for large-scale inference requirements. A full stack framework, compiler, and APIs (open-source as per the company’s website but couldn’t find the link). Their latest product Jayhawk II can scale up to 150 TOPS/W using 6nm technology and can handle LLM models up to 20$\times$ more inferences per second for LLM sizing to 40B parameters, compared to state-of-the-art GPUs.

Gyrfalcon Technology ⁸⁶ also leverages CNM to accelerate AI on the edge. They offer an AI processing in memory (APiM) architecture that combines a large MAC array directly with MRAM memory modules. As of the current date, their software stack is not available.

MemComputing ⁸⁷, founded in 2016, uses a computational memory based on its self-organizing logic gates (SOLG). SOLGs are terminal-agnostic elements (memristor or memcapacitor) that implement various logic gates. Their target applications comprise industrial computations associated with optimizations, big data analytics, and machine learning. MemComputing provides a software stack and offers it as a software-as-a-service.

Memverge ⁸⁸ is not directly doing any CIM or CNM but is relevant in the context. Backed by 9 investors including tech giants like Intel, SK hynix, the company’s main goal is to provide software designed to accelerate and optimize data-intensive applications. Their main target is to consider environments with “Endless Memory” and efficiently manage the memory to get more performance.

Mythic ⁸⁹ offers an analog matrix processor (Mythic AMP) that uses their analog compute engine (ACE) based on flash memory array and ADCs. Mythic ACE also has a 32b RISC V processor, SIMD vector engine, and a 64KB SRAM along with a high-throughput network-on-chip (NoC). Mythic workflow in Figure 26 shows that the software stack takes a trained NN model, optimizes it, and compiles it to generate code for Mythic AMP. The optimization suit also transforms NN in a way that can be accelerated on the analog CIM system.

Founded in 2018, NeuroBlade offers the SPU (SQL Processing Unit), the industry’s first, proven processor architecture that delivers orders of magnitude improvement by departing from Von Neumann model ⁹⁰. Neuroblade is also a CNM architecture (more closed to near-storage computing) where they integrate custom RISC processors on the DRAM chip (very similar to UPMEM). The SPUs are installed as PCI-e cards that can be deployed in data centers. As for the software stack, the company offers an SDK along with a set of APIs that hide the complexity and programming model for these cores from the end user and also allow optimizing for maximum parallelism and efficiency.

Founded in 2017, Rain AI also focuses on radically cheaper AI computing ⁹¹. The company has no hardware product yet but is aiming to be 100$\times$ better than GPU using their innovations in radical co-design (by looking at the algorithms and the CIM hardware at the same time). They are targeting AI training (along with the inference) on the edge with the ultimate goal of putting models the size of ChatGPT into chips of the size of a thumbnail. They are transforming the algorithms in a way that fundamentally matches the behavior of the analog memristive devices. As per the CEO, they have a few tap-outs planned for this year and the product (a complete platform) next year and they are working on a software stack for ease of use and ease of integration.

Founded in 2020, Semron ⁹² promises to offer 3D solutions powered by analog CIM. At the core of their technology is their innovative CapRAM devices which are semiconductor devices that store multi-bit values in their variable capacitances (unlike variable resistance states in memristors). Since CapRAM is capacitive, the noise in calculations is much lower and the energy efficiency, as per their website, is unparalleled. Although Semron has the device technology, there are no details of its products, architecture, and software stack.

Surecore ⁹³ is working on many low-power products including custom application-specific. They also have a product named “CompuRAM” that embeds arithmetic capability within the SRAM array to enable low-power AI on the edge. Besides working on SRAM-based solutions, in collaboration with Intrinsic, they have recently ventured into RRAM technology. No information is provided regarding the software stack.

Synthara is a Zurich-based Semiconductor company that was founded in 2017 ⁹⁴. Their latest product, ComputeRAM, integrates SRAM-based CIM macros with proprietary elements to accelerate dot products. The solution delivers 50$\times$ compute efficiency and can be used for AI, digital signal processing, and linear algebra-heavy routines. The CIM-powered SRAM array can be operated just like conventional SRAM. ComputeRAM is not married to a specific ISA and can work with any host processor. Synthara also provides what they call Compiler hooks that can transparently offload any input application to their ComputeRAM accelerator, without changing or rewriting the code.

Founded in 2017, Syntiant also leverages DRAM-based CNM and utilizes standard CMOS processes to design their neural decision processors (NDPs) that perform direct processing of neural network layers from platforms like TensorFlow ⁹⁵. Syntiant also mainly targets AI on the edge having applications in many domains, including always-on voice, audio, image, and sensor applications.

Syntiant’s TinyML platform, powered by NDP101, aspires to democratize AI by presenting a comprehensive system for those interested in initiating their own model training for edge computing.

Founded in 2018, TetraMem is set to offer the industry’s most disruptive CIM technology for edge application ⁹⁶. TetraMem is also leveraging memristors for analog MAC operations, aiming at inference on the edge. Their systems are built upon their patented devices and co-design solutions.

TetraMem offers (1) Platform as a service (PaaS), a complete hardware and software platform designed to integrate into your own system; (2) Software as a service (SaaS), to help develop your NN edge application and integrate it into your system. Their verified full software stack provides an unmatched experience on actual analog in-memory compute silicon; and (3) a neural processing unit (NPU) based on memristive technology.

TetraMem has recently announced a collaboration with Andes Technologies and together with their research collaborators have demonstrated a memristive device that can have thousands of conductance levels (unmatched) ⁹⁷.

Founded in 2022, EnCharge AI promise to offer an end-to-end scalable architecture for AI inference ⁹⁸. They leverage SRAM-based CIM arrays for analog MVM operations and combine them with SIMD CNM logic to perform custom element-wise operations. The architecture comprises an array of CIM units (CIMUs), an on-chip network interconnecting CIMUs, buffers, control circuitry, and off-chip interfaces. Each CIMU is equipped with an SRAM-based CIM array featuring ADCs to convert computed outputs into digital values. Additionally, CIMUs house SIMD units and FPUs with a custom instruction set, along with buffers dedicated to both computation and data flow. According to the company’s official website, they offer a software platform that fits with standard ML frameworks, such as PyTorch, TensorFlow, and ONNX. This also allows the implementation of various ML models and their customizations. Specific implementation details about the software stack are not available.

Re(conceive) is another CIM startup founded in 2019 that promises offering “the most power AI accelerator” ⁹⁹. As per their website, re(conceive) are pioneers in realizing the complete potential of CMOS-based analog in-memory AI computing, achieving the utmost efficiency among all known AI accelerators. However, no specific details are available on the company’s funding and technology (hardware/software).

Established in 2022 by a team of Oxford University scientists, Fractile ¹⁰⁰ aims to transform the world by enabling large language models’ (LLM) inference at speeds up to 100 times faster than Nvidia’s most recent H100 GPUs. This increase in performance primarily arises from in-memory computations. However, the details of the technology, both hardware and software, as well as the company’s funding particulars, remain undisclosed.

Founded in 2018 ¹⁰¹, Untether’s main design integrates RISC-V cores on the SRAM chips for processing AI workloads. Their latest product, the tsunAImi accelerator card provides a phenomenal 2 POPS of compute power, twice the amount of any available product. This compute power translates into over 80,000 frames per second of ResNet-50 throughput, three times the throughput of any product on the market. Untether AI provides an automated SDK for its products. The SDK takes a network model implemented in common machine learning frameworks like TensorFlow and PyTorch and lowers it into the kernel code that runs on these RISC-V processors. It automatically takes care of low-level optimizations, providing extensive visualization, a cycle-accurate simulator, and an easily adoptable runtime API.

Founded in 2015, UPMEM is a tech company offering programmable CNM systems for data-intensive applications. See more details on the architecture and programmability in Section 4.1.1.

Table 3 and Figure 27 summarize the discussion in this section and provides a landscape of CIM, CNM companies, their products, technologies, and funding status. Please note that this compilation is not exhaustive; it includes only companies known to us and those that, based on our understanding, fall within the CIM and CNM categories. As Figure 27 clearly illustrates, the current landscape is predominantly characterized by conventional technologies, with a notable absence of a comprehensive software ecosystem.

CIM and CNM systems have already entered the market, yet a series of open challenges are expected to become more pronounced as time progresses. It will take years to understand how these units will harmonize within the overall system architecture and determine their optimal utilization. In the following, we briefly discuss the important categories.

While every challenge holds significance and demands attention, programmability and user-friendliness are the most important ones from the user’s standpoint. Following is an excerpt from Facebook’s recent article on their inference accelerator that highlights the same.

“We’ve investigated applying processing-in-memory (PIM) to our workloads and determined there are several challenges to using these approaches. Perhaps the biggest challenge of PIM is its programmability”.

In response to the challenges associated with programmability, we have ourselves been working on high-level programming and compilation frameworks for CNM and CIM systems ¹⁰², ¹⁰³, ¹⁰⁴, ¹⁰⁵. We have developed reusable abstractions and demonstrated compilation flows for CIM systems with memristive crossbars, CAMs, CIM-logic modules, and for CNM systems like UPMEM and Samsung CNM. However, much more cross-layer work is needed to improve automation ¹⁰⁶, in particular for heterogeneous systems integrating several paradigms and technologies.