Heterogeneous Mapping for Analog In-Memory Computing Accelerators: A Unified Workflow
For researchers and engineers deploying large language models on analog in-memory computing hardware, this work provides a structured workflow and initial sensitivity insights for decoder-only transformers, though it only applies the first two stages and does not evaluate full deployment.
The paper classifies heterogeneous mapping methods for analog in-memory computing accelerators into a unified workflow and applies its first two stages to GPT-2, producing the first AIMC-specific precision sensitivity profile for a decoder-only transformer. It finds that sensitivity is dominated by 4 of 49 projections, with the first decoder block's attention output dominating by an order of magnitude.
Analog In-Memory Computing (AIMC) accelerators execute matrix-vector multiplications directly within memory arrays, reducing data movement and improving DNN inference efficiency. Their limited effective precision motivates heterogeneous architectures that combine analog compute tiles with digital processing units. This letter classifies existing methods for partitioning DNN workloads across these resources by mapping granularity, optimization strategy, and model support, and distills them into a unified four-stage workflow. To demonstrate the workflow on a model class not yet addressed by existing methods, we apply its first two stages to GPT-2, producing the first AIMC-specific precision sensitivity profile for a decoder-only transformer. Sensitivity is dominated by 4 of 49 projections, with the first decoder block's attention output dominating by an order of magnitude. This suggests that projection-level mapping and selective digital execution of early-block and output-facing projections are important for reliable decoder-transformer deployment on AIMC hardware.