A Unified Lightweight Compute Engine for Softmax, GELU, and SiLU Based on a Shared Exponential Unit

The self-attention mechanism and feed-forward networks of the Transformerarchitecture heavily rely on three non-linear activation functions: Softmax,GELU, and SiLU. Traditional hardware acceleration schemes typically deploy these three as independent modules, each equipped with dedicated exponential arithmetic units, look-up tables, and normalization logic, therebyleading to signicant area redundancy and power overhead. To address thisissue, this paper proposes a unied lightweight hardware compute enginefor the aforementioned three operators. Distinct from existing works thatuniversally adopt the tanh approximation, this paper introduces a sigmoidapproximation form of GELU. This mathematical transformation reveals theintrinsic structural identity between GELU and SiLUboth can be equivalently expressed as the product of an input operand and a two-elementsoftmax operator, with their dierence lying merely in a scalar pre-scalingfactor. Based on this architectural unication, Softmax, GELU, and SiLUcan share the same set of core hardware resources encompassing exponentiation, logarithm, and normalization. Switching across modes only requires theminor logic overhead of a single multiplier and a two-way selector. Building upon this, the design synergistically integrates Canonical Signed Digit(CSD) encoded shift-add arithmetic, a Mitchell approximate logarithm withrst-order error correction, a binary reduction adder tree, and mode-awaredata gating techniques, while oering an optional Time-Division Multiplexing (TDM) mechanism to systematically optimize Power-Performance-Area(PPA) metrics. The SystemVerilog hardware implementation, based on 16-bit xed-point inputs and 32-bit internal arithmetic formats, provides twodesign space congurations: the full-throughput conguration maintains thehigh eciency of processing one vector per cycle, reducing the area by 34.9%;the TDM conguration compromises with a halved throughput, further reducing the area by 50.2% and dynamic power by 40.6%. Synthesis evaluationsbased on the SMIC 55 nm CMOS process demonstrate that both congurations, while adding the SiLU functionality absent in baseline designs, achievean over an order-of-magnitude improvement in computational precision (Softmax MSE improved by 24.4×, GELU MSE improved by 17.8×).