High-Accuracy Inference in Neuromorphic Circuits using Hardware-Aware Training

The first architecture considered is an XNOR [9] implementation using FeFET-based dynamic logic. The array architecture is shown in Fig. 1, while the FeFET-based XNOR circuit block is illustrated in Fig. 2. XNOR-based networks could also be realized in SRAM, with identical training considerations. The FeFET-based approach is shown as an example here because other cell characteristics (such as area) are more desirable than in the SRAM case.

The cell of Fig. 2 implements the function $XNOR(X,W)$ where $X$ is a logic input, applied to the sources of the FeFETs, while $W$ is the weight, stored as the state of polarization of the BEOL FeCaps (connected to the gates of the underlying FETs of the overall FeFET). The FeFETs are programmed by applying moderately high voltage pulses to the program lines. Write disturbs for XNOR cells which share program lines are prevented using the selector FETs. Inference is performed by applying input signals ($X$, $\bar{X}$) with the PRG inputs grounded. This causes the storage node SN to either charge up to VDD or stay at GND. At the same time, the bitline BL is pre-charged. This performs the binary multiply portion of the MAC operation. The accumulate portion is performed in the second phase of the inference; the EVAL signal is enabled, and the driver FETs perform a partial discharge of the bitline. This particular approach to MAC eliminates (to a great extent) the variability problem associated with the FeCaps (which may be significant for scaled FeCaps), since the final voltage of the SN is either $\approx$ V_DD or $\approx$ 0. Variability is nevertheless present due to the Vt variation of the driver FETs, leading to variability in the discharge rate of the BL. This scheme is therefore useful when parallel accumulation is desired and FeFET variability is much greater than that of the standard FETs.

The Ternary Conductive Crossbar (TCC) architecture is a slightly modified resistive cross-bar array [5, 6], as shown in Fig. 3. The individual weight cells are shown in Fig. 4. The standard approach of using two weights to represent positive and negative conductances is used. The modification of the standard approach arises only in the use of dedicated program lines (Fig. 4), one for each row of weights. The program lines are shared across the entire row; write disturb prevention is accomplished by activating individual select lines. In inference mode, the program lines are grounded, and the weights behave like two-terminal devices, forming a cross-bar between the signal input and output lines. The weights themselves are FeFETs; the conductance level of the FeFETs (each with a grounded gate terminal) determines their weight value. Details of the programming can be found in [2].

In principle, it is possible to store analog or multi-bit weights into a single FeFET, if sufficiently accurate programming is available, and FeCap variability is sufficiently controlled. In this work, however, the FeFETs are assumed to be purely digital devices, programmed into either a strongly ON-state or strongly OFF-state. Such a scheme avoids the complexities of multi-bit or analog programming, as well as the challenges of variability when defining multiple programming states. Since the $G^{+}/G^{-}$ conductance pair is utilized, three useful conductance states are available: ($G_{ON}$, $G_{OFF}$), ($G_{OFF}$, $G_{ON}$), and ($G_{OFF}$, $G_{OFF}$), corresponding to the mathematical states 1, -1, 0. Since the FeFETs themselves are acting as the weight elements, all FeFET non-idealities impact the weight as well. While programming is similar to that of the XNOR cell, inference must be performed with low signal voltages which ensure that the FeFETs are in the linear regime throughout the inference event. Larger voltages create non-linear distortions which must be taken into account in the network model. Due to the presence of the $G^{+}/G^{-}$ pair, the nominal value of the ternary “zero” is exactly zero (due to $G^{+}/G^{-}$ cancellation). Statistically, however, there will be some variability of the ternary zero due to process variability. Additionally, any imperfections of the current mirror used to obtain the $G^{-}$ conductance will result in $G^{+},G^{-}$ asymmetry.

A possible approach to multi-bit crossbars (two bits in this case) is illustrated in Fig. 5 [2]. The weight cell consists of multiple branches, each comprised of a series combination of an FeFET and a passive resistor. The conductance values of the resistors follow a binary ladder. The total conductance of the weight is then given by:

The proposed training algorithm takes into account the appropriate HW model in a way which permits discontinuities, such as those caused by quantization, without sacrificing continuous differentiability of the cost function. All other non-idealities of the weights and activations are likewise included. As summarized in Alg. 1, the HW-aware training algorithm starts with a pre-trained neural net, resulting from FP-based training. Any suitable training method can be used to obtain $w_{FP}$. Regularization may or may not be performed for $w_{FP}$ at this stage. As discussed in [2], regularization without the appropriate HW model results in a loss of generalization capability, so re-training with HW-aware regularization is necessary. Next, the algorithm proceeds through several iterations of HW-aware weight refinement. At each iteration, the weight matrices are replaced by HW-description functions $g_{HW}(w,X,\alpha)$. The g-functions represent the non-ideal, HW-induced weights, and are therefore functions of the “ideal” weights, as well as the input vector $X$. Additionally, the g-functions depend on the variable $\alpha$, which serves as a continuation parameter. With $\alpha=0$, $g(w,X,\alpha)=w$; with $\alpha=1$, $g(w,X,\alpha)=G_{HW}(w,X)$, i.e. an accurate representation of the HW. With any intermediate value of the $\alpha$ parameter, the g-functions represent an approximation of the true HW model, becoming increasingly accurate as $\alpha$ is increased. A similar approach is used for activation functions, though these are generally functions of the pre-activation value $z$ only. Hyperparameter optimization forms the outer loop of the algorithm. Hyperparameters include standard parameters for $L^{1}$, or $L^{2}$ regularization, but also parameters related to the $g_{HW}$ functions. The latter include parameters such as the quantization step $\Delta$, but not parameters related to the physical model of the circuit such as asymmetries and non-linearities. In general, hyperparameters should only include variables which control the training, not those that characterize the hardware. It should also be noted that the network model used for validation should always contain the exact (or “best”) model of the HW, not the HW model used for the current training iteration. This ensures that regularization is being used in a context most similar to the final test evaluation.

A key property of the g-functions is that they are continuously differentiable w.r.t. $w$ for all values of $\alpha$, except possibly for $\alpha=1$. This ensures that the exact gradient is available at all steps in the iteration, enabling gradient descent-type optimizers (such as the suggested ADAM) to perform efficiently. The weights obtained in each iteration are used as the initial guess in the next iteration. After a few iterations of the algorithm (often just one), the obtained weights $w_{iter}$ are optimized for an accurate representation of the HW model. The algorithm can be illustrated with a simple example: a neural network model with undistorted Ternary weights. Discrete weight levels of the Ternary system can be approximated as follows:

The top figure of Fig. 6 illustrates the hardware approximation function $g^{3}_{HW}$ at an early stage in the iteration; $w_{sc}$ is relatively large, and the steps across discrete weight levels are smooth. The bottom figure of Fig. 6 shows the $g^{3}_{HW}$ at the end stage of the iteration; the scale parameter $w_{sc}$ is small, and the steps are nearly abrupt. Both plots also show the derivative of $g^{3}_{HW}$ w.r.t. the mathematical weight. It is apparent that the derivative is non-zero only near value transitions. Thus, in the late stages of the iteration, only mathematical weights near the transition boundary are impacted by the Gradient Descent family of algorithms (or any other gradient-driven algorithm). Mathematical weights are therefore “forced to choose” which side of the transition boundary they need to be on in order to minimize the cost function. The exact value of the mathematical weights (beyond the choice of side w.r.t. transition boundaries) ultimately does not matter in a discrete level system. The “choosing” effect of the algorithm on weight distributions is illustrated in Fig. 7. A Ternary NN using the function $g^{3}_{HW}$ of Eqn. 2 is trained on MNIST, and the weight distribution of the hidden layer is compared to that obtained by standard FP training.

Two examples are shown: using a small hidden layer (50 neurons, top plot of Fig. 7), and a larger hidden layer (200 neurons, bottom plot of Fig. 7). In both cases, the effect of HW-aware training for the ternary weights is apparent: weight distributions show gaps near the level transition boundaries. At each step in the iteration, weights near the transition boundary experience a strong gradient (scaled by the derivative of the $g^{n}_{HW}$ function, $>>1$ near transitions), which pushes them away from the boundary in whichever direction minimizes the cost function. Thus, after a few iterations of this algorithm, no weights (or negligibly few) remain near the boundaries.

The HW-aware training process is usually completed in two or three iterations. A typical training curve is shown in Fig. 8. The training accuracy is shown vs. Epoch, for a two-iteration training process. The “zeroth” iteration, labeled “FP” in Fig. 8 is just the standard SGD with FP-based weights and activations. After 10 epochs, the FP iteration reaches 9̃9% training accuracy. The “zeroth” iteration is terminated, and the next iteration begins. A new neural network is created using a Ternary HW model for the weights ($g^{3}_{HW}$), with the initial values for the weights obtained from the final iteration of the FP network. The Ternary step parameter $\Delta$ is set to 0.45 (a hyperparameter), while the $w_{sc}$ parameter is set to 0.05. This results in a smooth Ternary discretization that produces a significantly different model and cost than the FP case. This can be seen by the abrupt drop in training accuracy at the start of epoch 11. The same set of (mathematical) weights that produced 9̃9% training accuracy with the FP model now only produces 9̃1% with the approximate HW model (T1). However, after additional training with the initial ternary model T1, the training accuracy is increased to 9̃8%. The weight transition parameter $w_{sc}$ of 0.05 is not quite small enough to adequately model discrete levels, so an additional iteration is performed with $w_{sc}$ set to 0.005. This is shown as the “T2” step, starting with epoch 15. Due to the change of the model to a better approximation of the HW, there is another drop in the inference accuracy, which now equals 97%. A few epochs of additional training with the final HW model push the inference accuracy up to 98%. The transition scale $w_{sc}$ of 0.005 is sufficiently abrupt that further reductions of the parameter make no difference (as would be shown by an additional iteration, T3, not included here), so the the inference accuracy achieved at epoch 20 is the final training accuracy with a realistic HW model (in this case, and ideal Ternary). The neural net evaluated on the accurate HW model has only a 1% loss in inference accuracy (training) relative to the FP-based one. In Sec. IV, the test (not training) accuracy is investigated for several HW and network examples.

As described in section II-A, the particular choice of XNOR circuit results in binary weights and activations, with no significant distortions of weights due to hardware non-idealities. This is a consequence of the dynamic logic implementation of the signal-weight product operation, which results in either 0 or V_DD if the evaluation time is sufficiently long. The accumulation (summation) operation likewise does not have any weight-induced distortions, although it does, of course, suffer from errors due to FET Vt-related process variability. Similar considerations would have resulted had SRAM-based weights been used instead. Ignoring the variability aspect in this analysis, the weight model can be described as follows:

Applying the XNOR weight model of Eqn. 3 and Fig. 11 to a pair of simple benchmark networks shows that near-SW level accuracy is achievable. This is in contrast to direct binarization of SW-weights, which is shown to result in a significant loss of inference accuracy (Fig. 12).

The direct binarization approach is not able to do better than 30% error on MNIST w.r.t. the SW implementation, in spite of a certain degree of optimization of the binarization procedure itself (the magnitude of the weights is scaled to maximize validation accuracy separately for each network). The situation is even more dire with EMNIST: the obtained results are barely better than a random network. Direct binarization acts as a non-linear error amplifier for weights near the weight transition boundary. Small uncertainties for FP-based weights (from software training) which have negligible impact on inference accuracy become dramatically amplified by binarization. As indicated in Fig. 12, this problem is solved by HW-aware training, in which weights are iteratively pushed to either side of the transition boundary in a way that maximizes inference accuracy. As seen in Fig. 12, HW-aware training reduces the error to the 1% range or less, for both MNIST and EMNIST. In addition to assigning weights to the optimal side of transition boundaries, the HW-aware algorithm is also able to re-purpose unused weights to improve inference accuracy. As can be seen from weight distributions shown in Fig. 7, many (approximately 90%) of the weights are near zero, and not contributing to inference. These “unused” weights can be put to use by a HW-aware trained network consisting of non-ideal weights (as in the examples in this work). Thus, a larger number of “primitive” weights can perform at the same level as small number of more complex weights. As long as the network consists of a sufficient number of “unused” weights (as is typical), this is accomplished without adding neurons to the network; simply re-training in a HW-aware manner is sufficient. While the training algorithms are quite different, a similar conclusion regarding the capability of binary networks is reached in [9].

The Ternary crossbar approach of Sec. II-B provides three weight levels ({$-2\Delta$, 0, $2\Delta$}, with $\Delta$ a hyperparameter) which are achieved by signed summation of currents through pairs of conductances ($G^{+}$, $G^{-}$). The negative sign for the $G^{-}$ conductances is attained by a current mirror for the entire crossbar column, which sums $G^{-}$ currents. With an ideal current mirror, the mirrored current is indeed multiplied by $-1$; in practical implementations of current mirrors, however, some degree of error is to be expected. A systematic error in the current mirror results in an asymmetry between the $G^{+}$ and $G^{-}$ weights. For the purposes of this example, $G^{-}$ will be assumed to be the smaller effective conductance due to imperfect mirroring.

A suitable hardware model function $g^{3}_{HW}$ is:

As can be seen in Fig. 14, using HW-aware training is of essence for this problem, although for somewhat different reasons than in the case of the Binary XNOR. The approach of direct quantization yields very poor performance, with the error relative to the ideal NN never less than 30% on MNIST (and considerably worse with EMNIST). Comparing to results in [2], it is clear that most of the error is a result of the asymmetry, not quantization (unlike in the Binary XNOR case). Direct ternarization in [2] with symmetric weights was able to show errors on the order of 10% (with MNIST), indicating the severe penalty imposed by asymmetry in the current example. However, it is also clear that using HW-aware training almost completely suppresses both the asymmetry and quantization error; the discrepancy between the ideal SW-based network and the HW-aware trained network is well under 1% on both benchmarks (MNIST and EMNIST). It should be noted that HW-aware training can only re-distribute weights across weight categories; there is no change in the hardware values themselves. Thus, the asymmetry in weight magnitude persists even in the HW (i.e. the current mirrors are still imperfect), but the weight distribution has been optimized to take this into account. With larger hidden layers, the reduction in the HW-induced error is particularly striking, falling to less than 0.1%, even for the more challenging case of EMNIST. This is an illustration of the weight re-purposing nature of the HW-aware training algorithm. With a large number of weights available, most of the weights are essentially unused in an FP-trained net (as seen by the weight distributions of Fig. 7, where FP-based weights cluster around zero). During HW-aware training, these unused weights are re-purposed, helping to overcome their more primitive nature in HW. The larger the network, the more potential for re-purposing, and consequently the smaller the resulting HW error.

The Quinary crossbar, with its 2-bit weights, has been shown to be quite accurate when the non-linear distortion of the weights is small [2, 3]. For this example, a large non-linearity will be assumed. The large non-linearity may result from FET and resistor properties that are less ideal than assumed in [2]. In the context of [2], the most obvious source of non-linearity is the finite FET conductance, which should ideally be much larger than that of the passive series resistor. Depending on the implementation details, that may not be easily realizable, leading to a larger non-linearity than is described in [2]. The effects of quantization and non-linearity can be handled through HW-aware training, as shown next. The $g^{5}_{HW}$ function associated with a non-linear Quinary weight is shown in Fig. 15, and expressed mathematically as:

The performance of the Quinary weight with and without HW-aware training is shown in Fig. 15. The non-linearity degrades the expected high accuracy of the directly-quantized neural net. As discussed in [2], the expected inference error of the Quinary weight is as low as 1% on this problem (on MNIST). With the non-linearity included, the HW inference accuracy is at best within 10% of the SW version (for MNIST, and considerably worse with EMNIST). Using the HW-aware training algorithm with the model of Eqn. 5 reduces the error to the 0.1% level for both the MNIST and EMNIST benchmarks. Much like the case of the asymmetric Ternary weight, the network composed of non-linear Quinary weights benefits considerably from larger layer sizes. As before, the root cause of this behavior is the re-purposing of near-zero weights during HW-aware training. This excess of essentially unused weights (with FP-training) is a common feature of MLPs.

While all examples shown so far have highlighted the use of Alg. 1 for cases of non-ideal weights, it is also possible to apply the algorithm to other scenarios in which the desired behavior of weights and activations does not match that of the originally trained neural net. As an example, the training algorithm can be used to prune an already trained network in an optimal way. The pruning results in a much sparser neural net. While the sparsity is not straightforward to exploit in a crossbar array, it is certainly desirable if the neural net is implemented with a digital ALU. If the network is made to be sufficiently sparse, the number of weights may be small enough to fit into a local cache, thereby avoiding the energy and delay penalty associated with DRAM access of weights. For the sake of this example, the weights are considered to be implemented in a multi-bit digital fashion, with a sufficient number of bits to be approximated as a continuum. The goal of pruning is to remove as many of the near-zero weights as possible without significantly impacting the inference accuracy of the resulting sparse net. In order to accomplish this, the Alg. 1 is applied with the hardware function shown in Fig. 17.

The pruning hardware function of Fig. 17 exhibits two distinct behaviors: in the un-pruned region, the computed weights are equal to the mathematical weights. In the pruned region, the computed weights are identically zero. During the training process, the transition between the two regions becomes increasingly abrupt. The expression for the pruning function of Fig. 17 is given as:

The effect of the two pruning approaches is illustrated in Fig. 18. As the size of the pruning window is increased, the number of eliminated weights increases, and inference accuracy is reduced. However, while the naive algorithm shows a steep degradation of inference accuracy with increasing sparsity, the HW-aware pruning algorithm enables high sparsity levels with only a minimal impact to accuracy. Unlike the naive algorithm, the HW-aware pruning is free to re-distribute weights for each size of the pruning window. As was the case in other HW-aware training examples of this paper, only a very small number of weights are actually moved across transition boundaries. However, the choice of which weights to modify is guided by cost function optimization; this results in superior performance to simply eliminating “small” weights.