An Open-Source ML-Based Full-Stack Optimization Framework for Machine Learning Accelerators

We divide the problem of full-stack optimization of ML hardware accelerators into two subproblems. Given an ML architectural configuration \(({x_1, x_2, \ldots , x_n})\), the logical hierarchy graph (LHG) of the RTL netlist, the target clock frequency and the floorplan utilization, two subproblems are as follows.

To solve above two problems, we build two learning-based prediction frameworks:

For Equation (1), we train three separate models to predict each of power, performance and area. For Equation (2), we train two separate models to predict system-level runtime and energy.

We propose to leverage the RTL netlist produced by the ML hardware generator, together with architectural and backend features (\(f_{{target}}\), util), as a strategy to enhance performance of our ML model. We apply the GCN modeling, which is adept at extracting relevant features from a graph. We generate an LHG from the RTL netlist, and incorporate this as an additional input into our GCN model, alongside the architectural and backend features.

Next, we define the problem of design space exploration. Specifically, for a given target workload, our objective is to find the optimal architectural and backend configuration, which minimizes the cost function:while satisfying the following constraints:

Here, \(\alpha\) is a user-specified parameter and can be set to any positive real value. A higher value of \(\alpha\) will prioritize area cost and a lower value will prioritize energy cost. In the experimental evaluation of our approach, when the energy is on the order of mJ and the chip area is on the order of \(\text{mm}^2\), we set \(\alpha = 1\) (as in VTA). Similarly, when the energy is on the order of \(\mu \text{J}\) and the chip area is on the order of \(\mu \text{m}^2\), we set \(\alpha = 0.001\) (as in Axiline).

We demonstrate our approach on the accelerator engines from four parameterizable open-source ML hardware generators.

TABLA [26] implements non-DNN ML algorithms such as linear regression, logistic regression, support vector machines (SVMs), backpropagation, and recommender systems.

GeneSys [9] executes DNNs using an \(M \times N\) systolic array for GEMM operations such as convolution, and an \(N \times 1\) SIMD array for vector operations such as ReLU, pooling, and softmax.

Axiline [9, 40] builds hard-coded implementations of small ML algorithms (e.g., SVM, logistic/ linear regression) for training and inference.

VTA [3, 28] is a DNN accelerator where a compute module includes a GEMM core for convolution, along with a 2D array of PEs. VTA is integrated with Apache TVM [5], a deep learning compiler stack.

Integrating system simulations with backend data. tThe simulators are integrated with the backend analyses, where they receive PPA characteristics generated by our SP&R flow. The simulators used in our study are obtained from the GitHub repository of the VeriGOOD-ML project [41] and VTA hardware design stack [43]. For a specific hardware configuration point, provided as an input, the PPA characteristics feed the simulator with data such as the clock frequency, energy per access for each of the on-chip buffers, and dynamic and leakage power of systolic and SIMD hardware components, or GEMM and ALU hardware components. The performance statistics provided by the simulator are combined with these backend data from the SP&R flow to produce end-to-end runtime, energy, and power for execution of a given ML algorithm.

Sampling is a very important step for data generation. Improper sampling leads to an unbalanced dataset, resulting in poor performance of the trained models. Moreover, sampling techniques can help reduce the number of data points needed to train a model without degrading the performance. In this work, we have studied three sampling techniques, including (i) Latin Hypercube sampling, (ii) low-discrepancy sequence using Sobol, and (iii) LDS using Halton.

Latin Hypercube sampling. LHS divides the parameter space into equally spaced intervals along each dimension and then randomly selects points in the intervals such that each interval is selected exactly once. During the sampling process, we maximize the minimum pairwise distance of the sampled points.

LDS using Halton. An LDS, also known as a quasi-random sequence, is a sequence of points in a multi-dimensional space that exhibits more uniform and evenly distributed patterns, as compared to random sequences. These sequences are generated using deterministic algorithms that carefully distribute points across the space to minimize undesirable patterns. The Halton [30] sequence relies on unique prime number bases to generate uniformly distributed samples.

LDS using Sobol. The Sobol [30] sequence is also an LDS that utilizes primitive polynomials and bitwise operations to generated uniformly distributed samples. In the case of a high-dimensional parameter space, the Sobol sequence is expected to exhibit a more uniform distribution with a smaller sample size than the Halton sequence.

We use the scikit-optimize package [13] to generate samples for all three sampling methods. Both LHS and LDS yield superior results compared to random sampling, particularly when dealing with smaller sample sizes. One of the benefits of LHS is its ability to uniformly sample from the parameter space with a smaller sample size than LDS. However, a limitation of LHS is that to increase the number of samples, one needs to regenerate all the samples and cannot reuse the previously sampled data points. This is because adding new points to existing LHS samples disrupts the LHS property of maximizing the minimum pairwise distance between sampled points. However, LDS, which generates samples from a sequence, only needs to extend the sequence to generate new samples. This feature allows LDS to utilize previously sampled data points when the sample size is increased. In our experiments below (Section 7), we sample backend configuration util and \(f_{{target}}\) for all designs, and architectural configuration size and number of cycles for Axiline designs.

In our framework, we employ a range of regression models. A brief description of each model is as follows.

We train GBDT, RF, and ANN models using the open-source platform H2O [42]. H2O enables fast, distributed, in-memory, and scalable machine learning-based model training. In Section 7.3, we provide a detailed step-by-step guide on how to train GBDT, RF, ANN, and Stacked Ensemble models. We train GCN using PyTorch [32] and PyTorch Geometric [11]. In Section 6, we provide a detailed step-by-step guide on how to generate graphs, and in Section 7.3, we outline the process for training the models.

Design space exploration seeks an optimal implementation of an accelerator for a specified workload, to achieve better backend PPA and system metrics. To explore the design space more efficiently, we pay more attention to a region of interest in the design space, i.e., the region that contains the “optimal” implementation. Figure 3 gives an example of how to determine the ROI. In the example, we use the Axiline platform to generate two accelerators (Design-I and Design-II) with different configurations, to implement a recommender system algorithm. Here the size, number of cycles, number of units, and bit width for Design-I are, respectively, 9, 7, 8, and 8, while for Design-II they are 17, 4, 5, and 16. We run full SP&R flows for these two accelerators under 21 different \(f_{{target}}\) values, and compute corresponding energy and runtime system metrics. Figure 3(a) shows energy versus runtime of Design-I and Design-II for different \(f_{{target}}\). The data show a typical division of the design space into three regions: (i) region of runtime where we can reduce the runtime at the cost of increasing energy; (ii) region of balance where we can achieve lowest energy without introducing too much runtime overhead; and (iii) region of energy where the energy will also increase when the runtime increases. In our work, we set the ROI to be the region of balance, which is defined for each design. To visualize the ROI in terms of \(f_{{target}}\), we show plots of runtime versus \(f_{{target}}\) in Figure 3(b).² We observe that the ROI excludes both extremely high and low \(f_{{target}}\). Then, by examining \(f_{{effective}}\) versus \(f_{{target}}\) (Figure 3(c)), we further characterize the ROI in terms of the difference between \(f_{{target}}\) and \(f_{{effective}}\): post-routed designs tend to have smaller negative slacks at higher \(f_{{target}}\) and larger positive slacks at lower \(f_{{target}}\). Given the above, we define our ROI in terms of the difference between \(f_{{target}}\) and \(f_{{effective}}\), as follows:

The ROI for Axiline, VTA, and TABLA designs is identified by the data points situated between the red and blue lines in Figure 4. In Figure 4(a), for Axiline designs, we observe that for \(f_{{target}}\) values between 0.4 and 0.7, many data points lie above the red line, i.e., outside the ROI. This indicates that some of these data points fall on the left boundary of the ROI for different Axiline designs. Similarly, the right boundary of the ROI appears to range from 1.7 to 2.2 GHz. This exemplifies how the ROI can vary and may have different ranges for different designs.

We include floorplan utilization as a separate knob to control chip area. We observe that high utilization significantly impairs the outcomes of the backend P&R tool, leading to poor PPA. Figure 4(a) presents the \(f_{{effective}}\) vs. \(f_{{target}}\) plot for the Axiline design on GF12. For \(f_{{target}}\) values of 1.03 and 1.33 GHz, the floorplan utilization hovers around 90%, which results in poor postRouteOpt performance for most of the Axiline designs. Additionally, when the \(f_{{target}}\) is very high, the P&R tool struggles to achieve the desired performance, resulting in a poor \(f_{{effective}}\). Conversely, when \(f_{{target}}\) is very low, the tool yields a higher \(f_{{effective}}\) than \(f_{{target}}\). These extreme \(f_{{target}}\) values are also undesirable as the outcomes from the P&R tool for these frequencies tend to vary significantly, making them challenging to model accurately. So, these data points are considered as outliers, since they do not correspond to relevant design points, and since their inclusion in the training dataset deteriorates the performance of the model.

We modify the two-stage inference model proposed in Reference [10] into a two-stage model based on the ROI definition above. The goal is to efficiently detect outliers and prevent them from impacting the performance of the model. As shown in Figure 5, the two-stage model functions as follows. (1) First, we generate classification labels for all training data points, each corresponding to a different design configuration, using Equation (4) to assess whether they fall within the ROI. (2) Next, we train a binary classification model to identify the data points that are within the ROI. (3) Subsequently, we train our models to predict PPA and system-level metrics, focusing only on the data points within the ROI. (4) During inference, we use the trained classification model to determine if the data points are within the ROI. (5) If the data points are within the ROI, then we apply the trained backend and system-level models to predict PPA and system-level metrics; if not, these data points are excluded. In Equation (4), \(\epsilon\) is a parameter used to define the size of the ROI. For smaller ML accelerators such as Axiline, where the deviation of the \(f_{{effective}}\) from the \(f_{{target}}\) is generally small, we set \(\epsilon\) to 0.1. For larger ML accelerators such as GeneSys, VTA, and TABLA, where the deviation of \(f_{{effective}}\) from \(f_{{target}}\) tends to be larger, we set \(\epsilon\) to 0.3.

In our methodology, we utilize the above two-stage model for DSE of both architectural and backend configurations. During this DSE process, for a given runtime and power constraints, our initial step involves the identification of the Pareto front for area and energy. This is accomplished via the MOTPE. Subsequently, the optimal configuration is selected according to a user-specified cost function.

MOTPE-based DSE. MOTPE [31] is a sequential model-based optimization technique. It iteratively builds a surrogate model to predict performance and gather additional data informed by this model. The estimator constructs distributions of good (G) and bad (B) samples, categorizing samples based on their relative position in the objective space with respect to the current Pareto front. Before forming these internal distributions, the algorithm collects multiple random samples. It utilizes a non-parametric multivariate density estimation model, commonly referred to as the Parzen window, to construct these distributions. Subsequently, the MOTPE selects the sample that maximizes the \(G/B\) ratio, which ensures superior sampling efficiency. A significant advantage of MOTPE is its capacity to handle both discrete and continuous-valued parameters, thereby facilitating its application across a broad range of optimization scenarios. When conducting DSE of ML accelerators, backend parameters such as floorplan utilization and target clock frequency are continuous in nature, whereas architectural parameters such as the number of processing units and processing elements are discrete. This makes the MOTPE-based optimization well suited for our DSE problem. For a given workload, runtime, and power constraints, we use the MOTPE-based DSE approach to find the Pareto front for area and energy. Utilizing the cost function provided in Equation (3), we then select the best design configuration of the ML accelerator. Identifying the Pareto front allows us to understand the tradeoffs among different metrics, which is independent of user-specified weight in the cost function (i.e., \(\alpha\) in Equation (3)).

We divide the data generation process into three substeps: (i) sampling of architectural parameters and RTL netlist generation; (ii) sampling of backend parameters; and (iii) PPA and system-level metric data generation.

Sampling of architectural parameters and RTL netlist generation. All platforms, namely, TABLA, GeneSys, VTA, and Axiline, are parameterizable, with corresponding tunable architectural parameters shown in Table 1. We use a variety of strategies to generate multiple configurations for each platform. Prior works on DNN accelerators based on systolic arrays [14, 29] and vector dot-products [3] report architectural parameters such as array dimension, data bitwidth, on-chip buffer size and off-chip bandwidth. For GeneSys and VTA, we use such insights from prior works to guide our choice of architectural parameters. We proportionally scale buffer size and bandwidth parameters based on array dimensions. For each array dimension, we select other architectural parameters by changing ratios of the buffer sizes, to to exercise various data reuse tradeoffs in DNNs. For example, a larger WBUF facilitates more weight reuse while a larger OBUF biases a design toward more on-chip reduction of the partial sums. For TABLA, we explore multiple configurations using variations of the structures shown in Reference [26]. For Axiline, we use Latin Hypercube Sampling for integer architectural parameters, such as dimension and number of cycles, thus achieving uniform coverage in each dimension, and we simply enumerate all the combinations for all other remaining architectural parameters. In Section 8.1, we conduct a detailed analysis of the impact of various sampling methods and sample sizes on the prediction of unseen architectural configurations, specifically for the Axiline design. In Section 8.3, we show that our model underperforms when the testing dataset falls outside the range of the training dataset. Therefore, it is essential to ensure that the range of training dataset covers the testing dataset.

Sampling of backend parameters. As mentioned in Section 4, we incorporate the floorplan utilization as another backend parameter alongside the target clock frequency. This allows for the prediction of chip area. If we were to separately sample the target clock period and floorplan utilization and enumerate all configurations, then it would significantly inflate the total number of configurations and be highly inefficient. Thus, we employ LHS to sample backend configurations. The performance of a chip is directly proportional to the clock frequency, while it is inversely proportional to the clock period. So, we sample from the target clock frequency space during backend parameter sampling. We then convert the frequency to the target clock period for the SP&R data generation. Figures 7(a) and 7(b) show the backend configurations sampled for Axiline, and for TABLA, GeneSys and VTA designs. We sample 30 backend configurations (\(f_{{target}}\), util) for training and 10 for testing. We conduct \(f_{{target}}\) sweeps and set the \(f_{{target}}\) range to ensure that the worst slack is less than 200 ps and more than \(-200\) ps. For the util sweep, we set the upper boundary at the highest floorplan utilization that leads to a DRC count less than 100, and set the lower boundary at a difference of 0.4 to 0.5 from the upper boundary. Figure 4 presents how \(f_{{effective}}\) varies with \(f_{{target}}\) for Axiline, TABLA and VTA designs. Based on the results of these sweeps, for Axiline, we sample floorplan utilization from 40% to 90% and target clock frequency from 0.4 to 2.2 GHz. For the macro-heavy TABLA, GeneSys and VTA designs, we sample floorplan utilization from 20% to 60% and target clock frequency from 0.2 to 1.5 GHz.

PPA and system-level metric data generation. After sampling the architectural configurations, generating the RTL, and sampling the backend configurations, we run commercial synthesis, place and route, and collect post-route optimization PPA metrics. We execute the logic synthesis using Synopsys Design Compiler R-2020.09, generating the synthesized netlist. We then carry out place and route using Cadence Innovus 21.1 to capture the post-route optimization PPA metrics. All of our tool scripts are available in the VeriGood-ML GitHub repository [44]. For macro-heavy designs, we employ Innovus’ concurrent macro placer to automatically place all the macros. We conduct all of our studies using the GLOBALFOUNDRIES 12LP (GF12) enablement. For Axiline, we also generate PPA metrics on the open enablement NanGate45 [45] to demonstrate the adaptability of our framework across different platforms. Once we have the post-route optimization database, we run simulations to capture the system-level runtime and energy metrics. Here, we set the workloads to be the widely-used ResNet-50 and MobileNet-v1 networks for the GeneSys and VTA designs, respectively. The workload of each TABLA or Axiline design is determined by its benchmark parameter (see Table 1). For the training and testing of the ML model, we generate around 9800 data points for Axiline across both NG45 and GF12 enablements, and approximately 400, 320, and 190 data points for TABLA, VTA, and GeneSys, respectively.

After data generation, we divide the data into training and testing datasets. Our model is trained and tested using two distinct types of datasets, categorized based on backend and architectural configurations. These are referred to as the unseen backend dataset and the unseen architectural dataset. In the subsequent subsections, we first explore the significance of having a separate validation dataset rather than solely depending on cross-validation. We then delve into the details of the unseen backend dataset and the unseen architectural dataset.

Preference for validation dataset over cross-validation. We find that one of the primary causes of the poor performance of the ANN model in our previous study [10] is related to the use of cross-validation. Using cross-validation does not necessarily ensure adequate coverage of the design space, despite the fact that the complete training dataset has this property. This leads to a bias in the model that develops during the training process, which results in high \(\mu\)APE and MAPE. As mentioned in Section 7.2, by separately sampling a validation dataset that provides a thorough coverage of the design space, we achieve a substantial improvement in the performance of the model on the testing dataset. During hyperparameter tuning, we employ the validation dataset to select the best hyperparameters and then deploy them on the testing dataset for evaluation. We apply this approach to the model training of Axiline designs. However, for TABLA, GeneSys, and VTA designs, generating the RTL netlist for different architectural configurations requires significant manual effort. Although using a validation dataset offers additional benefits, due to the substantial manual effort required, we have chosen to employ fivefold cross-validation for these designs.

Unseen backend dataset. As the name implies, the training and testing datasets encompass entirely different sets of backend configurations, though the architectural configurations within both datasets remain the same. We sample 30 data points for training and 10 for testing, making sure there is no overlap between any of these datasets. This distinct sampling helps to ensure coverage of the entire backend design space. Figure 7 displays the backend configurations sampled for Axiline, TABLA, GeneSys and VTA for training, and testing with GF12 enablement. For Axiline, an additional 10 data points are sampled for model validation during the training process.

Unseen architectural dataset. In this case, the training and testing datasets comprise entirely of different sets of architectural configurations, while the backend configurations remain consistent across both datasets. Just as with unseen backend configurations, we ensure no overlap between the training and testing datasets. For the Axiline design, the training dataset contains 24 configurations, while the validation and testing datasets each consist of 10 configurations. Again, we separately sample all three (i.e., training, validation, and testing) datasets using LHS to help ensure a coverage of each dataset over the architectural design space. Section 8.1 discusses in detail the impact of sampling training configurations using different methods for varying sample sizes.

For Axiline, architectural configurations are sampled separately for training and testing, ensuring that both datasets uniformly cover the design space. Effective performance of the trained model on the test dataset suggests that the model will be reliable during DSE. However, for TABLA, GeneSys, and VTA, the substantial manual effort required to generate RTL necessitates a random division of the dataset into a 4:1 training-to-testing ratio, based on architectural configurations.

In this work, we train and evaluate a total of 200 ML models, spanning four platforms (Axiline, TABLA, GeneSys, and VTA), five metrics (power, performance, area, energy, and runtime), five types of machine learning models (GBDT, RF, ANN, Stacked Ensemble, and GCN) and two datasets (unseen backend and unseen architectural). We tune the hyperparameters of each model except GCN, using the H2O package and a random discrete search method. Table 2 presents the hyperparameters of each machine learning model type that we tune, and for other parameters, we use the default values unless mentioned otherwise. In the following subsections, we delve into the detailed steps of hyperparameter tuning for each model.

Training of GBDT and RF. We use H2O and adopt the same strategy to train both the GBDT and RF models. Hyperparameter tuning using H2O random discrete [4] grid search is executed in two stages. In the first stage, we set the number of trees to a very large value (500 for RF and 300 for XGB) and tune the remaining hyperparameters. From the best hyperparameter configuration, we find the best \(max\_depth\) to narrow down the search space for \(max\_depth\). For RF, we also reduce the search space for mtries. The range for \(max\_depth\) is reduced to best \(max\_depth \pm 3\). For RF, we retain the mtries value determined in the first stage. In the second stage, we conduct another round of hyperparameter search with the updated search space for \(max\_depth\) in both RF and GBDT and for mtries in RF. During the random discrete search process, we choose the model with the lowest Root-mean-square Error (RMSE), computed using Equation (5) on the validation dataset of size n. In this equation, \(y_{actual}\) represents the actual value of the target metric, and \(y_{predicted}\) is the value predicted by the trained model. This optimized model is then applied to the testing dataset. During hyperparameter tuning with random discrete grid search, we set the wall time to 300 s for each stage. Therefore, the overall model training time of the GBDT and RF models is limited to 600 s or less:

Training of ANN. For the ANN model, determining an effective hidden layer configuration is crucial. In previous work [10], hidden layer configurations are generated such that all hidden layers have identical node configurations, which leaves further room for improvement. The key idea in our present work is to map the features to a higher dimensional space and then gradually reduce them to a smaller dimension. This allows the model to extract complex representations and make the data more separable. To achieve this, we apply Algorithm 2 to create the hidden layer configurations. In Algorithm 2, nodeCount represents the count of nodes in the first hidden layer, and hLayerCount denotes the total number of hidden layers. The values minP and maxP are used to constrain the minimum and maximum number of nodes in the hidden layer, respectively, to \(2^{minP}\) and \(2^{maxP}\):

Our algorithm consistently uses a power of two for the node count in each hidden layer, which is more resource-efficient. We employ the parameter values provided in Table 2 to generate all the configurations for the hidden layers. We perform the hyperparameter search for these hidden layer configurations and the set of activation functions presented in Table 2. During model training, we leverage adaptive learning rates. The model exhibiting the lowest RMSE value (Equation (5)) on the validation dataset is selected. We carry out this hyperparameter tuning of ANN models using the H2O random discrete [4] grid search. The selected model is then applied to the testing dataset for evaluation. During hyperparameter tuning, we use the default batch size of 1 and set the number of epochs to 500. The size of the training dataset is on the order of several thousands, so employing a batch size of 1 does not slow down the training process. We implement an early stopping criterion: if there is no improvement in the validation dataset for 20 consecutive iterations, the training process is stopped. During the hyperparameter search, the wall time is set to 600 s. Therefore, the overall training time of the ANN model is limited to 600 s or less.

Training of Stacked Ensemble. We employ the stacked ensemble model of H2O, where we use the trained models of GBDT, RF and ANN as base learners, with linear regression acting as meta learner. Here, only the top seven models from the hyperparameter search process of GBDT, RF, and ANN are chosen as the base learners.⁴ This approach ensures a sufficient degree of variation in the selection of base learners, while filtering out the poorly performing models.

Training of GCN. We have implemented the GCN model using PyTorch and PyTorch Geometric. The GCN architecture used to train our model is depicted in Figure 8. For both the graph convolutional layers and the fully connected layers, we use the ReLU activation function. The convolutional layers generate node embeddings, and we then employ the GlobalMeanPool function that computes the average of all the node embeddings, i.e.,

The GCN models incorporate not only architectural and backend features but also extract features from LHGs. Given that ML accelerators are inherently modular and have shared building blocks, this enables the GCN models to produce more insightful embeddings for PPA and system-level metric predictions. Figure 9 shows the t-SNE plot of the graph embeddings generated for TABLA, VTA, and Axiline designs. Here, different colors are used to plot graph embeddings of different architectural configurations. In the t-SNE plot, we see clear distinctions between different architectural configurations. Since the PPA and system-level metrics differ for these configurations, it indicates that the GCN models are well-trained.

Following the generation of graph embeddings, we input them into the fully connected layer to produce the final predictions. The configuration of the fully connected layer is generated using the getNodeConfig function, with the graph embedding size and the number of fully connected layers serving as inputs, corresponding to nodeCount and hLayerCount, respectively. While training our model, we employ the mean absolute percentage error (\(\mu APE\)) loss, defined as

Throughout the training of the GCN model, we utilize the Adam [16] optimizer with decaying learning rate; decaying factor is set to 0.7 with a patience of 5. We also implement early stopping if no improvement in the model’s performance on the validation dataset is observed for 20 consecutive epochs. Upon completion of the training, the model exhibiting the lowest validation error is chosen for testing on the test dataset.

We employ the HyperOptSearch function of Ray Tune [23] to automatically optimize various parameters such as the type of graph convolutional layer, the number of convolutional layers, the number of fully connected layers, the batch size, and the learning rate as shown in Table 2. The best configuration is determined based on the following loss function,⁵ which captures both the average and worst-case performance:During the training of our GCN model, we set the number of epochs to 400. However, most of the runs complete earlier due to the early stopping mechanism. We have observed that with 100 samples, the total runtime of Ray Tune is approximately 1,800 s.

The selection of an appropriate sampling method and sample size is crucial, as a poor choice may result in non-uniform sampling. This can subsequently lead to the development of biased models or necessitate a larger volume of data points to achieve the desired performance. To identify the appropriate sampling method and sample size, we train our models using data obtained from various sampling techniques and a range of sample sizes. Figure 10 presents the design configurations of Axiline generated using LHS, Sobol sequence, and Halton sequence. Subsequently, we assess the performance of the trained models on unseen test configurations, capturing the mean absolute percentage error (\(\mu APE\)) and maximum absolute percentage error (MAPE). Additionally, we compute standard deviation of APE (STD APE) on the test configurations to measure stability of model performance on unseen configuration across the architectural parameter space. A smaller value of STD APE indicates stable model performance on the unseen configurations.

Table 3 shows the performance of different ML models for Axiline-SVM on the testing dataset when the training data is sampled using Latin Hypercube Sampling, Sobol Sequence, and Halton Sequence with sample sizes of 16, 24, and 32. For a given sample size and machine learning model, we use bold font to indicate the result of the top-performing sampling method. From Table 3, we make the following observations.

To summarize: in the remaining portion of our experiments, we employ LHS with a sample size of 24 to generate the training dataset for Axiline designs.

We now present the results of our model’s performance for predicting backend PPA metrics, and system-level runtime and energy for both unseen backend and architectural configurations, over a set of benchmark workloads. Our performance evaluation is based on \(\mu APE\), MAPE, and STD APE. The latter helps us understand how consistently our model performs on the testing dataset.

Results for unseen backend configurations. Table 4 presents the performance of the ML model for unseen backend configurations in predicting post-SP&R PPA and system-level metrics for TABLA, Genesys, VTA, and Axiline designs, implemented on GF12 and/or NG45. For the ROI classification task, all models for the GF12 implementation achieve at least 96% accuracy and an F1 score of 0.97. For the Axiline NG45 implementation, all models achieve at least 94% accuracy and an F1 score of 0.96. These results demonstrate the excellent performance of our models in identifying whether data points belong to the ROI. In Table 4, the best-performing model based on \(\mu APE\) for each design and each metric is highlighted in bold. Based on this table and for these models, we make the following observations.

As shown in Figure 7, we sample our test dataset so that it covers the design space uniformly. High accuracy and low \(\mu APE\) on the test dataset suggests that our training dataset is large enough. Table 6 in Appendix A details the performance of GBDT, RF, and ANN models for unseen backend configurations.

Results for unseen architectural configurations. Table 5 shows the performance of the ML model for unseen architectural configurations in predicting post-SP&R PPA and system-level metrics for TABLA, Genesys, VTA, and Axiline designs, implemented on GF12 and/or NG45. For the ROI classification task all the models for GF12 implementation achieve at least 95% accuracy and 0.97 F1 score. This indicates that models are performing very well in determining whether the data points belong to the ROI. In Table 5, the best-performing model based on \(\mu APE\) for each design and each metric is highlighted in bold. Based on this table and for these models, we make the following observations.

For some unseen architectural configurations, we observe MAPE values higher than 30%. However, further investigation reveals smaller standard deviation values for the APE. This indicates that our model delivers reliable results for most of the data points. Table 7 in Appendix A details the performance of GBDT, RF, and ANN models for unseen architectural configurations. We also observe that for both unseen backend and unseen architectural configuration datasets, the GCN model outperforms other models in most scenarios. In instances where it is not the best model, it still yields results very similar to those of the best-performing model.

We have additionally studied the performance of our model in terms of extrapolation. To put it more precisely, we study how well our model performs when the test data points fall outside the range of the training dataset. For this experiment, we sample architectural configurations of the Axiline design as displayed in Figure 11, where the blue, red, and cyan data points correspond to training, testing, and validation data, respectively. For each architectural configuration, we run 30 SP&R jobs to prepare the training, testing, and validation datasets. We observe that the model performs poorly on the validation dataset and produces similar poor results for the testing dataset, confirming the limitations of the ML model we use for dataset extrapolation.

However, our training dataset covers the architectural design space for Axiline and backend design space for all accelerators, including GeneSys, VTA, TABLA, and Axiline. The count of features handled by the Axiline design is computed as \(num\_cycles \times size\). The Axiline designs are expected to handle up to 800 features [41] and our sample space already covers up to 900 features. As seen in Figure 4, the \(f_{{effective}}\) does not change for higher \(f_{{target}}\), indicating that we have effectively covered the backend design space. Therefore, even though the models fail to produce satisfactory results for the extrapolation dataset, it is not considered a drawback. Our training’s configuration space includes the entire design space, eliminating the need for the model to function outside the scope of the training configurations.

We now apply our trained models for DSE on two different platforms: Axiline and VTA. We apply the MOTPE method and our trained models for the DSE of the ML accelerator, with the objective of minimizing chip area and system energy. We also employ an additional flag during the DSE process to indicate whether the explored data points fulfill design configuration, runtime, and power requirements. The specifics of these two DSE experiments are as follows.

DSE of Axiline-SVM in NG45 enablement. We optimize the implementation of an accelerator that executes the SVM algorithm with 55 features. The flow is as follows.

During the DSE process, we vary size from 10 to 51, \(num\_cycle\) from 5 to 21, \(f_{{target}}\) from 0.3 to 1.3, and floorplan utilization from 0.4 to 0.8. Figure 12(a) displays the predicted runtime, area, and energy plot for all the data points sampled during the DSE process. The data points highlighted with red dots do not meet the ROI, power and runtime requirements. We then identify the best configuration that minimizes Equation (3). Figure 12(b) presents Pareto front of area versus energy plot. For ground truth analysis, we also generate the RTL netlist and run SP&R for each of the top three configurations, and confirm that the predicted metrics for these top three configurations are within 7% of post-SP&R values.

DSE of VTA design in GF12 enablement. We also apply our the DSE method to optimize the backend configuration for the VTA design. The process is the same as outlined above for Axiline-SVM, except we only vary \(f_{{target}}\) from 0.3 to 1.3 and floorplan utilization from 0.25 to 0.55. Figure 13(a) displays the predicted runtime, area, and energy plots for all the data points sampled during the DSE process. The data points marked with red dots do not meet the ROI, power, and runtime requirements. We identify the best configuration that minimizes Equation (3) when \(\alpha\) is 1. We adjust these values, because the units of energy and runtime in Figure 13(a) differ from those in Figure 12(a). Figure 13(b) presents Pareto front of area versus energy plot. Running SP&R on the top three backend configurations identified from the DSE confirms that the predicted metrics for these top three configurations are within 6% of the post-SP&R values.

In this Appendix, we present the performance of GBDT, RF, and ANN models, along with the Ensemble model and GCN model, for predicting PPA, as well as system-level energy and runtime for TABLA, GeneSys, VTA, and Axiline designs on GF12 or Nangate45 enablement. Table 6 details the model performance for unseen backend configurations, while Table 7 showcases the model performance for unseen architectural configurations.