Automated Software Protection for the Masses Against Side-Channel Attacks Automated Software Protection for the Masses Against Side-channel Attacks

In this section, we give an overview of our approach, which is illustrated in Figure 1. The overall approach is divided into two parts: a static part and a runtime one.

Fig. 1. The overall compilation flow. The role of the user is reduced so that our approach can be deployed as fast as possible on pre-existing code base. The polymorphic functions’ binaries are generated at runtime by their respective SGPCs.

The user starts by annotating the target functions to be secured with polymorphism. Then, he chooses a configuration of polymorphism. The choice of such a configuration will be discussed in Section 6.2. The annotated C file (file.c in Figure 1) is compiled by our tool, Odo, into another C file (file.odo.c in Figure 1). The code of each function to secure has been replaced by (1) a wrapper that interfaces with the rest of the code, and (2) a dedicated generator. The wrapper handles the calls to the dedicated generator and to the produced code. The wrapper is also in charge of making the link between the original function, identically called by the rest of the code, and the polymorphic instance, i.e., to call the polymorphic instance with the right arguments and to return its return value. One dedicated generator is created for each polymorphic function; the implementation of the generator, which is automatically generated by Odo, is entirely dependant on the initial code of the function, and also depends on the polymorphic configuration chosen by the user. We refer to these generators as SGPC (Specialized Generator of Polymorphic Code) later on.

The produced C file is then compiled along with the Odo-runtime library, which provides the architecture support and the code transformation framework, into a binary file that is then loaded on the platform. A dedicated zone in RAM is statically reserved to host the runtime-generated code of a polymorphic function. This memory zone is called an instance buffer. In Figure 1, two functions are annotated in the source code. As generators are specialized, there are one SGPC and one instance buffer for each annotated function.

At runtime, the SGPC is called by the wrapper whenever the code has to be regenerated to generate a new polymorphic instance in the instance buffer. Calling it regularly gives the property of polymorphism to the function: Its code changes at each call. The frequence of the regenerations can be controlled by the user. We call regeneration period, denoted as $\omega$, the number of consecutive executions of the same polymorphic instance before a new regeneration. When the SGPC is called, the permissions of its instance buffer are switched from execute-only to write-only, and switched back at the end of generation. This ensures that the instance buffer is never writable and executable simultaneously (Section 5.3).

In this section, we present how Odo generates a SGPC for a function chosen by the user. The runtime code transformations implemented to leverage code polymorphism are presented in the next section. Odo is a standard compiler, based on LLVM. It performs a normal compilation to produce a suite of assembly instructions corresponding to the targeted function body, and then it generates the SGPC dedicated to this function from this suite of instructions. The code of a SGPC is composed of a sequence of calls to binary instruction emitters that targets the sequence of ARM assembly instructions generated by the normal compilation flow.

In Odo, the generation of SGPCs is executed by a new backend, which is fully identical to the ARM backend except for the code emission pass. As only this pass differs between our backend and the ARM backend, the suite of instructions from which a SGPC is built benefits from all previous optimisations of the compiler. The emission pass emits the C code of the SGPCs of the annotated functions instead of emitting assembly code. The SGPCs produced with polymorphism activated are quite similar to the ones obtained with polymorphism disabled, the differences are highlighted in the next section.

Listing 3 represents the code generated by Odo for the function annotated in Listing 1, when polymorphism is deactivated. This code is composed of a SGPC for f_critical named SGPC_f_critical and a new function f_critical, which interfaces with the rest of the code. The SGPC of f_critical, SGPC_f_critical, is designed to emit a suite of binary instructions identical to the assembly code that LLVM would have generated for the function (Listing 2). The SGPC is composed of a suite of function calls to the Odo-runtime library, including one call for each binary instruction to be generated. The encoding of all the ARM Thumb1 and Thumb2 instructions are available through the library. For instance, in Figure 3, the call eor_T2(r[4],r[1],r[0]) writes in the instance buffer the binary instruction eor r4,r1,r0. The suffix _T2 indicates that the Thumb2 encoding is used. All the binary instruction emitters defined in the Odo-runtime.a library (Figure 1) take the instruction operands, physical register names and/or constant values, as parameters, as would be for regular machine instructions. This enables the SGPC to change the operands from one generation to another one (e.g., r[4] can refer to a different physical register).

In addition, the SGPC raises interruptions at the very beginning and the very end of its execution so that the interrupt handler changes the access permissions of the dedicated instance buffer. This is illustrated by the calls to raise_interrupt_rm_A_add_B functions in Listing 3 . In practice, these interrupt calls are inlined using assembly primitives. The mechanisms enabling memory permissions management are presented in details in Section 5.3.

Listing 1. Original C file annotated by the user.

Listing 2. ARM instructions suite generated by LLVM for f_critical.

Listing 3. The C file generated by Odo when all polymorphism transformations are turned off.

Listing 4. The C file generated by Odo when all polymorphism transformations are activated.

Listing 5. Example of an ARM assembly code generated by SGPC_f_critical with all polymorphism transformations activated.r5 is used instead of r4 due to the register shuffling, a semantic variant is inserted for the exclusive or and several noise instructions were inserted. No instructions were shuffled due to the dependencies between the instructions.

In this section, we present the code transformations used to generate a different code each time the SGPC is called. We explain how compilation flow is leveraged to enhance the code transformations at runtime without requiring costly code analysis. We also present how the code of the SGPC generated by Odo differs when these transformations are activated.

Five different transformations can be used by the SGPCs to vary the code of polymorphic instances: (1) register shuffling, which is a random permutation among the callee-saved registers, (2) instructions shuffling, which consists in emitting in a random order independent instructions, (3) semantic variants, which refers to a random replacement of some instructions by a sequence leading to the same result, (4) noise instructions, which are useless instructions inserted in between the original instructions of the function, and (5) dynamic noise, which consists of a sequence of noise instructions preceded by a random forward jump so that the number of executed instructions varies at each execution.

Listing 4 shows the output of Odo for the input of Listing 1, when all polymorphic transformations are activated. Listing 5 is an example of a polymorphic instance that can be generated by the SGPC at runtime.

Register Shuffling. Contrary to what was proposed previously [16], where random register allocation was performed at runtime, here register allocation is done statically by the compiler, resulting in a better allocation and a faster runtime code generation. SGPCs make registers vary by relying on a permutation, which is done at the beginning of each code generation (shuffle_regs in Listing 4), between the general purpose callee saved registers (r4-r11), and which is a fast operation. Only instructions that encode registers on 4 bits are used. The effects of register shuffling are illustrated in Listing 5: in this example, register r5 is used instead of register r4.

Instruction Shuffling. This transformation aims at shuffling independent instructions prior to their emission in the instance buffer. The emitted binary instructions are first stored in a temporary buffer, named shuffling buffer, of configurable size (32 instructions in our experiments). Each binary instruction is associated with its defs and uses registers. Before an instruction is added to the shuffling buffer, Odo-runtime performs a dataflow analysis, starting from the last instruction in the shuffling buffer, to compute the list of possible insertion locations. A random location is then selected among the possible insertion locations. The shuffling buffer is flushed into the instance buffer at the end of each basic block or when it is full. The instruction shuffling transformation is transparently carried out by Odo-runtime, the code Odo generates for the SGPC is identical whether the transformation is activated or not.

Semantic Variants. Some instructions can be replaced by a suite of instructions that achieves the same result and leaves all the originally alive registers unmodified (status registers included). Odo-runtime currently provides semantic variants for instructions that are frequently used in cryptographic ciphers to manipulate sensitive data: instructions belonging to the families of eor, sub, load and store, it can be easily extended. Currently, each original instruction can be replaced by 1 to 5 variant instructions. Odo generates specific function calls to the Odo-runtime library for the emission of these instructions when semantic variants is activated. In Listing 1, green bold calls are in charge of the emission of semantic variants. At runtime, the SGPC emits the binary code of one variant randomly chosen among available ones (which include the initial instruction). The call variant_eor_T2(r[4], r[1], r[0]); from Listing 4 can generate the original instruction eor r4, r1, r0 or, e.g., a sequence eor rX, r0, #rand; eor r4, r1, #rand; eor r4, r4, rX (as illustrated in Listing 5) where rX is a randomly chosen free register and #rand is a random constant. Semantic variants for arithmetic instructions (e.g., sub and xor) are based on arithmetic equivalences, variants for stores use smaller stores like store-bytes or store-halfwords, and variants for loads use unaligned loads in addition of load-bytes and load-halfwords.

Noise Instructions. We call noise instructions functionally-useless instructions that are generated in between the useful instructions. The insertion of noise instructions is performed by the calls to gennoise (in blue italic in Listing 4). Similar to other transformations, Odo generates such calls only when this code transformation is activated.

The side-channel profile of noise instructions should be as close as possible to the profile of useful instructions, so that the attacker cannot distinguish them and filter them out from the side-channel measurements [19]. We selected noise instructions among instructions that are often used in programs, such as addition, subtraction, exclusive or, and load. The user can specify a particular range of addresses for the loads, which can allow random loads from the AES SBox for instance. Otherwise, a small static random table is used.

Each insertion of noise instructions is guided by a probability model. Odo-runtime currently offers two different configurable models. In both models, a random draw determines if noise instructions are inserted or not. The models are presented in Table 1. The number p is the probability of insertion of one or more noise instructions, and P[X=i] represents the probability of inserting i noise instructions. The parameter N controls the maximum number of noise instructions that can be inserted at once. The first model, named low-var, follows a uniform probability law, combined with the random draw of probability p. It has a low variance, which implies that the overall execution time of the function will always remain relatively close to the theoretical mean. The second model, referred as high-var, was specifically designed to have a much higher variance and a comparable mean. It is based on a binomial probability law. The number of inserted instructions, if not null, is 2 to the power of the number obtained from the binomial law. The variables N and p as well as the model can be chosen by the user when choosing a polymorphism configuration. The mean value of a model impacts the overall execution time therefore it should be kept low for the sake of performance, whereas having a high variance increases the attacks complexity [28]. The low-var model is interesting for time constrained applications where the execution time should not vary too much, otherwise the high-var model should be preferred. The high level of configurability of the insertion of noise instructions allows the user to tune the level of variability according to his will.

At every insertion of one noise instruction, the SGPC randomly chooses one instruction among add, sub, eor, and load instructions. Then, it randomly chooses the operands and allocates a free register for the destination register.

Dynamic Noise. We call dynamic noise a dynamic mechanism that provides a variable execution from one execution to another even without runtime code re-generation. We use a sequence of noise instructions whose starting point varies from one execution to another thanks to a random branching mechanism: A forward jump whose size is randomly chosen precedes the noise instructions sequence and skips a random number of noise instructions of the sequence. Such sequences for dynamic noises are inserted by the SGPC during runtime code generation following the same procedure as the insertion of noise instructions. The difference resides in the fact that at every insertion of one noise instruction, the SGPC randomly chooses to insert either an add, a sub, an eor, a load, or a dynamic noise sequence instead. Each time the SGPC is executed, different sequences of dynamic noise are generated, at variable locations in the code of the generated polymorphic instance.

This transformation has two advantages. First, it enables us to partly decorrelate the executed code to what happens during the generation, preventing an attacker to gain precise knowledge of the generated code during code generation. Second, as the execution of the same polymorphic instance varies, it enable us to lower the constraints on the frequency of regeneration owing to security requirements. As a consequence, the approach can be used even on systems that cannot afford to regenerate the code too frequently.

The jump size of every jump associated with a dynamic noise sequence of a polymorphic instance must be efficiently determined and must randomly vary from one execution of the polymorphic instance to another one. The size of all jumps associated with dynamic noise sequences of one polymorphic instance should also be different during the same execution. To this end, a register is reserved to hold a random number used to efficiently compute a random jump size at every execution of every dynamic noise sequence throughout an execution of a polymorphic instance. Also, the number of instructions in a dynamic noise sequence can only be a power of two. The jump size is then computed by masking the random value held in the reserved register with an immediate, by using a Boolean and. Figure 2 shows an example of such sequence with four noise instructions. The part before the bx handles the branch offset,¹ and four randomly chosen noise instructions are generated after the bx. Although the number of noise instructions in the sequence is fixed, it is configurable by the user, even within a function. As an example, in our experiments, we selected a dynamic noise sequence size of 32 instructions at the beginning and at the end of the function, and of four instructions in the middle. The rationale behind this choice is the need of a higher variability at the beginning and the end of the function making the synchronisation with the function execution more difficult.

Fig. 2. Example of sequence of instructions produced with dynamic noise. The value of Rrand changes regularly.

The register that holds random values is managed as follows. A dedicated place is reserved in memory to hold a seed value that will vary from one execution to another one. At the beginning of every execution of the function (polymorphic instance), the stored value of the seed is loaded to initialise a fast PRNG. The fast PRNG is then used to set a new random value in the reserved register. Then, throughout the execution, the register value can be updated by some other noise instructions (e.g., a noise addition instruction inserted by the SGPC can add a random immediate to the register). This makes the value of the register change throughout the execution of the function. Finally, at the end of any execution of the function, the value of the register is stored, it will be used as a seed for the fast PRNG at the beginning of the next execution.

Theoretical Number of Variants. To give an idea of the variability achievable by our approach, one can easily compute an underestimate of the theoretical number of variants $N_v$. Considering only the insertion of noise instruction, with the low-var model, $N_v \ge (\sum _{i=0}^{N}4^i)^{number\_instructions-1}$. The 4 comes from the fact that noise instructions are selected among four different instructions (add, sub, eor, load). Taking p=1/7 and N=4 this gives $N_v \ge 341^{number\_instructions-1} \gt 6\times 10^{22}$ for a code of only 10 instructions, and roughly $10^{704}$ for a code of 278 instructions as the aes T-table used later in the experimental evaluation. This number is an underestimated number of variants as it does only take into account the classic noise (not the dynamic noise nor the other transformations), and it does not take into account the randomness used within the noise instructions (as their immediate values are randomly chosen).

Management of Register Availability. The insertion of noise instructions and the use of semantic variants may require free registers. A liveness analysis performed statically by the compiler is used to allocate registers at low cost at runtime for these instructions.

During the compilation of SGPCs, Odo performs a static register allocation ignoring any polymorphism aspect. Then, Odo performs a backward register liveness analysis right before code emission of the SGPC. During the code emission of the SGPC, Odo emits additional calls throughout the SGPC's code to transfer the liveness information. These calls indicate which registers are free (or not) in between two useful instructions. In our SGPC example in Listing 4, add uses r1, thus r1 is alive right before this instruction. Then sdiv defines r1 without using it. Thus r1 is dead before the sdiv instruction and alive right after. As a consequence, r1 is free to be used (written) between the add instruction and sdiv one.

Thanks to these calls, the SGPC is made aware of the liveness state at any point of the program. It can then select free registers when needed. All registers allocated for the noise instructions and for the semantic variants are chosen randomly among free registers. As the extra registers used in the sequence of instructions resulting from these polymorphic transformations are dead immediately after their use, the static liveness analysis remains unmodified whatever the extra registers used.

Branch Management. Target offsets for branch instructions are computed at runtime, as the insertion of noise instructions and the use of semantic variants make the size of the code vary. If the offset becomes too large for the initially selected encoding, then the SGPC chooses another encoding that allows larger target offsets.

The allocated size of an instance buffer is computed during the generation of the associated SGPC, by computing the size required for useful instructions and the size required for noise instructions. For useful instructions, Odo computes the sum $S_u$ of the size of the useful instructions, considering the largest semantic equivalents in case of available semantic variant. For noise instructions, Odo computes the size $S_n$ to allocate by considering the probability law SP that results of the $n_i-1$ draws of law P (the law used to determine the number of noise instructions to be inserted), where $n_i$ is the number of original instructions. Given SP, we can compute the size to allocate so that the probability of having an overflow is below a given threshold: this size $S_n$ corresponds to the size of a noise instruction multiplied by the smallest integer $i$ that checks the condition $\sum _{j=i+1}^{\infty }SP[j] \lt threshold$.

Odo automatically computes SP from the knowledge of P and then finds the appropriate size to allocate, considering either a threshold provided by the user or a default threshold set to $10^{-6}$. As a result, with the default threshold, the probability of directly generating a code that fits into the allocated memory is more than 999,999 chances over a million, whatever the original size of the code. Thus, the probability to introduce an exploitable bias in the probability models used for the polymorphic runtime code transformations is controlled such that this could not lead to an exploitable vulnerability. Moreover, the SGPC prevents buffer overflows at runtime, as explained in next section, to guarantee that the code always fits into the allocated instance buffer.

The gap in terms of size that results from the proposed allocation policy instead of a worst case allocation policy is huge. Figure 3 illustrates that the gap is asymptotically constant as the number of original instructions increases; for the high-var model considered here, and considering one probability draw following P in between each pair of consecutive instructions, the difference of the allocated sizes represents about 58 bytes for each original instruction of the function, while about 10 bytes are saved for each original instruction for the low-var model. Considering an original function of 200 instructions, this results in a difference of 2kB for the low-var model and 11.6kB for the high-var model considered.

Fig. 3. Difference in terms of bytes per instruction between our policy (with threshold = $10^{-6}$) and worst case policy. For the high-var model taken as an example, this difference is approximately equal to 58 bytes per instruction. For a code of 500 instructions, this represents a difference of 29kB.

$S_u$, the maximal size of the useful instructions, is statically computed by Odo. At runtime, the SGPC initialises with $S_u$ a variable in charge of keeping track of the remaining needed space for the useful instructions and the longest semantic variants. This variable is decremented throughout the code generation, after the emission of each useful instruction. Moreover, at every generation of noise instructions, the noise instructions generator computes the maximum number of instructions it can insert by considering this variable and the available buffer space. This information enables the runtime to constrain the generation of noise instructions to guarantee that no overflow can occur.

Runtime code generation requires that code buffers are accessed with write and execute permissions, possibly exclusively. However, in embedded systems, write permissions are systematically disabled on program memory to prevent the exploitation of buffer overflows attacks. To overcome this issue, JITs typically provide an access to program memory exclusively with write or execute permissions [12, 13, 24]. In this work, we follow the same approach, but in addition the buffer of the polymorphic instance is protected so that only the legitimate SGPC can write into it.

In this section, we propose a mitigation technique based on the facts that the instance buffers are statically allocated, and that each instance buffer has a unique associated SGPC. We rely on the memory protection unit (MPU) of the target platform to switch access permissions related to the instance buffer from execute-only to write-only (and vice-versa) whenever needed. By default, a code buffer allocated for a function only has the execute permission. At the beginning of the SGPC execution, the SGPC raises an interruption to be granted by the write permission (Figure 4). The interruption handler checks the address where the interrupt has been raised. If the check passes, then it exchanges the execute permission with the write one only for the instance buffer associated with the requesting SGPC. Thanks to the static allocation of the instance buffers, the interrupt handler is made aware of which memory zone is associated with which SGPC (to allow a switch of permissions only for correct pairs of interruption address and memory zone) and of the addresses of each instance buffer (to switch the permissions for only a buffer zone). At the end of the SGPC, the write permission is removed and the execution permission added by following the same principle. This solution is lightweight (see Section 6.3.2), and can be easily extended to systems that provide a MMU instead of a MPU. It is suitable only if the system does not have preemptive multitasking, however.

Fig. 4. Permissions handling using interruptions and the MPU (Memory Protection Unit). The instance buffers are never writable (W) and executable (X) simultaneously. Their permissions can be switched thanks to the static allocation of the instance buffers and the specialisation of SGPCs that allow comparing the address where the interrupt has been raised to hardcoded values.

We discuss now the cases where the system support is different from the one we used. For systems with OS that have preemptive multitasking, the OS must be in charge of managing the access permissions to guarantee exclusive access to the code buffers. Just as the interrupt handler in the presented approach, the OS will take advantage of the information statically available to perform legitimacy validation. For any platform without Memory Protection Unit (MPU) nor OS, control flow integrity techniques can be used to ensure that an instance buffer can (1) only be modified by the dedicated SGPC and (2) only be jumped to from the address where the polymorphic code is called, e.g., in the case of Listing 4 the address corresponding to the call code_f(a,b). The principle is to insert checks before each stores and each branches (direct or indirect) to verify the validity of any write to a code buffer and of any jump into a code buffer. This idea was presented in the CFI extension called SMAC presented by Abadi et al. [1], and induces a much higher performance cost.

We considered a constrained embedded platform. We used a STM32VLDISCOVERY board from STMicroelectronics, fitted with a Cortex-M3 core running at 24MHz, 8kB of RAM, and 128kB of flash memory. It does not provide any hardware security mechanisms against side-channel attacks.

Our setup for the measurements of electromagnetic emission includes a PicoScope 2208A, an EM probe RF-U 5-2 from Langer, and a PA 303 preamplifier from Langer. The PicoScope features a 200MHz bandwidth and a vertical resolution of 8 bits. The sampling rate is 500Msample/s (which gives 20.83 samples per CPU cycle), and 24,500 samples were recorded for each measurement.

For the study on AES below, both for the t-test and CPA, a trigger signal was set via a GPIO pin on the device at the beginning of the AES encryption, and after runtime code generation by the SGPC, to ease the temporal alignment of the measurement traces. We verified that our measurements covered the full time window of interest, both for the reference and for the polymorphic implementations of AES. Note that the SGPC does not manipulate the secret encryption key, and hence would not be vulnerable to the side-channel attacks used here. Note also that our trigger setup makes the attack easier than it would be in practice for an attacker, as she would have to align the measurements.

Odo is based on LLVM 3.8.0. All C files produced by Odo were compiled using the -O2 optimisation level, which offers a good compromise between code size and performance. Because of the limited memory available on our target, we had to compile the rabbit and salsa20 benchmarks using the -O1 optimisation level to produce code whose size is more suitable for the platform. All the programs executed by the platform (the initial ones or the ones generated by Odo) were cross-compiled with the LLVM/clang toolchain in version 3.8.0 using the compilation options -O2 -static -mthumb -mcpu= cortex-m3. Execution times were measured as a number of processor clock cycles, . The size of the programs (data, text, and bss sections) was measured in bytes with the arm-none-eabi-size tool from the gcc toolchain.

As previously explained, the level of variability provided by Odo can be configured with transformations to use and their potential parameters. We first analyse the performance and security of 17 different configurations on the AES T-table, and we discuss the possible trade-offs between security and induced overheads. Then, we apply four polymorphic configurations for hardening 15 representative benchmarks (Table 2). These four selected configurations include a configuration with polymorphism disabled, and three configurations using different variability options. They are used to evaluate the performance and code size overheads for none to high variability at runtime.

In the following, configurations are designated with acronyms of the transformations used, presented in Table 3.

This section presents a performance and security study of the AES implementation from the mbed TLS library [28] (AES T-table in our benchmarks). Nowadays, this implementation is widely used in many embedded systems from IoT devices to mobile and desktop computers, and its original implementation does not feature any countermeasure against side-channel attacks. The security against side-channel attacks of the AES cipher has been studied for several years. It is often used as a reference for security evaluation .

6.2.1 T-test-Based Security Evaluation.

Presentation of the t-test. The t-test is a statistical method applied on two sets of side-channel measurements to determine if the two sets are statistically distinguishable. In the case of side-channel attacks, the evidence of distinguishability reveals the presence of an information leakage, which could potentially lead to a successful attack, for example, by means of a CPA. The t-test tries to differentiate the means and standard deviation of the two sets of measurements by computing values called t-statistic. These values are computed all along the measurement traces. If they remain in between ]$-4.5$,4.5[, then the two sets are considered to be statistically non-distinguishable with a confidence of 99.999% [22, 34].

In this article, we use the non-specific t-test [34], because it is independent of the underlying architecture and of the model of an attack. The t-test was performed by measuring the electromagnetic emission of our device during the execution of two randomly interleaved groups of $10^4$ encryptions. The two groups of measurement traces gather the electromagnetic emissions retrieved during encryptions of random plaintexts and fixed plaintexts, respectively.

Analysis of the Effects of the Transformations on the t-value. We performed 10 times the t-test, for 17 different configurations to evaluate the resulting security of the AES T-table on our platform. For all configurations, the period of regeneration was set to 1. For each process, we picked the maximal t-value (in absolute). In the following, t-value refers to the maximal for one t-test. When this value is below 4.5, the configuration passes the t-test.

Figure 5 shows a violin plot of the t-values obtained for the 10 t-tests performed for each configuration. A violin plot shows the minimal, maximal as well as the median of the maximal t-values obtained during the 10 t-tests as well as the distribution of the maximal t-values. Without any protection, the t-test fails and the maximal t-values are very high: the configuration none exhibit a median of the maximum t-values of 110. The five polymorphic transformations we deploy have two goals: to introduce desynchronisation (instruction shuffling, semantic variants, noise and dynamic noise) and to change the profile of the leakage (register shuffling, semantic variants, noise and dynamic noise). As more transformations get activated, the number of passed t-tests increases, and the maximal t-values strongly decrease. As polymorphism does not remove the leakage, one could expect all the t-tests to fail. However, we see that as the variability of the code grows, the maximal t-values decrease progressively, and that they quickly reach a point where more t-tests pass than fail. One of the configuration even passes all the 10 t-tests, which means that all t-tests failed to detect the hidden leakage. Note that we limited the number of configurations for the sake of clarity, but as our approach is fully configurable, one could get other configurations that are close to the studied ones, or with even more variability.

Fig. 5. Maximum t-values obtained for 10 t-tests for 17 different configurations. The distribution of these t-values is represented, as well as the minimum, maximum and median. The configurations are sorted by the medians obtained. Several configurations passed the t-test more times than they failed it, and one configuration passed all the t-tests.

Each of the polymorphic transformations used in isolation has a different impact on security (Figure 5). Register shuffling seems to have very little impact on the t-value on our platform but may be of interest for other platforms as the register index can have an impact on measurements [35]. Instruction shuffling has an higher impact on reducing the t-value than register shuffling, but has a lower impact than semantic variants. Noise instructions have a higher impact than semantic variants on reducing the t-value, and dynamic noise has the highest impact. When transformations are combined, the resulting effect on the t-value is harder to analyse, their combination can exhibit some complex interactions. For instance, dynamic noise could lower the effect of instruction shuffling as it disables instruction shuffling around the dynamic noise sequences. Yet, alone or combined, dynamic noise seems to be the one that has the higher impact on the t-value: it clearly increases the number of passed t-tests compared to noise instructions, as DN1 and DN2 pass more t-tests than N1 and N2, respectively. We also note that using a stronger noise model (high-var instead of low-var) improves the observed security; DN2 and N2 show a better security than DN1 and N1, respectively.

The best configuration is the one where all transformations are activated. However, as the impact of transformations depends on the targeted application (mix of instructions, available semantic variants, etc.), the impact of a configuration on security depends on the platform and also on the application.

Analysis of the Effects of Dynamic Noise on the t-value as the Period of Regeneration Increases. We study more precisely the effect of dynamic noise on the observed maximal t-value. In particular, we show that dynamic noise enables the user to increase the period of regeneration way more than the user could using non-dynamic noise.

Figure 6 shows how the median of the maximum t-values gathered on 10 t-tests evolves as $\omega$ grows, for the N1, N2, DN1, and DN2 configurations. Please note the log scales, both for y-axis and x-axis. Configurations DN1 and DN2 show better security than N1 and N2, respectively, for all the tested period of regenerations. In addition, the configurations where dynamic noise is activated maintain the security well better than the ones where it is not activated. For example, while N2 and DN1 show similar values for small periods of regenerations, the security provided by N2 starts to drop from a period of 100, while the security provided by DN1 starts to drop from a period of 1,000. The DN2 configuration exhibits an even greater ability to maintain the security level when the period increases, as the t-values observed start to increase significantly only for periods greater than 8,000.

Fig. 6. Observed maximum t-values (taken as a median of the maximum t-values of 10 t-tests) for configurations without and with dynamic noise, in function of the period of regeneration. The configurations with dynamic noise show overall better security, and better preserve security as the period regeneration increases.

Thus, the dynamic noise makes the choice of the period of regeneration wider, which may be useful to lower the generation cost.

6.2.2 Performance and Code Size Overheads. We discuss in this section the impact of the different transformations on performance and code size. We measured performance overheads in terms of relative execution time w.r.t. the reference code with and without taking into account the generation cost. Execution time overhead denotes the ratio of the averaged execution time of $10^4$ variants without the code generation cost to the execution time of the reference code. Global overhead refers to the ratio of the execution time averaged with $10^4$ variants, generation time included, to the reference execution time. For evaluating the global overheads, we considered different regeneration periods. We also measured the code size overhead as the relative code size w.r.t. the reference code.

Execution Speed and Code Size. Table 4 presents the execution time overheads, global overheads with various period of regeneration, generation time in clock cycles, and size overheads obtained for the 17 configurations.

Table 4. Overheads Obtained in Execution Time and Size for All Configurations

The global overheads are prohibitive when $\omega =1$, this period of regeneration is probably of interest only if the generation cost can be hidden during waiting times. Yet, the global overheads become more reasonable as the period of regeneration is increased.

The different transformations influence differently the overheads. Activating instructions shuffling in addition to some other transformations roughly doubles the generation time. For instance, the generation time is 82,077 cycles for RS+SV+DN1 163,873 cycles for RS+IS+SV+DN1, and 107,820 cycles for RS+SV+DN2 218,909 cycles for RS+IS+SV+DN2. Yet, this transformation has very little influence on execution time overhead. It also has little influence on size overhead. Thus, it is of interest if generation cost can be hidden, or if the period of regeneration is large enough to minimize its impact on performance. Activating register shuffling has a low impact on both execution time and the lowest impact on the global overhead as it benefits from the static analysis performed during the generation of the SGPC.

Semantic variants impact both the execution time, generation time and size overhead. Its impact has to be considered specifically for a use-case, as from one use-case to another the number of instructions (and their positions in the code) for which variants are available varies. Finally, the overheads due to noise and dynamic noise depend a lot on the probability law P used. For instance, the frequency at which the generator executes the code responsible for inserting strictly more than 0 noise instructions directly depends on the value of p.

Impact of Dynamic Noise on Overheads. Figure 7 shows the global overheads obtained with and without dynamic noise when $\omega$ varies from 1 (top left) to 10,000 (bottom right) plotted as a function of the security provided (maximal t-value). The sweet spot is at the bottom left, where hardened codes are the most secure and have the lowest overheads.

Fig. 7. Global overheads obtained with increasing $\omega$ values with and without dynamic noise in function of the obtained maximal t-value. The configurations with dynamic noise show similar or better overheads at a given security level compared to the ones with classic noise. Dynamic noise allows us to reach much better security at a given overhead.

The configurations with dynamic noise get closer to the sweet sport than the ones with classic noise. In particular, at a given global overhead, they offer a much better security than the configuration with classic noise. This shows that dynamic noise relaxes the security constraints on the period of regeneration. Thus, dynamic noise allows to reduce the regeneration period i.e. the code generation cost.

6.2.4 CPA Based Security Evaluation.

Methodology. We performed a first order CPA against both the reference implementation and an implementation protected using the low configuration. The considered attack targeted the output of the first SubBytes function of the AES encryption. The conducted attack aimed at the retrieval of the first byte of the key as retrieving the other key bytes can be similarly performed with a similar attack complexity. We used the hamming weight as model of the EM emission of the SubBytes.

Results. Figure 8 presents the success rate obtained for CPA against both the reference unprotected AES and an implementation protected by Odo with the configuration low. The success rate represents the statistical proportion of attacks to succeed given a number of traces. A success rate of 1 means that all attacks succeed. The results show that the reference implementation is highly vulnerable, as a success rate of 0.8 (80% of attacks are successful) is reached with about 290 traces on our test platform. Such a number of traces is considered as very low for side-channel attacks, even on unprotected implementations. Furthemore, using 290 traces, the correct key leaks with a correlation value of 0.53, which suggests that our measurement setup provides very good attack conditions.

Fig. 8. Success rate of CPA for both the unprotected implementation and the configuration low. The number of traces required for the attack is multiplied by $1.3 \times 10^4$, the execution time is multiplied by 2.5 (including generation cost).

The implementation protected by Odo with the configuration low is far more secure, as $3.8 \times 10^6$ traces have to be collected to reach a success rate of 0.8, in the same experimental conditions as for the reference unprotected implementation. Compared to the reference, that represents $1.3 \times 10^4$-fold more measurements.

The performance evaluation has been conducted by considering 15 different test cases, presented in Table 2. The benchmarks considered for the evaluation are mostly cryptographic benchmarks, either from the mbedtls library [28], the eSTREAM project [22] or a home-made 8-bits implementation of AES. As any C function could be secured by Odo, these selected cryptographic functions only represent a tiny panel of the possible usages of our approach. Hence, we also considered the bytecompare function from a verifyPin implementation from FISSC [18]. We evaluated these test cases against the four polymorphic configurations previously selected. We first analyse the execution time overhead (of the polymorphic variants). Then, we present an analysis of code generation speed and size overheads.

6.3.1 Execution Time Overhead. Figure 9 presents the execution time overhead (as defined in Section 6.2.2) for the 15 benchmarks. Depending on the benchmarks and on the configuration, execution time overheads range from 1, which means that the hardened code executes as fast as the reference one, up to 7. Execution time overhead depends on the instruction mix of the initial code. First, as semantic variants are not available for all instructions, it impacts more the performance of some codes than others. Second, as the code is generated in RAM, in our platform, instruction fetch and data loads use the same bus. As a consequence, load instructions take a longer time to execute than if the code was in Flash. More precisely, when stored in RAM a Thumb 1 (16 bits) load instruction takes 1.5 cycles in average and a Thumb 2 (32 bits) load instruction takes 2 cycles (instead of 1 cycle when stored in Flash memory).

Fig. 9. Execution time overhead compared to statically compiled version (clang 3.8.0 -O2).

The overheads observed in Figure 9 indicate that the code produced remains generally efficient considering the amount of variability that it gets. The overall impact of execution time overhead for an application that has one or several polymorphic functions depends a lot on the proportion of the execution time initially spent in the transformed functions. The different configuration possibilities allow to adapt the polymorphism to the application requirements.

6.3.2 Generation Speed. To analyze the cost of code generation, we correlated the number of useful instructions to the time needed for code generation for all the considered configurations (Figure 10). We also measured the minimal cost of code generation, which can be obtained with a call to an SGPC with no instruction to generate. This minimal cost is about 1,500 cycles. This cost includes the time taken for changing the instance buffers permissions with the MPU ($\approx 100$ cycles per generation). Figure 10 shows that the generation time evolves almost linearly with the number of useful instructions to generate. The reader should note that the number of useful instructions only depends on the source program and the optimisation level but does not depend on the polymorphism options. The slopes of the trend lines in Figure 10 indicate the mean number of cycles required to generate one useful instruction. For the configuration high, the generation of each instruction requires about 709 cycles and only 22 cycles per instruction when no polymorphism is introduced. As an indication, static compilers like LLVM take about 3 million cycles per instruction [11], while the specialized code generator deGoal requires 233 cycles per instruction [10, 11]. Thus, the runtime code generation is really efficient.

Fig. 10. Generation time vs. number of useful instructions generated. The slopes represent the cost of generation per useful instruction. Our approach can easily scale as the functions get bigger.

6.3.3 Size Overhead. Figure 11 illustrates the size overhead computed as the ratio between the size of the protected binary and the size of the unprotected binary. Each bargraph gives the breakdown of the .text, .bss, and .data sections. The results show that increasing the variability increases the size overhead. This is due to several factors. First, as new mechanisms are enabled, the library and SGPC codes that handle these mechanisms are added into the binary, which impacts .text sections. Second, a greater polymorphic variability generates larger instance buffers, as noise instructions are more probable or as the use of semantic equivalences requires to keep more space for individual instructions, which impacts .bss sections. The increase of the .data sections is due to the use of a shuffling buffer for instruction shuffling, and to some private data of the Odo-runtime library.

Fig. 11. Code size overhead compared to statically compiled version (clang 3.8.0 -O2).

The overhead in code size may appear too high for constrained systems; however, we considered only benchmarks that correspond to one functionality of an embedded system. By limiting the part of the embedded software to be secured for a given product to the minimum one, the overhead should be far lower. Also, if several polymorphic functions are deployed on a same platform, the library could also be shared among the SGPCs. Moreover, all benchmarks were able to fit in our platform, which has 8kB of RAM and 128kB of flash memory.