HeterogeneousRTOS: A CPU-FPGA Real-Time OS for Fault Tolerance on COTS at Near-Zero Timing Cost

Ionizing particles in the atmosphere may strike circuits causing Single Event Upsets (SEU), affecting the output correctness. Critical real-time systems are traditionally custom-designed, featuring redundancy for guaranteeing fault resilience. The downsides of such custom systems are typically weight, power, energy, space, and cost, compared to Commercial Off-the-Shelf (COTS) solutions. We explored the use of COTS in critical real-time environments by designing a CPU-FPGA heterogeneous system, which features an ARM CPU, running a modified version of FreeRTOS and an FPGA, on which the fault-detector and the scheduler are synthesized, in a redundant configuration for increasing fault resiliency. Moving the scheduler to the FPGA increases its fault resiliency while removing the periodic scheduler execution overhead from the CPU, making the scheduler overhead negligible and allowing for an elevated time resolution: the tasks can almost completely utilize the CPU time. Similarly, synthesizing the fault detector on the FPGA allows the execution of the fault detection in a fault-tolerant way without wasting CPU time. Transient fault resiliency in application tasks is achieved via fault detection and the subsequent fault recovery via re-execution. The fault detector implemented on FPGA uses a machine learning technique to model the behavior of tasks (offline and possibly online) and analyses it during their execution. Regarding fault recovery, the scheduler on the FPGA features a novel mixed-criticality scheduling algorithm that manages re-executions, ensuring the meeting of tasks’ timing constraints. The fault detection showed noticeable results while providing a lower overhead than general-purpose software techniques for improving fault resiliency. To the best of our knowledge, the integrated CPU-FPGA version of the system, featuring fault-tolerance and real-time scheduling, is a novel contribution that may enable the use of low-cost and fast COTS components in critical real-time environments. The source code for both hardware and software was released as open source.