Camaroptera: A Long-range Image Sensor with Local Inference for Remote Sensing Applications

Remote sensing devices are increasingly able to include a long-range radio, such as a LoRa chirp spread-spectrum radio [16, 18, 40]. LoRa integrated circuits (ICs) are commercially available, inexpensive, and offer long range (i.e., kilometers) at relatively low power (i.e., hundreds of mW). Extensible receiver infrastructure, like OpenChirp [16], affords simple, publish/subscribe data management (e.g., MQTT) with simple endpoints. The ability to communicate kilometers at low power creates the opportunity for more devices to be deployed in remote environments than is possible using other radio technologies. 4G/LTE incurs per-byte subscription costs [52], Bluetooth has limited range [8, 20], and WiFi requires many access points for wide-area coverage. Backscatter [37, 38, 69, 73, 76] is appealing, although limited by the need for large, powered transmitter infrastructure that provides wireless power and a communication carrier signal. LoRa provides a critical balance between long range and low-power operation, making it suitable for remote IoT deployments. While LoRa consumes relatively low power (hundreds of mW), this power draw can still be expensive for energy-constrained remote sensors, who must judiciously use their radio link or risk exceeding their constrained energy budget.

Batteryless, energy-harvesting devices are emerging to support sensing, communication, and computation with no batteries for energy storage [11, 25, 27, 63, 66]. These devices harvest energy from light [11, 34], radio waves (RF) [63], user interaction [36], or other sources, storing the energy in a capacitor or super-capacitor and using it when sufficient energy accumulates to do useful work. Some batteryless devices operate only intermittently, as energy accumulates [10, 28, 44, 45, 48, 62, 77]. A key advantage to batteryless operation is eliminating the need for per-device maintenance to replace batteries, which effectively extends deployed device lifetimes to the lifetime of the ICs on the board. Eliminating batteries also has secondary benefits: reducing size, weight, and environmental footprint (battery waste). In contrast, battery-powered systems have a number of disadvantages. Batteries require maintenance, because they need to be replaced over a long lifetime: Fixed batteries deplete, and rechargable batteries have a limited number of recharge cycles. Some recent work combines rechargable and fixed battery storage [34] using LTO batteries. These batteries bring more recharge cycles, but smaller capacity (40 mAh) than Lithium-Ion batteries, impeding their use in multi-decade deployments with energy-hungry radios.

Batteryless devices are becoming useful in image sensing platforms [57, 58]. Existing devices are limited, however, in their reliance on an instrumented environment with installed wireless power and a backscatter communication medium precludes their application in wide-spread, long-range (i.e., kilometer-scale) deployments. Avoiding complex wireless power infrastructure, which increases cost, is a key problem that we address in this work.

Communication is energy-hungry. Communicating large image data over multiple kilometers has a high energy cost, making it challenging to deploy on batteryless devices typically designed for ultra-low-power operation. Camaroptera’s main goal is to use local computation to address this challenge. Computing to process a QQVGA image using an image-classifying neural network consumes around \(65 mJ\) and transmitting an image consumes \(288 mJ\) (Section 5 describes our platform in detail, and Section 8 describes these data in more detail). High energy costs translate to high time costs in a batteryless system, because a batteryless system must spend its time collecting the energy that it uses to operate. Figure 2 shows the collection time at different power levels for the quantity of energy required to transmit an image: At 10–20 kLux (a typical cloudy day outdoors), recharging takes 45 seconds to two minutes. A batteryless device is unavailable to sense new data during this recharging period, and a long recharging latency could cause it to miss important events.

Many existing sensor designs commonly employ a Sense-and-Send approach, transmitting data as they are collected [57, 58]. Sense-and-Send is, however, a poor match for batteryless, remote-sensing, because transmitting all data leads to a high aggregate recharge time, which may miss important events. Moreover, all data are not equally interesting and sending these data at high cost is not useful to an application.

Camaroptera reduces unnecessary communication (and recharging) by computing locally on sensed data to find interesting data that should be transmitted, rather than indiscriminantly transmitting data in the Sense-and-Send model. While the computational capability of low-power microcontrollers is limited, prior work shows promise for sophisticated sensor data processing on batteryless, energy-harvesting devices [23]. With architectural support for yet more efficient computing in ultra-low-power MCUs [14, 24], the computational capabilities available to batteryless, energy-harvesting sensor nodes will further increase. The increase in computational capability of batteryless systems motivates Camaroptera, as it allows the use of local computing for reducing costly long-range communication, consequently enabling maintenance-free, multi-decade deployments of batteryless, long-range sensing systems.

Existing systems meet some, but not all, of the requirements for designing effective remote-sensing systems, as we show in Table 1. Prior attempts at batteryless image sensing [57, 58] rely on short-range, RF-energy-transmission infrastructures, violating requirements R1, R3, and R4. While prior work on batteryless communication often relies on short-range RF-backscatter [19, 37, 38, 76], a prior work has proposed using LoRa signals for energy-harvesting and backscatter communication [69], eliminating costly RF-energy-transmission infrastructures. However, LoRa backscatter supports a maximum separation of 475 m between the source and receiver, failing R1 and precluding its use for remote-sensing applications. In contrast, we equip Camaroptera with an active, long-range LoRa radio, enabling kilometer-range communication (R1) and a decrease in environmental impact and infrastructure requirements (low-power receivers only—R3).

A recent system (Magno et al. [22]) presents a batteryless image-sensing platform, designed for face recognition. While it employs a LoRa radio to achieve a long deployment range, it uses an ARM Cortex-M4 core with FLASH memory, whose limited write endurance precludes long-lifetimes and fails (R2). In contrast, Camaroptera uses an MSP430 microcontroller with embedded Ferroelectric RAM (FRAM) for long-lifetime non-volatile storage. Magno et al. also is designed for indoor operation, resulting in a power system with \(\sim 6\times\) larger solar panels and a larger cost and environmental impact (R3, R4).

Permamote [34] presents a battery-powered solution for long-term, low-cost sensing. However, it achieves a long lifetime by restricting operation to low-data-rate sensors (accelerometers and temperature sensors) combined with short-range, low-power communication, making it unsuitable for image-sensing applications with high-power, kilometer-range radios. Camaroptera achieves a maintenance-free, multi-decade lifetime (R2) by relying on batteryless operation, which also lowers per-device cost (R4) compared to equivalent, battery-powered systems.

The sensor board incorporates the main active components of Camaroptera’s remote sensor system, including a microcontroller (MCU), a low-power image sensor, and a LoRa transceiver with a ceramic patch antenna. The sensor board hardware is agnostic to its power system and can be powered by Camaroptera’s power board or by a standard 3V supply.

MCU. Camaroptera’s MCU is a Texas Instruments Ultra Low-Power MCU MSP430FR5994 [71] running at 16 MHz with 8 kB of SRAM and 256 kB of embedded non-volatile Ferroelectric RAM (FRAM). We select this MCU specifically for the embedded FRAM memory, as its non-volatility allows Camaroptera to save image data across power failures, and the higher write endurance of FRAM (compared to FLASH) allows Camaroptera to be deployed for multiple years without memory corruption. The limited memory, low clock frequency, and simple architecture of the MSP430FR5994 are key challenges to supporting sophisticated computations, as observed in prior work [23], and require device-specific software optimizations (described in Section 6.2).

Image Sensor. Camaroptera uses a Himax HM01B0 [30] image sensor, which is an ultra-low-power CMOS image sensor. Its sensor has an active area of 320 \(\times\) 320 pixels each of which with a side dimension of 3.6\(\mu\)m. Camaroptera configures the camera to operate in QQVGA (160 \(\times\) 120) mode, capturing 8-bit grayscale images. We use this image sensor for its ultra-low-power operation (\(\le \!1.1 mA\) for a QQVGA image).

Transceiver. Camaroptera uses a Semtech RFM95W LoRa chirp spread spectrum [64] modulation transceiver IC [31]. The chip incorporates an ultra-low-power 20 dBm power amplifier with a sensitivity over \(-\)148 dBm. Camaroptera connects this LoRa IC to a ceramic chip antenna with a maximum gain of 3.42 dBi. Using a LoRa transceiver allows Camaroptera to communicate the captured images to remote base stations located multiple kilometers away.

The power board implements Camaroptera’s energy harvesting power system. The power system uses a two-stage voltage boosting circuit with hardware voltage comparators to keep system voltage in the most efficient operating range for the boosters. The power board also houses Camaroptera’s supercapacitor-based energy storage.

Boosting. The first voltage boosting stage connects the solar cells to the supercapacitor using an LTC3105 [42]. The booster is a high-efficiency step-up DC/DC converter that operates down to 225 mV input and supports maximum power point control (MPPC). Both of these features are important for operating on variable solar energy.

The second boosting stage connects the energy storage capacitor to the sensor board using a TPS61070 [70] synchronous voltage boost converter. This boost IC provides efficiency over 85% with input as low as 1.2V for a regulated output of 3V. Camaroptera controls the booster operation, keeping it powered off when the supercapacitor voltage is outside its maximum efficiency region.

Voltage Thresholding. Camaroptera uses two MIC841 [53] voltage comparators with an externally adjustable hysteresis to drive the enable input of Camaroptera’s second booster and the reset line of the MCU. The first comparator ensures that the second boosting stage is enabled only when the energy storage capacitor is charged between a lower threshold of 1.24 V and its rated maximum of 3 V. Camaroptera’s second comparator holds the MCU in reset while the \(V_{CC}\) output of the second boosting stage stabilizes. We empirically determined that the system stabilizes at 2.2 V, which is the the minimum voltage to operate the LoRa modem. The second comparator is set to a low threshold of 2.2 V and a high threshold of 3 V.

Energy Storage. Camaroptera stores energy in a high-density supercapacitor. Camaroptera’s supercapacitor must store sufficient energy to ensure that its longest energy-atomic task completes without exhausting the device’s stored energy. Camaroptera’s largest energy-atomic task is sending a single 255-byte LoRa packet. Under this constraint, we equipped Camaroptera with a BestCap [4] 33 mF high-density supercapacitor. This supercapacitor also has a low Equivalent Series Resistance (ESR) enabling it to provide the high radio current with a smaller drop in voltage.

Camaroptera’s solar board is covered by four solar cells connected in parallel and mounts perpendicularly to the sensor and power boards. The solar cells are an array of IXYS [33] high-efficiency monocrystalline cells, measuring 22 mm \(\times\) 7 mm each. The solar board provides structure and power for the assembled device with mechanical and electrical connections to the other boards.

CamSW locally processes every image after its capture. The goal of processing is to find images of interest to the application that should be transmitted, otherwise discarding uninteresting images. Avoiding transmission of uninteresting images enables Camaroptera to judiciously use its time and energy. CamSW supports arbitrary image processing pipelines that can vary by application. A pipeline may use general filtering operations, such as differencing, or application-specific ones, such as a deep neural network trained for a specific task.

We built a representative, prototype processing pipeline designed to identify and transmit images containing people. This representative person-detection pipeline has both general and application-specific stages. The person-detection pipeline includes four stages. Figure 4 shows the software flow of this person-detection pipeline.

Image Differencing. After capturing an image, Camaroptera compares it to the most recently captured image to determine if the new image differs. If the image differs, then it may be of interest to the application and should continue through the pipeline. If not, then Camaroptera can safely discard the image. After comparison to a previous frame, Camaroptera takes the new frame and saves it as a reference for future differences. Difference computation is generally applicable.

DNN Inference. If a new image differed from a previous one, then Camaroptera runs an application-specific inference routine on that image to identify whether it is interesting to the application. For our person-detection application, Camaroptera runs a deep neural network trained to detect images containing people. Inference using statistical methods or learned inference models are inherently application-specific and each application requires its own model. For person detection, we run a single model, but Camaroptera supports cascading models, as MCU resources permit.

Pre-transmission Transformation. If an image is deemed interesting by inference, then Camaroptera transforms the image to prepare it for transmission. This transformation could include a variety of encoding and encryption, depending on application requirements. In our person-detection prototype, we compress each interesting image before transmission.

Transmission. Once transformed for transmission, Camaroptera packetizes the image and transmits each of an image’s packets in sequence using its LoRa radio.

We implemented Camaroptera’s processing pipeline, including differencing, inference, and compression.

Differencing. The goal of this processing stage is to act as a fast filter for images of static or slowly changing scenes. We implemented a simple image differencing algorithm that explicitly compares corresponding pixel values between images. We chose this approach for its simple and fast operation, where a more sophisticated approach (e.g., SSIM-based) would be considerably slower on the MSP430 MCU, which lacks a floating-point unit and must rely on software emulation of floating-point operations.

In our simple approach, we deem images different from one another if the number of different pixels exceeds a heuristically defined threshold. We set the threshold empirically to 400 pixels by observing that human figures in our images tend to be around 20 \(\times\) 20 pixels in size.

Inference. Our Camaroptera prototype uses a Deep Neural Network (DNN) for image classification. The DNN’s structure is derived from the LeNet [39] digit classification convolutional neural network. The architecture of the network we used is shown in Figure 5. We trained the LeNet-structured DNN using a set of images collected using our Camaroptera prototype. We collected 4,000 images around our university campus in five different locations in a wide variety of lighting conditions. We used Amazon Mechanical Turk [2] workers to label them as containing a person, not containing a person, or not being a valid image. The dataset included 60% negative images and 40% positive images. After labeling the dataset, we trained the network using a subset of 3,600 of the images, holding 10% aside for testing and validation.

We optimized the network to decrease its size because the model, as trained, does not fit in the 250 kB of available memory on our MSP430 MCU. First, we stored the network weights in a sparse format, using 16-bit fixed-point integers to save space and increase compute speed. The MSP430 MCU does not natively support floating point operations and software emulation is extremely slow. All computations for the network were therefore performed in fixed-point arithmetic, with kernels written for sparse matrices. Second, we reduced the size of the network by passing the 160 \(\times\) 120 input image through a 4 \(\times\) 4 average pooling layer, before the network’s first convolutional layer. The pooling layer effectively reduces the input to 30 \(\times\) 40 pixels. We then use the Genesis [23] network minimization tool to perform hyperparameter (i.e., structural) optimization on our trained, modified network. Genesis applies aggressive near-zero pruning and layer separation to reduce a network’s weight storage requirements, the size of its intermediate activiations, and the expected inference latency and energy. Camaroptera encodes the optimized network from Genesis as Compressed Sparse Row (CSR) for space efficiency. After optimization, the network’s weights consume 20 kB of memory, which is a substantial reduction compared to its initial 3.6 MB of network weights. The final network additionally requires around 80 kB for intermediate activations and yields 78% accuracy on a test set, with 40% False Positives (FP) and 1% False Negatives (FN). Section 8 evaluates Camaroptera with different false positive and false negative ratios.

Compression. Camaroptera implements JPEG compression, based on an existing implementation [56]. As with inference, we modified the JPEG implementation to use fixed point arithmetic instead of floating point, given the lack of floating point support in the MSP430 MCU. Shifting to fixed point reduced the latency to compress a 160 \(\times\) 120 image from 25 seconds to 7 seconds. Fixed point JPEG degrades image quality, but not excessively. Figure 6 compares an uncompressed image and ones compressed using floating and fixed point.

We additionally optimized the transmission of JPEG headers. The first 500 bytes of the compressed bit stream is always the same, using the same quality factor and frame resolution. We store the header on the receiver and avoid sending the 500 bytes of header data, which amounts to the transmission of two LoRa packets and seconds or minutes of device operation.

Transmission. We operate the LoRa radio with the following parameters: Frequency = 915 MHz, Bandwidth = 500 kHz, Spreading Factor = 7, Coding Rate = \(\frac{4}{5}\), Preamble Length = 8 symbols, Output TX Power = 17 dBm, Packet Size = 255 bytes.

Energy Management. We manage the energy usage of different tasks on Camaroptera in software by charging the capacitor to 3 V before the start of each task. A task can be either image capture, any single stage of the processing pipeline, or transmission of a single LoRa packet. Once the execution of a task is complete, if the capacitor voltage is below 3 V, Camaroptera goes into deep-sleep and charges the capacitor before resuming operation from the next task. We have sized the capacitor to atomically support the largest energy task (transmission of single LoRa packet), avoiding mid-task power failures.

Our prototype implements a representative person-detection application using a deep neural network and sends images containing people to the base station, which we refer to as operating in the Local Inference mode. We demonstrate the effectiveness of this Local Inference mode by comparing it with two alternate modes: Sense-and-Send and Basic Thresholding. The software flowcharts for these three modes are shown in Figure 4. Sense-and-Send indiscriminately transmits all the images it captures, only running JPEG compression for each image. Basic Thresholding sends JPEG compressed images that signficiantly differ from the device’s previous image, with 2% or greater number of pixels differing by 15% or greater. Local Inference uses local DNN inference to detect images of people. Local Inference first runs Basic Thresholding and for significantly different images, invokes the trained DNN person detector from Section 6. Local Inference sends images classified as containing people only and discards other images.

We ran three multi-week, in-lab experiments, with Camaroptera handling images in one of Sense-and-Send, Basic Thresholding, and Local Inference modes (Section 7.1) for the respective experiment. We evaluated how Camaroptera operates at three different, realistic outdoor light conditions ranging from overcast to bright sun (15, 30, and 45 kLux), setting these light levels using a dimmable incandescent bulb and a URCERI MT-912 light meter. Each trial had 5,000 image events emulated via an MSP430 experimental control board connected to Camaroptera. We emulated image delivery to ensure repeatable experiments. To emulate an event (i.e., a significantly different image), the control board raises the event GPIO pin, connected to Camaroptera. To emulate an interesting event (i.e., an image containing a person), the control board raises the interesting GPIO pin, connected to Camaroptera. The control board generates events and interesting events with Gaussian (\(\mu = 3 s\), \(\sigma = 1 s\)) durations and Poisson (\(\lambda = 10 s\)) inter-arrival times, based on our informal profiling of real, pre-COVID19 human activity on a college campus during daytime. We use the same event sequences across trials with different system configurations to ensure fair comparison.

As outlined in Section 7.1, we studied Camaroptera in three operating modes. In the Sense-and-Send mode, Camaroptera captures and transmits images continuously. In the Basic Thresholding and Local Inference modes, Camaroptera first captures an image. If Camaroptera captures an image with the event GPIO raised, then the image is treated as an event. Basic Thresholding transmits an image if it is an event, assuming perfect discrimination of events using differencing. For the Local Inference mode, an image captured with both the event and interesting event GPIO pins raised indicates the presence of a person, and not otherwise. The number of event images that are interesting is dictated by the True Positive (TP) rate: \(TP\%\) of all event images will be marked as interesting. For every event image, Camaroptera runs its classifier and makes a classification judgment uniformly randomly based on its false positive (FP) and false negative (FN) ratio: An uninteresting image has \(FP\%\) chance of being transmitted, and an interesting image has \(FN\%\) chance of begin discarded. We ran the multi-week experiment with a FP:FN = 10:10 classifier that emulated a reasonable operating point and a 1% TP rate.

We present the results of the multi-week experiment described in Section 7.2, showing that the Local Inference mode yields better data than Sense-and-Send and Basic Thresholding, providing more interesting images, fewer uninteresting images, and processing more images overall.

Local Inference sends more interesting images: Figure 7(a) shows the number of interesting images—images containing people—that Camaroptera captures in the Local Inference mode, compared to the Sense-and-Send and Basic Thresholding modes. Across all three light levels, Camaroptera with Local Inference captures and sends a larger number of interesting images. This difference is greatest at low input power (15klx), with the Local Inference mode enabling Camaroptera to send around \(12\times\) more interesting images than the commonly used Sense-and-Send mode, and around \(3\times\) more than the Basic Thresholding mode. The Local Inference mode outperforms the other two modes as it uses energy judiciously, avoiding the costly transmission of uninteresting images.

The data show that in Local Inference mode, Camaroptera transmits as many as \(50\%\) of all interesting images, which is 12x more interesting images than Sense-and-Send transmits. There are two main reasons that Camaroptera does not capture \(100\%\) of interesting images in Local Inference mode. First, Camaroptera spends time processing each image, creating a risk of not capturing an interesting image while processing an uninteresting one. Section 9 discusses strategies to further reduce this risk and capture more interesting events. Second, even in Local Inference mode, Camaroptera spends time recharging energy spent transmitting interesting events, which blocks capturing new data.

Local Inference sends fewer uninteresting images: Camaroptera’s local inference avoids transmitting uninteresting images more effectively than other modes. Figure 7(b) shows how the number of uninteresting images sent varies with input power across the three operating modes. Local Inference mode sends up to \(6.5\times\) fewer uninteresting images than Sense-and-Send mode. Additionally, while Local Inference mode transmits a roughly constant number of uninteresting images across input power levels, Sense-and-Send and Basic Thresholding send more uninteresting images as input power increases. Local Inference avoids the problem of eager transmission faced by Sense-and-Send and Basic Thresholding: As power increases, recharging becomes faster, enabling sending more images. However, without the ability to discriminate interesting from uninteresting, Sense-and-Send and Basic Thresholding more quickly send more uninteresting images.

Local Inference captures more total images: Operating in the Local Inference mode allows Camaroptera to avoid costly transmission of uninteresting images and use that energy to capture and process newer images. Figure 7(c) shows the total number of images that Camaroptera processes in all the operating modes across the whole trial. Local Inference mode enables Camaroptera to capture upto \(14.7\times\) more total images than the Sense-and-Send mode, and upto \(2.8\times\) more than the Basic Thresholding mode. Processing more raw images by using local inference decreases the chance that Camaroptera misses a critical, interesting event.

Transmitting images is energy-expensive, and collecting that energy takes significant time. Avoiding unnecessary image transmission allows the Local Inference mode to significantly reduce the energy collection time, resulting in less time spent on each image than the other two modes. Figures 7(d) to 7(f) show the distribution of total time spent on an image frame, across three light levels for each mode. The total time includes the time to capture, process and, if applicable, transmit the frame, as well as the time to collect the energy required for these tasks. As radio transmissions are energy-expensive, the frames that are transmitted incur a large latency, dominated by energy collection. This can be observed in Figure 7(d), which shows the Sense-and-Send mode transmitting every image, and thus incurring large frame latencies (\(60 s\) to \(90 s\) at 15 klx) on all the frames it captures. Further, the frame latencies are higher when input power is low (\(\ge \!\!60 s\) at 15 klx), due to slower energy collection, and lower at higher input power (\(30 s\) to \(40 s\) at 45 klx), where energy collection is faster. The Basic Thresholding mode (Figure 7(e)) avoids transmitting unchanged images, which is reflected in the large number of frames with latency \(\le \!\!10 s\), as these frames avoid large energy collection delays. However, the uninteresting images that the Basic Thresholding mode transmits still incurs this large delay, as represented by the frames having large total times (between \(50 s\) and \(90 s\) at 15 klx). In contrast, the Local Inference mode (Figure 7(f)) minimizes the total per-frame latency by using the high-energy radio only for transmitting the images it classifies as interesting for the application. This results in very few frames incurring the high energy collection cost of radio transmissions (frames with latency between \(60 s\) to \(90 s\) at 15 klx). Majority of the frames in the Local Inference mode either take \(\le \!\!10 s\) when eliminated by Image Differencing or take \(10 s\) to \(20 s\) when eliminated by the Inference module. Faster frame latencies allow the Local Inference mode to capture more total images, increasing the effectiveness of Camaroptera in detecting interesting events.

We studied Camaroptera’s sensitivity to varying the True Positive (TP) rate of images encountered across Figures 8(a) and 8(d). We restrict this study to Camaroptera’s Local Inference mode with a DNN having a ratio of False Positives to False Negatives of 10:10. The trials for this sensitivity study had 100 image events, and the results were averaged across three repetitions of each trial. We varied the TP rate from 20% (few people) to 80% (crowded area), representing different amounts of interesting event traffic. The data shows that higher interesting event traffic (TP rate) degrades Camaroptera’s ability to detect interesting events only when input power is low. We expected that a higher TP rate would be detrimental in the Local Inference mode, since it would require more energy-expensive image transmissions. This expectation holds true for total images captured, as seen in Figure 8(a), where the most images are captured for the lowest TP rate (\(20\%\)). Figure 8(d) shows that as the TP rate increases, Camaroptera spends more time on image transmission and less on image capturing, resulting in fewer total images.

Figures 8(b) and 8(c) provide additional insights into how Camaroptera operates at different TP rates. As we expect, Camaroptera reports more total interesting events when there is a higher amount of interesting event traffic; however, the fraction of interesting events it reports varies. At 10 klx, energy efficiency matters most, and a low TP rate helps Camaroptera avoid activating the expensive radio. Camaroptera reports the largest fraction of interesting events for the \(20\%\) TP rate at 10 klx. At higher input power, a TP rate of \(50\%\) presents a higher fraction of interesting events. Camaroptera reporting a larger fraction of interesting events at \(50\%\) TP than \(80\%\) TP is expected; more interesting events use the radio more frequently, missing the capture of newer interesting events. When comparing against a \(20\%\) TP rate, Camaroptera misses consecutive interesting events due to its long processing latency. While Camaroptera misses consecutive event for all TP rates, this degrades performance the most at \(20\%\) TP rate (since there are few interesting events to begin with). Figure 8 shows that Camaroptera’s ability to report interesting events is robust across a reasonable amount of event traffic (especially at \(50\%\)) and can be deployed across a variety of environments.

We measured Camaroptera’s sensitivity to variations in the False Positive and False Negative rate of its DNN classifier, evaluating its effectiveness with different learned inference models (Figures 9(a) to 9(c)). We evaluted three classifiers, representing differently tuned versions of the DNN in Section 6 with the same memory footprint. We identify a classifier by the ratio of its False Positives to its False Negatives, e.g., FP:FN = 10:10. Similar to Section 8.2, this sensitivity study was also conducted with trials comprising 100 image events, averaged across three repetitions.

Figures 9(a) and 9(b) show that a 10:10 classifier captures the most total images. However, the classifier that reports the largest fraction of interesting events depends on the input power. At low input power (\(10-20 klx\)), a low FP rate (\(10\%,20\%\)) leads to fewer uninteresting images transmitted (Figure 9(c)) than a \(40\%\) FP rate, preserving the limited available energy. Camaroptera uses this energy to capture and report a higher fraction of interesting events. At higher input power (\(\ge \!\!30 klx\)), energy is more abundant, and lowering the FN rate takes precedence; a \(40\!\!:\!\!1\) classifier reports the largest fraction of interesting events, even when its high FP rate leads to transmitting the most uninteresting events. Figure 9 shows that designers using Camaroptera must tune the classifier to match their application requirements (e.g., suffering higher FP for achieving lower FN), and also the deployment environment (e.g., prioritizing a lower FP when input power is low).

We characterized and compared the lifetime of our energy-harvesting, batteryless Camaroptera system with different battery-powered Camaroptera systems, as well as the Permamote [34] system, in Table 2. Non-rechargeable batteries are a poor choice for a long-range, visual-sensing system like Camaroptera given their limited lifetimes of a few weeks. Rechargeable batteries allow Camaroptera to achieve longer lifetimes, but still require expensive battery replacements every 4–5 years. Permamote combines a rechargeable LTO battery with a non-rechargeable backup CR2477 battery for powering operation during periods of no input power (e.g., night time). While this does extend the operation to night time, we argue that this fits our Camaroptera system poorly for two reasons. First, the 5-year lifetime even with a rechargeable LTO battery requires expensive battery replacements, failing our requirement for a multi-decade lifetime. Second, a visual-sensing-based system like Camaroptera can capture useful images only when the scene is well-lit, rendering night-time operation unnecessary, especially in remote areas where artificial lighting will be rare.

We also characterized Camaroptera’s latency and energy for its main operations, with data in Figure 10. The data show that while inference takes the most time to complete, transmitting using LoRa consumes the most energy, because the radio system has much higher power consumption. With low input power, the high energy cost of transmitting requires a long recharging latency (Figure 2) despite the low transmission latency. With high input power, recharge times drop and the time spent computing exceeds the time spent transmitting and recharging.

Prior work developed batteryless devices to sense their environment and transmit acquired data. The most related of these devices are batteryless image sensing systems [57, 58] described in Section 3.1. These systems rely on backscatter communication, requiring heavy RF infrastructure and are not appropriate for pervasive, wide-spread deployment, which is the motivation of Camaroptera. A recent system [22] presents a batteryless image sensing system for face recognition. While using a similar camera and radio to Camaroptera, its architecture and deployment targets are entirely different. It targets indoor face detection, resulting in a different power system design with significantly (roughly \(6\times\)) larger solar panels and uses a time-of-flight sensor to trigger image captures. More importantly, its computing subsystem is designed around an ARM Cortex-M4 core, which does not support byte-addressable non-volatile memory, making this system unsuitable for long-lifetime, energy-harvesting applications. Other batteryless systems [9, 11, 25, 27, 36, 63, 79, 80] are designed to collect low-data-rate time series data and do not support image collection. Additionally, by relying on backscatter, Bluetooth Low Energy (BLE), or other short range communciation media, these devices do not communicate over long distances, making them inapplicable to Camaroptera.

Other prior work on batteryless communication relates to Camaroptera. Some work uses active radios [8, 11, 20] focusing primarily on BLE. We are unaware at the time of writing of any batteryless device supporting active LoRa communication, making Camaroptera the first device with this capability. NeoFOG [46] relies on high-powered radios and focuses on optimizing sensor-to-sensor communication, not long-range backhaul. Other work uses passive communication, primarily based on RF backscattering. Some work aims at short-range backscattering of directed or ambient RF energy [37, 38, 63, 73, 76], which are inapplicable to the demands of wide-spread sensor deployment. Longer-range passive systems [59, 69, 75] extend the range of passive networking, but to insufficient range (tens of meters) [75] and with dependence on large, powered RF transmitters. Camaroptera has the full range of LoRa (hundreds of meters to kilometers) and requires only inexpensive receivers, not complex, high-power RF power transmitters.

Recent work improved the computational capability of batteryless devices with software and hardware models for intermittent computing. Most relevant is recent work on bringing deep learning to energy-harvesting devices [23]; Camaroptera directly leverages this work for its DNN hyper-parameter optimization. Binary networks [13, 41] and bit-serial [17] approaches are an appealing alternative option for DNNs on intermittent devices. Other work on intermittent computing focused on correctness using software support for tasks [10, 28, 48, 62, 77] and checkpoints [5, 6, 35, 49, 50, 61, 72], some with support for approximation [21, 47]. Other systems provide hardware support for designing intermittent architectures [29, 54], circuits [55], and platforms [11, 35]. Some work on intermittent computing targets the safety of I/O operations [3, 7] and concurrency [62, 77].

Camaroptera is largely orthogonal to this prior work on intermittent computing because, while batteryless, Camaroptera avoids intermittent operation by provisioning its capacitive energy storage for communication, which is an order of magnitude larger than what computing requires. Consequently, Camaroptera avoids unpredictable intermittent operation, instead Camaroptera is designed to have sufficient stored energy before attempting any computation.

Prior work on edge computing has studied early image discard for constrained image sensing systems, similar to Camaroptera’s processing pipelines. Systems like WULoRa [51] use a secondary wake-up receiver to trigger a sensing task on the LoRa-based sensing device, whereas Camaroptera triggers its LoRa radio upon detecting interesting events in its environment. Edge systems process image and video data [32, 68, 74] efficiently on inelastic deployed resources. Camaroptera differs in scale from most prior edge systems (with at-sensor computing being “beyond the edge,” according to prior work [23]). Edge computing on larger, yet inelastic systems represens an important future research topic. We assume Camaroptera’s base stations are cheap receivers, but in the future Camaroptera base stations could include sophisticated edge computing resources. Managing the division of labor between Camaroptera-scale sensor nodes, larger, base-station-scale edge computing resources, and clouds is an open problem.