Adapting LoRa Ground Stations for Low-latency Imaging and Inference from LoRa-enabled CubeSats

Conventional LoRa ground infrastructure (operating at the 915 MHz ISM band) is not designed to receive LoRa signals from CubeSats. Furthermore, any approach that requires us to build custom ground infrastructure would negate the latency gains of leveraging existing infrastructure. Primarily, there are two key differences between the signals received at LoRa base stations from terrestrial sensors and those from the CubeSats. First, the satellites are traveling at speeds of several km/s (\(\sim\)7 km/s for our satellite), much larger than any terrestrial mobile LoRa clients. Second, is the fact that CubeSats operate in the low earth orbit (300–800 km altitude; 525 km for our CubeSat) which is a significantly larger distance. At the core, the base station needs to perform two tasks: (1) Detect and Decode the CubeSat packets, and (2) Know when the CubeSat is overhead.

Detecting and Decoding LoRa CubeSat packets: The attenuation of the transmitted signal at the base station in a good pass (\(\sim 90^o\) elevation peak) is roughly 150 dB at an average. This leads to an average received power of –123 dBm significantly above the detection threshold of off-the-shelf LoRa base stations (–148 dBm). Fundamentally speaking, by leveraging the existing infrastructure (high gain antennas and LNAs), it is entirely feasible to detect and decode signals from the LoRa CubeSats. Indeed, prior deployments on hot-air balloons have demonstrated communication as far as 832 km using LoRa [83]. Thus, existing LoRa infrastructure with high-gain antennas can easily overcome its range bottleneck.

However, the massive amount of Doppler shifts in the CubeSat context present a difficult challenge. While LoRa was developed to be inherently resilient to terrestrial Doppler (a few Hz), the amount of Doppler seen from these CubeSats is on the order of several kHz (a significant proportion of the bandwidth). This means up to a third of the signal might be outside the receiver bandwidth. Along with the large communication range, this renders a large fraction of the packets undetectable (see Figure 3). If one were to use a software-defined radio with wide bandwidth, it would be easy to collect the raw signal data and post-process it to overcome this bottleneck. Instead, Vista takes the approach of re-configuring existing LoRa base stations with a simple software update to estimate and overcome Doppler.

Packet Detection using wideband correlators: The key reason for packet loss due to Doppler in LoRa base stations is because the correlation peak is far outside the bandwidth of the sampling unit at the receiver. To overcome this challenge, we tradeoff some noise resilience for improved Doppler robustness. Specifically, we set the receiver to the widest bandwidth correlators (250 kHz) and leverage the fact that even low-bandwidth LoRa preambles can correlate with these wideband correlators if configured properly. To see why this is possible, consider Figure 4: Notice that carefully configured correlators at 250 kHz share common signal fragments with those at lower bandwidths. However, only some fragments match (four out of eight preamble signals in Figure 4 (top)), meaning that this process results in a 3 dB reduction of detection SNR—trading off some SNR resilience for improved detectability. At this point, Vista can simply estimate Doppler using the SYNC symbols of a larger bandwidth (much like traditional LoRa) to obtain a coarse estimate of Doppler frequency shift. We illustrate this process in Figure 4 (bottom).Adapting off-the-shelf hardware for wideband correlators: One crucial aspect of the above approach is the wide-band correlators that it requires for detecting the high-Doppler packets. Thus, to enable this approach on off-the-shelf ground stations it is critical to demonstrate that there exist correlators at higher bandwidths with the same frequency slope as that of narrow bandwidth transmissions. Most off-the-shelf ground stations [17, 76] support multiple spreading factors and bandwidths while CubeSat deployments [21] typically operate on the narrowest bandwidth configurations. As shown in Figure 5, all low-bandwidth configurations at lower spreading factors have an equivalent higher bandwidth correlator (preamble) that provides the same frequency slope as the original low-bandwidth transmission. In fact, this is not really that surprising as increasing bandwidth quadruples the slope while increasing spreading factor halves the slope. Thus, we can leverage these higher bandwidth configurations with the same frequency slope to detect the Doppler offset and use interrupts on the base station to decode the rest of the packet in the right configuration with the Doppler offset corrected.

Satellite trajectory to inform Doppler and vice-versa: As the above approach of estimating and correcting Doppler requires us to use wider preambles and accumulates more noise, it becomes critical for the base station to know when the CubeSat is overhead to switch into high Doppler-resilient mode—trading off SNR for detectability. However, most small CubeSats cannot be tracked by ground radars for the first few weeks (the most critical time for a CubeSat mission) [4]—owing to being highly clumped up in a volume with other entities launched with them. However, the few initial Doppler measurements obtained from our LoRa ground stations can help us indirectly inform satellite orbits, much earlier than ground radars can capture them. These orbital measurements can then directly inform future expected Doppler values for these same satellites in subsequent passes.

We use a simplified version of prior Doppler-to-orbit models [53, 54] to estimate the trajectory from our coarse Doppler measurements using the algorithm described in Algorithm 1. While the precise orbit of the CubeSat may not be known, one can still make very good predictions about its general structure and allowable shape. Mathematically, the variation of Doppler in an overhead pass from a CubeSat can be written in terms of two main parameters besides time (t) [28]: \(\phi\), the inclination angle and \(\theta _{max}\), which is the maximum elevation achieved by the satellite as viewed by the receiver:where \(R_e\) is the radius of Earth, r is the orbital radius of the satellite, \(\omega _f\) is the angular velocity of the satellite (function of spacecraft trajectory and Earth’s rotation), and c is the speed of light. Given these parameters, we formulate our optimization as finding the best orbital parameters that fit our Doppler observations (\(f_o^i\)) across packets (\(i= 1, \dots , N\)), specifically:

The above objective function is not easy to optimize given the complex expression of the Doppler behavior. Specifically, while the expression is convex along the \(\phi\) and \(\theta _{max}\) dimensions, the randomness of \(t_{start}\) stops us from using gradient descent. However, since the possibilities of \(\phi\) and \(\theta _{max}\) are bounded, we can directly iterate on them as shown in Algorithm 1.

While there are many more complex and accurate models for estimating the trajectory from Doppler estimates, our results in Section 7.2 highlight the improvement in trajectory estimation due to better Doppler estimation by Vista. Upon estimation of this trajectory, we can leverage this information in future packets to get more accurate Doppler estimates of the incoming packets. Note that we can also use this information to reduce the bandwidth of correlator symbols to improve the SNR of reception. Finally, we can use this modified Doppler estimate to correctly decode the communicated information on the ground infrastructure and leverage it for decoding images and performing inference tasks.

As previously mentioned, the packets from LoRa-enabled LEO satellites are severely attenuated when they reach the ground station infrastructure. However, despite that stark 150dB of attenuation, these signals are still above the detection noise floor of the commercial off-the-shelf LoRa base stations in the absence of other radio interference. Yet, these ground stations are also designed to serve several terrestrial clients—their primary and original purpose. These clients are much closer to the base stations than the CubeSats meaning that their signals are usually larger in signal power compared with the CubeSat signals. Thus, the presence of terrestrial clients significantly reduces the capability of ground stations to detect packets from CubeSat clients.

One approach to overcome this problem of packet collisions is to leverage existing solutions for parallel decoding of collided LoRa packets such as Choir [48], MaLoRa [59], CoLoRa [84], and so on. However, these solutions are designed to decode packets that can be detected. Yet, our scenario, is plagued by an extreme near-far effect where the cubesat signal is undetectable owing to being overwhelmed by ground clients. We, therefore, need a solution that somehow specializes in detecting packets from CubeSats and filters out packets from terrestrial clients.

At first glance, one might presume such filtering cannot be done given that CubeSat LoRa packets use the exact same modulation as ground clients. Yet, while the transmitted signal is the same, the channel between the satellite and the ground station is tightly coupled with the trajectory of the satellite leading to an important effect on every individual packet—intra-packet Doppler shifts (see Figure 8(a). Specifically, given that LoRa packets and symbols are several milliseconds long, the large amount of Doppler drift that affects these packets changes within a packet. This means instead being bound inside a bandwidth of the signal the signal may drift as much as several tens of Hz (see Figure 6). Thus, given the trajectory of the satellite (see Section 3.1), we can develop special packet detection filters that detect only the packets from a speci fic satellite instead of detecting packets from all LoRa clients.Doppler-division Multiple Access (DDMA): The key idea behind DDMA is that while the packets face different Doppler frequency shifts, they are not random. Further, these shifts can be estimated across time using NORAD trajectory parameters or our own trajectory estimates above. Thus, the construction of these CubeSat packet detection filters given a Doppler trajectory \(\Delta f_t\) can be formulated as follows. Given a detection time \(t_0\), we create the series of upchirps and downchirps in the preamble \(x(t)\) similar to conventional LoRa packet detection. Given this basic preamble we multiply this vector by a satellite trajectory compensation factor described by \(e^{-2\pi j(\Delta f_t-\Delta f_{t_0})t}\). We then use our new preamble \(y(t)=x(t)e^{-2\pi j(\Delta f_t-\Delta f_{t_0})t}\) to detect packets from that specific satellite.

Why does this work? Similar to how normal preambles correlate well with standard preambles, this new preamble is designed to correlate perfectly with the packets of a specific CubeSat but partially with conventional LoRa packets. This improves the packet detection of CubeSat by reducing the correlation of noise/interference and improving the correlation with the desired signal.

Combating interference from other LoRa CubeSats: Another source of interference would originate from other satellites using the same modality as Vista to communicate information downlink. However, the trajectories of these various LoRa satellites will always be different (otherwise, they would collide!) either in time, or the axis of trajectory. Thus, by using the unique trajectories of these satellites, we can detect and decode the packets from these satellites.

Theoretically, these trajectory-driven preambles can achieve up to 13.12 dB (mean 5.54 dB, see Figure 6) of SINR improvement per interfering client for packet detection in presence of terrestrial interference. We evaluate these preambles in Section 7.2 for various combination of terrestrial clients and LEO satellite trajectories.

There are three key sources of information loss over this wireless link between the LoRa CubeSat and the ground station:

Noise causes LoRa chirps to be misclassified randomly across the frequency shifts. This means that each bit has an equal likelihood of bit-flip regardless of the location of the chirp. To estimate this error, we use the LEO satellite channel model as shown in Ref. [71]. We estimate the symbol error rate (SER) using the theoretical model in Ref. [26]. Based on measurements from our LoRa CubeSats launched in 2021 [21], our average SNR of –10 dB from these traces at LoRa ground stations correspond to roughly 10\(^{-4}\) SER. This directly corresponds to a BER of 10\(^{-4}\).

Intra-packet Doppler affects bit errors asymmetrically. To understand how, we first need to understand how a LoRa chirp is decoded: The receiver typically deconvolves an incoming chirp with another upchirp to ascertain the frequency shift relative to the frequency shift of the preamble. Thus, generally much of the Doppler is overcome by the preamble and one would assume there would be no effect on the decoding process. While this is true for traditional wide-band satellite transmissions lasting a few millisecond, many LoRa packets last several hundreds of milliseconds. This means a significantly larger amount of Doppler accumulates over the duration of the packet, causing the relative offsets of chirps near the end of a packet to be severely deviated from the start of the packet (Figure 7(a)). Doppler-induced frequency shifts lead to the decoded chirps shifting into nearby bins. This phenomenon of misclassification into nearby bins instead of random frequency bins leads to the asymmetric effect across bits (Figure 7(c)). Remember that most adjacent bins only differ in the least significant bits, while the most significant bits (MSB) only change across massive frequency shifts (LSB). In fact, going from the MSB to the LSB, the likelihood of each bit getting affected increases exponentially. Figure 8 shows the distribution of symbol errors across symbols and the likelihood of amount of frequency bins shifted.

Another source of error is discretization. Remember that we are limited to communicating only 2048 discrete bits for every packet. Thus, we need to reduce the analog sensed information to a few bits causing information loss. This restricts any information sent over the wireless link to a very strict constraint.

While all satellite wireless links face the above challenges, these effects are magnified for LoRa CubeSats due to the small bandwidth of LoRa (125 kHz vs. several MHz) and limited size of a LoRa packet (256 bytes per packet). However, we show a unique opportunity that LoRa provides in optimizing applications over this wireless link due to the deterministic nature of information loss and improve data efficiency for inference over images.

Current approaches to communicate images: Traditional approaches to communicate images such as JPEG, TIFF and PNG are not designed for such low packet sizes (hundreds of bytes). This is due to the fact that much of the compression achieved using these approaches relies on carefully designed dictionaries, hash tables and encoding schemes, all of which take space to store for decoding the image. Thus, to achieve any coherent image communication, CubeSats typically rely on two conventional compression approaches: subsampling and reducing bit resolution of every pixel. Subsampling reduces the spatial resolution of the image while the reduced bit resolution adds corruption due to discretization. Figure 9 demonstrates a few configurations that demonstrate significant loss of features, motivating the need for better solutions.

Data-Channel-Task Aware Approach: To ensure useful data compression within the constraints of LoRa packet size, Vista’s key idea is to leverage important known information about the images collected by CubeSats, the wireless channel that the signals propagate over, and tasks performed on the ground. Specifically, we consider: (1) features extracted from a large dataset of typical satellite images, (2) a broad set of tasks that are typically performed on those images, and (3) a well-modelled LoRa-CubeSat channel.

Data Awareness: We first rely on the vast amount of Earth images taken by larger satellites in similar orbits (NOAA [11], Sentinel [18]) to learn how best to compress the images. One approach would be performing a statistical analysis and identifying the patterns across images. Vista instead chooses a more automated approach that learns these patterns using a convolutional autoencoder. Task Awareness: Furthermore, this autoencoder can be optimized not only for reconstructing the images but also directed toward preserving maximal task-specific information. We design a common encoder for both image and task-recovery that is actively aware of the set of tasks that are intended to be performed at the ground. This loses a small amount of performance in both tasks but retains maximal information feasible for each task.

Channel Awareness: Finally, training and evaluating this autoencoder must account for bit-errors that these LoRa wireless packets typically encounter. We design a channel-task-aware image encoder that maximizes the output accuracy while being constrained to the limits of a LoRa packet. We do so by a two-stage process where the auto-encoder learns about noise, intra-packet Doppler shift, and discretization losses. As shown in Figure 10, the output of our solution preserves much of the useful information in the images for inference and imaging.

Vista Autoencoder Design: Our design of the autoencoder (shown in Figure 2) derives heavily from existing conventional wisdom in designing autoencoders over images for inference applications. We adopt a 3-layer convolutional neural network design on the satellite to maximize the accuracy-compute tradeoff. On the ground, we are relatively less resource constrained, and hence can have more layers. Our architecture co-optimizes for both tasks and image reconstruction using two parallel CNNs via a fully connected layer.

Our autoencoder design builds on conventional wisdom from traditional convolutional autoencoder designs. As we go from the image towards the signature, we increase the number of convolution features captured by each layer and decrease the size of filters. This allows the Vista autoencoder to preserve the maximum amount of information while ensuring progressive compression. After each layer, we pass the output via a Rectified Linear Unit (ReLU layer) to capture non-linear behaviors which in turn maximizes the ability of Vista to improve over simple subsampling and other linear baseline approaches. Finally, we use a tanh layer at the output. LoRa Payload and Channel: This output is then discretized to a low-bit representation and sent in a single packet with 256 byte payload. We note that encoder output size and discretization is primarily limited by available bandwidth. These tradeoffs can be task specific, as well as dictated by the wireless channel and we discuss these issues in Section 4.3. At the receiver, the LoRa base station decodes the packet and retrieves the communicated values.

Decoder Design: At the decoder, this signature is then passed on to a fully connected layer to distribute the input to two separate networks optimized for image reconstruction and labeling tasks, respectively. For image reconstruction, we have successive layers of upsampling, convolution, and ReLU to increase the spatial dimensions to that of the original image. For the labeling task, we have successive layers of convolution and ReLU layers with a final sigmoid layer to output the probability of belonging to a class.

Training the Network: We use the Mean Squared Error loss and Cross Entropy Loss functions for image reconstruction and multi-class classification, respectively:

We then train our network with five-fold cross validation on the training data to avoid overfitting while ensuring maximum accuracy. We have restricted our discussion in this section to primarily build a data-task-aware encoding for images based on our constraints. In the next section, we show how we can make this encoding robust to wireless impairments and improve inference performance.

Given the above data-task-aware autoencoder design, we next endeavor to co-optimize the encoding with the wireless impairments described in Section 4.1. Broadly, our approach modifies the training of the autoencoder given the data, task and channel information by training it in two stages. As shown in Figure 11: In the first stage, the network learns about the noise and Doppler related bit-errors. In the second stage, it compensates for the discretization loss. Note that both these stages of training will occur on the ground a priori and, hence, can be performed over a long time to converge.

Overcoming noise and intra-packet Doppler: To model the effect of noise and intra-packet Doppler, during the first stage of training, we add an additional layer that multiplies the input by a bit-flip operation layer. For a one bit output, we achieve this by creating random bit vectors of bit flips and then convert them to a (–1,1) vector (–1 signifying bit flip). Note that in a one-bit output scenario, the output of the tanh layer lies between (–1,1) which means a simple multiplication layer with our bit mask will work. Further, as this bit-mask remains constant during an epoch, the multiplication layer is both continuous and differentiable. Thus, during emulation of the bit-error, the gradient will also propagate over our custom layer and hence get corrected automatically. Note that this makes the network aware of the behavior of these errors and maximizes accuracy for the resulting image and task.

A new problem arises when each output is represented by multiple bits: The correct way to emulate bit flips would be to take our bit mask and XOR with the output of the encoder. However, it is well known that the XOR operation is not continuous or differentiable. To address this, we use a known approximation of the XOR operation [8] by a 2-layer neural network for the number of bits required to XOR. Hence, we add this XOR black box in place of the multiplication layer where the inputs to the XOR are given by our bitmask. This enables multi-bit outputs to learn the behavior of the bit error distributions caused by intra-packet Doppler.

Overcoming Discretization Loss: Discretization requires us to reduce these 4 byte float output elements to a few bits causing information loss. This means that if we use a 2\(\times 32\times\)32 sized output, we can only send one bit of each floating point number to the actual decoder. On the other hand, using a 1\(\times 16\times\)16 output for the encoder would allow us to communicate 8 bits of information to the output. As one can imagine, both of these scenarios severely affect the ability of the decoder to decode this input into coherent output.

To address the problem of discretization, we first attempt to understand how it affects the output of the decoder. We observe that the discretization loss at the input of the decoder propagates to roughly equal information loss in the output of a well-trained CNN. For example, the output of the 2\(\times 32\times\)32 encoder becomes a single bit-mask on the output of the decoder. On the other hand, 1\(\times 16\times\)16 image output suffers significantly less loss. However, this image output looks half the resolution of the image output of 2\(\times 32\times\)32 image, as each of the distorted pixels in the image is much larger, spanning double the number of underlying pixels.

The average information loss for k-bit discretization is equal to \(\frac{1}{2^{k+1}}\). However, it is quite difficult to quantify the amount of information lost due to reduction in spatial resolution. The only potential way to solve this dilemma is to evaluate the loss on the prior available data across both axes and use the least error configuration. Our evaluation shown in Figure 12 shows that the optimum configuration näively operates at 4-bits.

However, we can do even better. We ask a simple question: “Can the decoder learn and specialize for this discretization to maximize the accuracy of the output of the image and tasks?” The first obviously direct approach would be to create a differentiable model of such discretization and add it as an additional layer to the neural network. However, such discretization is neither continuous nor differentiable. Furthermore, unlike XOR, there is no guarantee that it can be estimated with a small enough neural network. Instead, we take an indirect approach to teach this behavior to our decoder: We first freeze the encoder as it is already optimized to best represent the image in the limited set of bins. We affect the output of the encoder with the necessary discretization to create an intermediate input. We then relearn the decoder parameters for the image output on these affected intermediate inputs. Our results show how our approach minimizes information loss and maximizes the accuracy across configurations.

Setup: We evaluate the potential latency benefits of using existing LoRa ground stations (operating on the 915 ISM band) over specialized ground infrastructure such as large generic satellite base stations or LoRa ground stations in lower UHF bands. Now, it is virtually impossible to know the locations of every possible ground station in all of these categories and thus we endeavor in procuring the publicly known locations of these ground stations on three platforms : The Things Network [24] (terrestrial LoRa network), TinyGS [20] (LoRa grounds stations operating at lower bands), SatNogsDB [16] (Satellite Ground Stations). Assuming the TLE information above obtained from NORAD, we use the location of our satellite across a year in space to evaluate the potential latency of retrieving information from any of these networks. We further analyze the benefits geographically from the satellite point-of-view (POV). Since, there is no clear definition of continental boundaries, we used the map described in (Figure 16, top right) to classify locations of the satellite across continents and geographically analyze the benefits provided by Vista.Results: Figure 15 shows the latency across trajectory of the satellite using various ground infrastructure during two days of operation. It is quite evident that across many developing regions where it is difficult to develop and deploy expensive specialized ground infrastructure, leveraging existing LoRa deployments in ISM bands provide a clear win. Our evaluation across a year of Vista trajectory (Figure 16, left) demonstrates that that leveraging the widely deployed 915 MHz LoRa ground infrastructure will enable significantly lower latency of image downlink compared to using generic satellite ground stations or specialized LoRa infrastructure. Across the orbit of an year, LoRa infrastructure at 915 MHz provides a coverage of 44.92% compared to 21.23% of TinyGS and 33.27% of SatNogsDB. In terms of 90%ile latency, using Vista for LoRa CubeSats can potentially reduce latency by up to 55.55%. This benefit indeed is distributed across the globe with improvement across continents. As shown in Figure 16 (bottom right), the benefits are most pronounced in Africa where there is a lack of satellite reception infrastructure as well as Europe which has the largest deployment of LoRa infrastructure. This demonstrates the potential of LoRa transceiver using Vista to reduce the latency of transmitting a quick snapshot to ground infrastructure.

Setup: We evaluate the efficacy of Vista’s wider correlators and trajectory-driven preambles in detecting Doppler offset packets. We use off-the-shelf Semtech SX1262 base stations to receive LoRa packets with increasing Doppler offsets and measure the limits of packet detection limits. We measure the received signal power across offsets using SemTech SX1257 base stations (with access to I/Q samples) for Vista’s wideband correlators over standard narrowband decoding. We use the state-of-the-art orbit models [53, 54] to emulate the behavior for LEO trajectory by varying values of maximum elevation (\(\theta _{max}\)), inclination angle (\(\phi\)) and start time of transmission (\(t_{start}\)). We also add additional frequency noise as expected from a real satellite due to angular velocities, offset estimation resolution and other factors. We then use Vista’s approach to estimate the trajectory using Algorithm 1. We then use this trajectory to test the efficacy of the trajectory-driven preambles.

Doppler-Resilience: Our measurement study in Figure 17(a) demonstrates that the frequency offset estimation limit of our wideband correlators is as good as the widest correlators supported by the hardware. Furthermore, using these correlators adaptively shows large gains in SNR reaching up to 15 dB when Doppler is as large as the bandwidth. Further, packet detection limits quadruple the resilience to Doppler shifts compared with baseline LoRa base station. As shown in Figure 13, this could lead to large increase in packet throughput when receiving signals from a single CubeSat pass.

Trajectory Optimization: Our results show that using the preamble along with the sync symbols achieves 4× better frequency offset estimation resolution compared with only two sync symbols. Further, due to the better resolution in measuring the offset estimate, we can estimate the trajectory with a resolution of 1.2\(^\text{o}\) in inclination angle (\(\phi\)) and 0.5\(^\text{o}\) in max elevation (\(\theta _{max}\)) which is comparable to the accuracy of available TLE files for much bigger satellites [4] (see Figure 17(b)). Furthermore, the inclination angle remains consistent within a margin of 1\(^\text{o}\) over the lifetime of a CubeSat, which can be further used to refine Doppler measurements.

Setup: In this section, we evaluate the efficacy of our trajectory-dependent preambles’ based Doppler-division multiple-access (DDMA, Section 3.2) to improve the interference resilience and scalability of LoRa-enabled CubeSats. We first demonstrate the signal-to-interference-noise (SINR) gain (in dB) by taking multiple 2 satellite configurations (by varying trajectory parameters) along with 2 terrestrial clients (one each deployed close and far from the ground station) and demonstrate the SINR benefits of leveraging a DDMA based packet reception. We take channel data for ground clients by choosing the clients from publicly available datasets [51] and take the channel data for clients in space by simulating various satellite trajectories. We then scale this evaluation to multiple (upto \(10^5\) satellite orbits and demonstrate how both Vista’s packet detection solutions (namely, wideband correlators and DDMA) work together to improve packet reception. While this simulation assumes all the colliding packets are transmitted simulteneously, the true scalability of LoRa will further amplify the number of satellites that can be simulteneously serviced by a single ground station.

Interference Resilience: Our results in Figure 18(a) demonstrate the SINR gains of leveraging trajectory-driven preambles described in Section 3.2 for detecting packets in presence of interference. As we can clearly see, when there are two concurrent CubeSat transmitters along with two terrestrial clients, Vista preambles improves the interference resilience of packet detection by 12.02 dB for CubeSat clients and 11.08 dB for terrestrial clients at an average. Note that the baseline absolute SINR of configurations with more clients will be lower and thus this valuable gain would mean the difference between detecting a packet and missing it completely.

Scalability: Our results in Figure 18(b) show the key benefits of using the wideband preamble detection and DDMA for multiple access in improving the packet reception rate of LoRa-based CubeSats. First, as demonstrated in Figure 13, only 56% of packets transmitted across a single CubeSat pass can be detected using conventional off-the-shelf ground stations leading to the lower packet detection rate. Further, this improvement in packet detection SINR enables more packets to be detected by the ground base station. Across the various packet detection rates, we can clearly see an average 4–4.5× increase in number of CubeSat clients that can be supported. Note that these numbers are bottlenecked by the noise floor which plays a crucial role in determining if the packets are detected. This means there is a new opportunity to further improve scalability by improving the noise resilience of the core LoRa technology (for instance, longer preambles and interspersing the preamble symbols between the packet).

Setup: Our model (Figure 2) uses a fully-connected layer to co-train two parallel CNNs optimized for image reconstruction and multi-class land-use classification with 50–50 (5,120 images each) train-test split. The dataset is taken from Ref. [56], which contains labeled images captured from Sentinel-2 [18] satellite from an altitude of 800km. We only choose the RGB spectrum inputs to classify the image into the 10 labeled land-use patterns: Sea & Lake, Annual Crop, Forest, Herbaceous Vegetation, Highway, Industrial, Pasture, Permanent Crop, Residential and River. For the baseline, we leverage a subsampled image with 8 bit resolution as input. For the data-aware approach, we specially train a Vista autoencoder specifically for reconstructing images. Finally, Vista’s approach co-trains two parallel CNNs optimized for image reconstruction and multi-class land-use classification. Improving Image Retrieval: Figure 19(a) shows that the baseline’s per pixel error is significantly larger than Vista’s approach. One of the crucial characteristics to observe is that the 90^th percentile error is especially larger for the baseline which means while in some cases the baseline may do well, as more fine grained features get involved the reconstruction becomes much worse. For example, a lake may be easier to identify from even a low-resolution image of the baseline while an industrial complex is far more challenging to make out. This enables Vista to achieve a lower mean squared pixel error of 5.42 compared with 9.17 which is 40% lower error. This corresponds to an average 4.56 dB improvement in image SNR (note that the ideal image SNR is \(\infty\)). Note, however, that Vista’s approach was not specialized for image reconstruction but also for accomplishing task using the same 256 bytes of information. A specialized network such as data-aware approach which is specialized for image reconstruction may perform even better at image reconstruction but will perform poorly at tasks. This is due to the fact that the latent space of compression is not necessarily tuned in the right dimensions for the task. Thus, we do not compare Vista with this method as a fair comparison.

Improving Inference Quality: As shown in Figure 19(b), Vista’s approach outperforms both the baseline approach as well as data-aware approach significantly leading to 72.92% accuracy in classifying the images compared to 52.5% and 57.09% accuracy of the former over 5120 test images (512 images for each class). The classification tasks where Vista’s approach achieves much of its gains are those which require high resolution intricate details such as outlines of buildings for residential and industrial complexes (higher spatial resolution). Further, industries and highways are both made of concrete (grey colored) and face larger errors in the baseline. Note that prior work on the complete 13-channel dataset achieves 84.48% classification accuracy in the original work [56]. But since we use only RGB channels, our classification accuracy is lower.

In this subsection, we present the impact of Vista on the CubeSat compute and memory resources as well as the efficacy of training an autoencoder aware of the channel error characteristics.

Channel-aware Autoencoder vs. Autoencoder Performance: We evaluate the efficacy of our training approach in improving the noise resilience of Vista’s channel-aware encodings. Note that there have been several works that have developed and deployed better and more comprehensive data-and-application aware encodings on satellites such as [43], and thus we will primarily focus this evaluation on the channel-aware nature of our encoding. Setup: We train two autoencoders on the same training data as follows: one trained with uniformly distributed errors (AWGN) and one trained on Doppler noise errors as described in Section 4 and Figure 8. We then evaluate both of these autoencoders using separate test data on both of these error distributions across bit errors. Improvement of Channel-aware Autoencoder: As we see clearly see, an autoencoder trained on the right distribution of Doppler errors does perform better compared with the autoencoder that was not trained on the same distribution. In fact, this is not surprising as training on the right distribution allows the autoencoder to ensure that the least valuable information is encoded on those bits. As shown in Figure 20, we see that as the number of bit-errors reach 10 bit errors (out of 2048 bits), the channel-aware autoencoder has a lower image reconstruction error in the presence of Doppler errors.

Power, Compute and Memory Overheads: The three key resources on the CubeSat are power available to use, compute resources available for processing the data collected, and memory to store the data prior to downlink transmission. (1) Power: It is a well known fact, that wireless communication dominates the power consumption of CubeSats as enough transmit power is required for the signal to reach the ground. Typically, a CubeSat generates energy with solar and consumes it when in eclipse. While the compute and sensors consume power on the order of a few \(\mu\)W for each image capture, the power required for transmission downlink by the RF frontend (even for a low-power technology like LoRa) is 1-2 W to enable efficient reception on the ground infrastructure. Thus, the extra compute overhead of performing Vista has negligible impact on the power budget of downlink transmission. Indeed, recent works [43] have deployed more complex designed data-driven algorithms to compress images. (2) Compute: At first glance, one might feel the extra compute required to implement Vista will be significant. However, remember that the training of Vista will be performed on the ground prior to the launch of the satellite. Thus, the primary impact on compute latency would be to run Vista pre-trained algorithm on images. Our evaluations on a replica of V-R3X [21] compute resources show that it requires an average of 4.83 ms to encode one image.² (3) Memory: The final constraining factor on the CubeSat to deploy Vista is the amount of memory required to store the encoder model that creates the encodings. Our encoding model with 8-bit weight representations requires only 36 kB to store.²