From Demodulation to Decoding: Toward Complete LoRa PHY Understanding and Implementation

Figure 2 shows the mainstream LoRa network architecture. LoRaWAN [23] or Symphony Link [24] protocol controls the whole LoRa network and provides a data interface to the application server. There are three main communication entities in the LoRa network, i.e., network server, gateway, and end device. The network server communicates with the gateway using a traditional internet connection while the end devices exchange data with the gateway by using LoRa PHY wireless protocol. Nearly all components and protocols used in Figure 2 have complete open source implementations except LoRa PHY. Though there exists some reverse-engineering works, the understanding of LoRa PHY is not yet complete. In the following, we demonstrate some basic concepts of LoRa PHY and discuss the details about demodulation and decoding in Sections 3 and 4.

LoRa PHY employs CSS to reach long-range transmission. The basic communication unit in LoRa PHY is a chirp symbol. As shown in Figure 1(b), we denote this unshifted chirp symbol as base chirp. When the frequency increases with time, we name it up-chirp and otherwise down-chirp. The novelty of LoRa modulation is using the start frequency of a cyclically shifted symbol to encode data. Figure 1(c) shows a LoRa symbol with two chirp segments. In LoRa specifications [22, 25, 26], the number of bits encoded by a symbol is SF. The SF satisfies \(2^{SF} = B\cdot T\), where \(B\) is bandwidth and \(T\) is chirp period. There are at most \(2^{SF}\) cyclically shifted up-chirp symbols given a specific SF. An up-chirp can be represented aswhere \(A\) is amplitude and \(f_0\) is start frequency (\(t=0\)).

Figure 3 shows the basic understanding of a standard LoRa packet. Overall, it has three parts: preamble, start frame delimiter (SFD), and data symbols. The preamble is a series of base up-chirps followed by two up-chirp symbols indicating network ID.³ The SFD is 2.25 down-chirps indicating the start of data symbols. Data symbols include PHY header, payload, and payload CRC (header and CRC are optional).

LoRa supports the following configurations. Code Rate (CR): LoRa applies Hamming code of coding rate \(\frac{4}{5}, \frac{4}{6}, \frac{4}{7}\), or \(\frac{4}{8}\) [22]. Low Data Rate Optimization (LDRO): When sending long packets, LoRa enables LDRO mode for better stability at the data rate cost. Implicit/explicit mode: In explicit mode, there is a PHY header in the packet, while in implicit mode, there is no PHY header.

Unknown: ⁴ (1) The ideal demodulation of LoRa, i.e., how a symbol is translated to bits, is still unknown, which is the key for LoRa to achieve low SNR and long-distance communication. (2) None of the existing works proposes a full understanding of the LoRa packet structure. (3) The details of LoRa specific modes are unknown, e.g., LDRO mode.

The space of prior research in LoRa communication is quite rich, including LoRa collision decoding [1, 2, 4, 9, 10, 27], LoRa chirp based backscatter [11, 12, 13, 28], LoRa network optimization [15, 29], and so on. Unfortunately, most of them do not release public accessible implementations. Considering that the LoRa network architecture is brand new and not maturely understood by the community, any new open source efforts deserve appreciation. Therefore, we first summarize the public available LoRa related open source projects.

Upper Layer Implementations. The upper layer of the LoRa network has rich open source implementations. For LoRa gateway and end device, Semtech corporation provides official embedded implementations [30, 31]. For network servers, ChirpStack [32] is a widely used implementation compatible with LoRaWAN. LoRaWAN-Server [33] provides a compact combination of network server and application server. The lowest level code here is the driver for LoRa chips. Therefore, it does not include the physical layer mechanism of LoRa communication.

Physical Layer Implementations. Since LoRa PHY is a proprietary protocol belonging to Semtech corporation, there is no public document explaining all the details of LoRa PHY. Existing researches and systems mainly rely on three open source SDR implementations, i.e., BR [17], RPP0 [16] and TAPP [18]. BR focuses on the implicit header mode of the LoRa packet. BR applies dechirping based demodulation. It ignores some crucial details about dechirping and is not reliable in low SNR. We will discuss the details in Section 3. Besides, the implementation of BR only supports decoding LoRa packets with SF=8. Our tests show that BR can only decode the first four bytes of a packet and fails to decode any complete packet with more than four bytes. RPP0 targets explicit header mode LoRa packet decoding. As shown in Figure 1(c), RPP0 demodulates LoRa symbols by finding sharp frequency changes and then uses them to estimate the start frequencies of the symbols. RPP0 relies on time-domain information, and it cannot decode LoRa packets for SNR\(\lt\)0. TAPP is a recent implementation of LoRa. TAPP is not optimized for LoRa demodulation, which results in high computation overhead. PRR of TAPP decreases when the decoder cannot consume the input data in time. According to our evaluation, its packet loss reaches 70% for a data rate of one packet per second. The average PRR of TAPP is only 66.7%. For decoding, none of those existing works can provide full decoding capability, and thus they can only decode a portion of LoRa packets. Besides, there are other LoRa PHY related works that provide open source implementations. Charm [14] focuses on weak signal decoding by leveraging the multi-gateway structure of the LoRa network. However, the Charm decoder [34] is only implemented in offline mode with MATLAB. Moreover, the author ignores the decoding procedure and the phase alignment problem (see Section 3). TinySDR [35] is an SDR platform tailored for internet-of-things networks. The author implements a LoRa modulator [36] in FPGA, which shows us a convenient hardware chirp generation method. But the demodulation process is commonly much more complicated than modulation, and there is no open source FPGA-based LoRa demodulator currently. In addition, the decoding process is also not considered in TinySDR. We see that existing works have not provided a fully usable implementation and complete understanding of LoRa PHY.

Fundamental limitations of existing works. (1) Existing works cannot work for low SNR, and thus they do not have the crucial advantage of LoRa. (2) Existing works cannot provide full decoding capability for LoRa, and their performance is significantly lower than the commodity LoRa hardware. They cannot provide full support of LoRa, i.e., LDRO mode, all SFs, and so on.

Our goal. We aim to provide a complete LoRa PHY implementation, including demodulation and decoding, to enable extremely low SNR signals receiving and all parameters/modes of LoRa.

The LoRa demodulator converts chirp symbols to bitstreams by first dechirping the received signal. The dechirping process mainly consists of two steps: First, it multiplies each received chirp with a base down-chirp, and then it performs the Fourier transformation on the multiplication results, translating chirps to energy peaks in the frequency domain. Ideally, the multiplication will result in single tone signals with continuous phase, whose energy can be accumulated to a single peak, as shown in Figure 4(a3) (denoted as IDEAL). However, in practice, there is an inevitable phase misalignment when the chirp frequency drops from its maximum to minimum. This phase misalignment is mainly induced by the hardware instabilities of inexpensive LoRa devices and is randomly distributed between 0 and \(2\pi\). With the phase misalignment, energy peaks from the Fourier transformation are severely distorted, as shown in Figure 4(a4), which limits the LoRa demodulation performance, especially under low SNR.

Existing works fail to resolve the phase misalignment problem in dechirping. Thus, their performance deteriorates dramatically in low SNR scenarios, far from the ideal sensitivity of LoRa demodulation. Our goal is to approach the ideal sensitivity of LoRa demodulation under phase misalignment. The design of our LoRa demodulator mainly consists of the following components.

Window Alignment. We need to accurately align the demodulation window (sample points level) with each symbol to retain the energy of the entire symbol for the demodulation process. Traditional works use preamble correlation for window alignment. However, merely using preamble correlation cannot achieve accurate alignment due to Carrier Frequency Offset (CFO). We leverage the SFD part in the LoRa packet for window alignment while eliminating the impact of CFO. The key operation is to combine the down-chirps in SFD together with up-chirps in the preamble. We apply dechirping to both of them (the down-chirp is multiplied with a base up-chirp). If the demodulation window is perfectly aligned with the signal, then the peak of the up-chirp and peak of the down-chirp appear at the same frequency, regardless of the CFO, as shown in Figure 5. Otherwise, there is a significant difference in the frequency of the two peaks, depending on the time offset between the chirp and the demodulation window. Therefore, we can achieve accurate alignment based on the frequency of peaks corresponding to the preamble and SFDs.

Oversampling and Peak Merging. We have previously revealed that the fundamental reason for SNR loss in demodulation is the peak distortion due to the phase misalignment when the chirp frequency drops from its maximum to minimum. To address this problem, we propose an oversampling-based approach that recovers the peak distortion in the existence of phase misalignment and approaches the ideal sensitivity of LoRa demodulation.

As illustrated in Section 2, LoRa modulates signals by cyclically shifting the frequency of a base up-chirp. Each modulated chirp consists of two chirp segments, one with the initial frequency of \(f_0\) and the other with \(f_0-B\). Existing works demodulate the received chirp symbols using a sampling frequency equaling to the chirp bandwidth \(B\). Thus, after multiplying with the base down-chirp, both chirp segments are translated to single tone signals of the same frequency due to frequency aliasing. However, there exists an inevitable phase misalignment between these two chirp segments. When performing the Fourier transformation on the whole symbol, the energy of these two chirp segments adds up destructively, leading to peak distortion and SNR loss for LoRa demodulation. Existing works have no idea of the value of phase misalignment. Thus, they cannot distinguish the signal of these two chirp segments and fail to correct the peak distortion.

We propose an oversampling-based strategy for efficiently separating the two chirp segments in the frequency domain. Specifically, as the initial frequencies of the two chirp segments are \(f_0\) and \(f_0-B\), when oversampling the signal with a frequency higher than \(2B\) (i.e., \(F_{s}\ge 2B\)), the dechirping results in two distinct peaks separately locate at \(f_0\) and \(F_{s}-B+f_0\), as shown in Figure 4(b3). Since there is no phase misalignment within each chirp segment, both of these two peaks are distortion-free. This gives us a chance to perfectly concentrate the energy of the whole chirp by coherently adding up these two peaks of chirp segments and thus eliminating the influence of phase misalignment. For the rest of this section, we illustrate how to merge these two energy peaks in the frequency domain efficiently with small sensitivity loss for LoRa demodulation.

Due to the existence of phase misalignment, the phase of the two peaks in Figure 4(b3) are out of coherence. Thus, directly adding up those two complex peaks will not increase the peak height or even cancel each other due to the phase difference of these two peaks. We propose Fine-grained Phase Alignment (FPA) to search for all possible values of phase misalignment \(\Delta \varphi = \frac{i\times 2\pi }{k} (i=0,1,\ldots , k-1)\) and compensate it before adding up two peaks. We iterate \(k\) possible phase offset and select the \(\Delta \varphi\) that generates the highest peak. The result of FPA is shown in Figure 4(b4). Compared with Figure 4(a3), FPA can approach the performance of IDEAL as the phase difference is compensated.

The searching process for determining the value of the phase misalignment has high computational complexity. To reduce this overhead, we further propose Coarse-grained Phase Alignment (CPA), which requires much less computation overhead and thus can work on low-cost hardware with very limited computation resources. The key observation is that the peak amplitude is in proportion to the energy of the corresponding chirp segment. Given the energy of the whole symbol is exactly the sum of the two chirp segments, if we directly add up the absolute value (amplitude) of part A and part B, then the resulted peak will have the same height as that of the IDEAL peak. The result of CPA is shown in Figure 4(b5). We see that CPA significantly changes the shape of the peak, i.e., the width of the main lobe increases. Besides, CPA also lifts the noise level during the adding up process, thus slightly degrading the demodulation performance compared with FPA.

Depending on the computing power of the receiver, we can switch between FPA and CPA methods. In the following, we theoretically analyze the symbol error rate (SER) with respect to different SNRs for IDEAL, FPA, and CPA.

We assume that the LoRa signal is transmitted over an additive white Gaussian noise (AWGN) channel, where the noise is modeled as following the complex Gaussian distribution. During the dechirping process, both the chirp symbols and the noise are transformed into energy peaks in the frequency domain. A symbol error happens when any of the noise peaks exceeds the peak of the target symbol. Suppose the peak height of the target symbol is \(h_d\), and the maximum peak corresponding to noise is \(h_n\); we can derive the expected symbol error rate asSER is a function of SNR theoretically. The calculation details of \(h_d\) and \(h_n\) for IDEAL, FPA, and CPA are listed in Section 8. We plot the theoretical performance of IDEAL, FPA, and CPA under SF8/12 in Figure 6.⁵ The searching step length in FPA (i.e., \(1/k\)) is set to \(1/16\) empirically (a tradeoff between performance and computation overhead). We see that both FPA and CPA have performance very close to IDEAL for SF12, while the theoretical SER of CPA in SF8 is slightly larger than other two methods. In summary, the theoretical analysis tells us that FPA and CPA are two practical and effective methods in LoRa demodulation, which is further validated in experiment results in Section 6. Our theoretical analysis also shows that LoRa could work under extremely low SNR (i.e., SNR as low as \(-\)20 dB), as can be seen from Figure 6(b).

Peak Refinement. Previously, we have illustrated how to convert received chirps to frequency domain peaks. Next, we need to estimate the highest peak frequency to recover the modulated data bits. However, in practice, height estimation for peaks is closely related to the frequency resolution after the Fourier transformation. The black line in Figure 7(a) shows the result of ideal Discrete-Time Fourier Transform (DTFT) on the target signal, which is a continuous function of frequency. In practice, the receiver only performs Discrete Fourier Transform (DFT), a sampled version of DTFT in the frequency domain, on the target signal. The output of DFT is marked as red dots in Figure 7. The naive DFT has a low-frequency resolution. Thus, its outputs are sparsely distributed over the ideal DTFT curve, decreasing the derived peak height and also impacting the peak estimation. We apply zero-padding to the signal in the time domain to obtain accurate peak estimation, equivalent to interpolation in the frequency domain. As shown in Figure 7(b), with zero-padding, the frequency resolution of DFT improves significantly, benefiting our peak estimation. We show that fourfold zero-padding achieves a balance between computation overhead and demodulation sensitivity in Section 6.3. The benefit of more zero-padding is marginal.

Since LoRa is commonly applied on low-cost devices, the equipped oscillators may not have high accuracy, which results in inaccurate bin values. Table 2 shows the first eight dechirped peak bins of two packets from a LoRa device having about 16-kHz CFO. We observe that the fractional part of the peak bins keeps drifting with time. If we do not cancel that drift, then the fractional bins will accumulate and incur errors in the integer bin. Such bin drifts mainly come from Sampling Frequency Offset (SFO). Due to the difference in sampling frequency, the actual LoRa symbol is \(\tau\) shorter (or longer) than the standard symbol period \(T\). Therefore, the start frequency of the next symbol has an additional drift as

Because the carrier frequency and sampling frequency are generated by the same oscillator, we havewhere \(f_{RF}\) is reference carrier frequency, \(f_{osc}\) is reference oscillator frequency, and \(\Delta f_{osc}\) is oscillator frequency bias. After translating to bin values, the estimated bin drift between two consecutive symbols is⁶For example, when CFO\(=16\) kHz, \(f_{RF}=470\) MHz, and \(SF=10\), the estimated bin drift \(\Delta bin \approx 0.035\), which matches the tendency in Table 2. We subtract accumulated \(\Delta bin\) from the peak bins before decoding.

Another thing we need to consider here is the LDRO mode. Typically, when a chirp period is too long (i.e., \(T\gt 16\) ms), LDRO is automatically enabled in LoRa chips [22]. In LDRO mode, as seen in the bottom of Table 2, the expected bin values always have the form of \(4n+1\), which means the two Least Significant Bits (LSBs) do not encode data. Such a design aims to better protect the data, since lower bits are more vulnerable in long packet transmitting. For LoRa signals lasting a short time, the two LSBs are stable enough to encode data (e.g., SF10 and bandwidth 125 kHz). Nevertheless, in the experiment, we find that the first eight symbols are always in LDRO mode to protect the header information (Section 4).

This section about decoding is not a review of previous works. We aim to provide a provable inference about the LoRa packet structure, decoding order, and configurations. From official LoRa datasheets [22, 25, 37] and patents [26], we know Equation (6) for calculating the number of symbols in a packet. In contrast, the packet structure is unknown. We also know that there are four main processes in decoding: Gray coding (G), deinterleaving (I), Hamming decoding (H), and dewhitening (W). In contrast, the order of the processes and the configuration parameters of each process are unknown. To show how LoRa PHY translates the bits from the demodulation module to meaningful packets, we analyze the LoRa decoding by three steps. First, we infer the packet structure from the black-box LoRa leveraging the number of symbols formula. Next, we reveal the order of the four main decoding processes. Finally, we infer the configuration parameters of each decoding process.

The official document [22] provides a formula to calculate the number of symbols in a packet:where \(PL\) is the number of payload bytes, \(SF\) is spreading factor, \(CRC\) is 1 if CRC check is enabled and otherwise 0, \(IH\) is 1 if implicit header mode (without header) is enabled and 0 if explicit mode (with header) is enabled, and \(DE\) is 1 if LDRO is enabled and otherwise 0.

We can derive the following properties from Equation (6):

Based on the properties, we then manipulate the data packets to infer the packet structure. We first gradually increase packet size and observe a stairlike increase of the number of data symbols in real signals. If \(CR=\frac{4}{7}\), then the number of symbols \(n_{sym}\) would increase following an arithmetic sequence (i.e., \(8,15,22,\dots\)). We also observe that when continually increasing the packet size, the newly added bytes are encoded in the last \(\frac{4}{CR}\) symbols, and the first \(\frac{4}{CR}(n-1)+8\) symbols do not change. It reveals that data bytes are encoded by a \(\frac{4}{CR}\) symbols block.

Further, we find that the first eight symbols are specially treated. Whether in explicit or implicit header mode, the first eight symbols’ values always appear in the form of \(4k+1(k=0,1,2,\dots)\), which means the last two bits of data are discarded, and there are \(SF - 2\) bits data in each symbol. We can infer that the first eight symbols are in LDRO mode, i.e., \(DE=1\). LDRO gives better protection, and the header may be in the first eight symbols. We know that LoRa uses coding rate \(CR\) to protect the payload, and the payload bytes are encoded by a \(\frac{4}{CR}\) symbols block. Similarly, the first eight symbols form an eight symbols block, and we guess that it uses a coding rate \(\frac{4}{8}\) to give the header the highest protection (4 parity bits for a nibble). We also observe that the first eight symbols encode payload bits in implicit header mode. Such a design can reduce the complexity of the decoding hardware complexity, because the explicit and implicit header mode can now be processed with the same procedure.

Figure 8 shows the inferred packet structure of a packet with LDRO mode disabled/enabled. In LDRO mode, the last two bits in each symbol are discarded. No matter whether the LDRO mode is enabled or disabled, the first eight symbols are in LDRO mode and use coding rate \(\frac{4}{8}\). The following symbols are in blocks containing \(\frac{4}{CR}\) symbols. The first four symbols encode data bits while the rest symbols encode parity bits.

Packet Structure Verification. We verify the above packet structure by comparing Equation (6) and the calculated symbol number using the inferred packet structure. For \(SF\ge 7\), the first eight symbols contain \(8\cdot (SF-2) \cdot \frac{4}{8} = 4SF - 8 \ge 20\) data bits. Therefore, the first eight symbols could always include all header information. Except the header, the first eight symbols can contain \(4(SF - 2) - 20(1-IH) = 4SF +20IH -28\) bits payload. Suppose there are \(PL\) bytes (\(8PL\) bits) payload in total. If CRC is enabled, then additional 16 bits are needed. Except the first eight symbols, the total number of required data bits for the rest symbols is \(8PL + 16CRC - 20IH - 4SF + 28\). Whitening and Gray decoding do not affect the total number of bits and the total number of symbols. The Interleaving operation only requires us to pad redundant bytes when the rest payload bytes cannot fill a block, thus not affecting the total number of symbol blocks. Each block has \(\frac{4}{CR}\) symbols, and each symbol in the block could encode \(SF-2DE\) bits. Due to the existence of parity bits, only \(4(SF-2DE)\) bits in a block are actual data bits. Then the total number of symbols in the packet isIt is obvious that formula (7) is the same as Equation (6), which means the inferred packet structure is correct.

In this section, we try to reveal the order of the four main processes in LoRa decoding: Gray coding (G), deinterleaving (I), Hamming decoding (H), and dewhitening (W). It is easy to know that Gray coding should be the first step, because it intends to solve the symbol adjacent drift problem (see Gray Coding in Section 4.4). Hamming decoding operates on bytes stream while a LoRa symbol contains \(SF-2DE\) bits. So it relies on the transformed bytes stream after deinterleaving. Thus, the order of “G, I, H” should be “G\(\rightarrow\)I\(\rightarrow\)H.” Hence, dewhitening has three possible positions, i.e., after “G,” “I,” or “H.” The implementations of BR, RPP0, and TAPP assume three different dewhitening positions, respectively.

We prove that the position of dewhitening does not affect the decoding results. Because the encoding process order is the reverse of decoding order, we use the operation in encoding to show the influence of the position of “W.” Consider the data as a bit vector \(D\), then interleaving is an operation that rearranges the position of bits. We can represent interleaving as a matrix \(I\) with each row or column containing only one “1.” Hamming coding, as a linear coding method, can be represented as a matrix \(H\). We have \(H\cdot D = [P\cdot D ~~ D]\), where \(P\) is the parity matrix. Whitening is \(W \oplus D\), where \(W\) is a random bit vector and \(\oplus\) is exclusive-or (XOR). Following are the three possible orders of decoding and the corresponding encoding operations:

where \(W_1\), \(W_2\), and \(W_3\) are three different random bit vectors. Formula (8b) can be rewritten asTherefore, formula (8a) and formula (9) are equivalent (let \(W_1 = I\cdot W_2\)). Formula (8c) can be rewritten asFormula (10) is a special case of formula (8b) (let \(W_2 = [P\cdot W_3 ~~ W_3]\)). Therefore, both “I\(\rightarrow\)W\(\rightarrow\)H” and “I\(\rightarrow\)H\(\rightarrow\)W” can be represented by “W\(\rightarrow\)I\(\rightarrow\)H,” which means the position of “W” does not affect the final decoding results. We will show the correct position for “W” in Section 4.4.

For the convenience of analysis, we first assume the decoding order is “G\(\rightarrow\)W\(\rightarrow\)I\(\rightarrow\)H.” Because the even parity of all-zeros is zero and interleaving in LoRa is just a diagonal realignment of all bits, the output codewords after “H” and “I” are zeros when the input bits are all zeros. Here we assume that LoRa adopts commonly used even-parity check, and the final results show that this assumption is correct. “W” is the XOR of data values and a pseudo-random sequence. When the decoding order is “G\(\rightarrow\)W\(\rightarrow\)I\(\rightarrow\)H,” as \(x \oplus 0 = x\), if we set all transmission bits to zeros, then the output values after “G” are the dewhitening sequence. Then, we could use the derived dewhitening sequence to recover the temporal results before “I” and “H” for further analysis.

This section reveals the key configurations of the four operations in LoRa decoding: Gray coding, deinterleaving, Hamming decoding, and dewhitening. We also show how to reveal the header structure and CRC polynomial.

Gray Coding. Gray coding is widely used in many wireless communication systems, which is a mapping from a bit vector to a binary representation. The adjacent representations of Gray coding only have a one-bit difference. In wireless communication systems, it is more likely to happen that we may misidentify a symbol to its adjacent symbol rather than another random symbol. For example, in LoRa modulation, the adjacent bin drift is common as described in Section 3.2. With Gray coding, the bit error caused by adjacent misidentification is reduced to one bit per symbol, which has a high possibility to be corrected by the error correction mechanism. The standard Gray coding can be expressed aswhere \(v_0\) represents the raw demodulated bin value, \(\gt \gt\) is the bitwise right shift operation, and \(v\) is the Gray coding output. However, when we continue our analysis based on natural symbol mapping and standard Gray coding, we cannot decode packets with a 100% decoding success rate. Project RPP0 suffers from this problem and still contains an unfixed bug.⁷ To understand the reason, let us recall the LDRO mode of LoRa. when LDRO is enabled, the encoder will put data bytes into high \(SF-2\) bits of a symbol and then add “1” to reduce the influence of bin drift. From the hardware perspective, it is reasonable to add “1” under any condition, because we can save the circuit cost to judge whether LDRO is enabled. We guess that all symbols output from the encoder have one bin shift. If it is true, before we apply Gray coding, then we should subtract “1” from the demodulation results. Fortunately, our guessing is supported by the decoding results. The process of “G”in LoRa decoding could be modified asNote that TAPP also applies similar operations on Gray coding as we do, but they explain it as a different Gray coding mechanism and uses brute-force algorithm to derive the additional ”one.” However, it seems unnatural to use a non-standard Gray coding. By contrast, our explanation is more reasonable.

Deinterleaving. Interleaving is used to reduce the impacts of burst errors. With interleaving, the errors could be distributed to multiple bit groups and corrected by forward error correction (FEC). It is mentioned [26] that LoRa applies diagonal interleaving instead of conventional row-line interleaving. Figure 9(a) shows the column-major order row-line interleaving of eight symbols. For row-line interleaving, the LSBs (\(b_{i,1}\)) of the eight symbols are assembled into a byte. From Section 3, we know that the LSBs of a symbol are more fragile than the most significant bits (MSBs) of the symbol. From the perspective of FEC, it is a bad design to group fragile bits together. Figure 9(b) shows diagonal interleaving used in LoRa. Diagonal interleaving distributes the fragile LSBs into different bytes and is more robust. We then manipulate the transmitted packets to derive the detailed diagonal mapping. As an example, we send packets with \(SF=8\) and \(CR = \frac{4}{8}\) in implicit header mode. Thus, the interleaving block is an 8 \(\times\) 8 block as shown in Figure 9(b). First, we assume the FEC used in \(CR=\frac{4}{8}\) is the standard (7, 4) Hamming code with one bit extension. Therefore, after “G” and “W,” the codeword for nibble “0000” is “00000000,” and the codeword for “1111” is “11111111.”⁸ Suppose the sending bytes are all zeros except that the fourth byte is 0x0F, we observe \(b_{11}=b_{22}=\cdots =b_{88}=1\) in Figure 9(b). Therefore the main diagonal represents the fourth byte 0x0F. The one bin shift problem mentioned in Gray coding reflects here that we cannot always get eight ones in a block if we directly apply the standard Gray coding. Shifting the mapping by one solves this problem and perfectly matches our following decoding process. By changing the all-1 data bits in the transmitted packet, we can derive the entire mapping for interleaving as shown in Figure 9(b). For other parameters, the deinterleaving process is similar. The only difference is that the block size becomes \(\frac{4}{CR}\times (SF-2DE)\). As a result, we summarize the deinterleaving process aswhere \(c_{i,j}\) is the \(j{\rm th}\) bit of the \(i{\rm th}\) codeword after deinterleaving, \(b_{j,i}\) is the \(i{\rm th}\) bit of the \(j{\rm th}\) symbol after Gray coding, and \(i \in \lbrace 1, 2, \ldots , \frac{4}{CR}\rbrace , j \in \lbrace 1, 2, \ldots , SF-2DE\rbrace\).

Hamming Decoding. After deinterleaving, we get the encoded data in the form of codewords, but the position of data bits and parity bits in a codeword are still unknown. LoRa provides four valid CR values (\(\frac{4}{5}, \frac{4}{6}, \frac{4}{7}, ~\text{and}~ \frac{4}{8}\)), which determine the codeword length and thus FEC strength. When CR is set to \(\frac{4}{7} ~\text{or}~ \frac{4}{8}\), LoRa applies Hamming\((7,4)\) code or extended Hamming code with one additional parity bit. When CR is set to \(\frac{4}{5} ~\text{or}~ \frac{4}{6}\), LoRa can detect bit errors but cannot correct them. First, we assume that a nibble with code rate \(\frac{4}{8}\) is protected by the standard Hamming\((8,4)\) code. But our final analysis results show that this assumption requires some modifications. To determine the position of each data/parity bit in the codeword, we could vary the sending bytes but keep one specific bit fixed. For example, to test the position of the LSB of the fourth byte, we set the transmitting bytes as 0x01, 0x03, 0x05, \(\cdots\), 0x0F.⁹ Then the bit that is always 1 in the interleaving block is our target, i.e., LSB in this case. We find that the four LSBs in the codeword are data bits while the four MSBs are parity bits, which differs from the standard Hamming\((8,4)\) code \(p_1p_2d_1p_3d_2d_3d_4p_4\), where \(d_i\) is the \(i\)th data bit and \(p_i\) is the \(i\)th parity bit. Equation set (14) shows the relation between data bits and parity bits in standard Hamming\((8,4)\) code,

We denote the four LSBs as \(d_1, d_2, d_3, d_4\) sequentially. To find which bit in the MSBs is \(p_i\), we vary the data bits to keep \(p_i = 1\). For example, selecting the codewords with \(d_1 \oplus d_2 \oplus d_4 = 1\), the parity bit that always equals “1” is the parity bit \(p_1\). Similarly, the positions of \(p_2\) and \(p_3\) are derived. However, the remaining parity bit does not fit for the definition of \(p_4\). After careful observation, we find that it is a parity covering \(d_1, d_2\) and \(d_3\), i.e.,For other code rates, we can derive the bit position similarly. When \(CR = \frac{4}{7}\) is used, parity bit \(p_1\) is abandoned. When \(CR = \frac{4}{6}\) is used, parity bits \(p_1\) and \(p_2\) are abandoned. When \(CR = \frac{4}{5}\) is used, there is only one parity bit and it is natural to use \(p_4\) to cover all bits. The conclusion does not change when SF varies. Figure 10 shows the bit positions for different code rates. Note that LoRa applies non-standard Hamming code, the naming number of parity is arbitrary and our naming is just one kind of them.

Dewhitening. In Section 4.3, we assume the dewhitening operation happens after “G” and we prove that the dewhitening position does not affect the final results. We here discuss the “correct” position for dewhitening. The process of deinterleaving and Hamming decoding has been interpreted clearly above. We move the position of “W” to the other two positions and send all-zero bytes to derive the whitening sequence under different order selections. We find that “G\(\rightarrow\)W\(\rightarrow\)I\(\rightarrow\)H” gives different whitening sequences for various combinations of SF and CR while the whitening sequence of “G\(\rightarrow\)I\(\rightarrow\)W\(\rightarrow\)H” and “G\(\rightarrow\)I\(\rightarrow\)H\(\rightarrow\)W” keep the same. Using the same sequence for all packets is more reasonable. Thus we first exclude “G\(\rightarrow\)W\(\rightarrow\)I\(\rightarrow\)H.” Then we need to choose the correct order from “G\(\rightarrow\)I \(\rightarrow\)W\(\rightarrow\)H” and “G\(\rightarrow\)I \(\rightarrow\)H\(\rightarrow\)W.” LoRa chip datasheet [22] mentions that the whitening sequence in FSK mode is generated by a Linear Feedback Shift Register (LFSR). We guess that there also exists a LFSR generating the whitening sequence for LoRa mode. We apply the Berlekamp–Massey algorithm [38] on the whitening sequences of the two decoding orders to obtain the corresponding LFSR. For “G\(\rightarrow\)I\(\rightarrow\)W\(\rightarrow\)H,” no matter how we change the input order of the bits (e.g., LSB first or MSB first), the minimal LFSR size we get from the Berlekamp–Massey algorithm is at least 64. But we know \(2n\) bits sequence could always be constructed from an n-bits LFSR. Any sequence containing 128 bits could be represented as the output of a 64-bits LFSR. The LFSR of the “G\(\rightarrow\)I\(\rightarrow\)W\(\rightarrow\)H” sequence is too long. Carefully checking the whitening sequence of “G\(\rightarrow\)I\(\rightarrow\)H\(\rightarrow\)W” as shown in Figure 11, we find that it is not like typical random bits, and each byte seems to be the state of an LFSR. We collect the MSBs of the sequence bytes, as shown in the red box of Figure 11, to run the Berlekamp–Massey algorithm. The LFSR polynomial for deriving such sequence is \(x^8+x^6+x^5+x^4+1\). It can be seen from the literature [39] that it is the maximal-length polynomial for 8-bits shift-register. Figure 12 shows the structure of the LFSR, and we can see that all the eight bits of the register compose a byte of the whitening sequence, and each bit of the register is used to whiten one-bit data. Since the whitening sequence of “G\(\rightarrow\)I\(\rightarrow\)H\(\rightarrow\)W” can be generated by a LFSR with a much smaller length, we consider it the correct order.

After four steps of Gray coding, deinterleaving, Hamming coding, and dewhitening, the raw LoRa PHY packets are shown. In previous sections, we send packets in implicit header mode and disable CRC check. We next try to analyze the CRC algorithm used in the LoRa payload. From our observation in Section 4.2, CRC checksum occupies 16 bits at the tail of a packet. Therefore, we first send explicit header and implicit header packets with the same data content to check the coverage of CRC. The results show that the CRC is only performed on the payload. Calculating the CRC checksum of a series of bits, in theory, is considering it as a large binary number and calculating the remainder with respect to a “divisor.” The divisor is usually represented as a polynomial based on \(GF(2)\) (Galois field of two elements), e.g., \(x^3+x+1\). Analyzing the CRC part of LoRa PHY is indeed finding the polynomial used. A feature of CRC is that for a polynomial of degree \(n\), if the original dividend is 1, then the remainder is the polynomial itself. If we construct a packet only with last bit equaling 1, then the CRC checksum in the last two bytes tells us exactly what polynomial is used. Meanwhile, the CRC polynomial cannot be randomly selected; it must follow some principles to ensure some specific requirements. Therefore, it is natural to choose the polynomial from standard CRC polynomials. Comparing the polynomial we derive and the standard polynomial sets, it takes little effort to find the actual polynomial used. Based on the above analysis, we enable the payload CRC in implicit header mode and set the last two bytes of payload as 0x0001, 0x0080, 0x0100, and 0x8000, respectively (other bytes are set to zero). Since we are not sure the endian and LSB-MSB order in CRC hardware implementations, we test the four possible combinations. The result, however, surprisingly, is beyond our expectation. Whatever we set in the last two payload bytes, CRC checksum are exactly the same with the last two payload bytes. In common CRC implementation, before doing the remainder calculation, it will pad \(n\) zero bits after data to ensure that the last \(n\) bits are also under full CRC protection.¹⁰ The phenomenon we observed means that the zero-padding step is not implemented in LoRa PHY. Therefore, we send 0x0001, 0x0080, 0x0100, 0x8000 at the third and fourth bytes from last to derive the polynomial. The received CRC bytes are 0x1021, 0x9188, 0x3331, and 0x1B98, respectively. 0x1021 refers to the polynomial named CCITT-16, being \(x^{16}+x^{12}+x^5+1\). We apply CRC check with this polynomial and test packets with random bytes. The CRC checksums calculated by our implementation are consistent with the CRC bytes in all samples. Due to the characteristic of CRC, it is hard for a wrong CRC implementation to have the same checksum with the correct one even in a small group of samples, which means our result is correct.

The left unknown part about LoRa PHY is the header. In Section 4.2, we have already known that the header in explicit mode has 20 bits. Our goal is to find the organization and meaning of the 20 bits. Payload Length (PL) is said to be encoded in the header and occupies at least 8 bits, because the maximal payload length is 255. We vary the payload length and try to decode the packet using the same decoding process in implicit header mode assuming the header is also whitened. Unfortunately, we are not able to recover the data bytes in implicit header mode. The problem comes from the whitening sequence used for header. Nonetheless, by observing the bits that change with the payload length, we can identify the position of PL. Note that in the first 2.5 bytes, only the bits of PL and header CRC will change when varying packet payload length. We observe that the bits in positions 1–8 and 16–20 are possible PL bits. Additionally, the intermediate results show us that the header is not whitened and the first byte of a header before dewhitening is exactly PL. Our derived LFSR can only generate 255 possible whitening bytes. There are no more additional whitening bytes for header dewhitening. Hence it is reasonable not to whiten the packet header. Our following tests on finding other bits strengthen this assumption. Therefore, we do not apply dewhitening operation on header in the following. Changing one parameter and fixing other parameters, the position of code rate and payload-CRC-enable bit are determined similarly. The code rate bits are \(001, 010, 011, 100\) for \(CR=\frac{4}{5}, \frac{4}{6}, \frac{4}{7}, \frac{4}{8}\), respectively. The payload-CRC-enable bit is set to 1 if setting payload CRC on. In our experiments, there are five bits changing rapidly when setting different parameters. We assume these five bits as header CRC. There are three bits keeping zero whatever the parameter setting is, and we consider them as reserved bits. Figure 13 illustrates the header structure in the first three bytes.

We failed to find a standard CRC5 polynomial satisfying the header CRC results using the method in Section 4.5. But thanks to the linearity of CRC algorithm, we could use a CRC matrix to equivalently represent the CRC calculation.¹¹ We denote 12 parameter bits (PL, CR, and payload-CRC-enable bit) as a bit vector \(v_1\). We denote the five header CRC bits as a bit vector \(v_2\). Our target is to find a matrix \(M\) satisfying \(v2 = M\cdot v_1\). Since the value of \(v_1\) is under our control, we can design a series of \(v_1\), say, \(v_1^{(1)}, v_1^{(2)}, \ldots , v_1^{(12)}\). They form a matrix \(V_1\). The relative \(v_2\) series is \(v_2^{(1)}, v_2^{(2)}, \ldots , v_2^{(12)}\). They form a matrix \(V_2\). Therefore,\(V_2 = M \cdot V_1\). If \(V_1\) is an identity matrix, then \(M = V_2\).

How to make an identity matrix \(V_1\)? Despite the fact that we can control the header content, we cannot finely control the state of each bit. In our case, it seems impossible to send a packet with \(v_1\) containing only a single “1.” However, to our surprise, the LoRa chip unexpectedly supports sending a packet with zero payload length. The eight PL bits then become all zeros. If we disable payload CRC and set \(CR=\frac{4}{5}\) or \(CR=\frac{4}{6}\) or \(CR=\frac{4}{8}\), then only one code rate bit in \(v_1\) is 1. Is it possible that only the payload-CRC-enable bit is “1”? The answer is again using the linearity of CRC. Suppose we send two packets with zero payload and \(CR=\frac{4}{5}\). Packet A has payload CRC but packet B does not. Then the corresponding bit vectors are \(v_1^A = 000000000011, v_2^A = 01100, v_1^B = 000000000010, v_2^B = 00111\). Therefore,

A similar process can be applied for PL bits. Finally, we summarize the header CRC calculation as \(v_2 = M \cdot v_1\), where

We verify the effectiveness of our implemented LoRa PHY by testing if it can decode the LoRa packets transmitted from the commodity LoRa devices. We configure a commodity LoRa device to repeatedly transmit random packets to our implemented LoRa receiver. The ground truth of the transmitted bytes are recorded through serial port. For a comprehensive evaluation, we tune the LoRa transmitter with different parameters, including six SFs (\(7\sim 12\)), four CRs (\(\frac{4}{5}\), \(\frac{4}{6}\), \(\frac{4}{7}\), \(\frac{4}{8}\)), and two header modes (explicit/implicit), and with/without CRC, i.e., 6 \(\times\) 4 \(\times\) 2 \(\times\) 2 = 96 combinations. We use a carrier frequency of 475 MHz and a bandwidth of 250 kHz.¹² The payload length is set to 16 bytes. For each configuration, the transmission repeats 100 times. We place the LoRa transmitter close to the USRP receiver, and thus all signals are received with high SNRs. Since the communication environment is ideal, any packet missing reflects the incompleteness of implementation.

The result of the experiment is shown in Figure 15. Among all the configurations, BR only works at SF8 with implicit header, while RPP0 only supports the explicit header. Neither of these two methods supports CRC. As the decoding processes of BR and TAPP are both computationally expensive, they failed to work with large packet length under high transmission rate. Specifically, BR cannot decode any packets longer than 5 bytes, and TAPP starts losing packets when the duty cycle of the transmitter is higher than 0.5. Due to all these limitations, the overall PRR of BR only reaches \(4.2\%\). Besides, we can see that TAPP fails to decode any SF11/SF12 packets, and RPP0 cannot resolve SF11 packets. In summary, RPP0 covers \(1,\!924/9,\!600\approx 20.0\%\) packets and TAPP covers \(6,\!400/9,\!600\approx 66.7\%\) packets. On the contrary, our decoder can decode all packets under any LoRa configurations.

In this section, we verify the sensitivity of our LoRa decoder. We connect a commodity LoRa transmitter to our decoder through a 20-m RF cable with two step attenuators. Thus, we can finely control the signal attenuation by adjusting the value of these two step attenuators.

Using the value of the step attenuators as the channel attenuation is not accurate, because the wired links, including RF cables and connectors, also introduce attenuation leading to sensitivity estimation errors. Therefore, before the experiment, we use a spectrum analyzer, Keysight N9322C, to precisely estimate and calibrate the link attenuation. Then, we set the transmitting power to its lowest, i.e., \(-7.9\) dBm, to avoid wireless channel leakage. We experiment with the configuration of SF8 and bandwidth of 250 kHz. Since BR fails to decode any packets longer than four bytes, we do not compare its performance for the rest of this section. Figure 16(a) shows the sensitivity results of four SDR decoders. The criteria of sensitivity in LoRa is defined as the minimal RSSI achieving PRR > 90% when the payload is 32 bytes. As RPP0 and TAPP do not leverage the LoRa features for demodulation, their PRR drops quickly as the RSSI of the received signal goes down. The sensitivities of RPP0 and TAPP are \(-106\) and \(-108\) dBm, respectively. In contrast, the performance of our implemented decoder remains stable and achieves a sensitivity as low as \(-126\) dBm.

We further conduct an experiment under a larger SF, i.e., SF12. The result is shown in Figure 16(b). Our measured sensitivity is \(-142\) dBm, which approaches the optimal sensitivity of LoRa. Interestingly, we find that the sensitivity of the RPI gateway is only \(-139.5\) dBm. That is because the RPI gateway does not adopt the optimal design of the LoRa gateway. Therefore, our decoder has better performance than it.

Then we verify the validity of our proposed peak refinement strategy, which improves the frequency resolution by zero-padding. We use the same experimental setup as Section 6.2, evaluating both the FPA and CPA methods. Figure 17 shows the receiver performance under different levels of RSSI and zero-padding ratios. For both FPA and CPA, we see that PRR increases with higher zero-padding ratio \(r\). The performance improvement for \(r\) from 1 to 4 is observable, while for \(r\ge 4\) the performance improvement becomes negligible.

In this section, we present the evaluation of our implemented LoRa encoder, which is a reverse version of the LoRa decoder. The only difference in encoder implementation comparing to the decoder is the redundant bytes for filling the last \(\frac{4}{CR}\) symbols. Though these bytes seem fixed in LoRa chip implementation, they are meaningless on the receiver side. In our implementation, we pad zeros to fill the left bytes. When the encoder is up, it opens a UDP port and waits for data. We send 16 bytes to the encoder process through a python script. After that, the generated LoRa signal is sent from USRP to a LoRa end device. We check the serial port outputs from the device to see if the data are correctly transferred. We use different parameters (i.e., different SFs, CRs, and bandwidth) for sending data. Besides, for each parameter setting, the transmission is repeated 100 times. The final results show that 9,600 packets sent from our encoder could all be successfully decoded, which from another direction reveals that our analysis and implementation are reliable.

In Section 3, we propose FPA and CPA for LoRa demodulation. We prove that the performance of the two methods is close to the IDEAL demodulation theoretically. That is because they are resistant to phase jitter caused by non-perfect hardware and multi-path. So what is the actual demodulation method implemented in commodity LoRa chip? Does it adopt FPA and CPA? Interestingly, we find that the demodulation algorithm used in the LoRa chip is not as good as FPA or CPA. We construct LoRa packets containing specially designed five bytes. As a result, the four consecutive data symbols are the same with \(f_{start} = 0\), i.e., a symbol with a sharp frequency drop in the middle. Such a symbol is most vulnerable to phase misalignment between the two chirp segments. Before sending the packet via SDR, we manually add phase offset on the two segments. Our receiver using FPA and CPA are resistant to phase misalignment and their PPR is 100%. However, we find that the LoRa gateway faces frequent packet loss when receiving packets with significant phase misalignment. Figure 18 shows PRR with respect to phase misalignment. Its PRR is nearly 100% when the phase offset is between \((-\frac{\pi }{2}, \frac{\pi }{2})\). But it drops quickly when the phase offset becomes large. The result means the LoRa chip adopts a weaker demodulation method than our methods. In other words, the commodity LoRa chip is vulnerable to “phase misalignment attack.” Despite the fact that this kind of “attack” is hard to accomplish, any small defects are worth noting in the security area. We conjecture that the LoRa chip may compensate a fixed phase offset \(\Delta \varphi\) during the demodulation process, e.g., it estimates \(\Delta \varphi\) from the preamble and applies \(\Delta \varphi\) to the following data symbols. If \(\Delta \varphi\) changes, then the LoRa chip fails to demodulate the packet correctly. Thus, the PRR drops when the phase offset changes. Of course, we do not know the internal implementation in LoRa chip unless Semtech publicly releases the documents. But, in summary, the LoRa chip demodulation implementation is weaker than our FPA and CPA method.

We finally compare our decoder with a commodity RPI gateway in a real-world environment. We place the receiver outside a window on the second floor. We carefully select the locations of the transmitters, assuring that there is no Line-of-Sight path between the transmitter and the receivers, which reflects the common situation of LoRa communication. The transmitter repeatedly sends a 32-byte packet in explicit header mode with setting SF12, BW = 125 kHz, CR = \(\frac{4}{8}\), and CRC enabled. We lift the transmitter to 1.8 m in height with a tripod. Figure 19 shows the experiment setting mentioned above. Figure 20 shows PRR in different communication distances. Set PRR = \(95\%\) as a bar, our decoder supports a communication range as long as 3,600 m, while the commodity RPI gateway only reaches the maximum of 2,800 m. An interesting detail is the PRR fluctuation of RPI gateway in 2,400 m. We guess that the reason is the phase offset caused by the complex environment. As mentioned in Section 6.5, the commodity LoRa chip is vulnerable to phase misalignment problem. Therefore our decoder seems more reliable in outdoor communications.