6.1 Indoor Experiments
We conduct indoor experiments with co-located nodes in a lab environment, which allows us to better control the process and cover an extensive range of parameter settings. The experiments use 50 LoRa nodes. There are total of 16 CH/SF combinations used in the experiments (8 CHs and 2 SFs). In the experiments, we adopt the following default settings: 16-byte payload, 10-chirp preamble, and 2-dB transmission power. Each experiment lasts 1 minute and is conducted in different times of the days over 2 months.
Overall network performance. We evaluate the goodput, PRR, and node energy consumption when network demand varies under the four MAC protocols. Network goodput is the amount of meaningful data delivered per second. From Figure
14(a), we can see that compared with ALOHA, LMAC-1/2/3 improve the goodput by
\(1.52\times\) ,
\(1.87\times\) , and
\(2.21\times\) , respectively. Thus, LMACs can deliver data efficiently owing to the collision avoidance. Moreover, the performance increments of LMACs suggest the effectiveness of utilizing channel diversity and CH/SF selection. ALOHA goodput rises as the network demand increases but reduces once network demand is beyond 2,500 B/s. This is consistent with the existing understanding of ALOHA performance. The goodputs of LMACs saturate and remain flat once the demand is beyond 2,000 B/s. The saturation of LMACs goodput suggests the contention in channel utilization.
From Figure
14(b), the PRRs of LMACs are maintained above
\(90\%\) , whereas that of ALOHA decreases with demand. The decline is due to increasing collisions. Differently, LMACs maintain high PRRs through avoiding collisions and busy CH/SF.
Figure
14(c) shows the average energy consumed by a node per successfully delivered frame under the four MAC protocols. Compared with ALOHA, LMAC-1/2/3 reduce the energy by
\(2.08\times\) ,
\(2.37\times\) , and
\(2.38\times\) , respectively. Figure
14(c) also shows that LMAC-2 and LMAC-3 consume similarly low energy, whereas LMAC-1 consumes slightly higher energy than LMAC-2/3. This is because LMAC-1 nodes continuously perform DIFS to contend for the channel. Differently, LMAC-2/3 perform less CADs due to wise CH/SF selection. In the experiment logs, we find that LMAC-3 performs 1.92% less CADs than LMAC-2, whereas LMAC-2 performs 7.81% less CADs than LMAC-1. The goodput gain of LMAC-3 over LMAC-2 is due to the time saved from less CADs performed. Specifically, LMAC-3 nodes can utilize the saved time for frame transmissions, whereas the PRRs of LMAC-2/3 are similar.
Overall, LMAC outperforms ALOHA through collision avoidance and achieves up to \(2.21\times\) goodput improvement and \(2.38\times\) reduction in nodes energy consumption, while maintaining above 90% PRR.
Per-node performance. A closer look at the results from a single experiment will provide more insights into the advantages of LMAC over ALOHA. Figure
15 shows the cumulative distribution functions (CDFs) of the per-node network performance metrics. From Figure
15(a), the majority of ALOHA nodes suffer low goodputs that are smaller than 5 B/s, whereas the remaining ALOHA nodes achieve goodputs scattered from 28 to 74 B/s. The standard deviation of ALOHA’s per-node goodput is 26.79 B/s. In contrast, under LMACs, the CDFs do not exhibit undesirable long tails; the per-node goodputs are mostly within the range of 20 to 60 B/s, with a standard deviation of 14.89 B/s. The preceding results suggest that, compared with ALOHA, LMACs achieve better balance among the nodes in utilizing the communication medium.
Figure
15(b) shows the CDFs of the per-node PRRs. LMAC-2/3 achieve at least 80% PRR. The mean PRR values of all LMACs are above 90%. In contrast, about 40% of ALOHA nodes suffer zero PRR, as the frames are lost due to collisions. The remaining nodes have scattered PRRs from 0% to 95%. Experiment logs suggest that ALOHA nodes with high PRRs dominate the CH/SF and hinder the reception of other nodes.
Figure
15(c) shows the CDFs of the per-node energy consumption. Up to 30% of ALOHA nodes cannot deliver a single packet. Thus, they are not accounted for in the CDF. Another 40% of ALOHA nodes consume excessive per-frame energy due to collisions. LMAC nodes are rarely affected by frame losses. The total energy consumed for channel probing trades for the exemption of futile energy consumption for transmitting lost frames. Overall, all LMAC nodes outperform 85% of ALOHA nodes in terms of energy consumption.
In summary, a closer look into individual node performance reveals significant reduction in energy consumption per node, which is hidden in the earlier results due to averaging. Moreover, under LMACs, the goodput and PRRs of each node are more evenly distributed among nodes, whereas ALOHA nodes are clearly divided into the two extremes with a small number of exceptional nodes.
Balance and fairness. We evaluate the loads of all frequency channels under various MAC protocols. Since each channel supports multiple SFs, the frequency channel load ( \(\Psi _{CH}\) ) needs to be aggregated properly from the logic channels loads ( \(\Psi _{CH,SF}\) ) that are estimated by the gateway in the experiments. By averaging from SF7 to SF12 in each channel, the channel load for each frequency channel is obtained.
Figure
16 shows the scatter plot of PRR versus channel load. The area of PRR
\(\in\) [0.8, 1],
\(\Psi _{CH}\) \(\in\) [0, 0.1] is zoomed in to present a detailed view of LMAC performances. PRR of ALOHA drops quickly with the frequency channel load. The result of ALOHA shown in Figure
16 reinforces our understanding on the disadvantages of ALOHA’s completely uncoordinated transmissions. In contrast, the frequency channel loads of LMAC-1 are maintained within 0.1. This low load helps maintain satisfactory PRRs. This shows that LMAC-1 effectively avoids frequency channel saturation. More importantly, the channel selection mechanisms make LMAC-2 and LMAC-3 operating within the region with frequency channel load smaller than 0.1 and PRR within [80%, 100%].
Figure
17 shows the sorted average numbers of CADs per frame performed by LMAC-2 and LMAC-3. LMAC-2 performs more CADs and yields a steeper slope, which means higher variation. The additional channel information provided by the gateway helps LMAC-3 nodes avoid known congested channels and perform less CADs. Many CADs are performed for every frame due to the DIFS design. Recall that each DIFS slot contains 12 CADs. Moreover, CADs are also performed during the decrement process of
\(N_{BO}\) .
With the preceding experiments, LMACs yield a controlled and a fairer frequency channel utilization. This allows node performances to be more evenly distributed and ensure data delivery for each node. Moreover, the overhead of LMAC-2/3 is shown to be spread evenly among nodes where most nodes perform less than 250 CADs or 21 DIFS per transmission within a high contention network. The fewer CADs with high reception probability save radio energy for each successful frame delivery.
The latency introduced by LMAC is largely bound to the network’s contention levels. LMACs, under high contention levels, will incur higher latency due to the wait for free channels. However, ALOHA is not subject to this latency but suffers from much higher frame loss rate due to collisions, which translates to much higher delay if retransmissions are used to ensure the same level of PRR.
6.2 Outdoor Experiments
To investigate the feasibility and performance of LMAC in an outdoor environment, a set of experiments were conducted in our university area. The experiment makes use of four distributed controllers and 16 end nodes. To exhibit the advantages of LMAC in a high-density network with a low number of end nodes, the outdoor experiments use six CH/SF combinations. This set of experiments focus on LMAC-2 and LMAC-3 as the final yields of this work. We collect 30 minutes of experiment traces with 10 minutes for all protocols including ALOHA. Different from the indoor experiments, the outdoor experiments were conducted between 10AM to 10PM spanning across 7 days.
The results of the experiments are presented in Figure
18. Each point in the figure represents a node with the averaged SNR when the gateway received the frames. The standard deviation of SNR is between 0 and 3 dB. The PRR (Figure
18(a)) and goodput (Figure
18(b)) of LMAC-2/3 spread across vertically and begin to concentrate downward as SNR reduces. This shows that the nodes with higher SNRs (i.e., well placed or nearby nodes) can deliver their frames more reliably. However, nodes with lower SNRs (i.e., poorly placed or obstructed nodes) most likely cannot reach the gateway and fail to deliver their frames. In contrast, ALOHA nodes are scattered in the plots with no clear pattern. Although there are nodes that achieve very high PRRs and goodputs, the majority of nodes have low PRRs and goodputs. The average PRR and goodput of ALOHA are 0.22 and 15.98 B/s, respectively, whereas LMAC-2/3 reports 0.49 and 0.63 average PRRs with 25.17 and 34.83 B/s average goodputs, respectively. Notice that ALOHA has less points than LMAC-2/3. This is because more than
\(40\%\) of ALOHA nodes’ frames are lost at the gateway, resulting in no SNR values registered for those nodes.
To understand the actual traffic of the network for each protocol, Figure
19 presents each frame contribution to the effective traffic of the respective protocol. The results show that high SNR frames are more likely to be received by the gateway. Notice that the effective traffic of ALOHA only starts to increase when SNR reaches 4 dB, whereas LMAC-2 and LMAC-3 effective traffic starts to increase at an earlier SNR. The late increase of ALOHA shows that frames with lower SNR are often drowned by higher SNR frames. Differently, LMAC-2/3 allow low-SNR nodes to transmit only when the channel is assessed to have low collision probability. Moreover, the total effective traffic of ALOHA is lower than those of LMAC-2/3. This coincides with the earlier observation where more than
\(40\%\) nodes are not accounted for due to no frame reception.
The preceding results suggest that LMAC can be affected by environment dynamics but provides effective traffic control even for low-SNR nodes. Contrarily, ALOHA nodes are divided into two extremes where one yields good performance while the other suffers from low PRR hence low goodput.
6.3 Energy Overhead
LMAC energy overhead is mainly attributed to CADs performed under \(DIFS\) and \(BO\) windows. We derive the percentage energy overhead of LMAC in comparison with a conventional ALOHA transmission.
We review the indoor experiment results based on 50 nodes with 10 different traffic demands in Section
6.1 and derive the average number of CADs a node needs to perform to send out a frame with the different network contentions, denoted by
\(N_{CADs}\) .
\(N_{CADs}\) accounts to average CADs performed during
\(DIFS\) and
\(BO\) windows across multiple SFs. From that, the average energy overhead per frame of LMAC, denoted by
\(E_{LMAC}\) , can be computed as
\(E_{LMAC} = E_{CAD}\times N_{CADs}\) , where
\(E_{CAD}\) denotes the average energy consumption per CAD across all SFs. Therefore, the per-frame percentage energy overhead of LMAC is estimated as
\(\frac{E_{LMAC}}{E_{frame}}\times 100\%,\) where
\(E_{frame}\) denotes the transmission energy for a single LoRa frame averaged across all SFs. We estimate through energy measurements that
\(E_{CAD}\) and
\(E_{frame}\) correspond to
\(0.356\,\text{mJ}\) and
\(0.31\,\text{J}\) , respectively. We also find through measurements that
\(N_{CADs}\) is similar across LMAC-2 and LMAC-3, which is also signified by the per frame energy presented in Figure
14(c). Therefore, we compute the energy overhead of LMAC as a single entity for both LMAC-2 and LMAC-3. We then extend the percentage energy overheads of LMAC for varied network demands in Figure
20. We see an energy overhead of about 20% to 25%, which does not significantly grow with the increase of network demands (thus higher contentions).
Simply computing LMAC overhead in comparison with ALOHA frame energy
\(E_{frame}\) does not fairly represent a real LPWAN network’s performance that suffers from frame loss. We factor in the frame loss when considering the energy overhead of goodput and derive the PRR calibrated energy overhead as expressed in Equation (
1):
where
\(Er_{frame(LMAC)}\) and
\(Er_{frame(ALOHA)}\) correspond to the PRR calibrated per frame energy—that is,
\(\frac{E_{frame}}{PRR(LMAC)}\) and
\(\frac{E_{frame}}{PRR(ALOHA)}\) —respectively.
The two quantities characterize the energy costs per successfully transmitted frame with LMAC and ALOHA. We present the PRR calibrated energy overhead of LMAC atop ALOHA in Figure
20 as well. The results show negative overhead and attest that significant energy gains (40% to over 80% energy savings) are achieved with LMAC, especially under high network contentions where ALOHA is subject to much higher frame loss. Figure
20 suggests that at a light energy cost in CADs, LMAC is able to greatly improve the PRR, which gains energy savings in achieving the same amount of goodput.
6.4 QoS and Interoperability
This section discusses the benefits of adopting LMAC from a network service providers’ perspective. First, we experimentally show how the adoption of LMAC could incentivize network service providers by adding to their QoS. Second, as a single geographical region likely receives overlapping coverage from multiple network service providers, we give consideration to the fact that MAC-layer changes will be adopted gradually across different network providers. Therefore, from an interoperability perspective, both LMAC and ALOHA nodes are bound to co-exist until all ALOHA nodes are phased out. As such, we discuss the implications of gradually introducing LMAC to ongoing deployments. Third, the LoRaWAN standard must conform to global regulatory policies. We study regulations from multiple regions and identify possible restrictions that may limit the default use of LMAC. We propose simple modifications to LMACs to ensure full compliance under said regions.
For all evaluations in this section, we choose to compare the state-of-the-art ALOHA directly with LMAC-2. The choice of LMAC-2 over other LMACs is motivated by the fact that LMAC-2 allows for the easiest global regulatory compliance (discussed later) and the least changes to existing infrastructure (e.g., gateway functionality).
LMAC from a QoS perspective. A network’s ability to best serve all nodes is a qualitative indicator of the service provided by a network service provider. Under typical LoRa deployment conditions, frames from the farthest nodes arrive at the gateway with the weakest instantaneous signal power compared to those from nearby nodes. Therefore, to increase reliability, nodes farthest from the gateway are configured to utilize lowest data rates (e.g., SF12), whereas the closest utilize faster data rates (e.g., SF7) to save energy. However, not all node positions can be optimized for highest reception at the gateway. Often, a part of the network is placed farthest from the gateway or naturally receives higher environmental obstructions. Frames from these nodes undergo significant path loss, and we refer to those as weak nodes. The instantaneous signal strength of frames from the weak nodes is much lower compared to that from nearby nodes. A key problem with weak nodes is that their frames are most vulnerable to arbitrary inference. Therefore, improving the reception capabilities of the overall network including for those weak nodes is a key challenge faced by network service providers. Today, it is deemed the last mile problem in improving QoS for all LPWANs.
Improving QoS for weak nodes is non-trivial. Although a naive solution would be to augment network reception by simply adding more gateways, it is challenging to consistently keep up with moving nodes [
8,
38] and changing network dynamics [
40,
44]. However, Table
1 shows the minimum SNRs required for demodulation by a LoRa gateway for data rates ranging from SF7 to SF12. It can be seen from Table
1 that SF12 offers an additional link budget of 14.5 dB compared to SF7. SF12 frames are the most resilient of all SFs and are therefore chosen to be the most suitable data rate for weak nodes. Given that weak nodes mostly utilize slowest data rates, another question that arises is to what degree could LoRa’s orthogonality protect frames from weak nodes? One would expect that weak node transmissions will not be interfered by other concurrent orthogonal LoRa frames. However, this scenario has limitations. LoRa’s orthogonality works such that for a given concurrent transmission and a select SF, chirps from other SFs only contribute toward increasing
noise power during the process of demodulation. Recall that frames from weak nodes are the most vulnerable to interferences. This results in lowered SNRs for all concurrent frames, but this effect mostly impacts frames from weak nodes. Therefore, nearby concurrent transmissions, despite orthogonality, could impair the reception of weak nodes by increasing the noise floor. A study conducted by Semtech [
36] based on network traces collected from network providers estimates that as much as
\(18\%\) of SF12 frames can be affected by close proximity SF7 frames.
Given this phenomena, we conduct an experiment to evaluate the impact of using LMAC on the QoS for weak nodes. We set up an outdoor experiment with 16 nearby nodes and 1 weak node. The 16 SF7 nodes were placed within a radius of 300 m from the gateway to emulate interference from nearby nodes. A weak node was emulated by placing an SF12 node at a distance of 1,000 m from the gateway. Nodes within the close proximity utilized a fixed transmission power of 10 dBm and a faster data rate of SF7. The weak node communicated using a slower data rate of SF12 but at a higher transmission power of 15 dBm. All nodes utilized the same channel and none were positioned line of sight to the gateway, to ensure realistic conditions. During the experiment, the nearby nodes transmitted at varying demands under ALOHA and LMAC-2 while the weak node transmitted a known number of frames for each setting. At the gateway, the PRR and SNR of received weak frames (for SF12) were recorded.
Figure
21 presents the SNR and PRR variations of received weak frames. As shown Figure
21(a), the SNRs of weak frames under LMAC-2 is less impacted under increasing demands of nearby nodes. In contrast, when nearby nodes follow ALOHA, Figure
21(b) shows that the SNR of weak frames are decreased at higher demands. The SNR gains of weak nodes by adopting LMAC-2 translate directly to PRR gains as shown in Figure
21(c), as higher SNR frames are more likely to be received by the gateway.
The SNR improvements of weak nodes under LMAC-2 are a result of reduced collisions across nearby transmissions which in turn lead to reduced instantaneous interference to weak frames. Figure
22 further illustrates this phenomena in terms of a spectrogram that represents interference to a weak frame under two scenarios. CH1 shows interference to a frame from a weak node when nearby nodes adopt LMAC-2, whereas CH2 represents interference to a frame from a weak node when nearby nodes adopt ALOHA. Recall that LMAC-2 nodes perform carrier sensing and therefore spread potentially colliding frames across time. As such, the likelihood of same SF collisions across a channel is significantly reduced under LMAC, whereas ALOHA takes no effort to minimize collisions. Therefore, it can be observed that a weak SF12 frame is less likely to be interfered with two or more SF7 frames under LMAC-2. This results in less instantaneous interference on the weak SF12 frame by nearby SF7 frames. In comparison, under ALOHA, two SF7 frames colliding on top of a weak SF12 frame causes double the instantaneous interference.
Hidden terminals. CSMA relies on the ability of two nodes to sense each other’s transmissions. Hidden terminals occur when nodes are unable to sense each other’s transmissions but are likely to be received by the gateway. If two hidden terminals transmit simultaneously, a collision occurs at the gateway. Note that hidden terminals and ALOHA nodes behave indifferently as far as carrier sensing is concerned. LMAC, as a CSMA protocol, is also subject to the hidden terminal problem. In addition, the ISM band is not regulated to perform carrier sensing in all regions of the world. This allows the existence of some nodes that do not perform carrier sensing among nodes that choose to perform carrier sense. At the same time, even though some countries enforce strict Listen Before Talk, their regulations mandate the use of RSS-based sensing, which is not an efficient carrier sense mechanism for LoRa [
5,
18,
32]. Furthermore, the adoption of LMAC is also gradual, and more time is needed to transform an entire network to utilize CSMA capabilities. Therefore, at any given time, networks are likely to contain a combination of ALOHA and LMAC nodes. As a result, networks are forced to co-exist with both types of nodes.
To evaluate the impact on the performance of an LMAC-2 network when co-existing with hidden terminals, the quantity of hidden terminal needs to be controlled. However, emulating and controlling large numbers of naturally occurring hidden terminals is challenging. Therefore, for the sake of simplicity, we emulate hidden terminals with nodes that do not perform any type of carrier sensing (i.e., ALOHA). This enables experiments to be conducted such that the impact on LMAC-2 can be evaluated under different combinations of hidden terminals.
Figure
23 presents the performance of LMAC-2 under different hidden terminal combinations. Results suggest that even with 70% of hidden terminals, an extra 20% of throughput gains and 26% of PRR gains can be realized as compared to running pure ALOHA. In addition, with 40% of hidden terminals, an extra 40% of throughput gains and 71% of PRR gains can be realized as compared to a network that only conforms to ALOHA.
LMAC from a regulatory perspective. The choice of LMAC should not solely be driven by the need for higher network performance but also should ensure regulatory compliance.
Although not mandated by most, some regions may mandate equal use of channels for devices operating within the ISM spectrum, such as the Federal Communications Commission (FCC) [
5]. The FCC requires technologies that communicate using multiple PHY channels (e.g., LoRaWAN) to utilize each channel equally. The LoRaWAN standard under ALOHA achieves this by making a pseudo random choice for the transmission channel while regularly checking with local logs to ensure that no channels are overutilized. In contrast, LMAC-2 and LMAC-3 are designed with more intelligence into the transmitter and therefore make informed decisions for the transmission channel to minimize potential collisions. However, minor modifications to protocol are possible for full compliance. They are explained in the following.
As far as LMAC-1 is concerned, it simply controls a node’s behavior on time. LMAC-1’s channel utilization is no different from the existing practice of LoRaWAN. Specifically, instead of directly transmitting a frame, an LMAC-1 node first senses the channel and waits until the channel is free before attempting further transmissions. Therefore, LMAC-1 will be FCC compliant by simply conforming to existing LoRaWAN practices of channel utilization. However, LMAC-2 makes an effort to choose a less crowded channel. As such, the choice of channel in LMAC-2 is not pseudo random. However, recall from Section
5.2 that LMAC-2 still makes a random choice from a list of potentially free channels. Therefore, to comply with content presented in reference [
5], a node should memorize previous choices of channels (similar to current practice) and remove overutilized channels from the list of channels made available for future random selection. This allows all channels to be equally used on average by each transmitter in the long run. LMAC-3 can follow a similar approach to that proposed for LMAC-2 to ensure FCC compliance.