Holistic Energy Awareness and Robustness for Intelligent Drones

High discharge currents with high energy density, and flexibility in shape and size make Lithium-Ion (Li-ion) or Lithium-Polymer (LiPo) batteries ideal candidates for powering drones. Similar to any other battery chemistry, Li-based ones also have some peculiarities. Here, we highlight the effect of two main factors—(i) starting current, and (ii) age — on the LiPO battery used in our drones.

Impact of starting current: The initial battery charging happens at a constant current level (CC mode). After a while, the charging process reaches an inflection point beyond which the charging happens at a constant voltage level (CV mode). Higher currents allow faster charging, while at the same time, the battery capacity degrades more quickly over multiple cycles [Badam et al. 2015]. Over the entire empirical study, we experimented with different charging currents. Figure 4(a) illustrates the impact of the initial current on the charging time of the battery. The figure presents the change in the charging current, the battery voltage, and the cumulative energy delivered for the three different charging sessions. For this experiment, we considered the batteries of the same age, i.e., ones that had undergone the same number of charge-discharge cycles. Specifically, we looked at the time taken to deliver the last 140 Wh to charge the batteries fully.¹ The overall time taken with the initial current of 6.4A, 3.7A, and 1.8A is 72 minutes, 112 minutes, and 207 minutes, respectively. For applications that require higher drone uptime, with disregard for battery capacity degradation down the line, one must choose higher initial charging currents.

Observation. Higher starting current leads to a commensurate reduction in charging times.

Impact of Ageing: The energy density of Lithium-based batteries is affected by ageing [Badam et al. 2015]. Battery ageing is caused by cell oxidation—an irreversible process. Over multiple charge-discharge cycles, the usable capacity drops consistently. We observed the effects of ageing over 200 charging cycles spread across multiple batteries.

Figure 4(b) illustrates the impact of ageing on the time taken to deliver the final 140Wh of energy to charge two batteries fully. Both batteries are charged with the same initial charging current of 6.4A. Battery 1 is a new battery with fewer than five charging cycles, whereas Battery 2 had been put through over 130 charge-discharge cycles.As shown, the current and energy charging profiles are identical in the first 34 minutes, as both batteries are in the CC mode of operation. However, the voltage increase in Battery 2 is much quicker as it reaches the inflection point faster. After reaching the inflection point, Battery 2 continues to get charged in the CV mode at a much slower rate, as indicated by the flatter energy curve. Battery 1 switches from CC to CV mode much later—around the 45-minute mark. To fully charge Battery 1 and Battery 2, it takes 79 and 113 minutes, respectively. Thus, the time taken to fully charge batteries with the same quantum of energy depends on their respective age.

Observation. Older batteries reach the inflection point faster and take longer to charge a given amount of energy.

The charging process is non-linear (due to the CC and CV modes) and varies with charging current and battery age. Thus, there are opportunities for drone operators to create drone flight schedules that optimize drone uptime by knowing the charging profiles of various drone batteries in their fleet. Using the observations made through our empirical study, we now present our model to characterize the charging profiles accurately. Figure 4(a) shows how the current profile is flat until the inflection point and experiences an exponential reduction in the CV mode. On the other hand, the voltage profile follows a quadratic curve until it reaches the inflection point and stabilizes at a maximum value in the CV mode.

Let \(\rho\) represent the time elapsed since the start of charging until when the inflection point occurs. \(I^{start}\) is the starting current applied for charging, and the final maximum voltage attained in the CV mode is \(V^{max}\) , which is constant for a specific battery-charger pair. Let \(V_{t}\) and \(I_{t}\) represent the voltage and the current levels at time \(t \in [0,T]\) , where T is the total number of time intervals needed for charging. We model the current and the voltage profiles using a piece-wise curve fit. In the CC mode, when \(\rho \gt t\) , we have: \(\alpha\) and \(\beta\) are the quadratic coefficients that fit the voltage curve. These parameters varies for different starting current, battery age, and so on. In the CV mode, when \(\rho \lt t\) , we have:Here, \(\gamma\) is the exponential parameter to fit the current curve. Once again, this parameter vary for different starting current, battery age, and so on. The above equations can be combined as

For estimate the parameters such as \(\rho\) , \(\alpha\) , \(\beta\) , and \(\gamma\) , we use a Bayesian Inference approach that provides probability density functions for each of these parameters. We describe the complete Bayesian formulation in Table 1. Here, we assume that the error associated with fitting the voltage and current profiles are normally distributed \(\mathcal {N}(0, \sigma)\) . Also, we use weakly informative prior for the different battery charging parameter based on the range of values relevant to the domain. Using the Markov chain Monte Carlo (MCMC) method, we can utilize the evidence (i.e., known quantities such as \(V_{t}\) and \(I_{t} \forall t \in T\) ) to get the posterior distributions for the charging model parameters such as \(\alpha\) , \(\beta\) , \(\gamma\) , and \(\rho\) starting from the initial prior beliefs. The priors were set using the recommendations provided by Gelman et al. [Gelman et al. 2006].

Based on the past charging sessions, we can learn the different charging parameters. From our empirical study, we found that the value of \(\rho\) , changes with the age of the battery. However, for same-aged batteries, \(\rho\) corresponds to reaching a specific battery energy level—irrespective of the starting current. From Figure 4(c), we observe that both \(\alpha\) and \(\beta\) follow a linear relationship with the starting current, reflecting the fact that the higher the charging current, the more rapidly the voltage approaches \(V^{max}\) during the CC phase. This linear dependence helps in learning the parameters for a specific charging current by interpolating the parameter values for various other charging currents. On the other hand, the value of \(\gamma\) ( \(\approx\) 0.91), which pertains to the CV phase, does not change with starting current for the battery. While all (except \(\gamma\) ) these parameters change with battery age, the change is gradual. Thus, we can predict quite accurately the time taken to charge the battery.

Energy consumed to fly the drone depends on the current drawn by the motors to provide the lift and thrust needed to execute the various maneuvers. In turn, these depend on the prevalent environmental conditions (wind speed and direction), the weight of the payload carried, and so on. Here, we highlight the impact of two main factors—(i) the drone’s speed, and (ii) weight of the payload—on the battery drain observed. This study involved recording energy consumption data over 100 flights involving a diverse set of maneuvers in different wind conditions and carrying different amounts of payload.

Impact of speed: While flying the drone at a higher speed will result in a shorter time to complete a mission, it is unclear if the energy consumed will also reduce. Thus, to illustrate the impact of speed, we show the results from two specific flight paths—(i) linear horizontal flights between two points approximately 130 meters apart at varying speeds, and (ii) vertical flights between the altitudes of 10 and 50 meters above the ground at 2 m/s.

Figure 5(a) shows the horizontal speed and the power consumed of the drone moving between two points while switching its maximum speed from 8 m/s to 6 m/s and finally to 4 m/s. Clearly, the maneuvers are completed in a shorter time at higher speeds. However, the power consumption does not increase proportional to the increase in speeds. Thus, the reduction in energy consumed at higher speeds is the result of a reduction in time without an equivalent increase power. Figure 5(b) presents the relationship between vertical up and down movement and power consumption. Ascending higher is represented by a positive value of the vertical velocity component represented as \(V_{z}\) , whereas a negative value represents the drone descending toward the ground. As shown, working against gravity (while ascending) incurs a 30% higher power consumption on average compared with descending, assisted by gravity.

Observation. At higher horizontal speeds, the energy usage is lower due to reduction in time without an equivalent increase power. Ascending incurs a 30% higher power compared to descending.

Impact of weight: Drones often carry an additional payload for specific missions. Thus, it is essential to characterize the energy consumption when carrying different weights. To study the impact of weight in detail, we created two drone workloads, i.e., flight plans involving several maneuvers. We call the first workload Horizontal Oblique Triangle, in which the drone moves horizontally from the start point to the endpoint in a straight line separated by a distance of 240m at 10 m/s. After reaching the endpoint, the drone follows an oblique path upward to the center of this line but at a higher altitude of 20m at 5 m/s. Then, the drone returns to the start point by moving in an oblique path downward at 5m/s. This series of maneuvers forms a triangular flight path. The drone completes three such triangular flight paths for a given payload. This workload is completed in roughly 341 seconds. The second workload is called Vertical Shift Hover. Here, the drone ascends from a height of 10 to 30 meters in 10 meter increments. At each point, the drone hovers for 5 minutes before ascending. After hovering for 5 minutes at 30 meters height, the drone descends for landing. This workload takes approximately 920 seconds to complete. Table 2 summarizes our findings of executing these two workloads with different weights attached to the drone. We observe a linear relationship between the energy consumed and the drone weight.

Observation. The increase in weight results in a linearly proportional increase in energy consumption.

Having discussed the impact of speed and payload, we want to predict the instantaneous power consumed while performing various maneuvers. By integrating the predicted power values, we can easily calculate the energy spent on a given set of maneuvers. Below, we discuss our power prediction model in more detail.

We can think of the power required to maintain a stable flight as a combination of several components. Let \(P_{xy}\) , and \(P_{z}\) be the power needed to sustain the flight in the horizontal and vertical direction, respectively. Similarly, let \(P_{wind}\) be the power demand to resist the deviation due to the wind. Finally, \(P_{drag}\) is the power necessary to move against drag. We know that Power = Force*Velocity = Mass*Acceleration*Velocity. Note that while hovering, drone’s vertical acceleration ( \(A_{z}\) ) has to be equal and opposite to acceleration due to gravity, i.e., 9.8 \(m/s^2\) near sea-level.

Here, m is the mass of the drone, \(A_{xy}\) and \(V_{xy}\) are acceleration and velocity in the horizontal direction. Similarly, \(A_{z}\) and \(V_{z}\) are acceleration and velocity in the vertical direction. \(V_{wind}\) denotes the velocity of the wind. Further, from drag equations [at National Aeronautics and Administration 2019], we know that \(P_{drag}\) is proportional to \(V_{xy} \cdot (V_{wind} + V_{xy})^2\) . Note that acceleration and velocities are vector quantities and have both magnitude and direction associated with them. One can find a closed-form expression linking the overall power consumption of the drone, given by \(P_{total}\) . However, these components may not be independent of each other. Thus, we choose random forest (RF) regression to learn a power prediction model using the four components of power as features to predict instantaneous power use. We choose RF regression out of Support Vector Regression (SVR) and K Nearest Neighbour (KNN) as RF gives the best performance, as shown in evaluation Section 7.

We now turn to the inference performed on the drone, specifically for (aerial) vision-based tasks. We characterize the various tradeoffs in executing ML inference workloads, such as object detection on edge devices. We experiment with two edge platforms, a Raspberry Pi 4 with a Google Coral USB Accelerator (Coral+RPi4) and Nvidia Jetson Nano. To characterize the object detection performance, we fine-tune two Single Shot Detectors (SSD) [Liu et al. 2016], with MobileNetV1 [Howard et al. 2017] (MV1) and MobileNetV2 [Sandler et al. 2018] (MV2) backbone networks. The detectors are pretrained on MS-COCO dataset [Lin et al. 2014], which comprises natural and “on-ground” scenes. We conduct our experiments on the UAVDT dataset (details in Section 6.1). This combination of the SSD detection model with the MobileNet backbones is known to be accurate while being efficient and minimizing latency, even outperforming computationally more expensive models like FasterRCNN in UAV scenes [Du et al. 2018].

Impact of Altitude and Lighting conditions: Inference tasks are heavily dependent on the characteristics of the data. For instance, flying the drone at a lower altitude would consume more energy, as a smaller area is captured in the field of view of the camera, requiring the drone to travel more distance to cover a target area. A decrease in flying altitude will result in a quadratic increase in the distance to be covered by drone, and thus an altitude drop is costly. However, this may ultimately be necessary if the DNN model suffers from unacceptable degradation in accuracy from using the more distant imagery obtained at higher altitudes. Furthermore, changes in weather conditions, such as fog, could reduce visibility and necessitate flying at a lower altitude. We do not consider the direct power consumption attributable to computation, which could consume \(\approx\) 5-10W (E.g., Jetson Nano, etc.)—equivalent to less than 2% of the energy consumed by the drone used in this study.

Figure 6 shows detection accuracy across various altitudes and lighting conditions for the different hardware and model combinations. As expected, increasing altitude of the drone results in a drastic drop in accuracy, due to the smaller size of the objects in view. It is interesting to note that night time conditions yield better accuracy than day time conditions. This is possibly due to models learning to identify vehicles based on their headlights and taillights being on during the night. Also, foggy conditions are especially bad for model performance.

Observation. Altitude and lighting conditions significantly influence the accuracy of the sensing task.

Having discussed the tradeoffs between various on-board edge devices, model quantization and drone altitude, we present our framework for predicting accuracy, given the edge device, model quantization level, the lighting condition and the flying altitude of the drone. Consider altitude as a discretized variable alt (with a bin-size of say b metres), while the variables relevant to a model such as model quantization, lighting, and so on, are denoted by o. Here, we model the performance to be a set of piece-wise linear functions of altitude, conditioned on the other variables. We sample the value set for performance and altitude from the experiments we have conducted earlier,where m and c are computed from the accuracy values of the task after flying the drone at the specified altitude. Given a desired accuracy level, we can now figure out the highest altitude to fly our drone at. At this altitude, our sensing will be accomplished at above the expected accuracy level with the largest field of view, thus minimizing the energy consumed.

Concave Partitioning: The shape of the area to be surveyed could either be convex or concave. In case of a concave polygonal shape, optimum sweep direction may not guarantee coverage in minimum time due to redundant turns. Therefore, a set of convex sub-regions ensures that for each sub-region, we traverse in the optimum sweep direction to minimize energy consumption. Thus, we partition the concave space into a set of convex sub-regions [Keil and Snoeyink 1998].

Finding Transects: In this step, we determine the optimum sweep direction for each convex sub-region. We utilize the algorithm described in [Araújo et al. 2013], which determines the direction of the maximum width of the polygon obtained by rotating it at every orientation (iterated at 1 degree in our implementation). Furthermore, if two adjacent sub-regions have similar optimum sweep direction, we merge them into a single sub-region, via a threshold \(\theta _{min}\) . Transects are formed along the optimum sweep direction. The other factor determining the number of transects is the altitude (which defines the field of view) and it is estimated from the desired accuracy level of the sensing task (as described in Section 2.6). For a given sub-region, and given sweep direction and altitude, there are four ways in which the entire polygon can be surveyed. Our energy model requires acceleration data to estimate energy usage to cover various transects. Thus, we simulated acceleration data for different transects.

Connecting Sub-Regions: After obtaining the transects, we need to traverse between the sub-regions in such a way that energy consumption is minimized. We call the inter-region traversal distance as transition distance. For connecting the sub-regions, we should minimize the sum of transition distance from one sub-region to another. A brute-force approach compares the total transition distance for all the possible orderings of the sub-regions and for each permutation of four possible traversals of a sub-region. This approach is computationally expensive as run-time is exponential to number of sub-regions. Thus, we first decide the order of visiting every sub-region. We model the sub-region ordering problem as a graph problem. Each sub-region is considered as a vertex, and the edge weights are the inter-centroid Euclidean distance between them. The proposed formulation is equivalent to the travelling salesman problem (or the source-t min cost Hamiltonian Path problem) in a metric graph (due to the definition of edge weights). However, the formulation is NP-Hard and a polynomial time approximation heuristic exists, the Christofides algorithm [Christofides 1976; Hoogeveen 1991]. The upper bound is 3/2 for the general case and 5/3 for the s-t case. These are currently the best known bounds for the problem. This algorithm gives us the order of traversal of the sub-regions.

We have obtained the ordering of traversal between the sub-regions, however, there are four possibilities while traversing each sub-region. The traversed path can be found by comparing the distance between the four possible end points of the first convex sub-region to the four possible end-points of the second sub-region. After connecting the points with minimum distance, the two sub-region are combined and there are only two possible end-points of the combined sub-region. Then, compare the distance between the two end points of the combined sub-region with the four end points of the third sub-region and so on to get the final path.

We formulate an ILPprogram (described in Table 3) to solve task division among the available drones. The assumptions in the formulation of the optimization problem are as follows:

Let \(\tau \in \Gamma\) be an ordered list of transects and \(d \in D\) be the set of available drones. Each drone d takes time \(T_{\tau ,d}\) to cover transect \(\tau\) and consumes energy \(E_{\tau ,d}\) . As we know the traversed path and can simulate the drone dynamics, \(E_{\tau ,d}\) can be estimated by calculating the instantaneous power from simulated acceleration using the model described in Section 2.4, and then integrating the predicted power values. The drone d has to travel from the ending of transect \(\tau\) to the start of transect \(\tau +1\) (take a turn), and for this it has to waste time \(\hat{T}_{\tau ,\tau +1,d}\) and spend energy \(\hat{E}_{\tau ,\tau +1,d}\) . \(\hat{E}_{\tau ,\tau +1,d}\) is again estimated using the model described in Section 2.4, similar to how \(E_{\tau ,d}\) is estimated. Each drone is allocated a contiguous list of transects to decrease the number of turns (which are wasteful as noted earlier). Let the battery capacity of drone d be denoted by \(B_d\) and time taken to charge the exhausted battery till the constant current charging mode be \(C_d\) . We can reliably estimate the battery profiles ( \(B_d\) and \(C_d\) ) using the models described in Sections 2.2 and 2.4 as the battery charging process is linear (CC mode). Furthermore, we want to know if the drone has visited the transect \(\tau\) . Let the binary indicator variable \(V_{\tau ,d}\) denote if d has visited \(\tau\) , while another binary indicator variable \(\hat{V}_{\tau ,\tau +1,d}\) denote if d has covered the distance between \(\tau\) and \(\tau +1\) . In our formulation shown in Table 3, we would want to minimize the time ( \(\mathbb {M}^{time}\) ) in which the given task can be completed. To minimize \(\mathbb {M}^{time}\) , we only need to look at the drone that takes the maximum time to complete its task. Let \(E_d^{total}\) be the total energy and \(T_d^{total}\) be the total time taken by d to cover the task assigned to it. These two quantities can be defined in terms of the indicator variables, \(V_{\tau ,d}\) , \(\hat{V}_{\tau ,\tau +1,d}\) , the time \(\hat{T}_{\tau ,\tau +1,d}\) and energy \(\hat{E}_{\tau ,\tau +1,d}\) .

In the formulation, constraint (1) ensures that the variables \(V_{\tau ,d}\) and \(\hat{V}_{\tau ,\tau +1,d}\) are binary, i.e., a transect or transition between transects is either completed or not completed. Constraint (2) ensures that the intermediary distance between \(\tau\) and \(\tau +1\) is considered as visited by drone d, if and only if the same drone visits both the transect \(\tau\) and \(\tau +1\) . Such a constraint ensures that a drone covers contiguous transects. Also, any transect must be visited only once in total (see constraint (3)). As we handle cases where all drones are used for the task, the total number of transitions between transects is constrained by the number of transects \(\Gamma\) subtracting the number of drones D, and is represented by (4). (6) denotes the number the times that drone d has to charge, to complete its task. Constraints (5) and (7) represent the total energy consumed and time taken by a drone d to complete the task allocated to it. Constraint (8) denotes that the total time taken to finish all the tasks (i.e., \(\mathbb {M}^{time}\) ) is at most equal to the maximum time taken by any drone in \(d \in D\) .

We verify the task optimization formulation soon after assigning transects to individual drones. For input, we have time and energy taken by each drone to complete a transect. For verification, the output after execution provides a set of contiguous transects to each drone.

We verify the Task Scheduling Optimisation problem, as shown in Table 3, over a survey area shown in Figure 7, which generates 70 transects with 69 inter-transects that requires to be distributed among three identical class of drones. The input Table 4 shows the time and energy input required by a drone on transects and inter-transects. In the table, the even value of k represents transects while the odd value represents inter-transects.

Table 5 shows the results of the formulation. The first column of results Table 5 shows a set of transects assigned to each drone, while the second and third columns, respectively, depict the total time and energy taken by each drone. The fourth column shows the number of times a drone needs a recharge. The value \(N_d \gt 1\) for each drone implies that each drone needs recharging at least once. The total time is the sum of Transect time, inter-transect time and charging Time( \(N_{d} *C_{d}\) where \(C_{d} = 7200\) ). Hence, we can now verify the time and energy calculations for each drone. These calculations are presented in Table 6.

By calculation shown in Table 6 a, we get the total time as 16930.42s, 16315.97s, and 16933.93s for the three drones. Similarly, from Table 6 b, for the three drones, we get the total energy as 270.13, 257.81 and 268.62 after calculation. Compared with Table 5, we get a maximum error of less than 0.001% for both time and energy calculations.

Our current task scheduling formulation, shown in Table 3, minimizes the maximum time taken by a drone to complete its assigned transects. A situation may arise where the longest transect is significantly longer than other transects. Therefore, the time and energy taken to cover this transect are significantly high compared with the time and energy taken to cover other transects. We propose a further optimization to our algorithm and consider the possibility of partial transects, where the longest transect is split into smaller partial transects. These new partial transects and previous transects, together form a new set \(\Gamma ^{\prime }\) , are then reassigned to the drones by applying our formulation again, with \(\Gamma\) changed to \(\Gamma ^{\prime }\) . This process of splitting the longest transect and reassigning it to drones is iteratively repeated, to further reduce \(M^{time}\) , until a stopping criterion is met. The stopping criteria can be a finite number of iterations. Too many iterations can increase transects, which means more transition points. Drones would waste time and energy due to these unwanted transition points. This has been sown in Section 7.3

In this section, we show our new formulation where we consider both time and energy required to reach the charging station (unlike in Table 3 Assumption 2). Consequently, our goal is to provide feasible solutions by placing several charging stations at appropriate locations. The formulation considers the drone’s number of visits to charging stations as well as its residual battery energy. For larger survey areas, the drone requires several recharges to complete the task. A visit to a charging station is mandated whenever the drone’s battery level falls below a specified threshold value, denoted by \(E_{th}\) . For a trip to a charging station, Table 7 captures the required formulation as follows:

In the new formulation of the task scheduling optimization shown in Table 7, while Constraint (6) defines the variable \(X^c_{\tau ,d}\) as non-decreasing with respect to \(\tau\) , Constraint (7) ensures that a drone can visit only one charging station from a transect. Constraint (8) ensures that the drone goes for charging only after it has expended \(E^{*}_{th,d} = B_d - E_{th,d}\) amount of energy, which includes the energy required to visit a charging station ( \(E^c_{\tau ,d}/2\) ). Constraint (11) ensures that the total number of times a drone requires charging is equal to the sum of trips to the charging stations.

The formulation presented in Table 7 not only allocates a contiguous list of transects to a drone but also chooses a transect which offers the least energy required to reach a charging station. The output of the formulation depends upon the number of charging stations, their location and \(E_{th}\) (or \(E_{th,d}\) ).

Another point of consideration about the charging station is that there is neither delay nor contention in obtaining the service from the charging station. The charging station is expected to support fast charging to minimize service time. The Task Scheduling formulation shown in Table 7 has the ability to handle the allocation of the nearest charging station to a drone and assumes there is no contention.

In this section, we present the system architecture consisting of drones and the BS. We also mention a few readily available data from this system, which we exploit to provide robustness to our system. We explain our system using Figure 9, which represents our system architecture and is explained here. Each node fetches sensor state from connected sub-modules such as IMU, GPS, and Telemetry Radio. We choose our autopilot as the open-source ArduPilot firmware running on Pixhawk autopilot board. The autopilot features MAVLink, drone-specific protocol based on publish-subscribe model, to communicate between various external sub-systems such as BS and companion computers such as RPi. The drone communicates with the BS via a Telemetry Radio (SikRadio) wireless serial link at 433 Mhz using MAVLink protocol. The drones and BS form a wireless sensor network. Each drone acts as a wireless sensor node (henceforth drones are referred to as nodes) with the capability to transmit current state to other nodes either directly or through multi-hop transmission, as seen in Figure 8.

This section describes our methodology which is executed at the BS when a link failure is detected. We present our observations on the variation of link strength with the distance between nodes, obtained from our flight log dataset. The trigger point is determined using these observations.

When the communication link starts to degrade, the node requires alternate communication links via which it can send packets reliably to the BS i.e., a reliable communication link needs to be established between the affected node and BS. One way to accomplish this is for the BS to compute alternate routes based on PDR and get a disjoint set of routes called RouteList. It employs the wireless nature of the link to broadcast a ReqCom including the RouteList.

The BS broadcasts the ReqCom and waits for time \(TR_{th}\) to receive the status of \(D_{1}\) . Meanwhile, it computes the remaining transects during \(TR_{th}\) and uses the algorithm in Table 3 with the change in sets of transects from \(\Gamma\) to \(\Gamma\) ’ and set of drones from D to \(D^{\prime }\) to assign the unfinished transects to the remaining functional drones. For further optimization, drones near the faulty drone should be considered in set D’. What should be the value of \(TR_{th}\) ? Let the Maximum number of hops between a node and BS be Mhop. A ReqCom request requires Mhop time intervals to reach the farthest node, and a node then requires one time interval to start transmitting via an alternate route and the packets require another Mhop time intervals to reach the BS. Hence, We compute \(TR_{th}\) to be \(Mhop*2 + 1\) . If BS fails to receive any packet from node \(D_{1}\) during this time, it uploads the re-assigned tasks to the nodes.

Existing GCS softwares are highly sophisticated and versatile as they provide support for various platforms, such as Pixhawk [Meier et al. 2015]. They are capable of handling detailed telemetry and uploading flight plans using multiple connection options such as TCP/UDP, and so on. Thus, we implemented our system as an extension in QGroundControl (QGC). QGC is open-source, and the most widely used GCS with a significant active contributors base. Specifically, we implemented our algorithm as an additional tab called Smart Survey, which is an improvement over the existing Survey tab that implements a basic path planning algorithm with no energy considerations. Figure 14 shows the UI design for our Smart Survey tab. The entire implementation involves around 2000 lines of C++ code and around 400 lines of QML code (our open-source implementation: https://github.com/t-sriyen/qgroundcontrol). For simulating accelerometer values for the energy predictions, we implement Allan variance algorithm [gnss 2021].

To learn the battery charging parameters, we used PyStan, a Bayesian modeling library with several MCMC samplers. We use Sklearn, a Python-based machine learning library to train our energy discharging models. As part of this work, we release an open-source toolkit consisting of the battery charging and discharging models. Furthermore, the toolkit also contains a framework to calculate the optimal altitude that minimizes energy usage of drones for any UAV-based sensing, given enough domain information. Moreover, we also release our detailed drone flights datasets containing GPS and IMU logs along with flight plans (our open-source toolkit: https://github.com/t-sriyen/DroneEnergyModeling).

Our object detection models are trained and fine-tuned on a server equipped with a Nvidia Tesla K80 equipped with 64 GB RAM using Tensorflow 1.12 with the Tensorflow Object Detection API. Our trained models (SSD-MV1 and SSD-MV2) cannot be executed as-is on either edge platforms and thus were converted to their respective proprietary formats—i.e., tflite model for Google Coral and UFF model for Jetson Nano.

For our experiments, we use TATTU Premium 22.2V 6-cell LiPo batteries with an 8000mAh capacity and discharging rate of 15C. We used a quadcopter with an onboard Pixhawk flight controller along with a RaspberryPi 3B+ as a companion computer to log IMU data, along with energy used to perform different drone maneuvers. For observing charging trends of the battery, we used a Turnigy Reaktor D6 Pro Duo charger and a coulomb counter, designed using IC LTC2944[LTC2944 2017]. Figure 13(a) shows this energy monitoring setup. For experiments involving ML inference on the edge platforms, our power logging setup involved a current sensor (ACS 712) and an SMPS regulated at 5 volts, shown in Figure 13(b).

We leverage P4[Bosshart et al. 2014] i.e., Programmable Protocol-independent Packet Processing language to implement our reliability algorithm and process the MAVLink messages. We use Mininet-WiFi[Fontes et al. 2015], a network emulation tool to emulate three drones and a BS. We assume for our article that the BS has acquired the topology information from MAVLink messages using P4 and computed a RouteList. Additionally, we assume BS has notified the drone \(D_{1}\) with a ReqCom containing the RouteList. There are two cases: one where D1 has failed and hence, no heartbeat packets can be transmitted. In the other case where it is alive, it generates packets and appends the RouteList header having a defined header structure. The interface replicates the packet to the first hop of the two routes Route1 and Route2.

Each node in the network runs our P4 program. In addition, each node periodically receives broadcast information about neighboring nodes together with signal strength, from the control plane using the OS’s wireless interface driver, and is stored in data plane memory in the NIC of the node toward fast lookup using P4 match-action tables. We parse each incoming packet’s header in the data plane. We use the IP header’s ID field to identify if a packet is a duplicate. If it is an original packet, the next hop’s PDR is compared with \(PDR_{th}\) and if it is found to be higher, then it is forwarded to the next hop. However, if it is lower, it replicates the packet and sends it to the next hop of a neighboring route. Figure 21 demonstrates that our algorithm ensures reliability due to a node continuously going out of range and still maintaining PDR using replication and elimination in comparison to no replication.

ML Inference: We conduct our experiments on the UAVDT [Du et al. 2018] object detection dataset, which consists of 80,000 frames of UAV flights. The frames are labeled with object bounding boxes of vehicles and other attributes—such as weather conditions, flying altitude, and so on—that are useful in characterizing performance of object detection algorithms in various real-world scenarios.

Path Planning: We created a dataset of prominent public places, including parks, piers, and city squares, and so on spread across four cities—New York, San Francisco, Bangalore, and London. These areas vary greatly in shapes and sizes, ranging from 0.2 to 5.7 sq. km. These sites form the set of shapes given as input to the algorithms. We will release this dataset as part of the toolkit to aid benchmarking.

Energy Modeling: For this, we use Mean Absolute Percentage Error (MAPE) as our performance metric.

ML Inference: We measure the accuracy of inference using Mean Average Precision with \(IOU\ge 0.5\) ([email protected]) [Everingham et al. 2010].

Path Planning: We characterize the input shapes using convexity, which is defined as the area of the shape divided by the area of convex hull of the shape [Sonka et al. 1993]. We characterize the performance by measuring both energy and the time taken for a planned path.

Energy Modeling: We compare the performance of our energy discharge model with the start of the art [Tseng et al. 2017], which modeled the drone energy consumption as a linear combination of several factors that include the drone weight, wind speed, velocity, and so on.

Path Planning: We compare the performance of our path planning approach with three existing algorithms - DroneDeploy (East-to-west algorithm) [DroneDeploy 2020], QGC [Zurich 2013] and FarmBeats [Vasisht et al. 2017] using energy consumption as the metric. DroneDeploy generates sweeping patterns from east to west or west-to-east regardless of the shape of the surveying area. FarmBeats generates the path that minimizes number of waypoints. In QGC, the sweep direction is left to the user to select. In our evaluation, we kept the sweep direction fixed from north-to-south.

Task Scheduling: We consider the equal transect division algorithm as our baseline, i.e., the transects are equally divided amongst all the drones available to us. This is in contrast to our system wherein a linear programming model is proposed. We also test the iterative greedy adjustment on one of the largest survey area. We ran the iterative approach for 15 iterations without any threshold.

Charging Station: We consider our formulation presented in Table 3 as baseline. We use \(M^{time}\) as a metric to compare the impact of the location and the number of charging stations.

Using our energy model, we tried different regression algorithms: RF, KNN and SVR. Our model consists of several components (i.e., \(P_xy\) , \(P_z\) , \(P_{wind}\) , \(P_{drag}\) ) as features. Our model outperforms baseline model as shown in Figure 22. We used over 100 different drone flight data and categorized them into four groups. The first two groups are called vertical and horizontal flight datasets. As the name suggests, these flights only involve drones either moving in horizontal or vertical directions. The vertical triangle group is similar to our workload discussed in Section 2.4. In the dataset group called full, every flight path was included.

Within each group, we performed a leave-one-out cross-validation approach to characterize the performance of our model (Proposed) compared with Tseng et al. [Tseng et al. 2017] (see Figure 22). As stated earlier, we model the energy consumption using a RF regression as it out performs the other (KNN and SVR) algorithms. Also, with more data points, as is the case in the full dataset group, the performance improves to an average MAPE of 5.6% for the predicted energy consumption across all manoeuvres. Clearly, our model significantly out performs the model presented by Tseng et al. [Tseng et al. 2017], as our average MAPE values are 2.5x lower than theirs in the average case. Our proposed method performs even better in horizontal and vertical manoeuvres, where we report 3 to 3.5x improvements in MAPE. We conducted a detailed evaluation of our charging models and observed an average MAPE of around 1.94% in predicting the transition point between the CC and the CV mode. Furthermore, the \(R^2\) fit of the charging models for voltage and current over 100 sessions never dropped below 0.98.

Summary: Our energy model demonstrates an average MAPE of 5.6% — a 2.5x improvement over the state-of-the-art.

To evaluate our system’s performance on path planning tasks, we compare it with three baseline algorithms (DroneDeploy, QGC, Farmbeats) on a dataset discussed in Section 6. In many different real-world surveying tasks, such as mapping districts, river deltas, and so on, the underlying shape is not convex. Thus, in Figure 15 we show the graph of the relationship between relative energy consumption w.r.t DroneDeploy and Area Convexity. Area Convexity is defined as the ratio of the survey area to the area of its convex hull. When the convexity is less than 0.95, the proposed algorithm shows significant improvement in energy consumption. The percentage improvement in energy when the convexity lies between 0.85 to 0.95 is 11.29%. The percentage improvement in energy consumption when the convexity is between 0.7 and 0.85 is 21.14%. The percentage improvement in the energy consumption when the convexity is greater than 0.95 is 3.09%, which is similar to the Farmbeats algorithm. Hence, when the shape is not convex, it is beneficial to partition the shape. When the shape is nearly convex, the paths generated by FarmBeats and our algorithm are very similar. Hence the energy consumption does not change too much.

Summary: Our system outperforms the three baseline algorithms in energy consumption. The improvement is more pronounced when the area convexity is lower.

To evaluate the performance of our proposed system for multi-drone scheduling, we use the same dataset used in the above path planning evaluation. We vary the number of drones and generate the path using our algorithm and the equal transect division algorithm. Figure 16 shows the relative time taken by three approaches—(i) DroneDeploy (Equal), where the path is generated using DroneDeploy and task division is done using equal transect splitting algorithm, (ii) Proposed (Equal), where our system does path planning and task division is done using equal transect splitting algorithm and (iii) Proposed (ILP), where the path planning and task division is done by our system using the proposed linear programming approach. As expected, the time taken decreases with an increase in the number of drones. In all the cases, the Proposed (ILP) based task division algorithm results in the least time taken. Specifically, with just two drones, the average reduction in time taken to complete surveying tasks in our dataset by Proposed (ILP) compared with DroneDeploy (Equal) the 35.97%. With four drones, the reduction in time is 46.91%. This is because the Proposed (ILP) algorithm considers various factors—such as charging profile, energy consumption for various maneuvers, time taken per transect, and so on—to divide the task among available drones optimally.

We use one of the survey areas (Figure 7) from the dataset, used above in path planning evaluation, for evaluating the greedy adjustment. We used 4 drones and used our formulation shown in 3 for task division. From our experimentation, the advantage of the iterative greedy approach depends on the survey area in consideration. The results can also be affected by the class of drones used. Figure 17 shows the values for 15 iterations for time and energy consumed by each drone on each iteration. There is no considerable improvement in this case, where all the drones are of the same class. This is because the splitting of the transect will be distributed to the same class of drones(maximum velocity is \(10 m/s\) ). This distribution will cause more energy consumed and time used as the number of transects increases. However, Figure 18 shows about a 5% reduction in maximum travel time and 3% in maximum energy consumed after the second iteration. This is because the redistribution of transects happens among 4 drones of different classes(maximum velocities are 10 m/s, 5 m/s, 10 m/s, and 12 m/s, respectively). The new transects are redistributed among better-capable drones, causing an overall decrease in time and energy usage. However, as we increase the iterations, more transects cause an increase in time and energy usage.

Summary: Our proposed system achieves about 47% reduction in mission times with multiple drones than the baselines. Our Iterative greedy adjustment can further provide a reduction of 5% in some cases.

We evaluated the new formulation presented in Table 7 on a survey area shown in Figure 7 of 570 Hectares. In Figure 19(a), we have 1 charging station at the launching site. In Figure 19(b), we have 2 charging stations on diagonally opposite corners. In Figure 19(c), 4 charging stations are located on the perimeter, while in Figure 19(d), the charging stations are placed in the middle of the survey area. The charging stations are marked in red circles.

Figure 20 shows the maximum time ( \(M^{time}\) ) increases with respect to the previous formulation (baseline) because the time for a trip to a charging station is included in the current formulation. Also, for more than 1 drone, increasing the number of charging stations from 1 to 4 reduces the time to complete the task by 1.7%. This is because the charging time is significant in comparison to the travelling time. A slight improvement in \(M^{time}\) is also observed on changing the configuration from 4 charging stations on the perimeter(Figure 19(c)) to the middle(Figure 19(d)) of the survey area. For a single drone covering the survey area, the appropriate location and number of charging stations can significantly reduce \(M^{time}\) by 15%.

As observed in Figure 20, a high value of \(E_{th}\) such as \(0.2*B_d\) , can increase visits to the charging stations, thereby increasing \(M^{time}\) significantly. For the cases, when the Number of Drones is 2 and 3, \(M^{time}\) has increased by 35% and 60%, respectively.

Summary: Our new proposed formulation considers the presence of charging stations. The location and the number of charging stations significantly impact the mission time of a single drone. A high value of \(E_{th}\) can significantly impact mission time.

To study the effect of the altitude on the accuracy of object detection models and the consequent impact on task-level energy consumption, we use UAVDT [Du et al. 2018] dataset for identifying vehicles. We choose three object detection models YOLO [Redmon et al. 2016], Tiny YOLO [Redmon et al. 2016] and SSD-MV2 [Liu et al. 2016; Sandler et al. 2018] to compare their accuracy at different altitudes. Figure 23 shows the variation of [email protected] of different models and task-level energy consumption with the altitude of drone. At higher altitudes, the field of view of the camera has increased coverage. Hence, the flight time needed to complete a surveying task could reduce. We observe that object detection models have lower accuracies at higher altitudes due to reducing object size (in terms of the number of pixels). YOLO has maximum accuracy out of the three models used at all altitudes. Thus, if an embedded device can run an expensive model like YOLO, it will yield greater accuracy. Furthermore, we see that the energy consumption decreases with the increase in altitude, but the decrease is not uniform. Instead, the energy decreases in a step-wise manner. This behavior is because the the flight time and hence the energy consumption depends on the cumulative length of the transects, and an increase in altitude does not necessarily mean a decrease in the number of transects. In some cases, the number of transects decreases with an increase in altitude, while in other cases, it remains unchanged, and so the energy consumption also remains essentially unchanged.

Summary: Increasing altitude hurts object detection accuracy. However, energy consumption decreases in a step-wise manner.

Using our Mininet-WiFi emulation of mobile drones and a stationary BS as wireless nodes, we demonstrate the effect of our reliability algorithm on the PDR even though drone D1 moves away from BS and experiences loss of connection. We run our experiments once with a direct drone-to-BS link, and then with our replication algorithm where other nodes in the vicinity are used as alternate routes to BS. The drones moved around randomly in the survey area. Figure 21 demonstrates that PDR remains greater than 0.75 for 90% vs 60% of the time and greater than 0.5 for 100% of the time for our replication strategy and direct link respectively. However, a drastic comparison is observed for PDR greater than 0.5, where it remains for 100% of the time vs 66% for replication vs direct link, respectively. We also observe zero PDR in the case of direct link when the drone goes more than 80m away from drone whereas it still maintains a link to the BS in the case of replication strategy.

Summary: Our algorithm outperforms the direct link in PDR. The improvement is more pronounced for longer distances.

Optimal usage of a drone’s battery can significantly increase its flight time. Past works have looked at creating a theoretical or regression energy model for the drone as studied by [Zhang et al. 2021]. One of them, [Tseng et al. 2017], proposes a regression model of energy consumption for drones. [Di Franco and Buttazzo 2016] presents an energy model derived from real-world measurements and then uses it to derive the path planning algorithm. A recent work [Kim et al. 2020] proposes a methodology for SOC estimation based operation policy for a drone flight. However, these systems do not talk about energy-efficient task division among multiple drones across multiple charging events—a fundamental requirement in large-scale surveying tasks. Moreover, none of the existing work considers the interplay between energy consumption and sensing quality.

Most approaches to solve drone path planning problem involve dividing the survey area into cells. Following which, a traveling salesman algorithm is applied to find the coverage path [Xie et al. 2019]. Furthermore, several algorithms find optimal sweep direction to minimize the number of turns while surveying an area [Araújo et al. 2013; Huang 2001]. All these works restrict themselves to only the geometrical aspects of the survey area and do not consider factors such as battery capacity, weight, and so on. Recently, an approach to energy-aware path planning was introduced [Franco and Buttazzo 2015]. However, the treatment was quite preliminary and did not consider factors such as wind, payload weight, and so on. Neither did it look at the impact of the plan on sensing accuarcy. There are also many powerful mission planning and GCS having various capabilities. QGround Control [Zurich 2013] is the most used open-source GCS having an ample amount of contributors on GitHub. UGCS [UGCS 2020] is closed-source but has some great features compared with open-source counterparts like multiple drone support even in simulation, 3D surveying, and so on. None of these have implemented energy-focused path planning and leave the choice of the mission flight altitude to the user.

Work in several areas (computer systems, robotics, etc.) has looked at developing drone orchestration, i.e., a system to manage a fleet of drones for a set of applications [Van’t Hof and Nieh 2019; Mottola et al. 2014; Mahmoud et al. 2015; He et al. 2020]. AnDrone [Van’t Hof and Nieh 2019] is a virtual drone computing platform that attempts to use drone-as-a-service to make it accessible like a cloud resource. Through this, a single physical drone can run multiple virtual drones instances for different applications simultaneously. Voltron [Mottola et al. 2014] presents a programming model to manage a fleet of drones using programming constructs. More recently, BeeCluster [He et al. 2020] looked at providing APIs to abstract complex drone tasks. However, none of these systems consider battery energy modeling (charging and discharging) and its implications in reducing mission times. Moreover, none of these focuses on the interplay between application objectives — i.e., model inference accuracy in surveying tasks— and the choice of altitude, with the resulting impact on the flight time for a mission and the battery drain. However, an interesting future direction would be integrating our energy-aware path planning and task scheduling into these drone orchestration systems.