Fine-grained Courier Delivery Behavior Recovery with a Digital Twin Based Iterative Calibration Framework
Article No.: 96, Pages 1 - 25
Abstract
Recovering the fine-grained working process of couriers is becoming one of the essential problems for improving the express delivery systems because knowing the detailed process of how couriers accomplish their daily work facilitates the analyzing, understanding, and optimizing of the working procedure. Although coarse-grained courier trajectories and waybill delivery time data can be collected, this problem is still challenging due to noisy data with spatio-temporal biases, lacking ground truth of couriers’ fine-grained behaviors, and complex correlations between behaviors. Existing works typically focus on a single dimension of the process such as inferring the delivery time and can only yield results of low spatio-temporal resolution, which cannot address the problem well. To bridge the gap, we propose a digital-twin-based iterative calibration system (DTRec) for fine-grained courier working process recovery. We first propose a spatio-temporal bias correction algorithm, which systematically improves existing methods in correcting waybill addresses and trajectory stay points. Second, to model the complex correlations among behaviors and inherent physical constraints, we propose an agent-based model to build the digital twin of couriers. Third, to further improve recovery performance, we design a digital-twin-based iterative calibration framework, which leverages the inconsistency between the deduction results of the digital twin and the recovery results from real-world data to improve both the agent-based model and the recovery results. Experiments show that DTRec outperforms state-of-the-art baselines by 10.8% in terms of fine-grained accuracy on real-world datasets. The system is deployed in the industrial practices in JD Logistics with promising applications. The code is available at https://github.com/tsinghua-fib-lab/Courier-DTRec.
1 Introduction
The prosperity of online e-commerce has put forward increasingly high requirements for logistics, where last mile delivery is one of the most important problems. Although crowd-sourced logistics is also a promising solution [5, 6], currently most delivery tasks are undertaken by express delivery systems built around the world with large amounts of professional couriers [8]. A common practice is that a courier is assigned a batch of waybills and then delivers them around the neighborhood according to the shipping address. The efficiency and quality of couriers’ labor is one of the most important factors that affect the cost and benefits of logistics enterprises. Recovering the couriers’ working processes, such as the delivery time [16, 19], service time [17], and the actual delivery locations,1 [18] and so on, is essential to various important applications, including route planning [12], parcel allocation optimization [22], and delivery service improvements [4].
Existing works exhibit two characteristics. First, they mainly focus on a single dimension rather than the whole delivery process. For example, most of the existing works concentrate on recovering the delivery time or actual delivery location [17, 19, 24]. Second, their results are typically coarse-grained. To be specific, many of them predict courier behaviors at a stay point level, which is at a temporal resolution of 10 minutes. For example, Ruan et al. [19] infer couriers’ delivery time by a stay point-based algorithm, which regards the midpoint of a stay point as the delivery time of a batch of waybills. Their results are demonstrated as the coarse-grained process in Figure 1.
Fig. 1.
However, these coarse-grained recovery results on a single dimension fail to meet business needs with the increasing requirements for cost reduction, efficiency increase, and meticulous operation management [2]. On the one hand, the data mined at a single dimension are not enough to recover the complete courier behaviors during the whole working process. Simply combining existing works from different dimensions often violates physical constraints and business logic since the complex correlations among the couriers’ behaviors are not considered, leading to unreasonable results such as a short time gap between far-away delivery locations. On the other hand, coarse-grained recovery cannot support meticulous operation management. For the grassroots terminal delivery depot managers to operate the daily task allocation and courier supervision, it is still hard to figure out what exactly the couriers are doing and analyze why some packages are delayed through coarse-grained inferred delivery location or delivery time data. In fact, in JD Logistics,2 which is China's leading technology-driven logistics services provider, despite many advancements made in coarse-grained inference or prediction on several key dimensions of delivery data [16, 19], the daily management decisions in the vast terminal delivery depots are still mostly based on human experience. Toward meticulous digital and intelligent management, there is a need for fine-grained courier working process recovery with detailed and comprehensive courier behaviors and quantified workloads, which benefits intuitive working process supervision and balanced task allocation.
Therefore, in this article, we propose to recover the fine-grained courier working process with the data commonly used in existing works, including the manually recorded waybill delivery data, which is typically inaccurate [18], and coarse-grained trajectory data collected from the couriers’ Personal Digital Assistant Devices (PDAs) [11, 18]. Here, the fine-grained working process is defined from two perspectives. One is that we cover all types of behavior in the couriers’ working process, including moving, arranging parcels, going up and down stairs, delivering, and resting. The other is that we recover the behaviors at a spatio-temporal resolution of trajectory points rather than stay points, which are typically composed of hundreds of trajectory points [19]. In other words, the model should infer the detailed behavior type of each trajectory point recorded by the PDAs, thereby giving the exact delivery time of each waybill and the complete and consecutive working process consisting of fine-grained courier behaviors. An example of the fine-grained working process is shown in Figure 1. The fine-grained results can provide intuitive and comprehensive exhibition to on-site managers by directly representing the complete and consecutive working process of each courier, which enables timely detection and detailed analyzing of any abnormal states or events. Besides, the workloads can be measured as traveled distance, climbed floors, and work time rather than only the number of waybills, which promotes task allocation equity and overall efficiency.
The fine-grained courier working process recovery problem is challenging. An intuitively straightforward solution is to combine the existing models that recover different dimensions of the process. However, this does not meet the spatio-temporal resolution requirements and cannot model the complex correlations among the couriers’ behaviors, which leads to coarse-grained results and often violates physical constraints and business logic. Overall, we summarize the challenges of the problem as follows:
—
Poor data quality. Spatio-temporal biases exist in both the manually recorded waybill delivery data and courier trajectory data. Many couriers often delay the delivery confirmation, and thus, the recorded delivery time is often inaccurate. Besides, the location parsed from the waybill shipping address can be different from the actual delivery location due to wrong address parsing or delivery to reception centers. Furthermore, couriers’ working environment is complex and the sheltering effects caused by buildings and even elevators can be strong and varying, so shifts in the GPS points are common, which makes it difficult to directly find out whether the courier is moving or staying, or which building is the courier in.
—
Lacking ground truth of fine-grained behaviors. Collecting the ground truth of fine-grained behavior data requires the data collectors to follow the couriers and record the precise time ranges and locations for each of their working procedures, which takes a high cost and is extremely difficult. Without fine-grained labels, hardly can we find the link to fine-grained behaviors from the waybill and trajectory data, since there is essentially no direct clue to fine-grained behaviors from data. To be specific, both the finish time and the location of the waybill are ambiguous and the trajectory is noisy which cannot directly hint the fine-grained behavior semantics behind the data. Although it is possible to infer the actual location and time range of a delivery event at a stay point level [18, 19], these coarse-grained results still provide no fine-grained knowledge on how the courier behaves during a delivery process such as arranging parcels, going up and down stairs, and delivering.
—
Modeling complex correlation between behaviors. The fine-grained recovery problem is not simply a finer-grained version of existing coarse-grained works. It requires that all the recovered fine-grained behaviors together make a complete, consecutive, and reasonable working process. This means the complex correlations among the couriers’ behaviors should be modeled to conform to inherent physical constraints and business logic. For example, couriers cannot move between buildings faster than their ability. By simply refining existing methods, there is no guarantee to obtain a complete and reasonable working process.
To solve these challenges, we design and implement a digital-twin-based iterative calibration framework for fine-grained courier working process recovery (DTRec), which can effectively model the complex correlations and inherent physical constraints to give accurate, fine-grained behavior recovery. The proposed system contains three modules, including the data preprocessing module, the coarse-grained recovery module, and the digital-twin-based fine-grained recovery module, with three designs. First, to correct the biases in the original data, we propose a spatio-temporal bias correction method, which systematically improves the existing methods in correcting waybill addresses and denoising trajectory points. Then we recover coarse-grained behaviors using the denoised data based on existing methods. Second, to mine the fine-grained behaviors without ground truth labels while modeling the complex correlations and inherent physical constraints, we propose an agent-based courier model and build the digital twin of couriers by fitting it to the coarse-grained behaviors. Without ground truth fine-grained behavior label data, this digital-twin-based approach naturally encodes physical constraints, human factors, and business knowledge to mine the fine-grained behaviors. Third, to further consider the behavior correlations to avoid unreasonable results, we design an iterative calibration framework that cleverly explores and exploits the real-world data and the digital-twin model. Its main idea is to leverage the inconsistency between the deduction results of the digital twin and the recovery results from real-world data to iteratively calibrate both the agent-based model and the recovery results. This inconsistency is discovered as unreasonable recovered behaviors according to the human factors and physical constraints modeled in the agent-based model. Then, the correlated behaviors and the digital twin model are iteratively calibrated to achieve the consistency of the two counterparts.
To sum up, the main contribution of this work is four-folds:
—
To the best of our knowledge, we propose and study the fine-grained courier behavior recovery problem for the first time and identify its significance and challenges.
—
We design and implement a digital-twin-based iterative calibration framework to solve the problem, which contains three novel designs.
—
Extensive experiments on large-scale real-world datasets demonstrate the effectiveness of our method. Specifically, it improves the recovery Accuracy (Acc) of the fine-grained processes by 10.8%.
—
We developed a system based on the proposed algorithm and tested its performance in JD Logistics.
2 Preliminaries
2.1 Definition
Waybill. A delivery waybill \(O\) is the basic unit of task for couriers. Typically, couriers will load the waybill packages onto a minivan at the depot and then drive to the target communities to start their delivery process. And the working process ends when all the waybill packages are delivered to recipients. The waybill attributes of our interest include the shipping address \(O.addr\) and the confirmed finish time \(O.t\), which is supposed to be recorded by couriers on the PDAs as soon as the waybill is delivered. \(O.addr\) is a text that is further parsed into a geographical coordinate \(O.loc\), the building unit number \(O.unit\), and the floor number \(O.floor\).
Trajectory. The GPS-based courier trajectory \(P\) is a time sequence \([p_{1},\dots,p_{n}]\), where each trajectory point \(p_{i}\) is a triplet of longitude, latitude, and time stamp \(p_{i}=(lng_{i},lat_{i},t_{i})\).
Stay Point. A stay point \(S\) is not a single trajectory point, but a sequence of consecutive points in a trajectory that stays within a spatial threshold \(D\) and lasts for at least a temporal threshold \(T\). We denote the start time and end time of \(S\) as \(S.ts\) and \(S.te\), and the average coordinates as \(S.loc\).
Coarse-Grained Delivery Process. Coarse-grained delivery process is a sequence of coarse-grained behaviors \(\mathbb{B}^{c}=[B_{1}^{c},\dots,B_{n}^{c}]\), where each behavior \(B^{c}_{i}\) is a triplet of “type” \(B^{c}_{i}.type\), “start time” \(B^{c}_{i}.ts\), and “end time” \(B^{c}_{i}.te\). The start time and end time of adjacent behaviors are continuous, namely, \(B^{c}_{i}.te=B^{c}_{i+1}.ts\). The behavior types include “rest,” “move,” and “deliver.” For deliver behaviors, which waybills are delivered should also be specified. When the courier is not staying, the behavior is labeled as move. If the delivery of any waybills is attributed to a stay point \(S\), then during the time range of \(S\), the behavior is labeled as deliver. Stay point without corresponding waybills delivered is labeled as rest.
Fine-Grained Delivery Process. Fine-grained delivery process is a sequence of fine-grained behavior \(\mathbb{B}^{f}=[B_{1}^{f},\dots,B_{n}^{f}]\). It is the same with \(\mathbb{B}^{c}\) for move and rest since no more valuable information of interest can be mined during moving or resting time. However, the coarse-grained deliver is further partitioned into several fine-grained behaviors including “arrange,” “up,” deliver, “down,” and “unit,” which specify the fine-grained delivery process inside a delivery stay point, including arranging packages, going upstairs, delivering to recipients, going downstairs, and moving between building units.
2.2 Problem Statement
Given the waybills \(\mathbb{O}=\{O_{1},\dots,O_{n}\}\) and courier trajectory \(P\) in a target day and several historical days, the proposed framework \(F\) is first fitted on the historical data and then used to recover the fine-grained courier working processes \(\mathbb{B}^{f}\) on the target daywhere the information in the collected data used by the approach includes \(O.addr,O.t,O.loc,O.unit,\) \(O.floor\), and all the trajectory points.
\begin{align}\mathbb{B}^{f}=F(\mathbb{O},P;\mathbb{O}_{historical},P_{historical}),\end{align}
(1)
3 Method
3.1 Overall Framework
Figure 2 shows the overall framework of DTRec. It takes the courier trajectory data and the waybill delivery data from real-world logistic depots as inputs. There are three main components, including the data preprocess module, the coarse-grained recovery module, and the digital-twin-based fine-grained recovery module. First, the data preprocess module preliminarily denoises the trajectory data and extracts stay points from it. And the waybill address is parsed into geographical coordinates. Second, in the coarse-grained recovery module, a spatio-temporal bias correction method is proposed to correct the original stay points and address locations. Then coarse-grained delivery behaviors are recovered by matching waybills to stay points. Third, in the digital-twin-based fine-grained recovery module, an agent-based digital courier model is proposed to bridge the gap from coarse-grained behaviors to fine-grained behaviors, which considers the complex correlations between behaviors by encoding inherent physical constraints, human factors, and business knowledge. Finally, we design a digital-real iterative calibration framework that leverages the inconsistency between the digital-world model and the real-world behaviors to calibrate the two counterparts jointly.
Fig. 2.
3.2 Data Preprocessing
3.2.1 Trajectory Denoising.
A trajectory point will be removed if the speed between its last non-noise point is larger than a threshold. The threshold is set as 12 m/s considering the capability of the courier's minivan and the residential community environment. We find the first non-noise point as the first point that has a speed within this threshold between each of its next three points. About 10% of trajectory points are filtered out in this way.
3.2.2 Stay Point Extraction.
We adopt the method [10] to preliminarily detect the stay points in trajectory. Recalling the definition of stay point, we loop with the following steps till going through the whole trajectory: (1) Choose a trajectory point \(p_{i}\) as an anchor, such that \(p_{i}\) is the first point that does not currently belong to any detected stay point yet. (2) Find anchor's next k points \(p_{i+1},\ldots,p_{i+k}\) such that \(dis(p_{i},p_{i+j}){\,\lt\,}=D,\forall j\) in (\(1,\ldots,k\)) and \(dis(p_{i},p_{i+k+1}){\gt}D\). (3) If \(p_{i+k}.t-p_{i}.t{\gt}=T\), then \(\{p_{i},\ldots,p_{i+k}\}\) is detected as a stay point. The thresholds \(D\) is set as 30 m and \(T\) is set as 60 s considering the setting of previous works [10, 19].
3.2.3 Waybill Address Parsing.
The shipping address of a waybill is a text that users fill in. We convert it into geographical coordinates using Geocoding, which maps a structured address to a Point of Interest (POI), and the coordinates can be found as the POI location. We adopt the Geocoding service3 provided by a major location-based service provider in China. We also apply a regular expression matching method to extract the unit number and the floor number.
3.3 Coarse-Grained Behavior Recovery
Recalling the definition of coarse-grained behaviors, recovering the coarse-grained behaviors is equivalent to matching waybills to stay points. However, the spatio-temporal biases in the waybills and raw stay points can deteriorate this matching. Therefore, we propose a spatio-temporal bias correction method to correct the raw stay points and waybill locations.
3.3.1 Stay Point Correction.
We design a stay point margin correction algorithm and a stay point merging algorithm to boost the precision of the stay point compared with the original method[10].
Stay Point Margin Correction. The nature of stay behaviors differs, and a consistent threshold \(D\) and \(T\) are not suitable for all stays. For example, a stay at a small building involves a smaller spatial range than \(D\), so moving points can likely exist at the margin of the detected stay point to reach \(D\). Considering the anchor point is always the first point in a stay, it is more likely that the noisy moving points are at the head. As Figure 3 illustrates, the actual stay point is \(\{p_{1},\ldots,p_{7}\}\); however, \(D\) is larger than its spatial scale, so \(\{p_{0},\ldots,p_{6}\}\) is detected as stay point instead, with \(p_{0}\) is a noise and \(p_{7}\) is missing. Based on this observation, after obtaining a raw stay point, we use its average location as a new anchor and use the average radius, namely the average distance from the stay points to the new anchor, as a new spatial threshold. Through this, the anchor is not biased toward the head, and the radius is adapted to the actual scale of the stay point. As Figure 3 illustrates, \(p_{0}\) is removed, and \(p_{7}\) is recalled.
Fig. 3.
Stay Point Merging. Note that margin calibration typically reduces the spatial threshold and tackles the stay points of a smaller scale. But how about a stay point with a larger spatial scale? For example, a stay at a huge building involves a larger spatial and temporal range than \(D\) and \(T\), and it will be split apart since \(D\) is not enough. Toward this, we merge two adjacent stay points if they are temporally adjacent, and all the points from the first point of the first stay to the last point of the last stay together make up a stay point with a larger spatial scale set as \(1.5D\). Note that the time range of a merged stay point will be at least \(2T\) so that the merged stay point keeps the feature of staying within some spatial area for a long time.
3.3.2 Waybill Address Correction.
We find the address location obtained by Geocoding can be far from the true delivery location for two reasons.
First, Geocoding can fail due to ambiguous inputs and the incomplete POI database. As Figure 4 illustrates, different buildings can be very similar in address and result in wrong Geocoding results. In our practice, we also find many waybill addresses parsed to community gates where no buildings are around, and this may be due to an incomplete POI database that lacks the residential building-level POIs. So the gate location is retrieved by the Geocoding API to respond to the residential building-level queries.
Fig. 4.
Second, even if Geocoding works well, the location may still be different from the actual delivery location. For example, when a recipient is not at home, a courier may deliver the package to an express package storage cabinet. For enclosed areas such as schools, office building regions, and industrial parks, couriers are likely to have no entrance permission, and packages may be delivered to reception shelves at the gate.
To obtain the actual waybill delivery location, we seek trajectory data for help. However, the waybill finish time is also noisy due to the delayed confirmation by couriers. Therefore, we can not simply query the trajectory position at the waybill finish time. Inspired by [18], we leverage the data from multiple historical days to alleviate the influence of the random confirmation delay, and we make three improvements to the original method [18]. First, our method is unsupervised, without the unendurable cost of collecting massive ground truth location data (e.g., asking couriers to label the actual delivery location during their working process). And it turns out to be efficient and effective using 5 weeks of historical data. Second, we consider the actual trajectory in the target day to avoid impossible corrected locations that the courier even doesn’t stay at. Third, we do not map an address to a unique corrected location. Instead, we reserve several possible locations which are more practical since the actual delivery location can be multiple. For example, the delivery can be to-door when the recipient is at home and to-cabinet otherwise.
Given a target day with waybills \(\mathbb{O}=\{O_{1},\dots,O_{m}\}\) and stay points \(\mathbb{S}=\{S_{1},\dots,S_{n}\}\), we correct the delivery location for any waybill \(O_{i}\) by discovering stay locations that frequently co-occurred with \(O_{i}.addr\) in historical days. The method consists of three steps:
(1) Location pool construction. The locations of each stay point from all days are clustered to generate the set of all the possible candidate locations \(\mathbb{L}\).
(2) Candidate location querying. For the target waybill \(O_{i}\), we assume the confirmed finish time \(O_{i}.t\) is delayed by no more than \(T^{-}\) and advanced by no more than \(T^{+}\). \(T^{-}\) is set as 1 hour since couriers can even forget to confirm some waybills till a later checking. \(T^{+}\) is set as 3 minutes since couriers are strictly prohibited from confirming in advance. So the actual delivery location of \(O_{i}\) must be in \(\mathbb{L}_{i}^{0}=\{S_{i_{1}}.loc,\dots,S_{i_{k}}.loc\}\) such that \(O_{i}.t-T^{-}{\,\lt\,}S_{j}.ts{\,\lt\,}S_{j}.te{\,\lt\,}O_{i}.t+T^{+},\forall j\in\{i_{1},\dots,i_{k}\}\). We say \(\mathbb{L}_{i}^{0}\) is “co-occurred” with \(O_{i}.addr\) in the target day. We further map \(\mathbb{L}_{i}^{0}\) to \(\mathbb{L}_{i}\subset\mathbb{L}\) by finding all the nearby locations in the location pool in \(\mathbb{L}\) such that its distance between any location in \(\mathbb{L}_{i}^{0}\) is less than 50 m.
(3) Candidate location validating. We check each location \(l\) in \(\mathbb{L}_{i}\) and evaluate its co-occurrence with \(O_{i}.addr\) in historical days. We define a co-occurring metric as the portion of days where \(l\) co-occurred with \(O_{i}.addr\) in those historical days that \(O_{i}.addr\) occurs. If this metric is larger than a threshold, which is intuitively set as \(0.5\), we adopt \(l\) as a valid candidate for correcting the actual delivery location of \(O_{i}\). Note that we reserve all the candidates as long as the metric is larger than the threshold rather than only taking the one with the highest metric.
3.3.3 Waybill-Stay Matching.
We match any waybill \(O\) to a stay point \(S\) on a target day by defining their matching score. We first define a temporal matching score which linearly decreases with the offset of the delivery time and the stay time range
\begin{align}Score^{t}_{O,S}=\begin{cases}1, & \text{if }S.ts < =O.t < =S.te \\1-(S.ts-O.t)/T^{+}, & \text{if }S.ts-T^{+} < O.t < S.ts \\1-(O.t-S.te)/T^{-}, & \text{if }S.te < O.t < S.te+T^{-} \\0, & \text{otherwise.}\end{cases}\end{align}
(2)
Then, we define a spatial matching score. Note that with Geocoding and delivery location correction, a waybill may have multiple possible locations: \(O.locs=[l_{geocoding},l_{cor_{1}},\dots,l_{cor_{n}}]\). We further pose a confidence score on each possible location. The confidence of the Geocoding location is set as 1 and the confidence of a corrected location is set as its co-occurring metric. Then, the spatial score when taking the ith possible location of waybill is formulated aswhere \(D_{match}\) is a spatial threshold tuned as 100 m, and \(C_{i}\) is its confidence.
\begin{align}Score^{s,i}_{O,S}=\max\left(0,1-\frac{dis(O.locs[i],S.loc)}{D_{match}}\right)\cdot C_{i},\end{align}
(3)
And the final spatio-temporal matching score between a pair of waybill \(O\) and stay point \(S\) is the product of the temporal score and the maximum spatial score for each of the possible waybill locations
\begin{align}Score_{O,S}=Score^{t}_{O,S}\cdot\max\limits_{i}Score^{s,i}_{O,S}.\end{align}
(4)
In this way, waybills are matched to the stay point with the largest spatio-temporal matching score.
3.4 DT-Based Fine-Grained Process Recovery
To obtain the fine-grained working process, we build an agent-based digital courier model that encodes physical constraints, human factors, and business knowledge to help mine the fine-grained behaviors. To further consider the complex correlations between behaviors, avoid unreasonable results, and boost final performance, an iterative digital-twin-based calibration framework is designed which leverages the digital-real inconsistency to iteratively calibrate each other.
3.4.1 Work Pattern Recognition.
Before building the agent-based model, we recognize that there are several essentially different working patterns that should be tackled differently. Thus, instead of posing a unified agent-based model on all the working processes for all couriers, we first recognize the working patterns and then build a tailored model for each pattern.
Based on data analysis and logic of the delivery task, three main kinds of working patterns exist due to the constraints of the target buildings: (1) Staircase building pattern. In normal residential buildings without elevators equipped, couriers walk to go upstairs and downstairs. (2) Elevator building pattern. In residential buildings with elevators, compared with staircase buildings, the time consumed between floors becomes more uncertain since the waiting time to get elevator service is uncertain while the actual traveling time between floors is relatively short. (3) Closure park pattern. For a closed area of an industrial park or a set of office buildings, it is common that couriers do not go inside the park due to security guards. Instead, couriers “make a stall” along the roadside. In other words, couriers stay outside of the gate, make phone calls, and wait for recipients to fetch their packages. Another possible working pattern is the express cabinet pattern, where couriers put packages into lockers rather than go into buildings. In this article, we do not especially model this pattern since in JD Logistics, couriers are required to deliver to the door as much as possible, and thus this pattern is rare. If necessary in other industrial practices, this pattern can be easily recognized by referring to the positions of the local express cabinets.
We first distinguish the closure park pattern by the feature of coarse-grained behavior because a “stall” typically lasts for a long time with lots of waybills matched to the corresponding stay point. Note that it is enough to recognize this pattern, and we don’t further construct a digital model for this since the fine-grained events during a stall, namely when the recipients come to fetch their packages are too random to be predicted. Then, we distinguish the staircase building pattern and elevator building pattern by the prior knowledge of the building type. We obtain whether the residential building is equipped with an elevator from a large-scale real estate rental website.4 In industrial practices, this can be easily labeled by couriers, since typically the building type in one community is the same, and the region each courier is taking charge of is stable across days.
3.4.2 Agent-Based Digital Twin Model.
The target model \(M_{W}(;\Theta)\), corresponding to working pattern \(W\), with parameters \(\Theta\) to encode courier features and physical constraints, is an agent-based model that acts as the digital twin of a courier to perform a series of fine-grained behaviors that together composes a coarse-grained delivery behavior. Given any coarse-grained behavior \(B_{i}^{c}\) as input, if \(B_{i}^{c}.type\) is deliver, the model recovers the corresponding fine-grained behaviors \(\mathbb{B}_{i}^{f}=[B_{i_{1}}^{f},\dots,B_{i_{n}}^{f}]\) where \(B_{i_{1}}^{f}.ts=B_{i}^{c}.ts\) and \(B_{i_{n}}^{f}.te=B_{i}^{c}.te\), and if \(B_{i}^{c}.type\) is move or rest, the model keeps it the same
\begin{align}\mathbb{B}_{i}^{f}=M_{W}(B_{i}^{c};\Theta)=\left\{\begin{array}{ll}[B_{i_{1}}^{f},\dots,B _{i_{n}}^{f}], & \text{if }B_{i}^{c}.type=``\text{deliver"}\\{[}B_i^c], & \text{if }B_{i}^{c}.type\neq ``\text{deliver".}\end{array}\right.\end{align}
(5)
For example, if \(B_{i}^{c}\) corresponds to a stay point at a staircase building with one waybill at floor 1 and another at floor 2, the types of each fine-grained behavior in \(\mathbb{B}_{i}^{f}\) are arrange, up, deliver, up, deliver, and down, respectively, and their time ranges are decided by \(\Theta\) to satisfy \(B_{i_{1}}^{f}.ts=B_{i}^{c}.ts\) and \(B_{i_{n}}^{f}.te=B_{i}^{c}.te\). Given the whole coarse-grained working process \(\mathbb{B}^{c}\), by applying the model to each behavior \(B_{i}^{c}\) in \(\mathbb{B}^{c}\) as Equation (5) shows and concatenating the results, the fine-grained working process is obtained
\begin{align}\mathbb{B}^{f}=M_{W}(\mathbb{B}^{c};\Theta)=Concat\{M_{W}(B_{i}^{c};\Theta), \text{ for each }B_{i}^{c}\in\mathbb{B}^{c}\}.\end{align}
(6)
The model parameters \(\Theta\) are learned by fitting the model on each \(B_{i}^{c}\) such that \(B_{i}^{c}.type\) is deliver. Based on Equation (5), the true time range \(T_{i}^{true}=B_{i}^{c}.te-B_{i}^{c}.ts\) should be consistent with the model prediction
\begin{align}T_{i}^{M_{W}(;\Theta)}=\sum_{j}\left(B_{i_{j}}^{f}.te-B_{i_{j}}^{f}.ts\right),\text{ for each }B_{i_{j}}\in M_{W}(B_{i}^{c};\Theta).\end{align}
(7)
Then, the optimized model parameters \(\Theta^{*}\) are obtained by minimizing the square error between \(T_{i}^{M_{W}(;\Theta)}\) and \(T_{i}^{true}\)
\begin{align}\Theta^{*}=\arg\min\limits_{\Theta}\sum_{i}\left(T_{i}^{M_{W}(;\Theta)}-T_{i}^{true }\right)^{2},\text{ for each }B_{i}^{c}\in\mathbb{B}^{c},s.t.B_{i}^{c}.type=``\text{deliver".}\end{align}
(8)
The model is trained independently for each courier and working pattern. In other words, several models will be trained respectively for each courier and working pattern, using the coarse-grained deliver behaviors of the courier and the working pattern as input, as Equation (8) shows.
Specifically, for the staircase building pattern, a basic assumption is presumed based on human knowledge that in a building, the courier will deliver packages for each unit, and in each unit, the courier will deliver by the sequence of floors. Besides, at the beginning of the delivery, the couriers will first take some time to prepare, such as arranging the packages in their minivans. Formally, given the unit number and floor number of each waybill in \(\mathbb{O}\) corresponding to a coarse-grained deliver behavior, denoting the number of different units as \(u=|\{O.unit|O\in\mathbb{O}\}|\), the couriers need to travel between units for \(u-1\) times. The total floor number that the couriers need to go upstairs (and downstairs) is the sum of the maximum floor in each unit, denoted as \(f=\sum_{i}\max(\{O.floor|O\in\mathbb{O},O.unit=i\})\). Therefore, Equation (7) can be calculated aswhere \(T_{b},T_{u},T_{f}\), and \(T_{o}\) consist model parameters \(\Theta\) which are decided by courier and building features. \(T_{b}\) is the basic preparation time, for example, arranging packages. \(T_{u}\) is the traveling time between units. \(T_{f}\) is the total time for a courier to go up and go down one floor. \(T_{o}\) is the time for the courier to deliver a package to the recipient.
\begin{align}T^{M_{W}(;\Theta)}=T_{b}+T_{u}\cdot(u-1)+T_{f}\cdot f+T_{o}\cdot|\mathbb{O}|, \text{when }W=\text{staircase pattern},\end{align}
(9)
For the elevator building pattern, similarly, we use \(T_{b}\), \(T_{u}\), and \(T_{o}\) to model the time taken to prepare, move between units, and deliver a package. The time between floors, however, is modeled by three parameters \(T_{e},T_{w_{1}}\), and \(T_{w_{2}}\). \(T_{e}\) is the total time for an elevator to move up and move down one floor, which is similar to \(T_{f}\), while \(T_{w_{1}},T_{w_{2}}\) models the waiting time to get the elevator service. Specifically, for the first time that a courier waits for an elevator in a unit, the waiting time is \(T_{w_{1}}\), and for the next few times in the same unit waiting for the elevator, the waiting time is \(T_{w_{2}}\). We suppose that \(T_{w_{1}}\) is different from \(T_{w_{2}}\) because \(T_{w_{2}}\) typically happens not long after the courier uses the elevator, and the elevator is likely not used by others during this time. Then, we derive that in any unit numbered \(u_{i}\), the times that the courier waits for an elevator is \(w_{i}=|\{O.floor|O\in\mathbb{O},O.unit=u_{i}\}|+1\) (once for reaching each floor and once for going back). So in the unit numbered \(u_{i}\), if the total waiting number \(w_{i}\) is larger than 1, then the “first time” waiting number \(w_{i}^{1}\) is 1, and the “non-first-time” waiting number is \(w_{i}^{2}=w_{i}-w_{i}^{1}\). We denote the total first time waiting number and non-first-time waiting number across all the units as \(w_{1}\) and \(w_{2}\), respectively. Then, the Equation (7) can be calculated as
\begin{align}T^{M_{W}(;\Theta)}=T_{b}+T_{u}\cdot(u-1)+T_{e}\cdot f+T_{o}\cdot|\mathbb{O}|+T _{w_{1}}\cdot w_{1}+T_{w_{2}}\cdot w_{2},\text{when }W=\text{elevator pattern}.\end{align}
(10)
Note that for both Equations (9) and (10), \(T^{M_{W}(;\Theta)}\) is a linear function concerning \(\Theta\), so we apply linear least square regression with non-negative parameters5 to solve the parameter optimization problem in Equation (8). To sum up, the digital courier model naturally encodes the knowledge of couriers’ working patterns, human factors such as courier ability, and the inherent physical constraints since it is based on the microscopic movement mechanisms of how the delivery operations happen in the real world. The model only uses coarse-grained information during training but can bridge the gap from coarse-grained behaviors to fine-grained behaviors as Equation (6) shows.
3.4.3 Digital-Real Iterative Calibration.
After building the digital twin model, we design a digital-real iterative calibration algorithm to leverage the inconsistency between the deduction results of the digital twin and the recovery results from real-world data to systematically improve recovery performance. A digital model calibration step and a real-world behavior calibration step are operated in an interleaving manner during iterations, which we refer to as the outer layer iteration. Besides, within the digital model calibration step, there is also an iterative process to achieve the convergence of model parameters and training masks, which we refer to as an inner layer iteration. There is also an inner layer iteration within the real-world behavior calibration step, which calibrates the waybill-stay matching and the stay time range in an interleaving manner. This process is formulated as Algorithm 1.
Digital Model Calibration Step. As Equation (8) puts, the model using all the deliver behaviors in \(\mathbb{B}^{c}\) as training samples. Because \(\mathbb{B}^{c}\) will be updated in every iteration of Algorithm 1 (lines 10 and 11), we also re-train the model in every iteration using the updated \(\mathbb{B}^{c}\) (lines 2–8). What's more, outliers are bound to exist in the training samples, since sometimes irregular events can happen during the delivery, for example, unexpected waiting for recipients or talking with recipients for a long time, which is out of the scope of the model assumption. To avoid interference from these outliers during model training, we mask those training samples of which the time range deviates largely from the model prediction (line 6), and then the model parameters are updated by fitting on the rest training samples (line 7). This process is looped until the mask indexes converge.
Real-World Behavior Calibration Step. With the calibrated digital courier model, the coarse-grained behaviors can be calibrated from two aspects, as the inconsistency between the predicted delivery time and the observed stay time can be attributed to two reasons:
(1) Wrong waybill-stay matching. The model can give a wrong prediction of stay time range as Equation (9) or (10) puts due to wrong waybills \(\mathbb{O}\) are matched to the observed stay point. The wrong matching can be caused by the noisy spatio-temporal information of both waybills and trajectory and the lack of consideration of the delivery time consistency during the coarse-grained waybill-stay matching.
(2) Wrong detected stay time range. The time range of the detected stay point is not the actual delivery time range, which may be due to the noisy trajectory or unsuitable spatio-temporal thresholds in stay point extraction.
Toward these, we perform an inner layer iteration here to iteratively calibrate the waybill-stay matching and the stay time range. This process is displayed in Algorithm 1 from lines 9–12. Intuitively, the waybill-stay matching is more error-prone than the detected stay time range. Therefore, in each inner iteration, we first assume the stay time range is basically correct, and the waybill-stay matching results should be calibrated to make the predicted delivery time consistent with the stay time. Then, we assume the waybill-stay results are basically correct, and we modify the margins of stay points to make the stay time consistent with the predicted delivery time.
Waybill-Stay Matching Calibration. A waybill exchanging algorithm is designed to exchange the matched waybills between two temporally adjacent stay points. Given the digital model \(M_{W}(;\Theta)\), we define the loss \(L\) of a stay point \(S\) with its corresponding behavior \(B^{c}\) to be the deviation between stay time and the predicted delivery time
\begin{align}L=(B^{c}.te-B^{c}.ts)-T^{M_{W}(;\Theta)}=(S.te-S.ts)-T^{M_{W}(;\Theta)}.\end{align}
(11)
We take the “character” of a stay point with positive loss as “consumer” and a stay point with negative loss as “producer.” A stay point without any waybills matched is also a consumer. The algorithm exchanges a waybill between two temporally adjacent stay points with different characters if their total absolute loss decreases after the exchange. The insights of the design are three-folds:
(1) The actual stay point can be split into multiple adjacent parts when performing stay point detection due to GPS error or large spatio-temporal scales. And when the courier confirms the finish time at the end of this delivery, the waybills can all be matched to one of them. Then, the one with lots of waybills matched is likely to become a producer since its loss is likely to be negative, and other stay points are consumer. Hopefully, the waybills can be spread to all these stay points during the exchange.
(2) Couriers can delay the confirmation till they finish delivering several buildings, so again the stay point at the last building may be matched with too many waybills, which are expected to be exchanged to some previous stay points.
(3) The exchange is restricted to temporally adjacent stay points, which implicitly considers the spatio-temporal matching score since if the waybills are initially matched to a stay point, the matching score with the adjacent stay point is also guaranteed to be not too small.
Through this algorithm, non-trivial calibrations to behaviors can happen. Not only the matched waybills of a deliver behavior can change but also the type of behavior may change. For example, a deliver behavior can become a rest behavior if it loses all of its matched waybills, and a rest behavior can also become a deliver behavior if it obtains some waybills.
Stay Time Range Calibration. The stay time range calibration algorithm extends the time range of a stay point \(S\) if its loss \(L\) is negative and shortens it otherwise to meet the consistency of stay time and the predicted delivery time. The value of extending or shortening time is proportional to \(L\). Normally we keep the midpoint of the stay time range unchanged by identically moving its two margins. However, when extending the margin, we may touch another stay point \(S^{{}^{\prime}}\). Here, if \(S^{{}^{\prime}}\) is not matched with any waybills, we just let \(S\) continue its expanding and taking over the time range of \(S^{{}^{\prime}}\). Otherwise, we try to extend more toward the other side of the margin. If we are “blocked” on both two sides, namely, we touch the margin of other delivery stay points on both two sides, these stay points will be merged. It is non-trivial to merge the stay points since all the fine-grained behaviors to deliver those waybills will be re-considered in the new merged time range and different results will emerge.
During the real-world behavior calibration step, the human knowledge and physical constraints encoded in the digital model are exploited to consider the complex correlations between behaviors. By trading the matched waybills (line 10) and the time intervals (line 11) between adjacent behaviors, non-trivial calibrations to coarse-grained behaviors can happen. Finally, with the calibrated model and the calibrated coarse-grained behaviors, the calibrated fine-grained behaviors are obtained by applying the model to the coarse-grained behaviors (line 13). This iterative calibration framework benefits from the power of our digital-twin-based approach, which enables the natural co-evolution and co-optimization of both the digital model and the real-world system.
4 Experiments
We conduct extensive experiments on a real-world dataset to answer four following Research Questions (RQs):
—
RQ1: How is the overall performance of DTRec compared with state-of-the-art baselines?
—
RQ2: How do the designed spatio-temporal bias correction module and the digital-twin-based iterative calibration module contribute to the performance?
—
RQ3: How do the spatio-temporal bias correction module and the digital-twin-based iterative calibration module play a role in specific real-world cases?
—
RQ4: How does our model take effect in industrial logistics system practices?
4.1 Experiment Settings
4.1.1 Dataset.
The dataset was collected in a delivery depot of JD Logistics in Tongzhou district, Beijing, over 37 days (from 1 August 2022 to 6 September 2022), composed of trajectory and waybills of each courier in each day. On the last day (6 September 2022), we collected the ground truth of the fine-grained courier working process by assigning data collectors to follow couriers and record all of their behaviors. The data on the last day serve as the testing set and others as the training set. After obtaining the raw dataset, We discard a courier's data in a day with a lack of waybills or trajectory data. After this filtering, each courier has about 29 days of data and each day sees about 15 couriers, and there are 15 couriers on the last day with 2,274 ground truth fine-grained behavior labels that are recorded by data collectors who are sent to follow through couriers’ working process. An afterwords checkup is also conducted by the data collectors to fix or remove any infeasible or contradictory fine-grained behavior labels. As a result, 2,141 valid ground truth fine-grained labels are obtained.
The statistics of the dataset on each date are demonstrated in Figure 5. The line chart above plots the number of waybills and the number of unique shipping addresses of waybills respectively on each day. The line chart below plots the number of stay points and the number of candidate stay points per waybill respectively on each day. The candidate stay points of a waybill are counted as those stay points that are within 70 m of the waybill address and 1 hour of the waybill's confirmed finish time. Normally for each day, there are around 1,700 waybills with 1,000 addresses, 900 stay points, and 5 candidates stay points per waybill, which suggests the challenge and complexity of our task. There are fluctuations among different days, for example, on some days only a small portion of couriers are working, while the last day that serves as the testing set is not an outlier, which sees around 2,000 waybills, 1,200 waybill addresses, 1,200 stay points, and 5 candidate stay points.
Fig. 5.
There are some limitations of the dataset. First, due to the extremely high cost of collecting fine-grained behavior labels, we only have 1 day of testing set with 2,141 ground truth labels. Nevertheless, from the analysis of dataset statistics, the data distribution on the testing day is expected to be representative. Second, we have no more datasets in different cities to examine the robustness of our methods, where the nature of the logistics working process could be heterogeneous due to management customs or physical environments. Nevertheless, we do have six couriers that mainly work in eight communities with staircase buildings and nine couriers that mainly work in nine communities with elevator buildings, which can preliminarily challenge the robustness of our methods. But probably for some special scenarios, more unique working patterns need to be recognized and corresponding tailored models should be designed (e.g., substitute for Equations (9) and (10)) before deploying the system in other cities. As the first to study the fine-grained courier delivery process recovery, we suggest developing and evaluating the robustness of solutions across extensive industrial scenario data in future works.
4.1.2 Metrics.
Courier working process recovery is evaluated at both the coarse-grained scale and the fine-grained scale. The first kind of metric is “Accuracy (Acc)” for each kind of coarse-grained and fine-grained behavior. Another metric is “Delivery Time Error (DTE)” to evaluate the finish time of waybills.
Acc. Consider the time axis during the whole working process. The recovery results partition the time axis into many continuous intervals and then label each interval to a kind of behavior. Therefore, we measure how accurate is the time axis partitioned and labeled as the overlap ratio with the ground truth. Specifically, consider the toy example illustrated in Figure 6, the Acc of move (Acc-move) is 100% since during the time interval of move on the ground truth axis, i.e., [0, 5], the behavior label on the recovery time axis is also move. However, the Acc of rest is Acc-rest \(=\frac{25-21}{25-20}=80\%\) since during the ground truth interval of rest, i.e., [20, 25], on the recovery time axis only in [21, 25] is the behavior label correct. Similarly, the Acc of “coarse-grained deliver” is \(\frac{20-6}{20-5}=93.33\%\). Thus, the overall Acc of coarse-grained results is \(\frac{5+14+4}{5+15+5}=92\%\). For the fine-grained results, the deliver is further partitioned into arrange, “go upstairs,” “deliver (fine-grained),” and “go downstairs” while the move and rest are the same as coarse-grained results. Similarly, the Acc of go upstairs is Acc-up \(=\frac{11-10}{11-9}=50\%\), and the overall Acc of fine-grained results is \(\frac{5+3+1+4+1+4}{5+4+2+7+2+5}=72\%\). It can be seen that the Acc of a certain kind of behavior cannot punish the “false positive” phenomenon, for example, the move behavior is not correctly recovered since it lasts longer than the truth, while the “Acc-move” is perfectly 100%. However, this will be captured in the overall Acc since the false positive of move will damage “Acc-arrange” and the overall Acc.
Fig. 6.
DTE. The DTE is calculated as the mean absolute error of the recovered delivery time of waybills. Specifically, in Figure 6, there is one waybill delivered at \(t=18\) and recovered to be delivered at \(t=19\), then its DTE is \(|18-19|=1\). Denote the ground truth delivery time of some waybill as \(O_{i}^{gt}.t\), the recovered one as \(O_{i}^{rec}.t\), and the total number of waybills as \(N_{o}\), we have
\begin{align*}DTE=\frac{1}{N_{o}}\sum_{i=1}^{N_{o}}|O_{i}^{gt}.t-O_{i}^{rec}.t|.\end{align*}
Pre. The “Acc” metric can be seen as the “recall” rate of ground truth behaviors, and here we introduce an extra “Pre” metric as the precision of recovered behaviors. It is similar to Acc to consider the overlap between the ground truth and the recovered results, but here the denominator of the overlap ratio becomes the recovered results. For example, consider the toy example illustrated in Figure 6, the precision of up (Pre-up) is \(\frac{11-10}{14-10}=25\%\). Note that the overall Pre will be the same as Acc since the total length of the compared time axis is the same. The behavior-wise Pre can directly punish the false positive, but after all Acc and DTE are considered the major performance metrics, since how accurate can the behaviors and delivery time of waybills be recovered are actually of interest in application.
4.1.3 Baselines.
Since there is a lack of literature on fine-grained courier working process recovery, we combine state-of-the-art coarse-grained recovery methods with some designed fine-grained methods to compare with the proposed DTRec. We implement four coarse-grained methods and four fine-grained methods. As a result, 16 combinations are produced as baselines.
Coarse-Grained Baseline Components.
—
“MS” matches a waybill to the spatially nearest stay point within a maximum temporal distance.
—
“MT” matches a waybill to the temporally nearest stay point within a maximum spatial distance.
—
—
“DTInf” [19] is a strong coarse-grained delivery recovery baseline that performs waybill address correction and then matches the waybills to stay points based on spatio-temporal features.
Fine-Grained Baseline Components.
—
“mid” is what existing solutions do. It takes the midpoint of stay point as the waybill delivery time. Since essentially not a fine-grained baseline, it can only be used to compare the coarse-grained Acc and DTE.
—
“unf” considers the waybill delivery time to be uniformly distributed during the stay time range, then the fine-grained behaviors are filled to be equally carving up the time gap between these delivery times.
—
“smt” smartly takes use of the confirmed delivery time of waybills. It first adapts the confirmed times into the range of stay while keeping the relative ratio of time gaps between them, then fine-grained behaviors are filled in between similarly as unf.
—
“deep” uses a deep neural network to predict the time gap between two waybills in one stay point (as well as the time before the first one and the time after the last one). The distribution of the waybill on the unit and floor is encoded. Context features, including “Courier Id,” “Workday or Weekend,” and “Time of Day,” are also embedded. The model is trained using the total time of these time gaps, which should be equal to the stay time. In the inference phase, the model outputs are further scaled to be consistent with the stay interval.
4.2 Overall Performance (RQ1)
The performance of the proposed DTRec and the 4 \(\times\) 4 baselines is reported in Table 1. The major results w.r.t. Acc and DTE are shown in subtable (a) and extended results w.r.t. Pre are in subtable (b). In subtable (b), the overall Pre is omitted since it is equivalent to the overall Acc. We first draw observations from subtable (a):
—
DTRec consistently outperforms all the baselines on overall coarse-grained Acc, overall fine-grained Acc, and DTE. What's more, DTRec is especially powerful w.r.t fine-grained results, where the DTE is reduced by 6.8% and the fine-grained overall Acc is increased by 10.8% compared with the best baseline. Besides, for the most important behavior, i.e., deliver, DTRec achieves a significant gain of 5.7% and 20.4%, respectively, for coarse-grained results and fine-grained results. This validates the effectiveness and superiority of our method.
—
For some specific behaviors, DTRec is not the best, for example, MSTP is slightly superior to ours on move and rest. However, MSTP may have achieved better Acc on these behaviors due to the false positive phenomenon discussed in the Acc metric calculation, which is at the cost of damaging the performance of other kinds of behaviors and the overall performance. Indeed, MSTP suffers an obvious drop in the performance of coarse-grained “Acc-deliver” and also the overall Acc.
—
Comparing the coarse-grained baselines, MS performs worst, MT and MSTP achieve similar overall coarse-grained Acc while MSTP is better at overall fine-grained Acc, and the DTInf generally performs best. This indicates that the location error caused by address parsing and the actual delivery location gap is even worse than the DTE caused by courier confirmation, and both the spatial information and the temporal information are important for the recovery task.
—
Comparing the fine-grained baselines, mid and smt generally produce worse results, which indicates that neither the existing coarse-grained practice nor directly using the confirmed delivery time is advisable. unf and deep generally result in better DTE and fine-grained Acc, of which deep is generally even better, which indicates the awareness of the waybill distribution on units and floors is important to recover fine-grained behaviors.
—
Comparing DTRec with the best baseline, i.e., “DTInf+deep,” DTRec is still superior on the coarse-grained Acc and especially superior on DTE and fine-grained Acc. This suggests that our digital-twin-based approach that explicitly encodes the human factors and physical constraints with an agent-based model is favorable for the fine-grained recovery problem, and utilizing the correlations between behaviors to iteratively calibrate the digital-world model and the real-world behaviors can further boost the fine-grained and coarse-grained performance.
Table 1.
| (a) Major results over Acc and DTE | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Method | Coarse-grained | Fine-grained | |||||||||
| Acc | Acc-deliv | Acc-move | Acc-rest | DTE | Acc | Acc-deliv | Acc-arrg | Acc-up | Acc-down | Acc-unit | |
| MS \(+\) mid | 57.98 | 47.65 | 74.06 | 67.15 | 183.87 | - | - | - | - | - | - |
| MS \(+\) unf | 57.98 | 47.65 | 74.06 | 67.15 | 195.98 | 38.71 | 34.93 | 13.65 | 20.31 | 9.73 | 6.65 |
| MS \(+\) smt | 57.98 | 47.65 | 74.06 | 67.15 | 203.42 | 38.35 | 34.66 | 11.58 | 21.42 | 19.49 | 5.90 |
| MS \(+\) deep | 57.98 | 47.65 | 74.06 | 67.15 | 183.17 | 40.80 | 32.43 | 20.35 | 18.10 | 25.55 | 0.00 |
| MT \(+\) mid | 76.01 | 76.39 | 76.40 | 25.16 | 114.03 | - | - | - | - | - | - |
| MT \(+\) unf | 76.01 | 76.39 | 76.40 | 25.16 | 108.98 | 46.16 | 41.81 | 25.85 | 38.79 | 18.86 | 7.48 |
| MT \(+\) smt | 76.01 | 76.39 | 76.40 | 25.16 | 114.16 | 45.21 | 42.61 | 17.86 | 46.52 | 19.57 | 7.48 |
| MT \(+\) deep | 76.01 | 76.39 | 76.40 | 25.16 | 111.60 | 48.98 | 44.14 | 39.73 | 31.48 | 20.03 | 3.07 |
| MSTP \(+\) mid | 75.59 | 69.31 | 85.25 | 81.85 | 134.92 | - | - | - | - | - | - |
| MSTP \(+\) unf | 75.59 | 69.31 | 85.25 | 81.85 | 131.35 | 49.61 | 41.66 | 24.58 | 33.05 | 23.67 | 9.93 |
| MSTP \(+\) smt | 75.59 | 69.31 | 85.25 | 81.85 | 126.50 | 47.86 | 40.49 | 17.23 | 35.17 | 20.53 | 13.64 |
| MSTP \(+\) deep | 75.59 | 69.31 | 85.25 | 81.85 | 127.71 | 53.20 | 46.59 | 37.64 | 32.91 | 23.20 | 4.19 |
| DTInf \(+\) mid | 82.65 | 81.36 | 82.84 | 79.37 | 99.67 | - | - | - | - | - | - |
| DTInf \(+\) unf | 82.65 | 81.36 | 82.84 | 79.37 | 95.63 | 47.05 | 46.55 | 25.01 | 35.05 | 14.86 | 4.64 |
| DTInf \(+\) smt | 82.65 | 81.36 | 82.84 | 79.37 | 101.52 | 45.91 | 46.40 | 18.29 | 41.38 | 15.04 | 7.50 |
| DTInf \(+\) deep | 82.65 | 81.36 | 82.84 | 79.37 | 96.01 | 54.59 | 54.67 | 43.68 | 43.86 | 33.97 | 0.00 |
| Ours | 84.67 | 85.96 | 83.72 | 80.70 | 89.14 | 60.51 | 65.82 | 56.52 | 45.59 | 32.36 | 14.14 |
| Gain | 2.4% | 5.7% | \(-\) 1.8% | \(-\) 1.4% | 6.8% | 10.8% | 20.4% | 29.4% | \(-\) 2.0% | \(-\) 4.7% | 3.7% |
| (b) Extended results over Pre | ||||||||
|---|---|---|---|---|---|---|---|---|
| Method | Coarse-grained | Fine-grained | ||||||
| Pre-deliv | Pre-move | Pre-rest | Pre-deliv | Pre-arrg | Pre-up | Pre-down | Pre-unit | |
| MS \(+\) mid | 74.03 | 57.37 | 41.08 | - | - | - | - | - |
| MS \(+\) unf | 74.03 | 57.37 | 41.08 | 52.35 | 77.14 | 13.24 | 7.21 | 5.42 |
| MS \(+\) smt | 74.03 | 57.37 | 41.08 | 50.79 | 78.55 | 13.37 | 14.54 | 6.22 |
| MS \(+\) deep | 74.03 | 57.37 | 41.08 | 47.78 | 80.25 | 15.31 | 22.36 | 0.00 |
| MT \(+\) mid | 82.91 | 55.44 | 74.76 | - | - | - | - | - |
| MT \(+\) unf | 82.91 | 55.44 | 74.76 | 65.32 | 70.53 | 16.20 | 19.54 | 5.24 |
| MT \(+\) smt | 82.91 | 55.44 | 74.76 | 62.62 | 64.09 | 18.00 | 23.14 | 3.17 |
| MT \(+\) deep | 82.91 | 55.44 | 74.76 | 70.07 | 69.47 | 23.11 | 31.80 | 12.92 |
| MSTP \(+\) mid | 88.09 | 59.59 | 67.43 | - | - | - | - | - |
| MSTP \(+\) unf | 88.09 | 59.59 | 67.43 | 66.05 | 67.16 | 18.01 | 22.60 | 7.82 |
| MSTP \(+\) smt | 88.09 | 59.59 | 67.43 | 60.90 | 64.57 | 17.79 | 21.01 | 6.85 |
| MSTP \(+\) deep | 88.09 | 59.59 | 67.43 | 72.90 | 64.95 | 23.47 | 25.10 | 5.31 |
| DTInf \(+\) mid | 90.12 | 59.89 | 75.50 | - | - | - | - | - |
| DTInf \(+\) unf | 90.12 | 59.89 | 75.50 | 66.64 | 60.33 | 13.53 | 11.50 | 7.70 |
| DTInf \(+\) smt | 90.12 | 59.89 | 75.50 | 62.20 | 58.60 | 15.08 | 12.59 | 4.28 |
| DTInf \(+\) deep | 90.12 | 59.89 | 75.50 | 74.11 | 59.50 | 25.35 | 28.76 | 0.00 |
| Ours | 88.62 | 63.13 | 95.91 | 76.68 | 70.77 | 46.03 | 34.35 | 14.01 |
| Gain | \(-\) 1.7% | 5.4% | 27.0% | 3.5% | \(-\) 11.8% | 81.6% | 8.0% | 8.4% |
Table 1. Performance Comparison of Our Model and State-of-the-Art Baselines
deliv, Deliver.
Extended evaluation results in subtable (b) are also consistent with our above observations. Specifically, DTRec generally achieves superior performance considering both Acc and Pre for each type of behavior, while Acc and Pre can exhibit some trade-offs. Indeed, when DTRec is not the best at Acc for some type of behavior, DTRec is always the best at Pre of the corresponding behavior. And it also holds after exchanging the role of Acc and Pre. While MSTP achieves better Acc on move and rest, the corresponding Pre becomes relatively poor. Overall, the results show that DTRec can accurately and precisely “partition and label the time axis” of the courier working process.
4.3 Ablation Study (RQ2)
An in-depth ablation study is conducted with six variants of the proposed methods to validate the effectiveness of the designed spatio-temporal bias correction module and the digital-twin-based iterative calibration module. For each of the variants, we remove the corresponding module and observe how it affects the performance. Specifically, three variants named -StayCor, -AdrCor, and -Cor refer to removing the stay point correction module, the address correction module, and both modules, i.e., the spatio-temporal bias correction module, respectively. Other three variants named -MatchIter, -StayIter, and -Iter stand for removing the waybill-stay matching calibration module, the stay time calibration module, and both modules, i.e., the digital-twin-based iterative calibration module, respectively.
Figure 7(a) and (b), respectively, shows the coarse-grained and fine-grained overall Acc of each variant. We derive four observations. First, both the coarse-grained and the fine-grained performance consistently drop when any designed module is removed, and the more modules removed, the more significantly the performance deteriorates, which demonstrates the effectiveness of each of our designs. Second, the removal of spatio-temporal bias correction modules generally results in a greater performance decrease, which suggests that the correction of basic data is essential to the overall performance while the iterative modules can further boost the final performance. Third, the effect of the address correction module consistently outweighs the stay point correction module, which suggests that the delivery location bias is more serious than the stay point extraction error. Fourth, the waybill-stay matching calibration module affects the coarse-grained performance more than the stay time calibration module, while the opposite holds for the fine-grained performance, which is within the expectations since the coarse-grained results are exactly decided by the waybill-stay matching while the stay time calibration directly affects the time range of the recovered fine-grained behaviors.
Fig. 7.
4.4 Case Study (RQ3)
4.4.1 Address Correction Case.
In Figure 8, we show how the address correction method takes effect. There is a waybill with the Geocoding location (on the right) far away from the matched location proposed by our address correction method (on the left). The waybill finish time lies in the time range of a stay point at the proposed location, which suggests the matching result is correct. We further check the view street images6 at the Geocoding location and the delivery location and find that the Geocoding location is at the gate of a residential community, which suggests the Geocoding API may have matched the shipping address to the community gates due to incomplete POI database. This case shows the gap between the waybill address and the actual delivery location and the effectiveness of our address correction method.
Fig. 8.
4.4.2 Digital Twin Calibration Case.
To examine how the digital-twin-based iterative calibration takes effect in specific real-world cases, in Figure 9, we show a specific digital-twin-based calibration case. “Iter 0” shows the original result. The two stay points \(S_{1},S_{2}\) actually belong to the same delivery event which delivers all the three waybills. However, due to the large spatial scale of this building, it is split into two parts \(S_{1},S_{2}\). The courier confirmed the three waybills \(O_{1},O_{2},O_{3}\) after finishing all of them in a batch, so their finish times all lie in the range of \(S_{2}\). As a result, the three waybills are all matched to \(S_{2}\). In “Iter 1,” however, with the waybill exchanging operations in the waybill-stay matching calibration module, \(O_{1},O_{2}\) are exchanged to \(S_{1}\). This is because with no waybills matched, \(S_{1}\) is a “taker” and with too many waybills, \(S_{2}\) is a “giver.” And with \(O_{1},O_{2}\) exchanged to \(S_{1}\), better consistency between the stay time and the predicted delivery time is achieved. Finally in “Iter 2,” the stay time calibration module further takes effect. With two waybills matched to \(S_{1}\), \(S_{1}\) becomes a giver and our algorithm extends its time range. As a result, the time range of \(S_{1}\) and \(S_{2}\) touches and they are further merged into a whole stay point, which is exactly the case in real life and leads to more precise fine-grained behaviors.
Fig. 9.
4.5 Deployed System (RQ4)
An intelligent terminal delivery depot management system is built based on DTRec and tested in the industrial practices of JD Logistics. Figure 10 shows the system interface demonstrated to terminal managers, which consists of the main panel in the middle, the depot information panel on the left, and the courier information panel on the right. The main panel keeps presenting the fine-grained working status of each courier, including the real-time location and behavior semantics, which helps detect the abnormal state of couriers and analyze the delivery process of any waybill, especially those delayed, to optimize the working procedure. The left panel shows the depot information including the efficiency, quality, and cost at the corresponding time, which provides a bird's eye view of overall operation status. The right panel gives the detailed working information of a courier, including working process logs, workloads, and other metrics. With the system, a depot manager can easily perceive, analyze, and optimize the courier working procedure, which contributes to meticulous digital and intelligent management with detailed, comprehensive, and intuitive supervision and balanced task allocation.
Fig. 10.
5 Related Work
Couriers’ Working Process Recovery. Recovering the couriers’ working process is one of the most important problems for last-mile delivery systems. Existing works have addressed many dimensions of the process, including the delivery time [16, 19, 24], the service time [17], the delivery location [18], the delivery sequence [15, 23], and so on. For instance, Ruan et al. [17] designed a meta-learning-based model to predict the service time. They also propose to recover the actual delivery locations based on couriers’ trajectories, which corrects the recorded delivery time with delays [18]. Ren et al. [14] infers the map-constrained trajectories with a Seq2Seq model. Overall, most existing works only focus on a single dimension and can only recover the coarse-grained delivery events, while we recover the whole working process with fine-grained spatio-temporal resolution, with consideration of complex correlations between behaviors.
Digital Twin Systems in Logistics. The digital twin is a concept that was first introduced in the manufacturing industry, and it is now widely referred to as a technology that builds a virtual model that interacts with the physical system [20]. It is drawing increasing attention and has been widely used in logistics in recent years [1, 3, 7]. Many researchers and practitioners leverage it to support decision-making in logistics, building more sustainable and efficient delivery systems [3, 9, 13]. For example, Belfadel et al. [3] propose an adaptive modeling approach based on digital twins and build a digital twin framework for designing and assessing targeted urban logistics policies. Different from existing works, we propose to leverage digital twins to correct the recovered results by the interaction between our digital courier model and the working process recovering results from the physical systems, which successfully improves both the digital courier model and the working process recovering results.
6 Discussion
In this section, we discuss the scalability and generalization capacity of DTRec, then summarize what issues to note when practically implementing it, and finally suggest improvement directions in future works.
The proposed approach is expected to be adequately efficient and scalable for typical terminal deploy depots in large modern cities. Specifically, we analyze the efficiency both from the algorithmic complexity and the faced data scale. In respect of the algorithmic complexity, both operations in the real-world behavior calibration step (lines 10 and 11) are of O(n) complexity according to the stay points number. And the model parameter fitting operation in the digital model calibration step (line 7 in Algorithm 1) is a standard linear regression problem for which quick solvers are available. Besides, the associated matrix in solving the linear regression is of shape (m, n), where m is the number of coarse-grained delivery behaviors which is no more than stay points number, and n is the number of parameters which is 4 or 6, respectively, for Equations (9) and (10). In our data, their computations are all within several seconds. Although there are iterations to require repeated computations, the overall time consumption is still within several minutes for each courier. The more time-consuming operations are the stay point extraction and correction of O(n) complexity according to the trajectory point number and the waybill address correction involving clustering on stay point locations. In our study, it takes around an hour to process the dataset. However, these steps belong to data preprocessing, and the results can be kept in memory after a single calculation. In respect of the faced data scale, since the digital twin is respectively built for each courier, and the behaviors are recovered for everyday work, the faced data scale will be no more than the number of waybills/stay points per courier per day, which are bound to be limited (typically less than 300) due to the labor capacity even if different cities and logistics systems are considered. In other words, the computation is not sensitive to the total dataset scale owing to the factorization into each courier and day. Users may further parallelize the computation across the couriers at larger depots with more couriers.
For generalization capacity, fair efforts may be needed to effectively generalize the methods under scenarios where the couriers’ working patterns are out of our original assumption. For example, the prior assumption on behavior patterns, such as described in Equations (9) and (10), may be modified in a different logistics system compared with JD Logistics with heterogeneous courier behavior patterns. Also, some parameters may need to be tailored, such as the stay-waybill matching time tolerance \(T^{+},T^{-}\), and stay time threshold \(T\) that may vary with the property of the business of specific logistic systems. Nevertheless, we remark that the proposed digital-twin-based approach as well as the framework of Algorithm 1 is still generally applicable to various situations despite modifications to specific equations and parameters may be needed. Besides, the fact, that DTRec can directly work on the common waybill data and trajectory data without extra need for manual labels, has made it lighter and more feasible in industrial practice than the heavy purely data-driven approaches. Because by no means can the trained model directly generalize across different couriers and depots, labeling is needed for every different depot, and re-labeling is needed once the courier employee changes or even once the district in charge of the courier changes, while DTRec can simply adapt to these by using the latest data.
Then, we summarize the issues to note when implementing DTRec in practice. First is the resources to prepare, including the waybill address, waybill confirmed finish time, and courier trajectory. Make sure the couriers keep carrying a GPS device throughout the working process, such as the PDAs in JD Logistics. Make sure proper Geocoding services are available to parse the waybill address into GPS location, building unit number, and floor number. At least 1 month of the latest historical data is recommended. It is better to update the dataset and re-train the model regularly, to keep track of any changes in employees, districts of charge, or even the evolved personal capacity and habits. Then, preprocess the data with stay points extraction and correction and waybill address correction. It is recommended to visualize the trajectory data on map tiles to grasp how severe the GPS shifts are, the typical building scales and stay times, and then set suitable stay point spatial-temporal thresholds \(D\) and \(T\). Then match the waybill and stay points with spatial-temporal scores. Make sure to have basic knowledge of to what extent your couriers may advance or delay the confirmation of delivery and how far the delivery location can deviate from the address, to help tailor the thresholds \(T^{+},T^{-}\), and \(D_{match}\). Next, figure out the specific working patterns of couriers and develop the unique behavior pattern equation, such as Equations (9) and (10) if necessary. Users should combine both the courier's habits and the constraints of the working spots to figure out the standard procedures for the courier to conduct the delivery. Make sure to train the model on exactly those delivery events of the corresponding working pattern. Finally, to decide the iteration parameters \(N_{outer}\) and \(N_{inner}\), we recommend a straightforward grid search, since the computation is bound to be fast given the number of waybills and stay points are quite limited.
With the above discussion, we remark on the major limitations of the study and suggest improvement directions in future works. First, despite our analysis of the scalability and generalization capacity, DTRec should be evaluated on more datasets across heterogeneous logistics scenarios and cities to evaluate its robustness. Second, although the knowledge-encoded digital-twin-based approach makes the methods more feasible and explainable than purely data-driven methods, on the other hand, it requires more expertise from users to have fair knowledge of the nature of couriers’ working patterns to tailor specific behavior pattern equations and parameters. Future studies can provide more standard practice templates for users without adequate expertise under more different common working scenarios that we have not encountered in JD Logistics. Finally, we suggest extending the methods if more detailed and plenty of data is available. For example, height sensors or indoor/outdoor sensors [25] may also be available at couriers’ PDAs in future logistics systems, which can explicitly help infer fine-grained behaviors. For another example, given plenty of training data and more delicate features, such as package weights and even the building layouts, the courier-specific constants \(T_{b},T_{u},T_{f},T_{o}\) in Equation (9) can be replaced by functions depending on these features, which can be approximated by neural networks given plenty of training data. Future studies can extend DTRec to make use of more data sources to further relieve its dependence on human prior knowledge and user expertise, lifting its robustness and generalization capacity. Another important future direction is to further build simulation and optimization systems upon the solid fine-grained digital-twin of couriers, moving from perception and modeling to proactive rehearsal and decision [26].
7 Conclusion
In this article, we develop a digital-twin-based iterative calibration system for fine-grained courier working process recovery. Facing the three challenges of poor data quality, lacking ground truths of couriers’ fine-grained behaviors, and complex correlations among couriers’ behaviors, we propose three novel designs to effectively solve them, including a spatio-temporal bias correction algorithm, an agent-based courier digital twin model, and a digital-twin-based iterative calibration framework. Overall, extensive experiments show that our system achieves superior performance, demonstrating that digital twins for logistics are a promising future direction for both research and practice. To the best of our knowledge, we are the first study to identify the challenge and significance of the fine-grained courier working processes recovery problem and propose a digital-twin-based approach that enables the natural encoding of human knowledge, physical constraints, and the co-evolution and co-optimization of both the digital model and the real-world system. In future works, we suggest evaluating and improving the robustness of solutions with more extensive data across heterogeneous logistics scenarios and cities.
Footnotes
1
The address information on a parcel could be incorrect due to wrong address parsing. Furthermore, some couriers may deliver a parcel to a reception center rather than the shipping address according to the customer's preference.
References
[1]
Ahmed Zainul Abideen, Veera Pandiyan Kaliani Sundram, Jaafar Pyeman, Abdul Kadir Othman, and Shahryar Sorooshian. 2021. Digital twin integrated reinforced learning in supply chain and logistics. Logistics 5, 4 (2021), 84.
[2]
Martin Behnke. 2019. Recent trends in last mile delivery: Impacts of fast fulfillment, parcel lockers, electric or autonomous vehicles, and more. In Logistics Management: Strategies and Instruments for Digitalizing and Decarbonizing Supply Chains—Proceedings of the German Academic Association for Business Research. Springer, 141–156.
[3]
Abdelhadi Belfadel, Sebastian Hörl, Rodrigo Javier Tapia, and Jakob Puchinger. 2021. Towards a digital twin framework for adaptive last mile city logistics. In Proceedings of the 6th International Conference on Smart and Sustainable Technologies. IEEE, 1–6.
[4]
Techane Bosona. 2020. Urban freight last mile logistics—Challenges and opportunities to improve sustainability: A literature review. Sustainability 12, 21 (2020), 8769.
[5]
Chao Chen, Sen Yang, Yasha Wang, Bin Guo, and Daqing Zhang. 2020. CrowdExpress: A probabilistic framework for on-time crowdsourced package deliveries. IEEE Transactions on Big Data 8, 3 (2020), 827–842.
[6]
Chao Chen, Daqing Zhang, Xiaojuan Ma, Bin Guo, Leye Wang, Yasha Wang, and Edwin Sha. 2016. Crowddeliver: Planning city-wide package delivery paths leveraging the crowd of taxis. IEEE Transactions on Intelligent Transportation Systems 18, 6 (2016), 1478–1496.
[7]
Nadia Giuffrida, Jenny Fajardo-Calderin, Antonio D. Masegosa, Frank Werner, Margarete Steudter, and Francesco Pilla. 2022. Optimization and machine learning applied to last-mile logistics: A review. Sustainability 14, 9 (2022), 5329.
[8]
Sina Gläser, Hermann Jahnke, and Nikolas Strassheim. 2021. Opportunities and challenges of crowd logistics on the last mile for courier, express and parcel service providers. International Journal of Logistics Research and Applications 26, 1006–1034.
[9]
Dongmin Lee and SangHyun Lee. 2021. Digital twin for supply chain coordination in modular construction. Applied Sciences 11, 13 (2021), 5909.
[10]
Quannan Li, Yu Zheng, Xing Xie, and Wei-Ying Ma. 2008. Mining user similarity based on location history. In Proceedings of the 16th ACM SIGSPATIAL International Conference on ADVANCES in Geographic Information Systems. 1–10.
[11]
Ruiyuan Li, Huajun He, Rubin Wang, Yuchuan Huang, Junwen Liu, Sijie Ruan, Tianfu He, Jie Bao, and Yu Zheng. 2020. Just: JD urban spatio-temporal data engine. In Proceedings of the IEEE 36th International Conference on Data Engineering (ICDE’20). IEEE, 1558–1569.
[12]
Canhong Lin, King Lun Choy, George T. S. Ho, H. Y. Lam, Grantham K. H. Pang, and Kwai-Sang Chin. 2014. A decision support system for optimizing dynamic courier routing operations. Expert Systems with Applications 41, 15 (2014), 6917–6933.
[13]
Edoardo Marcucci, Valerio Gatta, Lisa Hansson, and Svein Bråthen. 2020. Digital twins: A critical discussion on their potential for supporting policy-making and planning in urban logistics. Sustainability 12, 24 (2020), 10623.
[14]
Huimin Ren, Sijie Ruan, Yanhua Li, Jie Bao, Chuishi Meng, Ruiyuan Li, and Yu Zheng. 2021. Mtrajrec: Map-constrained trajectory recovery via seq2seq multi-task learning. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1410–1419.
[15]
Siyuan Ren, Bin Guo, Longbing Cao, Ke Li, Jiaqi Liu, and Zhiwen Yu. 2022. DeepExpress: Heterogeneous and coupled sequence modeling for express delivery prediction. ACM Transactions on Intelligent Systems and Technology (TIST) 13, 6 (2022), 1–22.
[16]
Sijie Ruan, Xi Fu, Cheng Long, Jie Bao, Ruiyuan Li, Yiheng Chen, Shengnan Wu, and Yu Zheng. 2021. Filling delivery time automatically based on couriers’ trajectories. IEEE Transactions on Knowledge and Data Engineering 35, 2 (2021), 1528–1540.
[17]
Sijie Ruan, Cheng Long, Zhipeng Ma, Jie Bao, Tianfu He, Ruiyuan Li, Yiheng Chen, Shengnan Wu, and Yu Zheng. 2022a. Service time prediction for delivery tasks via spatial meta-learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 3829–3837.
[18]
Sijie Ruan, Cheng Long, Xiaodu Yang, Tianfu He, Ruiyuan Li, Jie Bao, Yiheng Chen, Shengnan Wu, Jiangtao Cui, and Yu Zheng. 2022b. Discovering actual delivery locations from mis-annotated couriers’ trajectories. In Proceedings of the IEEE 38th International Conference on Data Engineering (ICDE’22). IEEE, 3241–3253.
[19]
Sijie Ruan, Zi Xiong, Cheng Long, Yiheng Chen, Jie Bao, Tianfu He, Ruiyuan Li, Shengnan Wu, Zhongyuan Jiang, and Yu Zheng. 2020. Doing in one go: delivery time inference based on couriers’ trajectories. In Proceedings of the 26th ACM SIGKDD International Conference. 2813–2821.
[20]
Concetta Semeraro, Mario Lezoche, Hervé Panetto, and Michele Dassisti. 2021. Digital twin paradigm: A systematic literature review. Computers in Industry 130, 103469.
[21]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30, 5998–6008.
[22]
Yuan Wang, Dongxiang Zhang, Qing Liu, Fumin Shen, and Loo Hay Lee. 2016. Towards enhancing the last-mile delivery: An effective crowd-tasking model with scalable solutions. Transportation Research Part E: Logistics and Transportation Review 93, 279–293.
[23]
Haomin Wen, Youfang Lin, Huaiyu Wan, Shengnan Guo, Fan Wu, Lixia Wu, Chao Song, and Yinghui Xu. 2022. DeepRoute+: Modeling couriers’ spatial-temporal behaviors and decision preferences for package pick-up route prediction. ACM Transactions on Intelligent Systems and Technology (TIST) 13, 2 (2022), 1–23.
[24]
Fan Wu and Lixia Wu. 2019. DeepETA: A spatial-temporal sequential neural network model for estimating time of arrival in package delivery system. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 774–781.
[25]
Pengfei Zhou, Yuanqing Zheng, Zhenjiang Li, Mo Li, and Guobin Shen. 2012. IODetector: A generic service for indoor outdoor detection. In Proceedings of the 10th ACM Conference on Embedded Network Sensor Systems. 113–126.
[26]
Zhi-Hua Zhou. 2022. Rehearsal: Learning from prediction to decision. Frontiers of Computer Science 16, 4 (2022), 164352.
Index Terms
- Fine-grained Courier Delivery Behavior Recovery with a Digital Twin Based Iterative Calibration Framework
Recommendations
Stakeholders collaborations, challenges and emerging concepts in digital twin ecosystems
Abstract Context:Digital twin (DT) ecosystems are rapidly evolving, connecting many stakeholders, such as manufacturers, customers, and application platform providers. These ecosystems require collaboration and interaction between diverse actors to ...
Preventing Spam Email by Delivery Limitation in RMX
IDEAS '15: Proceedings of the 19th International Database Engineering & Applications SymposiumOn the rule-based email exchange system called RMX, similar to general mailing lists, anyone can send emails by sending to an address unique to RMX. However, there is a security problem that we cannot prevent spam emails and accidentally sending email ...
Comments
Information & Contributors
Information
Published In
Copyright © 2024 Copyright held by the owner/author(s).
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 05 November 2024
Online AM: 13 June 2024
Accepted: 14 April 2024
Revised: 23 March 2024
Received: 19 October 2023
Published in TIST Volume 15, Issue 5
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 373Total Downloads
- Downloads (Last 12 months)373
- Downloads (Last 6 weeks)76
Reflects downloads up to 16 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in