Enhancing Crowd Flow Prediction in Various Spatial and Temporal Granularities

The study of human mobility is relevant to a large variety of topics, including public safety, migration, on-demand services, pollution monitoring, diffusion of epidemics, geomarketing, and traffic optimisation [1, 2, 14, 16, 23, 24, 29, 30]. Thanks to the recent deluge of digital data and the striking development of artificial intelligence, there has been a vast scientific production on various tasks involving human mobility data [1, 22]. A notable example is crowd flow prediction, consisting in forecasting the aggregated incoming and outgoing flows of people that move across regions in a geographic area [37]. The main challenge in solving this task lies in capturing the close and far spatial and temporal dependencies in the data at the same time. To date, crowd flow prediction is tackled with two main approaches: statistical models based on time series, which generally cannot capture both the spatial and the temporal dependencies; and models based on deep learning, which outperform traditional statistical models thanks to their complex architecture [22]. Examples of such solutions are ST-ResNet [37], DMCSTNet [32] and ACMF [20]. All these models rely on convolutional networks to capture long and short term spatio-temporal patterns and on fully connected networks to capture external factors known to have an impact on human mobility (e.g., weather conditions). We argue that existing solutions may not satisfy policymakers needs as the models only predict aggregated inflow or outflow, no information is provided about the origin and the destination of these flows. However, that information is crucial when dealing with certain tasks like epidemic diffusion [3]. There is also another limitation. As the predictors are based on traditional convolutions, they can only deal with regular tessellations and it is not possible to work with geographic areas with irregular shapes like census tracts. It may be of interest to policymakers for a variety of reasons [3]. In this work, we propose CrowdNet, a deep learning approach based on spatio-temporal graph convolutions that solves crowd flow prediction and overtakes the aforementioned limitations. In particular, our model can predict crowd inflows and outflows in geographic areas with both regular and irregular shapes. Moreover, it corroborates the prediction with an origin-destination matrix.

We evaluate CrowdNet on the widely adopted datasets of bikes in New York City and taxis in Beijing. Regardless of the dataset, CrowdNet overtakes other state-of-the-art solutions providing also a set of new advantages to policymakers. We also provide the code to reproduce CrowdNet and our experiments on public datasets at (link to repo removed for double bling submission).The paper is structured as follows. First, we formally define the crowd flow prediction problem and flow prediction problem. In Section CrowdNet, we present an overview of the architecture of CrowdNet. In Section Experiments, we described the used datasets, the evaluation metrics and the settings of the experiments. In Section Results, we provide the obtained results and we show the performances of CrowdNet on irregular geographic areas and on the flow prediction task. In Section 2, we provide a brief description of the literature related to crowd flow prediction. Finally, we give a summary of the contributions of our work and possible future improvements.

Statistical-based methods for crowd flow prediction represent flows through equation matrices and adopt independent variables to represent adjacent areas and historical data. Autoregressive Integrated Moving Average (ARIMA) [17] uses a number of lagged observations of univariate time series to forecast new observations. Vector Auto-Regressive (VAR) exploits multiple time series to capture the pairwise relationships among flows [9]. Overall, autoregression approaches cannot capture neither complex temporal and spatial dependencies and they require feature engineering to transform raw data into appropriate internal representations for spatio-temporal dependency detection. In contrast, Deep Learning (DL) approaches can discover features from raw data automatically [11].

Deep Learning approaches. There are many DL algorithms that are specifically designed to solve the crowd flow prediction problem. Many of them are collected in a recent survey by Luca et al. [22]. Most of the solutions leverage convolutional neural networks (CNNs) and recurrent networks (RNNs) to capture spatio-temporal patterns and dependencies. Examples are [4, 6, 13, 19, 21, 26, 28, 31, 34, 37]. Some other solutions also rely on attention mechanisms. Examples are [4, 13, 28, 31]. In what follows, we introduce additional details of the models we will use as baselines in this study. DeepST [36] captures temporal patterns using the assumption that time series always respect temporal closeness, period, and seasonal trend. A convolutioanl neural network (CNN) module captures spatial dependencies, and a fusion mechanism combines the outputs. Spatio-Temporal Residual Network (STResNet) [37] improves on DeepST adding residual learning, a parametric and matrix-based fusion mechanism, and the consideration of external factors. Local-Dilated Region-Shifting Network (LDRSN) [28] combines local and dilated convolutions to learn the nearby and distant spatial dependency, which makes it more resilient to overfitting than approaches based on CNNs. Hybrid-Integrated DL Spatio Temporal network (HIDLST) [26] exploits an long-short term memory network (LSTM) to capture dynamic temporal dependency in time series and Residual CNNs to capture spatial dependencies. Attentive Traffic Flow Machine (ATFM)[21] captures spatial-temporal dependencies with two convolutional LSTMs (ConvLSTM) units and an attention mechanism able to infer the trend evolution exploiting dynamic spatial-temporal feature representation learning. Li et al. [19] proposes a model that is made up of densely connected CNNs to extract spatial characteristics, an attention-based long short-term memory module to capture temporal components and a fully connected neural network to extract features from external factors. Deep Spatio-Temporal Irregular Convolutional Residual LSTM (DST-ICRL) [6] integrates multi-channel traffic representations, irregular convolution residual networks and LSTMs to provide crowd flows forecasting. Multi-View Residual Attention Network (MV-RANet) [34] captures spatial dependencies by a double-branch residual attention network: one branch for small-scale dependency, the other one serves as an attention model, extracting spatial dependencies at large scale. External features are represented as three graphs of functional areas.

In this section, we formalise the problem of crowd flow prediction and introduce the main concepts used in the paper.

Mobility flows represent aggregated movements among geographic locations, and they are usually represented as an Origin-Destination matrix.

The origin and destination regions often coincides (n = m). In the Crowd Flow Prediction problem, flows are aggregated into crowd flows (either incoming or outgoing) and represented as a bi-dimensional matrix in which an element represents the crowd flow in a tile during a certain time interval.

Given the aforementioned, we define the problem as follows:

A variant of crowd flow prediction is flow prediction:

CrowdNet is a deep neural network whose input is a temporal origin-destination matrix that describes historical flows among different regions, allowing it to use tessellations of various shapes (e.g., irregular tessellations). First, the model solves flow prediction, forecasting the flows among all pairs of regions in the tessellation. Then, it solves crowd flow prediction summing all the flows in the predicted OD matrix that have as a destination (origin) k, so to obtain the crowd inflow (outflow) of region k.

Given a geographic area tasselled into n regions, we represent a set of flows at time t as tensors , where the first dimension represents the origin and the second dimension represents the destination of the flow. Hence, F_t(i, j) contains the flow at time t moving from region i to region j. A flow equal to 0 means that no people move from region i to region j at time step t.

We adapted CrowdNet from the work by Yu et al. [33] on traffic forecasting. In particular, we treat the problem as a weighted link prediction applied to temporal dynamic directed graphs, where each node represents a region and each weighted edge quantifies the flow between two regions. Formally, a weighted graph, at the t-th time step, is a triple G = (V, E_t, f_t), where V is a set of vertices, E_t is a set of edges, i.e., a set of ordered pairs (u, v) where (u, v) ∈ V × V, with u ≠ v and f_t is a function, assigning a value representing the weight of the edge [10]. The network outputs the graph at the t + 1 time interval, i.e., the triple G = (V, E_{t + 1}, f_{t + 1}).

4.1 Architecture

We can formalise CrowdNet as:

\begin{equation} \mbox{CrowdNet}(X_t, A) \rightarrow Y \end{equation}
(4)

where are the OD matrices with n nodes from time t to t + k. is the adjacency matrix of the graph (representing the Origin-Destination flows), in particular A _{i, j} = 1 if i and j are linked in at least one interval is the model's prediction, where l is the number of time intervals predicted and n is the number of regions. Therefore, CrowdNet's predictions are the adjacency matrices from time t + k + 1 to t + k + l. In our experiments, we fix l = 1. Y is then aggregated into a bi-dimensional matrix where each element represents the inflow and the outflow, solving the crowd flow prediction problem. Formally, the second output of CrowdNet is where n = q × q, q is the number of tiles in the x axis and q is the number of tiles in the y axis.

CrowdNet's architecture is composed of several spatio-temporal convolutional blocks, each made up of a multi-layer structure with two convolutional layers and one spatial graph convolutional layer in between. The former captures temporal dependencies and the latter catches the spatial dependencies. Since for crowd flow prediction it is necessary a good response to dynamic changes [33], we apply convolutions on the time axis to capture the temporal features of flows [8]. Figure 1 schematizes CrowdNet's architecture.

Time Block. Inspired by Gehring et al. [8], we exploit CNNs to capture temporal dependencies. This choice is justified by the fact that CNNs perform well when predicting flows dynamic changes. With respect to Recurrent Neural Networks (RNNs), CNNs have a faster training and the lack of dependencies constrains to previous steps allows a parallel and controllable training process through the multi-layer convolutional structure.

The Time Block (TB) contains a convolution followed by Gated Linear Units (GRUs) [5], which implement a gating mechanism over the output of the convolution. Formally:

\begin{equation} \Gamma = (X * \Theta _1 +b_0) \circ \sigma (X * \Theta _2 + b_1) \end{equation}
(5)

where X is the input, Θ ₁, Θ ₂, b ₀ and b ₁ are learnable parameters, σ is the sigmoid function, * is the convolution operator, and ○ denotes the Hadamard product. The sigmoid gate controls what is relevant for discovering the structure and the dynamics of the time series. To enable the use of deep convolutional networks, we use residual connections from the input X to the output of each layer. We use Rectified Linear Unit ( ReLU) as the final activation function, defined as ReLU( x) = max(0, x). In summary:

\begin{equation} TB = ReLU(X * \Theta _3 + b_2 + \Gamma). \end{equation}
(6)

Spatial Block. We define a Graph Convolution as:

\begin{equation} X^{\prime } = \hat{D}^{-\frac{1}{2}} \hat{A}\hat{D}^{-\frac{1}{2}} X \Theta \end{equation}
(7)

where $ \hat{A} = A + I$, i.e., $\hat{A}$ is the adjacency matrix of the directed graph G with self-loops. I is the identity matrix. $\hat{D}$ is a diagonal matrix:

\begin{equation} \hat{D}_{i,i} = \sum _{j=0}^n \hat{A}_{i,j} \end{equation}
(8)

where the element ( i, i) is the number of adjacent nodes for the node i. All the other elements are equal to 0. X is the input, i.e., the origin-destination matrix for a defined time interval, as defined in Section 4. This operation is better motivated by a first-order approximation of localised spectral filters on the graph [ 15].

The operation of graph convolution is used by a layer block, named Spatial Block (SB). It is a one-layer Graph Convolutional Network (GCN) having the following form as forward model:

\begin{equation} SB = ReLU(\hat{D}^{-\frac{1}{2}} \hat{A}\hat{D}^{-\frac{1}{2}} X \Theta) \end{equation}
(9)

where ReLU is the rectified linear unit activation function applied to the graph convolution operation.

ST-GCN Block. The ST-GCN Block is composed of a Time Block, a Spatial Block, and another Time Block. The Spatial Block is fed by the first Time Block and performs a graph convolution that can be expressed as:

\begin{equation} X^{\prime \prime } = SB(X, \hat{A}) = ReLU(\hat{D}^{-\frac{1}{2}} \hat{A}\hat{D}^{-\frac{1}{2}} X \Theta) \end{equation}
(10)

The output of the Spatial Block layer is provided to a Time Block, which in turn returns:

\begin{equation} X^{\prime \prime \prime } = TB(X^{\prime \prime }) = ReLU(X^{\prime \prime } * \Theta _3 + b_2 + \Gamma). \end{equation}
(11)

Finally, a batch normalisation is applied to the output of the the last Temporal Block. Batch normalisation is defined as

\begin{equation} y^{\prime } = \frac{X^{\prime \prime \prime } - E[X^{\prime \prime \prime }]}{\sqrt {Var(X^{\prime \prime \prime }) + \epsilon }} * \gamma + \beta \end{equation}
(12)

where γ and β are learnable parameters. It allows to use higher learning rates, to have faster training and permits to take less care to initialisation [ 12].

In summary, CrowdNet has two ST-GCN layers and one output layer. The last block maps the outputs of the last ST-GCN layer into a single step prediction output. The loss function used in CrowdNet is the Mean Squared Error (MSE).

In this section, we compare the results of CrowdNet with those of ST-ResNet for the crowd flow prediction problem. Moreover, the following Tables contain results related to the BikeNYC datasets. The same Tables and results for the datasets of BikeDC and TaxiBJ can be found in the Appendix. We report only the BikeNYC results as the behaviour of CrowdNet on the other datasets does not change.

Figure 2 visually compares the mean real crowd inflows with the mean crowd inflows predicted by STResNet and CrowdNet, with tessellations of 1000m and time interval of 60min. It is evident how the predicted crowd flows are strikingly similar to the real ones. For example, the predictions reproduce a notable pattern in Manhattan: the concentration of areas with large crowd flows in the middle of the island and areas with small crowd flows in its borders. In general, CrowdNet slightly underestimates large crowd flows.

Figure 3 compares the sum of the total real crowd inflows during one week (from the 22nd to the 28th of Semptember), for tiles of 1000m and time intervals of 60min, with those predicted by STResNet and CrowdNet. CrowdNet's predictions are closer to the real values than STResNet's predictions, therefore we can say that CrowdNet performs better in case of high time intervals. When external events occur such as the thunderstorm occurring on Thursday in Figure 3, the performance of the models is worse.

Table 8 summarises the results of both models, reporting the RMSE for all the possible combination of tile sizes and time intervals. Note how CrowdNet performs better for larger time intervals, while for smaller values the RMSE of the two models is comparable.

One of the advantages of using CrowdNet is the possibility to use irregular tessellations. STResNet, DMBSTNet and ACMP cannot be used with irregular tessellations because they take as input an image-like matrix; irregular tessellations cannot be represented easily in this way. For instance, considering an administrative tessellation, a region can have different neighbours, and it is difficult to represent this kind of relationship using an image-like relationship.

We illustrate the performance of CrowdNet on an irregular tessellation defined by an administrative tessellation defined by 29 neighbourhoods in Manhattan. Data of the administrative tiles are taken from an official tool of the municipality of New York City.

Figure 4 compares, the real crowd inflows and outflows with the crowd inflows and outflows predicted by CrowdNet. As in the case of the squared tessellation, CrowdNet's predictions are strikingly similar to the real one. Note how, also in this case, larger crowd inflows and outflows are concentrated in the middle of the island, while smaller ones concentrate in the southern part of the island.

Flow prediction. While crowd flow prediction aims to forecast the aggregated flows in each tile, flow prediction aims at predicting the flow between each pair of tiles, thus corresponding to the prediction of the entire origin-destination matrix.

In Appendix, Figure 6 compares the real flows with those predicted by CrowdNet: the two predictions are almost identical, meaning that the flows are correctly predicted by the model.

Providing detailed information about the crowd flow predictions is essential to acquire knowledge that can be useful to possible users, such as policymakers and urban planners. In this direction, CrowdNet enriches the predicted crowd flows with useful information, such as the origin and the destination of each flow. As an example, Figure 5a illustrates the predicted crowd flows in companion with the flows between the tiles in Manhattan: each node's size is proportional to its crowd inflow, while edge thickness represents the magnitude of the single flows between pairs of tiles. Figure 5b shows the same crowd flow prediction with a focus on the flows outgoing from node 17. The ability of CrowdNet to solve flow prediction allows us to enrich crowd flow predictions with information about the origin of a tile's inflow or outflow.

To investigate the robustness of CrowdNet to the variation in the spatial and temporal aggregation, we investigate how the model's performance changes varying time intervals (fixing a tile size of 1000m) or tile sizes (fixing time intervals to 60min). Results with others time intervals or tile sizes are similar.

CrowdNet is robust with respect to the temporal aggregation. As shown in Appendix through Table 3 the RMSE increases as the time interval increases. This is due to the fact that the flow magnitude increases as the time intervals becomes bigger. Similarly, considering the spatial aggregation, the RMSE increases as the tile size increases. This is due to the fact that as the tile size becomes bigger its flow increases.

We proposed CrowdNet, a solution to crowd flow prediction that represents crowd flows as a graph in which nodes are geographic regions and edges are people moving among them. CrowdNet allows policymakers to use spatial tessellations (even non-squared ones) and make predictions of flows at street or block levels. We also provided a characterisation of CrowdNet's behavior on different types of tessellation with different shapes and sizes and on different time intervals, so to find the combinations that lead to the most accurate predictions.

CrowdNet is a first step towards the adoption of more exhaustive prediction models. Our experiments may lead policymakers to the adoption of more transparent and trustable solutions thanks to their enriched prediction and parameterisation.