Dynamic 3D Point Cloud Sequences as 2D Videos

The recent years have witnessed the proliferation of various deep set architectures that directly operate on unstructured 3D point sets without any pre-processing procedures, as pioneered by [5, 6]. The follow-up researches further investigated rich varieties of convolution-based [14, 15, 16, 17, 18, 19, 20, 9, 10], graph-based [21, 7, 22], and transformer-based [23, 8, 24, 25] backbone learning architectures.

More specifically, PointNet [5] proposed to perform point-wise embedding and global aggregation via shared multi-layer perceptrons (MLPs) and channel max-pooling. PointNet++ [6] further incorporates multi-scale hierarchical feature abstraction mechanisms with farthest point sampling (FPS) and neighborhood interpolation. SO-Net [14] explicitly utilizes the spatial distribution of point clouds by building self-organizing maps. PointCNN [15] applies learned transformations for adaptively weighting and permuting point features in potentially canonical orders. FeaStNet [21] dynamically builds correspondences between filter weights and graph nodes. KPConv [16] learns to continuously locate convolution weights in Euclidean space through kernel points and further explores spatially deformable convolution operators. RS-CNN [17] focuses on capturing the high-level relations among local point sets via learning from predefined geometric priors. DGCNN [7] queries neighbors globally in the feature space for constructing graph edges and performing dynamic feature aggregation. PointASNL [18] adopts an adaptive sampling scheme together with local-nonlocal cells to improve its model robustness when dealing with noises and outliers. CurveNet [20] aggregates hypothetical curves initially grouped through guided walks to augment point-wise features. PointMLP [26] builds a pure residual MLP network equipped with lightweight geometric affine modules, which can perform competitively without any sophisticated learning mechanisms. PointNeXt [9] systematically revisits the classic PointNet++ processing pipeline and then improves model training and scaling strategies. PT [8] investigates highly-expressive self-attention layers for point clouds. FastPT [24] incorporates lightweight local self-attention with voxel hashing architectures to encode continuous 3D positional information to greatly enhance computational efficiency. Conclusively, these approaches above are specialized for learning from a single 3D point cloud input to extract geometric feature representations in the spatial domain. The adaptation to dynamic point cloud learning for achieving sequence processing and analysis is highly non-trivial, due to the complexity of joint spatio-temporal modeling.

Besides, another line of work resorts to a different perspective for overcoming the irregularity and unstructuredness of 3D point clouds by creating regular 2D geometry image (GI [27]) or GI-like representation structures, on which numerous mature 2D modeling architectures (e.g., convolutional neural networks (CNNs)) can be seamlessly applied, or adapted with minimal modifications, to achieve various point cloud learning tasks. Representatively, [28, 29, 30, 31] exploit traditional mesh parameterization techniques to implement the process of surface-to-plane mapping, while [32, 33] propose to directly learn deep 2D regular representations from 3D point clouds in an unsupervised manner. These works are also related to the previous folding-based methods [34, 35, 36]. However, these approaches are still limited to static geometric representations, and are thus inapplicable to dynamic point cloud sequences with requirements of spatio-temporal structurization.

DPMix [37] employs 2D depth video methods to enhance the understanding of point cloud videos. Other methods explore the temporal interpolation of the 3D point cloud sequences [38, 39, 40]. These advancements demonstrate the growing interest and diversification in approaches to effectively model the dynamic nature of 3D point cloud sequences.

Specifically, PSTNet [41] introduces the point spatio-temporal (PST) convolution, leveraging a ‘point tube’ concept to aggregate local spatio-temporal features from 3D point cloud sequences. Building upon this, PSTNet2 [42] merges spatial features into a unified spatio-temporal representation. And MaST-Pre [43] advances this concept with a masked point tube for pre-training on point cloud videos, incorporating the spatiotemporal masked autoencoder idea from [44]. P4Transformer [45] boosts the performance of PSTNet by employing transformers to bypass the need for point tracking while capturing spatio-temporal correlations across point cloud sequences. Similarly, $\text{PST}^{2}$ [46] introduces a spatio-temporal self-attention (STSA) module to grasp contextual information across adjacent frames. This concept is further enhanced by PST-Transformer [12] with spatio-temporal encoding. These architectures primarily follow an autoencoder-like framework to process dynamic point cloud sequences, akin to the role of PointNet++ [6] in handling static point clouds. While these methods introduce specially designed spatio-temporal feature aggregation structures to adapt to dynamic point cloud learning challenges, they do not fundamentally address the inherent irregularities of spatio-temporal characteristics in 3D point cloud sequences. In essence, the irregular data modality of 3D point cloud sequences remains unaltered.

Another line of research explores implicit representation methods. OFlow [47] adopts implicit strategies for 4D reconstruction, utilizing ground truth occupancy and trajectory information. CaSPR [48] introduces spatio-temporal representations for objects within normalized coordinate space. Similarly, CaDeX [49] tackles inter-frame deformations by employing continuous bijective canonical maps, focusing on a canonical shape. Notably, these methodologies primarily utilize mesh, voxel, or occupancy data, diverging from pure point cloud sequences. Our work, in contrast, concentrates on self-supervised explicit representations. Specifically, we structure the irregular 3D point cloud sequences as structured point cloud videos, aiming to achieve a compact representation that effectively minimizes both distortion and storage requirements.

A variety of methods have been developed for specific 3D point cloud sequence processing tasks, as well as related technologies. These technologies include flow-based [50, 51, 52], depth-based [53], GAN-based [54], kinematics-inspired [55], contrastive learning [56], and correspondence approaches [57, 58]. There are also some task-specific sequence processing methods, notably compression [59, 60] and interpolation [38, 39, 40].

For example, PV-RAFT [50] proposes a point-voxel recurrent all-pairs field transforms method to estimate scene flow from point clouds. NSFP [51] revisits the scene flow problem, emphasizing runtime optimization and regularization of the scene flow. SSFE [52] aims to predict 3D scene flow for all pairs of point clouds in a given sequence. 3DV [53] utilizes the dynamic 3D voxel for depth-based 3D action recognition. TPU-GAN [54] proposes a super-resolution generative adversarial network for upsampling dynamic point cloud sequences. Kinet [55] explores kinematics-inspired neural networks to capture 3D motions without explicit tracking. These methods employed various approaches, such as utilizing voxel and depth information for feature extraction, utilizing GAN-based methods to avoid the need for explicit point correspondence annotation, or relying on explicit scene flow annotation datasets. Overall, these methods did not directly address the underlying irregularity issues of 3D point cloud data. Furthermore, they lacked powerful insights into the explicit geometric aspects of 3D point clouds.

As mentioned earlier, learning from irregular and unordered point sets presents a significant challenge compared to feature extraction from 2D images or videos organized in a regular grid. Essentially, an RGB image of sizes $H\times W\times 3$ can be regarded as a set of 5-dimensional vectors denoted as $(r,g,b,h,w)$, where the color information (or pixel value) $(r,g,b)$ to be processed is indexed by the 2D coordinate $(h,w)$ uniformly distributed on a regular 2D grid of sizes $H\times W$, while for point cloud data, the spatial coordinates $(x,y,z)$ serve as both the geometry information to be processed and the indirect indexing cues to be used for determining inter-point relationships such as proximity and correspondence. Consequently, existing point-based deep learning architectures often fall short in efficiency when processing large-scale point data due to their involvement in complex learning mechanisms, including cumbersome pre-processing, extensive neighbor querying, and costly sampling, matching, and multi-scale abstraction processes. The significant computational complexity of the aforementioned time-consuming operations in turn hinders the development of deeper or more sophisticated architectural models for 3D point cloud sequences, thus limiting performance on downstream tasks. Additionally, these operations increase memory consumption, particularly when processing denser 3D point cloud sequences. Furthermore, measuring the discrepancy between two point clouds poses a significant challenge due to their irregular and unordered nature, making standard methods like CD and EMD either ineffective or inefficient [61].

The shift from static point cloud to 3D point cloud sequence modeling introduces additional challenges, exacerbated by the inherent lack of structure in both spatial and temporal dimensions. Current approaches rely heavily on intensive spatial neighborhood searches and the establishment of temporal correspondences, necessitating complex spatio-temporal feature processing. This intricate methodology is not just laborious and costly—it also substantially obstructs the creation of streamlined and efficient learning frameworks for 3D point cloud sequences.

Denote by $\mathcal{P}=\{\mathbf{P}_{t}\in\mathbb{R}^{N_{t}\times 3}\}_{t=0}^{T-1}$ a dynamic 3D point cloud sequence with $T$ point cloud frames, where $N_{t}$ refers to the number of points contained in the $t$-th frame¹¹1Without losing generality, we assign a fixed number of points to all frames in practice.. Note that the point-wise correspondence across the frames of $\mathcal{P}$ is unknown. Inspired by the above-mentioned structure of 2D images/videos and the fact that 3D geometric shapes are essentially 2D manifolds, we aim to structure dynamic 3D point cloud sequences to address the aforementioned challenges.

Generally, for each point of a typical point cloud frame $\mathbf{P}_{t}$, we plan to obtain a unique 2D coordinate $(u,v)$, thereby forming a 5-dimensional signal $(x,y,z,u,v)$. Meanwhile, points that are nearest neighbors in 3D space should have close 2D coordinates. When the $(u,v)$ coordinates are uniformly distributed across the regular 2D grid, the structure effectively resembles that of an RGB image. In other words, we can fill $[x,~{}y,~{}z]$ into the corresponding $(u,v)$ location, constructing an RGB image-like representation of dimensions $U\times V\times 3$ ($U\times V=N_{t}$). After processing all point cloud frames of $\mathcal{P}$ with this manner, we can derive a 2D video-like representation of $\mathcal{P}$, denoted as $\mathcal{G}=\{\mathbf{G}_{t}\in\mathbb{R}^{U\times V\times 3}\}_{t=0}^{T-1}$, named Structured Point Cloud Video (SPCV). Moreover, we constrain that the encoded points at an identical location across different frames of $\mathcal{G}$ correspond to the same or approximately the same position of the 3D object/scene, which is more advanced than traditional RGB videos. In the following, we also call an element of $\mathbf{G}_{t}$ a pixel for convenience. Note that $\mathbf{G}_{t}$ can be directly reshaped/reprojected back to a point cloud without any additional computation, i.e., the value of a pixel is taken as a 3D point.

In summary, we anticipate that the resulting SPCV should exhibit the following two characteristics, as illustrated in Fig. 4:

1. 1.

   Spatial Smoothness: the pixels in a local patch of a typical frame of an SPCV correspond to a set of neighboring points in 3D space;

2. 2.

   Temporal Consistency: the pixels with identical locations across the $T$ frames of an SPCV correspond to the consistent position of the object/scene.

The proposed representation modality is able to offer the following advantages:

1. 1.

   Improved Efficiency and Performance of Downstream Applications. The SPCV framework streamlines downstream processing and analysis tasks by replacing complex and time-consuming operations like FPS and K-NN aggregation with simpler and highly efficient pixel indexing and sliding window mechanisms. Meanwhile, the uniform structure of SPCV relives the challenge faced by neural networks in feature learning, thereby potentially generating more expressive features. Moreover, SPCV can seamlessly integrate with established learning techniques used in 2D image/video network designs, enhancing overall performance.

2. 2.

   Reduction of Memory Consumption. SPCV efficiently reduces memory usage by indexing frames using variable pixel locations, a contrast to traditional methods that necessitate storing multiple scales of point clouds for feature aggregation. This advantage becomes particularly evident when processing dense or large-scale point clouds.

3. 3.

   Simplified Design and Implementation. Adopting SPCV simplifies the design and implementation complexities in point cloud applications. For instance, encoders with specialized spatio-temporal feature extraction can be replaced with encoders that are already commonly used in the 2D image/video community.

4. 4.

   Free of the Limitations of CD and EMD. Traditional models often struggle with inefficiencies from point cloud loss functions like CD and Earth Mover’s Distance (EMD). The SPCV framework, however, enables more efficient and straightforward optimization using grid-based losses like the Frobenius norm or $\mathcal{L}_{1}$, thanks to its inherent structure.

To overcome this challenge, we consider constructing a self-supervised pipeline without relying on ground-truth 2D locations and temporal correspondences. Specifically, for a single point cloud, we first pre-define a set of 2D locations uniformly distributed on a regular 2D grid of sizes $U\times V$, each location storing/encoding $(u,v,0)$. We then feed $\{(u,v,0)\}$ into a learnable neural network to generate $\{(\hat{x},\hat{y},\hat{z})\}$ further assigned to the corresponding locations of their inputs. We can utilize the original point cloud as supervision to drive the learning of the network, ensuring that the generated $\{(\hat{x},\hat{y},\hat{z})\}$ closely resemble $\{(x,y,z)\}$. Moreover, owing to the inherent spectral bias property of neural networks [63], the generated data from spatially smooth inputs are expected to preserve spatial smoothness.

Furthermore, observing that a dynamic 3D point cloud sequence recording the motion of an object can be essentially thought of as a set of 3D points with a fixed topology deforming/evolving over time, we can adopt the following procedure to structure it. First, we represent $\mathbf{P}_{0}$ through the above-mentioned self-supervised learning approach for single point clouds, producing $\mathbf{G}_{0}$. We then obtain the representations of the remaining $T-1$ point cloud frames $\{\mathbf{P}_{t}\}_{t=1}^{T-1}$ by learning the deformation fields that deform $\mathbf{G}_{t-1}$ to $\mathbf{G}_{t}$ ($t\in[1,T-1]$) in a recursive fashion, i.e., $\mathbf{G}_{t}=\mathbf{G}_{t-1}+\bm{\Delta}_{\mathbf{G}_{t}}$, where $\bm{\Delta}_{\mathbf{G}_{t}}\in\mathbb{R}^{U\times V\times 3}$ stands for the deformation field for $\mathbf{G}_{t}$. To be specific, we can minimize the discrepancy between $\mathbf{P}_{t}$ and $\mathbf{G}_{t}$²²2Note that we reshape $\mathbf{G}_{t}$ into a point cloud and compute the discrepancy in 3D space. to estimate $\bm{\Delta}_{\mathbf{G}_{t}}$. Such a procedure aligns the resulting structures of $\{\mathbf{G}_{t}\}_{t=1}^{T-1}$ with that of $\mathbf{G}_{0}$, leading to the spatial smoothness of all frames. Also, the deformation mechanism establishes a fixed topology across all frames, thus naturally preserving temporal consistency across all frames.

Generally, the proposed SPCV shares a similar concept with previous parameterization-based geometry image/video representation for 3D triangle meshes/dynamic mesh sequences [27, 66, 67, 68], i.e., representing 3D geometry data with a 2D RGB image/video-like structure. However, our work significantly distinguishes itself from previous methods in terms of motivation, technical implementation, and application scenarios. First, previous methods map 3D vertices to the 2D domain through traditional parameterization algorithms optimizing the area and/or angle distortions of triangles. Due to the fundamental challenges of parameterization, these methods impose high requirements on the topological structures of 3D meshes to be processed, along with the requirement of correspondence across frames to model temporal information. Given that 3D point cloud sequences lack connectivity information and temporal correspondence and may have arbitrarily complex topological structures, these methods are largely inapplicable to our scenario. Second, driven by the objective of separating the indexing cues from the geometry information for improving efficiency, our pipeline re-organizes neighboring 3D points determined in the sense of the Euclidean distance into adjacent 2D locations on a regular 2D grid. The manner of determining a neighborhood is consistent with that of mainstream deep point-based architectures. In contrast, the parameterization-based methods essentially achieve the mapping in a geodesic distance-aware fashion. Accordingly, the theorems they employed are not suitable for our method. Finally, it is also worth noting that our method mimics the behavior of traditional parameterization when handling 3D objects with low curvatures, as demonstrated in Fig. 7.

In summary, unlike the previous representation mechanism pursuing accurate parameterization, the proposed learning-based pipeline leverages the impressive capability of deep learning to offer a practical and promising solution for real-world applications. The advantages and effectiveness of our proposed SPCV are extensively demonstrated through experiments on various downstream tasks in Section IV.

As a common sequence-level task in point cloud sequence processing, this task evaluates the model’s ability to extract discriminative features from each 3D point cloud sequence, thereby determining the most probable label for the sequence. Existing methods [69, 42, 45, 12] typically employ point-based spatio-temporal convolution operations, e.g., PST convolution [69, 42], for learning feature encodings across spatio-temporal dimensions, or transformers [45, 12] for spatio-temporal context interaction. However, due to the insufficient design to cope with the unstructured characteristic of 3D point cloud data, their capability of extracting discriminative features efficiently and effectively is still relatively weak, thus limiting their accuracy. With the video-like representation, our approach enables the use of mature 2D convolutional operations to obtain more discriminative and expressive features. We modify the feature embedding process, common in approaches like PST-Transformer [12], by employing an efficient convolutional encoder. This modification efficiently generates more expressive feature tokens for the transformer.

Specifically, as illustrated in Fig. 6 (a), the proposed SPCV-based framework first spatially downsamples an input SPCV to a quarter of its original scale before being passed into a video-based encoder, consisting of an 18-layer ‘UNet3D’ architecture, styled after ResNet [2], a design also used in conventional video-based methods [70, 71]. In the network’s bottleneck, the scaled SPCV frames, along with features scaled down to a quarter of their original size, are converted into sequences of point clouds and their corresponding features. These sequences are then fed into a decoder with transformer to enhance feature space interactions. Inside the decoder, the transformer’s output features undergo two layers of max-pooling to produce batches of feature vectors. These vectors are subsequently processed through an MLP head layer to generate the final scores, indicating the probabilities of various classes for the input sequence.

This task involves reconstructing a high temporal resolution 3D point cloud sequence from a low-resolution one, a counterpart of the 2D video frame interpolation, which targets to increase the frame rate of a video by generating the in-between frames. The unstructured nature presented within individual frames and lack of correspondence across frames compound the complexity of this task, especially for sequences with large non-rigid deformation. PointINet [38] addresses this task by adopting a scene flow estimator for interpolation. IDEA-Net [39] estimates point-wise trajectories for coarse interpolation and trajectory compensation. These methods suffer from inefficient and ineffective feature extraction and rely on distribution-based loss functions to interpolate point cloud frames with restricted fidelity. To address these issues, we develop an interpolation model based on an image encoder. Leveraging the structured nature of our representation, the mature image encoder efficiently generates more effective feature embedding tokens, which are then enhanced through downstream transformers in both temporal and spatial domains for feature interaction. And our model can be driven by common 2D loss functions like the Frobenius norm, enabling interpolating higher-quality point cloud frames efficiently.

Fig. 8 illustrates the overall learning pipeline of our proposed SPCV-based point cloud frame interpolation. Specifically, for an input pair of consecutive frames at time step $t_{1}$ and $t_{2}$, we consider interpolating a new frame at time step $t$, where $t_{1}<t<t_{2}$. To implement this, we feed the two point cloud frames into a shared 2D image-based encoder consisting of $4$ Conv2D layers to produce the corresponding feature maps $\mathbf{F}_{1}$ and $\mathbf{F}_{2}$. Then we can deduce an interpolated feature map, i.e., $\frac{t_{2}-t_{1}-t}{t_{2}-t_{1}}\mathbf{F}_{1}+\frac{t}{t_{2}-t_{1}}\mathbf{F}_{2}$, which is further passed into a residual decoder with $5$ Conv1D layers. Finally, we add the generated residuals to the input frame at time step $t_{1}$ to obtain the desired interpolation result.

With the rapid development of 3D sensing technologies and the broad deployment of consumer-level devices (e.g., depth camera, LiDAR, etc.), there is an urgent need to develop efficient 3D point cloud codecs to compress the corresponding huge data volume and effectively save transmission bandwidth and storage space. However, unlike its mature 2D counterpart, i.e., image/video compression, which has been investigated and developed for a long time, 3D point cloud compression is still a relatively young topic whose actual coding performance is far from satisfactory. Therefore, there exists an obvious contradiction between ever-growing 3D point cloud data volume and immature compression techniques. Compared to 2D image/video compression, 3D point cloud data compression³³3Note that we consider compression of the geometry information of 3D point clouds, rather than the attributes. poses a greater challenge due to the unstructured nature of spatial and temporal domains, i.e., the points of each frame are irregularly distributed, the number of points may change from one frame to another, and there is no consistent point-to-point correspondence between successive frames.

The Moving Picture Experts Group (MPEG) introduced two point cloud compression (PCC) standards: Geometry-Based PCC (G-PCC) and Video-Based PCC (V-PCC) for compressing static and dynamic point cloud data, respectively. We refer the readers to [72, 73, 60] for detailed introductions of these methods and [74] for a survey on deep learning-based point cloud compression.

Like [75, 76], the proposed SPCV naturally bridges the gap between dynamic 3D point cloud sequences and mature 2D image/video compression techniques, paving the way for efficient and effective compression pipelines. Specifically, given a 3D point cloud sequence, we represent it as an SPCV, which is further fed into the Versatile Video Coding (VVC) Test Model, the latest H.266/VVC video compression standard [77], to achieve compression. This pipeline is fully compatible with existing infrastructures for 2D image/video compression and transmission.

We compared our SPCV-based action recognition framework with existing state-of-the-art 3D sequence processing methods [45, 42, 12] on the challenging dataset of deformingThings4D [82], which consists of 1,972 animation 3D point cloud sequences across 31 categories of humanoids and animals with large non-rigid deformations. Each frame of the point cloud sequences contains 2,048 points. We segmented the whole sequence into multiple fixed-frame clips, which are fed into all competing methods. To adapt our method, we converted all 3D point cloud sequences to SPCVs with the height and width of each frame equal to 64 and 32, respectively. Following [45], during inference, the prediction result of the whole sequence is determined by averaging the probabilities of all clips.

As compared in Table III, our method outperforms existing state-of-the-art methods with the highest recognition accuracy, which reveals the enhanced feature extraction capability of our SPCV-based learning network. Moreover, our framework turns out to be much more efficient in terms of the GPU memory cost and inference speed, benefiting from the structured nature of SPCVs that can circumvent many expensive operations of indexing, sampling, and grouping.

Following the experimental protocols in [78], we performed temporal interpolation on the DHB dataset and made comparisons with existing state-of-the-art 3D point cloud sequence processing approaches [38, 42, 45, 78, 12], where we adopted CD and EMD as quantitative metrics.

In our implementation, we bench-marked our model against the original learning framework of [78], along with the point-level task models outlined in [42, 45, 12]. Similarly to [78], we utilized EMD as the training loss function for the compared methods. All the competing methods share the same training and testing data as prepared in [78]. As depicted in Fig. 8, we constructed an image-based learning framework to fully leverage the capabilities of our regular grid representation for temporal interpolation. Thanks to the structured nature of our SPCV representation, we can directly utilize the pixel-wise Frobenius norm as the loss function.

As shown in Table IV, our approach demonstrates significant improvements over the existing state-of-the-art methods, as indicated by its lower CD and EMD metrics. Besides, our learning framework shows less GPU memory consumption and forward pass time cost. As visualized in Fig. LABEL:intpl_swing, our method distinctly outperforms the others in preserving geometric fidelity within the interpolated frames. Compared with previous methods, our SPCV representation has explicit temporal consistency, which greatly reduces the difficulty of frame interpolation by the model, thereby achieving a lower geometric error. Thanks to the regular structure of SPCVs, our method can directly use sliding window convolution operations without the need for specially designed grouping and feature aggregation methods, thus improving the processing speed. Moreover, when extracting multi-scale features, due to the regularity of each frame, we do not need to pre-use FPS or other sampling methods to extract key points, which enhances the speed and reduces the GPU storage.

We adopted the popular MPEG [83] and MITAMA [84] datasets and bench-marked our SPCV-based point cloud compression framework for compressing both static and dynamic point clouds. For experiments on static point cloud compression where each point cloud uniformly contains 10K points, we made comparisons with the latest version of G-PCC (TMC13 v23.0-rc2). Besides, we also included Flattening-Net [33] and RegGeoNet [32] for comparison by feeding their generated 2D representations into the H.266/VVC video encoder. For experiments on dynamic point cloud compression where each sequence consists of 5 frames and each frame contains 250K points, we made comparisons with the latest version of V-PCC (TMC2 Release 18.0).

The quantitative results depicted in Fig. 20 indicate that our SPCV-based compression framework outperforms both G-PCC and recent point cloud structurization methods. This is substantiated by achieving lower CD and Point-to-Face (P2F) metrics at the same compression ratio. The enhanced performance can be attributed to the structured nature of the SPCV representation that is conducive to the utilization of the advanced video compression techniques, and the intrinsic spatial smoothness of SPCV that further boosts the compression performance. Visual comparisons in Fig. 21 show that our compression method more accurately preserves the geometric fidelity of the surfaces reconstructed from the decoded 3D point clouds.

For point cloud sequence compression, the results in Fig. 22 demonstrate that our SPCV-based compression framework significantly outperforms V-PCC in compression efficiency, as evidenced by much lower CD and P2F metrics at the same compression ratio (or much smaller compressed file size at the same CD and P2F metrics). The SPCV’s temporal consistency enhances its intra-frame compression capabilities, optimizing the overall performance. Visual comparisons in Fig. LABEL:vpcc_vis demonstrate that our SPCV more faithfully preserves the geometric fidelity in the reconstructed surfaces.

We conducted ablation studies to facilitate analyzing and understanding the proposed learning framework for structuring an arbitrary 3D point cloud sequence into a 2D video.