Search Results (224)

Most current point cloud super-resolution reconstruction requires huge calculations and has low accuracy when facing large outdoor scenes; a Dense Feature Pyramid Network (DenseFPNet) is proposed for the feature-level fusion of images with low-resolution point clouds to generate higher-resolution point clouds, which can be utilized to solve the problem of the super-resolution reconstruction of 3D point clouds by turning it into a 2D depth map complementation problem, which can reduce the time and complexity of obtaining high-resolution point clouds only by LiDAR. The network first utilizes an image-guided feature extraction network based on RGBD-DenseNet as an encoder to extract multi-scale features, followed by an upsampling block as a decoder to gradually recover the size and details of the feature map. Additionally, the network connects the corresponding layers of the encoder and decoder through pyramid connections. Finally, experiments are conducted on the KITTI deep complementation dataset, and the network performs well in various metrics compared to other networks. It improves the RMSE by 17.71%, 16.60%, 7.11%, and 4.68% compared to the CSPD, Spade-RGBsD, Sparse-to-Dense, and GAENET. Full article

LiDAR sensors have been shown to generate data with various common corruptions, which seriously affect their applications in 3D vision tasks, particularly object detection. At the same time, it has been demonstrated that traditional defense strategies, including adversarial training, are prone to suffering from gradient confusion during training. Moreover, they can only improve their robustness against specific types of data corruption. In this work, we propose LiDARPure, which leverages the powerful generation ability of diffusion models to purify corruption in the LiDAR scene data. By dividing the entire scene into voxels to facilitate the processes of diffusion and reverse diffusion, LiDARPure overcomes challenges induced from adversarial training, such as sparse point clouds in large-scale LiDAR data and gradient confusion. In addition, we utilize the latent geometric features of a scene as a condition to assist the generation of diffusion models. Detailed experiments show that LiDARPure can effectively purify 19 common types of LiDAR data corruption. Further evaluation results demonstrate that it can improve the average precision of 3D object detectors to an extent of 20% in the face of data corruption, much higher than existing defence strategies. Full article

In the process of using structured light technology to obtain indoor point clouds, due to the limited field of view of the device, it is necessary to obtain multiple point clouds of different wall surfaces. Therefore, merging the point cloud is necessary to acquire a complete point cloud. However, due to issues such as the sparse geometric features of the wall point clouds and the high similarity of multiple point clouds, the merging effect of point clouds is poor. In this paper, we leverage artificially enhanced features to improve the accuracy of registration in this scenario. Firstly, we design feature markers and present their layout criteria. Then, the feature information of the marker is extracted based on the Color Signature of Histograms of OrienTations (Color-SHOT) descriptor, and coarse registration is realized through the second-order similarity measure matrix. After that, precise registration is achieved using the Iterative Closest Point (ICP) method based on markers and overlapping areas. Finally, the global error of the point cloud registration is optimized by loop error averaging. Our method enables the high-precision reconstruction of integrated home design scenes lacking significant features at a low cost. The accuracy and validity of the method were verified through comparative experiments. Full article

The point clouds obtained directly from three-dimensional scanning devices are often sparse and noisy. Therefore, point cloud upsampling plays an increasingly crucial role in fields such as point cloud reconstruction and rendering. However, point cloud upsampling methods are primarily supervised and fixed-rate, which restricts their applicability in various scenarios. In this paper, we propose a novel point cloud upsampling method, named RE-PU, which is based on the point cloud reconstruction and achieves self-supervised upsampling at arbitrary rates. The proposed method consists of two main stages: the first stage is to train a network to reconstruct the original point cloud from a prior distribution, and the second stage is to upsample the point cloud data by increasing the number of sampled points on the prior distribution with the trained model. The experimental results demonstrate that the proposed method can achieve comparable outcomes to supervised methods in terms of both visual quality and quantitative metrics. Full article

To meet the requirement of autonomous driving development, high-quality point cloud generation of the environment has become the focus of 4D mmWave radar development. On the basis of mass producibility and physical verifiability, a design method for improving the quality and density of point cloud imagery is proposed in this paper, including antenna design, array design, and the dynamic detection method. The utilization of apertures is promoted through antenna design and sparse MIMO array optimization using the genetic algorithm (GA). The hybrid strategy for complex point clouds is adopted using the proposed dynamic CFAR algorithm, which enables dynamic adjustment of the threshold by discriminating and calculating different scanning regions. The effectiveness of the proposed method is verified by simulations and practical experiments. Aiming at system manufacture, analysis methods for the ambiguity function (AF) and shooting and bouncing rays (SBR) tracing are introduced, and an mmWave radar system is realized based on the proposed method, with its performance proven by practical experiments. Full article

In recent years, with the rapid development of unmanned aerial vehicle (UAV) technology, multi-view 3D reconstruction has once again become a hot spot in computer vision. Incremental Structure From Motion (SFM) is currently the most prevalent reconstruction pipeline, but it still faces challenges in reconstruction efficiency, accuracy, and feature matching. In this paper, we use deep learning algorithms for feature matching to obtain more accurate matching point pairs. Moreover, we adopted the improved Gauss–Newton (GN) method, which not only avoids numerical divergence but also accelerates the speed of bundle adjustment (BA). Then, the sparse point cloud reconstructed by SFM and the original image are used as the input of the depth estimation network to predict the depth map of each image. Finally, the depth map is fused to complete the reconstruction of dense point clouds. After experimental verification, the reconstructed dense point clouds have rich details and clear textures, and the integrity, overall accuracy, and reconstruction efficiency of the point clouds have been improved. Full article

LiDAR-based simultaneous localization and mapping (SLAM) offer robustness against illumination changes, but the inherent sparsity of LiDAR point clouds poses challenges for continuous tracking and navigation, especially in feature-deprived scenarios. This paper proposes a novel LiDAR/SINS tightly integrated SLAM algorithm designed to address the localization challenges in urban environments characterized in sparse structural features. Firstly, the method extracts edge points from the LiDAR point cloud using a traditional segmentation method and clusters them to form distinctive edge lines. Then, a rotation-invariant feature—line distance—is calculated based on the edge line properties that were inspired by the traditional tightly integrated navigation system. This line distance is utilized as the observation in a Kalman filter that is integrated into a tightly coupled LiDAR/SINS system. This system tracks the same edge lines across multiple frames for filtering and correction instead of tracking points or LiDAR odometry results. Meanwhile, for loop closure, the method modifies the common SCANCONTEXT algorithm by designating all bins that do not reach the maximum height as special loop keys, which reduce false matches. Finally, the experimental validation conducted in urban environments with sparse structural features demonstrated a 17% improvement in positioning accuracy when compared to the conventional point-based methods. Full article

Neural Radiance Fields (NeRFs) are an emerging approach to 3D reconstruction that use neural networks to reconstruct scenes. However, its applications for multi-view satellite photogrammetry, which aim to reconstruct the Earth’s surface, struggle to acquire accurate digital surface models (DSMs). To address this issue, a novel framework, Geometric Constrained Neural Radiance Field (GC-NeRF) tailored for multi-view satellite photogrammetry, is proposed. GC-NeRF achieves higher DSM accuracy from multi-view satellite images. The key point of this approach is a geometric loss term, which constrains the scene geometry by making the scene surface thinner. The geometric loss term alongside z-axis scene stretching and multi-view DSM fusion strategies greatly improve the accuracy of generated DSMs. During training, bundle-adjustment-refined satellite camera models are used to cast rays through the scene. To avoid the additional input of altitude bounds described in previous works, the sparse point cloud resulting from the bundle adjustment is converted to an occupancy grid to guide the ray sampling. Experiments on WorldView-3 images indicate GC-NeRF’s superiority in accurate DSM generation from multi-view satellite images. Full article

Point cloud registration is a critical problem because it is the basis of many 3D vision tasks. With the popularity of deep learning, many scholars have focused on leveraging deep neural networks to address the point cloud registration problem. However, many of these methods are still sensitive to partial overlap and differences in density distribution. For this reason, herein, we propose a method based on rotation-invariant features and using a sparse-to-dense matching strategy for robust point cloud registration. Firstly, we encode raw points as superpoints with a network combining KPConv and FPN, and their associated features are extracted. Then point pair features of these superpoints are computed and embedded into the transformer to learn the hybrid features, which makes the approach invariant to rigid transformation. Subsequently, a sparse-to-dense matching strategy is designed to address the registration problem. The correspondences of superpoints are obtained via sparse matching and then propagated to local dense points and, further, to global dense points, the byproduct of which is a series of transformation parameters. Finally, the enhanced features based on spatial consistency are repeatedly fed into the sparse-to-dense matching module to rebuild reliable correspondence, and the optimal transformation parameter is re-estimated for final alignment. Our experiments show that, with the proposed method, the inlier ratio and registration recall are effectively improved, and the performance is better than that of other point cloud registration methods on 3DMatch and ModelNet40. Full article

Although existing 3D object-detection methods have achieved promising results on conventional datasets, it is still challenging to detect objects in data collected under adverse weather conditions. Data distortion from LiDAR and cameras in such conditions leads to poor performance of traditional single-sensor detection methods. Multi-modal data-fusion methods struggle with data distortion and low alignment accuracy, making accurate target detection difficult. To address this, we propose a multi-modal object-detection algorithm, Snow-CLOCs, specifically for snowy conditions. In image detection, we improved the YOLOv5 algorithm by integrating the InceptionNeXt network to enhance feature extraction and using the Wise-IoU algorithm to reduce dependency on high-quality data. For LiDAR point-cloud detection, we built upon the SECOND algorithm and employed the DROR filter to remove noise, enhancing detection accuracy. We combined the detection results from the camera and LiDAR into a unified detection set, represented using a sparse tensor, and extracted features through a 2D convolutional neural network to achieve object detection and localization. Snow-CLOCs achieved a detection accuracy of 86.61% for vehicle detection in snowy conditions. Full article

Three-dimensional (3D) single-object tracking (3D SOT) is a fundamental yet not well-solved problem in 3D vision, where the complexity of feature matching and the sparsity of point clouds pose significant challenges. To handle abrupt changes in appearance features and sparse point clouds, we propose a novel 3D SOT network, dubbed CDTracker. It leverages both cosine similarity and an attention mechanism to enhance the robustness of feature matching. By combining similarity embedding and attention assignment, CDTracker performs template and search area feature matching in a coarse-to-fine manner. Additionally, CDTracker addresses the problem of sparse point clouds, which commonly leads to inaccurate tracking. It incorporates relatively dense sampling based on the concept of point cloud segmentation to retain more target points, leading to improved localization accuracy. Extensive experiments on both the KITTI and Waymo datasets demonstrate clear improvements in CDTracker over its competitors. Full article

In this paper, we propose a novel, vision-transformer-based end-to-end pose estimation method, LidPose, for real-time human skeleton estimation in non-repetitive circular scanning (NRCS) lidar point clouds. Building on the ViTPose architecture, we introduce novel adaptations to address the unique properties of NRCS lidars, namely, the sparsity and unusual rosetta-like scanning pattern. The proposed method addresses a common issue of NRCS lidar-based perception, namely, the sparsity of the measurement, which needs balancing between the spatial and temporal resolution of the recorded data for efficient analysis of various phenomena. LidPose utilizes foreground and background segmentation techniques for the NRCS lidar sensor to select a region of interest (RoI), making LidPose a complete end-to-end approach to moving pedestrian detection and skeleton fitting from raw NRCS lidar measurement sequences captured by a static sensor for surveillance scenarios. To evaluate the method, we have created a novel, real-world, multi-modal dataset, containing camera images and lidar point clouds from a Livox Avia sensor, with annotated 2D and 3D human skeleton ground truth. Full article

High-precision positioning and multi-target detection have been proposed as key technologies for robotic path planning and obstacle avoidance. First, the Cartographer algorithm was used to generate high-quality maps. Then, the iterative nearest point (ICP) and the occupation probability algorithms were combined to scan and match the local point cloud, and the positions and attitudes of the robot were obtained. Furthermore, Sparse Matrix Pose Optimization was carried out to improve the positioning accuracy. The positioning accuracy of the robot in x and y directions was kept within 5 cm, the angle error was controlled within 2°, and the positioning time was reduced by 40%. An improved timing elastic band (TEB) algorithm was proposed to guide the robot to move safely and smoothly. A critical factor was introduced to adjust the distance between the waypoints and the obstacle, generating a safer trajectory, and increasing the constraint of acceleration and end speed; thus, smooth navigation of the robot to the target point was achieved. The experimental results showed that, in the case of multiple obstacles being present, the robot could choose the path with fewer obstacles, and the robot moved smoothly when facing turns and approaching the target point by reducing its overshoot. The proposed mapping, positioning, and improved TEB algorithms were effective for high-precision positioning and efficient multi-target detection. Full article

This paper presents a methodology for generating virtual ground control points (VGCPs) using a binocular camera mounted on a drone. We compare the measurements of the binocular and monocular cameras between the classical method and the proposed one. This work aims to decrease human processing times while maintaining a reduced root mean square error (RMSE) for 3D reconstruction. Additionally, we propose utilizing COLMAP to enhance reconstruction accuracy by solely utilizing a sparse point cloud. The results demonstrate that implementing COLMAP for pre-processing reduces the RMSE by up to 16.9% in most cases. We prove that VGCPs further reduce the RMSE by up to 61.08%. Full article

In recent years, research on the preservation of historical architecture has gained significant attention, where the effectiveness of semantic segmentation is particularly crucial for subsequent repair, protection, and 3D reconstruction. Given the sparse and uneven nature of large-scale historical building point cloud scenes, most semantic segmentation methods opt to sample representative subsets of points, often leading to the loss of key features and insufficient segmentation accuracy of architectural components. Moreover, the geometric feature information at the junctions of components is cluttered and dense, resulting in poor edge segmentation. Based on this, this paper proposes a unique semantic segmentation network design called MSFA-Net. To obtain multiscale features and suppress irrelevant information, a double attention aggregation module is first introduced. Then, to enhance the model’s robustness and generalization capabilities, a contextual feature enhancement and edge interactive classifier module are proposed to train edge features and fuse the context data. Finally, to evaluate the performance of the proposed model, experiments were conducted on a self-curated ancient building dataset and the S3DIS dataset, achieving OA values of 95.2% and 88.7%, as well as mIoU values of 86.2% and 71.6%, respectively, further confirming the effectiveness and superiority of the proposed method. Full article