Search Results (52)

Camera arrays can enhance the signal-to-noise ratio (SNR) between dim targets and backgrounds through multi-view synthesis. This is crucial for the detection of dim targets. To this end, we design and develop an infrared camera array system with a large baseline. The multi-view synthesis of camera arrays relies heavily on the calibration accuracy of relative poses in the sub-cameras. However, the sub-cameras within a camera array lack strict geometric constraints. Therefore, most current calibration methods still consider the camera array as multiple pinhole cameras for calibration. Moreover, when detecting distant targets, the camera array usually needs to adjust the focal length to maintain a larger depth of field (DoF), so that the distant targets are located on the camera’s focal plane. This means that the calibration scene should be selected within this DoF range to obtain clear images. Nevertheless, the small parallax between the distant sub-aperture views limits the calibration. To address these issues, we propose a calibration model for camera arrays in distant scenes. In this model, we first extend the parallax by employing dual-array frames (i.e., recording a scene at two spatial locations). Secondly, we investigate the linear constraints between the dual-array frames, to maintain the minimum degrees of freedom of the model. We develop a real-world light field dataset called NUDT-Dual-Array using an infrared camera array to evaluate our method. Experimental results on our self-developed datasets demonstrate the effectiveness of our method. Using the calibrated model, we improve the SNR of distant dim targets, which ultimately enhances the detection and perception of dim targets. Full article

We study the multiviews of algebraic space curves X from n pin-hole cameras of a real or complex projective space. We assume the pin-hole centers to be known, i.e., we do not reconstruct them. Our tools are algebro-geometric. We give some general theorems, e.g., we prove that a projective curve (over complex or real numbers) may be reconstructed using four general cameras. Several examples show that no number of badly placed cameras can make a reconstruction possible. The tools are powerful, but we warn the reader (with examples) that over real numbers, just using them correctly, but in a bad way, may give ghosts: real curves which are images of the emptyset. We prove that ghosts do not occur if the cameras are general. Most of this paper is devoted to three important cases of space curves: unions of a prescribed number of lines (using the Grassmannian of all lines in a 3-dimensional projective space), plane curves, and curves of low degree. In these cases, we also see when two cameras may reconstruct the curve, but different curves need different pairs of cameras. Full article

PANDORA (Plasmas for Astrophysics Nuclear Decays Observation and Radiation for Archaeometry) is an INFN project aiming at measuring, for the first time, possible variations in in-plasma $\beta$-decay lifetimes in isotopes of astrophysical interest as a function of thermodynamical conditions of the in-laboratory controlled plasma environment. Theoretical predictions indicate that the ionization state can dramatically modify the $\beta$-decay lifetime (even of several orders of magnitude). The PANDORA experimental approach consists of confining a plasma able to mimic specific stellar-like conditions and measuring the nuclear decay lifetime as a function of plasma parameters. The $\beta$-decay events will be measured by detecting the $\gamma$-ray emitted by the daughter nuclei, using an array of 12 HPGe detectors placed around the magnetic trap. In this frame, plasma parameters have to be continuously monitored online. For this purpose, an innovative, non-invasive multi-diagnostic system, including high-resolution time- and space-resolved X-ray analysis, was developed, which will work synergically with the $\gamma$-rays detection system. In this contribution, we will describe this multi-diagnostics system with a focus on spatially resolved high-resolution X-ray spectroscopy. The latter is performed by a pin-hole X-ray camera setup operating in the 0.5–20 keV energy domain. The achieved spatial and energy resolutions are 450 µm and 230 eV at 8.1 keV, respectively. An analysis algorithm was specifically developed to obtain SPhC (Single Photon-Counted) images and local plasma emission spectrum in High-Dynamic-Range (HDR) mode. Thus, investigations of image regions where the emissivity can change by even orders of magnitude are now possible. Post-processing analysis is also able to remove readout noise, which is often observable and dominant at very low exposure times (ms). Several measurements have already been used in compact magnetic plasma traps, e.g., the ATOMKI ECRIS in Debrecen and the Flexible Plasma Trap at LNS. The main outcomes will be shortly presented. The collected data allowed for a quantitative and absolute evaluation of local emissivity, the elemental analysis, and the local evaluation of plasma density and temperature. This paper also discusses the new plasma emission models, implemented on PIC-ParticleInCell codes, which were developed to obtain powerful 3D maps of the X-rays emitted by the magnetically confined plasma. These data also support the evaluation procedure of spatially resolved plasma parameters from the experimental spectra as well as, in the near future, the development of appropriate algorithms for the tomographic reconstruction of plasma parameters in the X-ray domain. The described setups also include the most recent upgrade, consisting of the use of fast X-ray shutters with special triggering systems that will be routinely implemented to perform both space- and time-resolved spectroscopy during transient, stable, and turbulent plasma regimes (in the ms timescale). Full article

Curb detection tasks play a crucial role in the perception of the autonomous driving environment for logistics vehicles. With the popularity of multi-modal sensors under the BEV (Bird’s Eye View) paradigm, curb detection tasks are increasingly being integrated into multi-task perception networks, achieving robust detection results. This paper modifies and integrates the tri-plane spatial feature representation method of the EG3D network from the field of 3D reconstruction into a BEV-based multi-modal sensor detection network, including LiDAR, pinhole cameras, and fisheye cameras. The system collects a total of 24,350 frames of data under real road conditions for experimentation, proving the effectiveness of the proposed method. Full article

Maritime traffic monitoring systems are particularly important in Mediterranean ports, as they provide more comprehensive data collection compared to traditional systems such as the Automatic Identification System (AIS), which is not mandatory for all vessels. This paper improves the existing real-time maritime traffic monitoring systems by introducing a distance estimation algorithm for monocular cameras, which aims to provide high quality maritime traffic metadata collection for traffic density analysis. Two distance estimation methods based on a pinhole camera model are presented: the Vessel-Focused Distance Estimation (VFDE) and the novel Vessel Object-Focused Distance Estimation (VOFDE). While VFDE uses the predefined height of a vessel for distance estimation, VOFDE uses standardized dimensions of objects on the vessel, detected with a Convolutional Neural Network (CNN) for instance segmentation to enhance estimation accuracy. Our evaluation covers distances up to 414 m, which is significantly beyond the scope of previous studies. When compared to the distances measured with a precise instrument, VOFDE achieves a Percentage Deviation Index (PDI) of 1.34% to 9.45%. This advance holds significant potential for improving maritime surveillance with monocular cameras and is also applicable in other areas, such as low-cost maritime vehicles equipped with single cameras. Full article

This study proposes a Nipkow-based pinhole disk laser scanning confocal microscopic imaging system for ordinary optical microscopy, fluorescence microscopy, and confocal microscopy imaging of biological samples in order to realize the dynamic experimental monitoring of space-based life science experiments and the fine observation of biological samples. Confocal microscopic imaging is mainly completed by a scanning module that is composed of a spinning disk and other components. The parameters of the spinning disk directly determine the quality of the image. During the design process, the resolution and signal-to-noise ratios caused by different pinhole diameters in the spinning disk are the main considerations. Changes and image blurring caused by crosstalk due to the pinhole arrangement and different pinhole spacings are addressed. The high photon efficiency of the new EMCCD (electron-multiplying charge-coupled device) and CMOS (complementary metal-oxide-semiconductor) camera reduces the exposure time as much as possible, reduces damage to living cells, and achieves high-speed confocal imaging. It is shown in a confocal imaging experiment with a variable magnification of 1–40× that the imaging resolution of the system can reach a maximum of 2592 × 1944, the spatial resolution can reach 1 μm, and the highest sampling frequency is 10 fps, thus meeting the design requirements for high-speed live-cell imaging. Full article

Monocular depth estimation is a task aimed at predicting pixel-level distances from a single RGB image. This task holds significance in various applications including autonomous driving and robotics. In particular, the recognition of surrounding environments is important to avoid collisions during autonomous parking. Fisheye cameras are adequate to acquire visual information from a wide field of view, reducing blind spots and preventing potential collisions. While there have been increasing demands for fisheye cameras in visual-recognition systems, existing research on depth estimation has primarily focused on pinhole camera images. Moreover, depth estimation from fisheye images poses additional challenges due to strong distortion and the lack of public datasets. In this work, we propose a novel underground parking lot dataset called JBNU-Depth360, which consists of fisheye camera images and their corresponding LiDAR projections. Our proposed dataset was composed of 4221 pairs of fisheye images and their corresponding LiDAR point clouds, which were obtained from six driving sequences. Furthermore, we employed a knowledge-distillation technique to improve the performance of the state-of-the-art depth-estimation models. The teacher–student learning framework allows the neural network to leverage the information in dense depth predictions and sparse LiDAR projections. Experiments were conducted on the KITTI-360 and JBNU-Depth360 datasets for analyzing the performance of existing depth-estimation models on fisheye camera images. By utilizing the self-distillation technique, the AbsRel and SILog error metrics were reduced by 1.81% and 1.55% on the JBNU-Depth360 dataset. The experimental results demonstrated that the self-distillation technique is beneficial to improve the performance of depth-estimation models. Full article

This study presents a methodology for the coarse alignment of light detection and ranging (LiDAR) point clouds, which involves estimating the position and orientation of each station using the pinhole camera model and a position/orientation estimation algorithm. Ground control points are obtained using LiDAR camera images and the point clouds are obtained from the reference station. The estimated position and orientation vectors are used for point cloud registration. To evaluate the accuracy of the results, the positions of the LiDAR and the target were measured using a total station, and a comparison was carried out with the results of semi-automatic registration. The proposed methodology yielded an estimated mean LiDAR position error of 0.072 m, which was similar to the semi-automatic registration value of 0.070 m. When the point clouds of each station were registered using the estimated values, the mean registration accuracy was 0.124 m, while the semi-automatic registration accuracy was 0.072 m. The high accuracy of semi-automatic registration is due to its capability for performing both coarse alignment and refined registration. The comparison between the point cloud with refined alignment using the proposed methodology and the point-to-point distance analysis revealed that the average distance was measured at 0.0117 m. Moreover, 99% of the points exhibited distances within the range of 0.0696 m. Full article

In this work, we demonstrated a new type of biomimetic multispectral curved compound eye camera (BM3C) inspired by insect compound eyes for aerial multispectral imaging in a large field of view. The proposed system exhibits a maximum field of view (FOV) of 120 degrees and seven-waveband multispectral images ranging from visible to near-infrared wavelengths. Pinhole imaging theory and the image registration method from feature detection are used to reconstruct the multispectral 3D data cube. An airborne imaging experiment is performed by assembling the BM3C on an unmanned aerial vehicle (UAV). As a result, radiation intensity curves of several objects are successfully obtained, and a land type classification is performed using the K-means method based on the aerial image as well. The developed BM3C is proven to have the capability for large FOV aerial multispectral imaging and shows great potential applications for distant detecting based on aerial imaging. Full article

Train distance estimation in a turnout area is an important task for the autonomous driving of urban railway transit, since this function can assist trains in sensing the positions of other trains within the turnout area and prevent potential collision accidents. However, because of large incident angles on object surfaces and far distances, Lidar or stereo vision cannot provide satisfactory precision for such scenarios. In this paper, we propose a method for train distance estimation in a turnout area based on monocular vision: firstly, the side windows of trains in turnout areas are detected by instance segmentation based on YOLOv8; secondly, the vertical directions, the upper edges and lower edges of side windows of the train are extracted by feature extraction; finally, the distance to the target train is calculated with an appropriated pinhole camera model. The proposed method is validated by practical data captured from Hong Kong Metro Tsuen Wan Line. A dataset of 2477 images is built to train the instance segmentation neural network, and the network is able to attain an MIoU of 92.43% and a MPA of 97.47% for segmentation. The accuracy of train distance estimation is then evaluated in four typical turnout area scenarios with ground truth data from on-board Lidar. The experiment results indicate that the proposed method achieves a mean RMSE of 0.9523 m for train distance estimation in four typical turnout area scenarios, which is sufficient for determining the occupancy of crossover in turnout areas. Full article

A camera equipped with a transparent shield can be modeled using the pinhole camera model and residual error vectors defined by the difference between the estimated ray from the pinhole camera model and the actual three-dimensional (3D) point. To calculate the residual error vectors, we employ sparse calibration data consisting of 3D points and their corresponding 2D points on the image. However, the observation noise and sparsity of the 3D calibration points pose challenges in determining the residual error vectors. To address this, we first fit Gaussian Process Regression (GPR) operating robustly against data noise to the observed residual error vectors from the sparse calibration data to obtain dense residual error vectors. Subsequently, to improve performance in unobserved areas due to data sparsity, we use an additional constraint; the 3D points on the estimated ray should be projected to one 2D image point, called the ray constraint. Finally, we optimize the radial basis function (RBF)-based regression model to reduce the residual error vector differences with GPR at the predetermined dense set of 3D points while reflecting the ray constraint. The proposed RBF-based camera model reduces the error of the estimated rays by 6% on average and the reprojection error by 26% on average. Full article

A multichannel soft X-ray (SXR) array has been developed to measure the electron temperature in the Versatile Experiment Spherical Torus (VEST). To estimate electron temperature using the two-filter method applied to SXR intensity, we designed a pinhole camera that has two photodiode arrays with different metallic filters. We also adopted a filter wheel and tested various filter parameters to find the optimal filter set. Through tests, the combination of aluminum and beryllium was found to be the most suitable for the current experimental conditions in VEST. The filtered SXR signals were acquired with a low-noise preamplifier, exhibiting sufficient signal-to-noise ratios for electron temperature estimation based on the intensity ratio of two signals obtained with different filters. The estimated electron temperature from the developed two-filter SXR array showed reasonably matched levels and consistent trends with Thomson scattering measurements. Error contribution from impurity line emission is also discussed. Full article

Spatial frequency domain imaging (SFDI) is well established in biology and medicine for non-contact, wide-field imaging of optical properties and 3D topography. Especially for turbid media with displaced, tilted or irregularly shaped surfaces, the reliable quantitative measurement of diffuse reflectance requires efficient calibration and correction methods. In this work, we present the implementation of a generic and hardware independent calibration routine for SFDI setups based on the so-called pinhole camera model for both projection and detection. Using a two-step geometric and intensity calibration, we obtain an imaging model that efficiently and accurately determines 3D topography and diffuse reflectance for subsequently measured samples, taking into account their relative distance and orientation to the camera and projector, as well as the distortions of the optical system. Derived correction procedures for position- and orientation-dependent changes in spatial frequency and intensity allow the determination of the effective scattering coefficient ${\mathsf{\mu }}_s^{\prime }$ and the absorption coefficient ${\mathsf{\mu }}_a$ when measuring a spherical optical phantom at three different measurement positions and at nine wavelengths with an average error of 5% and 12%, respectively. Model-based calibration allows the characterization of the imaging properties of the entire SFDI system without prior knowledge, enabling the future development of a digital twin for synthetic data generation or more robust evaluation methods. Full article

Continuous, robust, and precise localization is pivotal in enabling the autonomous operation of robots and aircraft in intricate environments, particularly in the absence of GNSS (global navigation satellite system) signals. However, commonly employed approaches, such as visual odometry and inertial navigation systems, encounter hindrances in achieving effective navigation and positioning due to issues of error accumulation. Additionally, the challenge of managing extensive map creation and exploration arises when deploying these systems on unmanned aerial vehicle terminals. This study introduces an innovative system capable of conducting long-range and multi-map visual SLAM (simultaneous localization and mapping) using monocular cameras equipped with pinhole and fisheye lens models. We formulate a graph optimization model integrating GNSS data and graphical information through multi-sensor fusion navigation and positioning technology. We propose partitioning SLAM maps based on map health status to augment accuracy and resilience in large-scale map generation. We introduce a multi-map matching and fusion algorithm leveraging geographical positioning and visual data to address excessive discrete mapping, leading to resource wastage and reduced map-switching efficiency. Furthermore, a multi-map-based visual SLAM online localization algorithm is presented, adeptly managing and coordinating distinct geographical maps in different temporal and spatial domains. We employ a quadcopter to establish a testing system and generate an aerial image dataset spanning several kilometers. Our experiments exhibit the framework’s noteworthy robustness and accuracy in long-distance navigation. For instance, our GNSS-assisted multi-map SLAM achieves an average accuracy of 1.5 m within a 20 km range during unmanned aerial vehicle flights. Full article

This study presents a visual-only odometry technique of a low-profile pallet robot using image feature tracking in ground plane images generated from a fisheye camera. The fisheye camera is commonly used in many robot vision applications because it provides a larger field of view (FoV) around a robot. However, because of the large radial distortion, the fisheye image is generally converted to a pinhole image for visual feature tracking or matching. Although the radial distortion can be eliminated via image undistortion with the lens calibration parameters, it causes several side effects, such as degraded image resolution and a significant reduction in the FoV. In this paper, instead of using the pinhole model, we propose to generate a ground plane image (GPI) from the fisheye image. GPI is a virtual top-view image that only contains the ground plane at the front of the robot. First, the original fisheye image is projected to several virtual pinhole images to generate a cubemap. Second, the front and bottom faces of the cubemap are projected to a GPI. Third, the GPI is homographically transformed again to further reduce image distortion. As a result, an accurate ortho-rectified ground plane image is obtained from the virtual top-view camera. For visual odometry using the ortho-rectified GPI, a number of 2D motion vectors are obtained using feature extraction and tracking between the previous and current frames in the GPI. By calculating a scaled motion vector, which is the measurement of the virtual wheel encoder of the mobile robot, we estimate the velocity and steering angle of the virtual wheel using the motion vector. Finally, we estimate the pose of the mobile robot by applying a kinematic model to the mobile robot. Full article