Accurate and robust deep learning models for facial expression recognition are challenging to achieve, given the diversity of human faces and variations in images, including different facial poses and lighting conditions. In this work, we proposed a clock-driven convolutional Spiking Neural Network (SNN) and STDP fine-tune architecture, meticulously calibrated its hyperparameters, and experimented with various optimization methods. The best model resulted was trained and evaluated on the Fer2013 and FER+ database, obtaining an accuracy of 61.87% and 79.97% without requiring auxiliary training data or face registration. To our best knowledge, the proposed SNN achieved comparable accuracy to CNNs of similar depth and possessed the advantages of low energy consumption and high computational efficiency. The computational efficiency of the proposed SNN is approximately three times that of CNNs. Along with this, we introduced the very recent cumulative spike guided encoder visualization technique and revealed the strong encoding capability of the proposed SNN.

With the development of educational technology, machine learning and deep learning provide technical support for traditional classroom observation assessment. However, in real classroom scenarios, the technique faces challenges such as lack of clarity of raw images, complexity of datasets, multi-target detection errors and complexity of character interactions.Based on the above problems, a student classroom behavior recognition network incorporating super-resolution and target detection is proposed. To cope with the problem of unclear original images in the classroom scenario, SRGAN (Super Resolution Generative Adversarial Network for Images) is used to improve the image resolution and thus the recognition accuracy.To address the dataset complexity and multi-targeting problems, feature extraction is optimized and multi-scale feature recognition is enhanced by introducing AKConv and LASK attention mechanisms into the Backbone module of YOLOv8s algorithm. To improve the character interaction complexity problem, the CBAM attention mechanism is integrated to enhance the recognition of important feature channels and spatial regions. Experiments show that it can detect six behaviors of students raising their hands, reading, writing, playing cell phones, looking down, and lying on the table in high-definition images. And the accuracy and robustness of this network are verified.Compared with small target detection algorithms such as Faster R-CNN, YOLOv5, and YOLOv8s, this network has better detection performance and can efficiently deal with the behavior recognition of multiple students.

The task of human action recognition (HAR) based on skeleton data is a challenging yet crucial technique owing to its wide-ranging applications in numerous domains, including patient monitoring, security surveillance, and observation of human-machine interactions. While numerous algorithms have been proposed in an attempt to distinguish between a myriad of activities, most practical applications necessitate highly accurate detection of specific activity types. This study proposes a novel and highly accurate spatiotemporal graph autoencoder network for HAR based on skeleton data. Furthermore, an extensive investigation was conducted employing diverse modalities. To this end, a spatiotemporal graph autoencoder was constructed to automatically learn both spatial and temporal patterns from human skeleton datasets. The powerful graph convolutional network, designated as GA-GCN, developed in this study, notably outperforms the majority of existing state-of-the-art methods when evaluated on two common datasets, namely NTU RGB+D and NTU RGB+D 120. On the first dataset, the proposed approach achieved accuracies of 92.3\% and 96.8\% for the cross-subject and cross-view evaluations, respectively. On the more challenging NTU RGB+D 120 dataset, GA-GCN attained accuracies of 88.8\% and 90.4\% for the cross-subject and cross-set evaluations, respectively.

The symptoms of multiple sclerosis (MS) are determined by the location of demyelinating lesions in the white matter of the brain and spinal cord. Currently, magnetic resonance imaging (MRI) is the most common tool for diagnosing MS, understanding the course of the disease, and analyzing the effects of treatments. However, undesirable components may appear during the generation of MRI, such as noise or intensity variations. Mathematical morphology (MM) is a powerful image analysis technique that helps to filter the image and extract relevant structures. Granulometry is an image measurement tool of MM that determines the size distribution of objects in an image without explicitly segmenting each object. While, several methods have been proposed for the automatic segmentation of MS lesions in MRI, in some cases only simple data preprocessing, such as image resizing to standardize the input dimensions, has been performed before the algorithm training. Therefore, this paper proposes an MRI preprocessing algorithm performing elementary morphological transformations in brain images of MS patients and healthy individuals to delete undesirable components and extract the relevant structures such as MS lesions. Also, the algorithm computes the granulometry in MRI to describe the size qualities of lesions and trains two artificial neural networks (ANN) to predict MS diagnosis. The computing of differences in granulometry measurements of an image with MS lesions and a reference image (without lesions) can determine the size characterization of the lesions. Then, the ANNs were evaluated with the validation set and the performance results (test accuracy = 0.9753, and cross-entropy loss = 0.0247) show the proposed algorithm can support the decision of specialists for diagnosing MS and estimating the disease progress, based on granulometry values.

The present work titled QRGB consists of the development of a Python application for generating QR codes using the additive color generation method (RGB). This innovative method increases the information density stored in QR codes by using three color layers (red, green, and blue), each representing a different data set. QRGB offers an efficient and secure solution for storing and transmitting large amounts of information in limited spaces, significantly improving the capabilities of traditional black-and-white QR codes.By using three color layers (red, green, and blue), QRGB codes can store up to three times more information in the same space. This technique not only increases storage capacity but also enhances information security, making it more difficult to counterfeit or manipulate the code. The superposition of multiple data layers allows for redundancy implementation, increasing the code's robustness against damage or reading errors.QRGB codes are especially useful in applications that require the transmission of large amounts of data in limited spaces, such as in the packaging industry, digital business cards, and interactive advertising. Additionally, they have great potential in areas such as document and ticket security, where the authenticity and integrity of information are crucial.These points provide a solid foundation for understanding the innovation and advantages of colored QR codes (QRGB) compared to traditional QR codes, highlighting their applicability and potential in various sectors.The Python application developed for QRGB utilizes advanced color image processing techniques, applying models and methods to generate and decode QR codes using the RGB color system, thus enhancing information density and security through the innovative use of additive color layers.

Color distortion in an image poses a challenge for classification and regression when the input data consists of pictures. Because of that, a new algorithm for color standardization of photos is proposed, which serves as a base for a deep neural network regression model. This approach utilizes a custom color template, which was developed based on an initial series of studies and digital imaging. Using the equalized histogram of R, G, B channels from the digital template and its photo, a color mapping strategy is computed. By applying this approach, the histograms are adjusted, and the colors of photos taken with a smartphone are standardized. The proposed algorithm has been developed for a series of photos where the entire surface roughly maintains a uniform color, and the differences in color between the photographs of individual objects are minor. The optimized approach was validated in the colorimetric determination procedure of vitamin C. The dataset for deep neural network in the regression variant was formed from photos of samples under two separate lighting conditions. For the concentration range of vitamin C from 0 to 87.72 µg·mL-1, the RMSE for the test set ranged between 0.75 and 1.95 µg·mL-1, in comparison to the non-standardized variant, where this indicator was at the level of 1.48-2.29 µg·mL-1. The consistency of the predicted concentration results with actual data expressed using R2 was between 0.9956 and 0.9999, for each of the standardized variants. This approach allows for the removal of light reflections on the shiny surface of solutions, which is a common problem in liquid samples. The color matching algorithm has a universal character, and its application scope is not limited.

Object detection algorithms for open water aerial images present challenges such as small object size, unsatisfactory detection accuracy, numerous network parameters, and enormous computational demands. Current detection algorithms struggle to meet the accuracy and speed requirements while being deployable on small mobile devices. This paper proposes DFLM-YOLO, a lightweight small-object detection network based on the YOLOv8 algorithm with multiscale feature fusion. Firstly, to solve the class imbalance problem of the SeaDroneSee dataset, we propose a data augmentation algorithm called Small Object Multiplication (SOM). SOM enhance dataset balance by increasing the number of objects in specific categories, thereby improving model accuracy and generalization capabilities. Secondly, we optimize the backbone network structure by implementing Depthwise Separable Convolution (DSConv) and the newly designed FC-C2f, which reduces the model's parameters and inference time. Finally, we design the Lightweight Multiscale Feature Fusion Network (LMFN) to address the challenges of multiscale variations by gradually fusing the four feature layers extracted from the backbone network in three stages. In addition, LMFN incorporates the Dilated Re-param Block structure to increase the effective receptive field and improve the model's classification ability and detection accuracy. Experimental results on the SeaDroneSea dataset indicate that DFLM-YOLO improves mAP by 12.4% compared to the original YOLOv8s, while reducing parameters by 67.2%. This achievement provides a new solution for UAVs to conduct object detection missions in open water efficiently.

This article explores the efficiency of various clustering methods for image segmentation under different luminosity conditions. Image segmentation plays a crucial role in computer vision appli-cations, and clustering algorithms are commonly used for this purpose. The search for an adaptive clustering mechanism aims to ensure maximum symmetry of real objects with objects/segments in their digital representations. However, clustering methods performance can fluctuate with varying lighting conditions during image capture. Therefore, we assess the performance of several clustering algorithms — including k-means, K-Medoids, fuzzy c-means, Possibilistic C-Means, Gus-tafson-Kessel, Entropy-based Fuzzy, Riddler-Calvard, Kohonen Self-Organizing Maps, and Meanshift — across images captured under different illumination conditions. Additionally, we develop an adaptive image segmentation system utilizing empirical data. Conducted experiments highlight varied performance among clustering methods under different luminosity conditions. This research enhance better understanding of luminosity's impact on image segmentation and aiding in method selection for diverse lighting scenarios.

In recent years, research on adversarial attack techniques for remote sensing object detection (RSOD) has made great progress. Still, most of the research nowadays is on end-to-end attacks, which mainly design adversarial perturbations based on the prediction information of the object detectors (ODs) to achieve the attack. These methods do not discover the common vulnerabilities of the ODs and thus the transferability is weak. Based on this, this paper proposes a foreground feature approximation (FFA) method to generate adversarial examples (AEs) that discover the common vulnerabilities of the ODs by changing the feature information carried by the image itself to implement the attack. Specifically, firstly, the high-quality predictions are filtered as attacked objects using the detector, after which a hybrid image without any target is made, and the hybrid foreground is created based on the attacked targets. The images' shallow features are extracted using the backbone network, and the features of the input foreground are approximated towards the hybrid foreground to implement the attack. In contrast, the model predictions are used to assist in realizing the attack. In addition, we have found the effectiveness of FFA for targeted attacks, and replacing the hybrid foreground with the targeted foreground can realize targeted attacks. Extensive experiments with seven rotating ODs on the RSOD datasets DOTA and UCAS-AOD show that FFA achieves both targetless and targeted attacks with a high success rate and transferability.

Traffic sign detection plays a pivotal role in autonomous driving systems. The intricacy of the detection model necessitates high-performance hardware. Real-world traffic environments exhibit considerable variability and diversity, posing challenges for effective feature extraction by the model. Therefore, it is imperative to develop a detection model that is not only highly accurate but also lightweight. In this paper, we proposed YOLO-ADual, a novel lightweight model. Our method leverages the C3Dual and Adown lightweight modules as replacements for CPS and CBL modules in YOLOv5. The Adown module effectively mitigates feature loss during downsampling while reducing computational costs. Meanwhile, C3Dual optimizes the processing power for kernel feature extraction, enhancing computation efficiency while preserving network depth and feature extraction capability. Furthermore, the inclusion of the CBAM module enables the network to focus on salient information within the image, thus augmenting its feature representation capability. Our proposed algorithm achieves a [email protected] of 70.1% while significantly reducing the number of parameters and computational requirements to 51.83% and 64.73% of the original model, respectively. Compared to various lightweight models, our approach demonstrates competitive performance in terms of both computational efficiency and accuracy.

This paper highlights the importance of one-shot learning from prototype SKU images for efficient product recognition in retail and inventory management. Traditional methods require large supervised datasets to train deep neural networks, which can be costly and impractical. One-shot learning techniques mitigate this issue by enabling classification from a single prototype image per product class, thus reducing data annotation efforts. We introduce the variational prototyping-encoder (VPE), a novel deep neural network for one-shot classification. Utilizing a support set of prototype SKU images, VPE learns to classify query images by capturing image similarity and prototypical concepts. Unlike metric learning-based approaches, VPE pre-learns image translation from real-world object images to prototype images as a meta-task, facilitating efficient one-shot classification with minimal supervision. Our research demonstrates that VPE can significantly reduce the need for large datasets while accurately classifying query images into their respective categories, providing a practical solution for product classification tasks.

Traditional 3D reconstruction models have consistently encountered a challenge: attaining high recall of object edges while preserving precision. In this paper, we introduce a post-processing method Multi-Head Attention Refiner (MA-R) aimed at tackling this challenge by integrating a multi-head attention mechanism within the U-Net style refiner module. The 3D reconstruction model applying this method demonstrates excellence in parsing intricate image details, resulting in significant enhancement of boundary predictions and increased recall rates. In our experiments, our method significantly enhances the reconstruction performance of Pix2Vox++ when multiple images are utilized as input. Specifically, it achieves an enhanced Intersection Over Union score of approximately 1.1% compared to the original Pix2Vox++ model when 16-view images are employed.

Strip steel surface defect recognition research has important research significance in industrial production. Aiming at the problems of defect feature extraction difficulty, slow detection speed and insufficient dataset, YOLOv5 is improved on the basis of YOLOv5, and YOLO-LFPD (Lightweight fine particle detection) model is proposed. By introducing the RepVGG (Re-param VGG) module, the robustness of the model to the surface defect features of strip steel is enhanced, and the expressive ability of the model is improved. FasterNet is used to replace the backbone network, which ensures the accuracy and accelerates the inference speed, making the model more suitable for real-time monitoring. The use of pruning, GA genetic algorithm with OTA loss function method further reduces the model size while better learning the strip steel defect feature information, thus improving the generalisation ability and accuracy of the model. The experimental results show that the introduction of the RepVGG module and the use of FasterNet can well improve the model performance, with a reduction of 48% in the number of parameters, a reduction of 13% in the number of GFLOPs, an inference time of 77% of the original, and an optimal accuracy compared with the network models in recent years. The experimental results on the NEU-DET dataset show that the accuracy of YOLO-LFPD is improved by 3% to 81.2%, which is better than other models, and provides new ideas and references for the lightweight strip steel surface defect detection scenarios and application deployment.

Detecting cracks in structural health monitoring is crucial for ensuring infrastructure safety and longevity. Using drones to obtain crack images and automate processing can improve the efficiency of crack detection. To address the challenges posed by the limited computing resources of edge devices in practical applications, we propose CrackScopeNet, a lightweight segmentation network model that simultaneously considers local and global crack features, suitable for deployment on drone platforms with limited computational power and memory. CrackScopeNet features a multi-scale branch to improve sensitivity to cracks of varying sizes without substantial computational overhead and a stripe-wise context attention mechanism to enhance the capture of long-range contextual information, mitigating the interference from complex backgrounds. Experimental results on CrackSeg9k dataset demonstrate that CrackScopeNet leads to a significant improvement in prediction performance, with the highest mean intersection over union (mIoU) scores reaching 82.12%, while maintaining a lightweight architecture with only 1.05M parameters and 1.58G floating point operations (FLOPs). Besides, CrackScopeNet excels in inference speed on edge devices without GPU because of its low FLOPs. CrackScopeNet contributes to the development of efficient and effective crack segmentation networks, suitable for practical structural health monitoring applications using drone platforms.

Synthetic Aperture Radar has been extensively used in numerous fields and can gather a wealth of information about the area of interest. This large-scene data-intensive technology puts a high value on automatic target recognition (ATR) which can free the utilizers and boost the efficiency. Recent advances in artificial intelligence have made it possible to create a deep learning-based SAR ATR that can automatically identify target features from massive input data. In the last 6 years, intensive research has been conducted in this area, however, most papers in the current SAR ATR field used recurrent neural network (RNN) and convolutional neural network (CNN)-varied models to deepen the regime’s understanding of the SAR images. To equip SAR ATR with updated deep learning technology, this paper tries to apply a lightweight vision transformer (LViT)-based model to classify SAR images. The entire structure was verified by an open- accessed SAR data set and recognition results show that the final classification outcomes are robust and more accurate in comparison with referred traditional network structures without even using any convolutional layers.

The rapid advancements in satellite technology have led to a significant increase in high-resolution remote sensing (RS) images, necessitating advanced processing methods. Additionally, a patent analysis revealed a significant increase in deep learning and machine learning applications in remote sensing, highlighting the growing importance of these technologies. Therefore, this paper introduces the Kolmogorov-Arnold Network (KAN) model to remote sensing, aiming to enhance efficiency and performance in RS applications. We conducted several experiments to validate KAN's applicability, starting with the EuroSAT dataset, where we combined the KAN layer with multiple pretrained CNN models. The optimal performance was achieved with ConvNeXt, leading to the development of the KonvNeXt model. KonvNeXt was evaluated on the Optimal-31, AID, and Merced datasets for validation, and it achieved accuracies of 90.59%, 94.1%, and 98.1%, respectively. The model also showed fast processing speed, with the Optimal-31 and Merced datasets completed in 107.63 seconds each, while the bigger and more complicated AID dataset took 545.91 seconds. This result is meaningful since it achieved faster speeds and comparable accuracy compared to the existing study which utilized VIT and proved KonvNeXt's applicability for remote sensing classification tasks. Furthermore, we investigated the model's interpretability by utilizing Occlusion Sensitivity and by displaying the influential regions, it validated its potential use in a variety of domains including medical imaging and weather forecasting. This paper is meaningful in that it is the first use of KAN in remote sensing classification, proving its adaptability and efficiency.

Gastric cancer is the fifth most common and fourth deadliest cancer worldwide, with a bleak 5-year survival rate of about 20%. Despite significant research into its pathobiology, prognostic predictability remains insufficient due to pathologists’ heavy workloads and the potential for diagnostic errors. Consequently, there is a pressing need for automated and precise histopathological diagnostic tools. This study leverages Machine Learning and Deep Learning techniques to classify histopathological images into healthy and cancerous categories. By utilizing both handcrafted and deep features and shallow learning classifiers on the GasHisSDB dataset, we conduct a comparative analysis to identify the most effective combinations of features and classifiers for differentiating normal from abnormal histopathological images without employing fine-tuning strategies. Our methodology achieves an accuracy of 95% with the SVM classifier, underscoring the effectiveness of feature fusion strategies. Additionally, cross-magnification experiments produced promising results with accuracies close to 80% and 90% when testing the models on unseen testing images with different resolutions.

Public safety and intelligent surveillance systems critically depend on anomaly behavior detection for effective monitoring. In real-world pedestrian detection scenarios, prevalent challenges such as missed detections, complex background interference, and small target sizes hinder accurate anomaly identification. To address these issues, this study introduces YOLO-ABD, a lightweight method for anomaly behavior detection that integrates small target detection and channel shuffling. This method employs YOLOv8n as the baseline model, incorporating a small target detection mechanism in the Head part and utilizing GSConv convolution in the Backbone to enhance perceptual capability. Additionally, the SimAM attention mechanism is integrated to mitigate complex background interference, thereby improving target detection performance. Evaluation on the IITB-Corridor dataset demonstrated mAP50 and mAP50-95 scores of 89.3% and 60.6%, respectively. Generalization testing on the street-view-gdogo dataset further highlighted the superiority of YOLO-ABD over advanced detection algorithms, underscoring its effectiveness and generalizability. With a relatively small parameter count, YOLO-ABD presents an excellent lightweight solution for pedestrian anomaly behavior detection.

In this paper, we present a multi-task model that predicts disparities and confidence levels in deep stereo matching simultaneously. We do this by combining its successful model for each separate task and obtaining a multitask model that can be trained with a proposed loss function. We show the advantages of this model compared to training and predicting disparity and confidence sequentially. This method enables an improvement of 15% to 30% in the area under the curve (AUC) metric when trained parallel rather than sequential. In addition, the effect of weighting the components in the loss function on the stereo and confidence performance is investigated. By improving the confidence estimate, the practicality of stereo estimators for creating distance images is increased.

Infrared and visible light image fusion technology integrates image feature information from two different modalities to generate an image that integrates complementary information from the source images. However, in low-light scenarios, the degradation of lighting in visible light images leads to existing fusion methods being unable to effectively extract texture detail information from the scene. At this time, the target prominence information provided by infrared images alone is far from sufficient. To address this challenge, this paper proposes a lightweight infrared and visible light image fusion method based on low-light enhancement, named LLE-Fuse. Firstly, the method improves the MobileOne Block by embedding the Sobel operator in the Edge-MobileOne Block to perform feature extraction and downsampling on the source images. Then, intermediate features of different scales are obtained and fused through a cross-modal attention fusion module, followed by image reconstruction using a decoder with upsampling. Next, the CLAHE algorithm is used for image enhancement of infrared and visible light images, and the enhanced loss guides the network model to learn the capability of low-light enhancement. Finally, upon completion of the network training, the method optimizes the Edge-MobileOne Block into the direct connection structure of MobileNetV1 through structural re-parameterization, effectively reducing the computational resource consumption of the model network. Through extensive experimental comparisons, this paper's method demonstrates excellent performance in both visual appeal and evaluation metrics, providing satisfactory fusion results even in low-light scenarios.

Visualization of volumetric data holds great importance, particularly in medical and biomedical applications. Various imaging techniques, such as FMRI and 3D microscopy, are employed to generate volumetric data, used in medical practice, research, and teaching. Commonly utilized tools like 3D Slicer, Fiji, and MATLAB ® aid in rendering and analyzing 3D images. However, these tools may lack comprehensive rendering functionality and face challenges in handling computational demands as data sizes grow. To address these limitations, this work introduces a GPU-supported renderer with a MATLAB ® interface. This solution gives the user flexible control over rendering parameters and optimizes data transfer through a memory management system. By leveraging the computational power of NVIDIA GPUs, the renderer enables complex and high-quality renderings, enhancing speed and efficiency. It therefore facilitates the analysis and visualization of volumetric data within an integrated environment, namely MATLAB ®, streamlining their workflows. This advancement provides valuable opportunities for researchers and medical professionals to explore and comprehend volumetric data effectively.

Multi-threshold image segmentation is a relevant research field in further higher-level image preprocessing and computer vision. This paper adopts the Bayesian Forecasting Evolutionary Algorithm (BFEA) for natural scenery image segmentation using multilevel thresholding. An update method for the prediction vector is designed for image segmentation. The design idea of the BFEA algorithm for multilevel thresholding is based on the basic principle of Evolutionary Computation, the probability distribution of promising solutions, and the Bayesian theorem. BFEA could effectively solve the curse of dimensionality. Extensive experiments have demonstrated that the proposed algorithm dominated the state-of-the-art population-based thresholding method in term of the quality of image, function fitness value, optimal threshold value, SSIM value, PSNR value, computation time, dimensionality issue and convergence rate. The proposed method is very effective for color image segmentation.

In computer vision, human action recognition is a hot topic, popularized by the development of deep learning. Current models have achieved high accuracy results on public datasets. Despite this success, they require significant computational resources for training. Given that transfer learning based techniques allow reusing what other models have already learned and training models with less computational resources, in this work we propose using a transfer learning based approach for action recognition in videos. We describe a methodology for human action recognition using transfer learning techniques in a custom dataset. The proposed method consists of four stages: 1) human detection and tracking, 2) video preprocessing, 3) feature extraction (using pretrained models with ImageNet), and 4) action recognition using a two-stream model consisting of TCNs, LSTMs, and CNNs layers. The custom dataset is imbalanced with 189, 390, 490, 854, and 890 videos per class, respectively. For feature extraction, we analyzed the performance of seven pretrained models: Inception-v3, MobileNet-v2, MobileNet-v3-L, VGG-16, VGG-19, Xception, and ConvNeXt-L. We show that the best results were obtained with the last one. Finally, using pretrained models for feature extraction allowed training in a PC with a single GPU with an accuracy of 94.9%.

In this study, we investigate how dataset composition influences the performance and bias of age estimation models across different ethnic groups. We utilized pre-trained Convolutional Neural Networks (CNNs) such as VGG19, fine-tuning them on two datasets: UTKFace and APPA-REAL. UTKFace comprises 23,705 samples (12,391 males, 11,314 females) including 10,078 White, 4,526 Black, and 3,434 Asian individuals. APPA-REAL consists of 7,591 samples (3,818 males, 3,773 females) with 6,686 White, 231 Black, and 674 Asian individuals. We adjusted dataset composition by oversampling minority groups and reducing samples from the majority group to evaluate effects on model performance, measured using Mean Absolute Error (MAE) and standard deviation. Our findings demonstrate that oversampling alone does not guarantee equitable performance across ethnicities; reducing samples from the majority group often resulted in more balanced performance, evidenced by lower MAE standard deviations. These results underscore the importance of a nuanced approach to achieve fairness, including combining various sampling techniques. This study emphasizes the critical need for tailored dataset compositions to mitigate bias and suggests exploring more refined data processing methods and algorithmic adjustments for enhanced model fairness and accuracy in facial recognition technologies.

Drought stress severely affects the normal growth and development of wheat, leading to reduced yields and quality. To address the lag and limitations of traditional drought monitoring methods, this paper proposes a drought stress monitoring model for winter wheat during critical growth stages based on deep learning. By acquiring drought stress images of winter wheat during three critical growth stages: rise-jointing, heading-flowering and flowering-maturity, a dataset correlating these images with soil moisture monitoring data was established. The DenseNet121 network model was selected as the base network to extract phenotypic features of winter wheat under drought stress. Variables such as the model's training approach, changes in learning rate, and whether an attention mechanism was included, were used to conduct eight sets of model training and optimization strategy experiments. A drought stress monitoring model for winter wheat during critical growth stages based on an improved DenseNet-121 was constructed. The results showed that the average recognition accuracy of the eighth group in the test set reached 94.67%, indicating good application prospects in the recognition of drought stress levels during critical growth stages of wheat.

Real time camouflaged object detection so far using CAMO or COD10K datset has been able to achieve mAP50 less than 64%. We present a fourier transfom and Milkyway filament extraction based method extracting shape and edges of camouflaged object, achieving 88%+ mAP50 while continuing to deliver real time detection using YOLOv8s based models. This allows for search and rescue of wilelife and humans in smoke, fog, natural disasters possible by combining low compute Yolov8s model on headup display for rescue workers such as firefighters, first reponders etc.

Aiming at the problem that dynamic targets in indoor environments lead to low accuracy and large errors in the localization and position estimation of visual SLAM systems and the inability to build maps containing semantic information, a semantic visual SLAM algorithm based on the semantic segmentation network DeepLabV3+ and LK optical flow is proposed based on ORB-SLAM2 system. First, the dynamic target feature points are detected and rejected based on the lightweight semantic segmentation network DeepLabV3+ and LK optical flow method. second, the static environment occluded by the dynamic target is repaired using the time-weighted multi-frame fusion background repair technique. Lastly, the filtered static feature points are used for feature matching and position calculation. Meanwhile, the semantic labeling information of static objects obtained based on the lightweight semantic segmentation network DeepLabV3+ is fused with the static environment information after background repair to generate dense point cloud maps containing semantic information, and the semantic dense point cloud maps are transformed into semantic octree maps using the octree spatial segmentation data structure. The localization accuracy of the visual SLAM system and the construction of the semantic maps are verified using the TUM RGB-D widely used dataset and real scene data, respectively. The experimental results show that the proposed semantic visual SLAM algorithm can effectively reduce the influence of dynamic targets on the system, has a higher localization accuracy, and compared with other advanced algorithms, such as DynaSLAM, has the highest performance in indoor dynamic environments while considering both localization accuracy and real-time performance. In addition, semantic maps can be constructed so that the robot can better understand and adapt to the indoor dynamic environment.

This paper introduces a novel framework for zero-shot learning (ZSL), i.e., to recognize new categories that are unseen during training, by distilling knowledge from foundation models. Specifically, we first employ ChatGPT and DALL-E to synthesize reference images of unseen categories from text prompts. Then, the test image is aligned with text and reference images using CLIP and DINO. Finally, the predicted logits are aggregated according to their confidence to produce the final prediction.Experiments are conducted on multiple datasets, including CIFAR-10, CIFAR-100, and TinyImageNet. The results demonstrate that our model can significantly improve classification accuracy compared to previous approaches, achieving AUROC scores above 96\% across all test datasets. Our code is available at https://github.com/1134112149/MICW-ZIC.

Early disease detection is crucial for maximizingcrop yields and minimizing financial losses in agriculture. Inorder to categorise leaf diseases in pepper, tomato, and potato plants, this study suggests a revolutionary CNN design. The algorithm collects disease-specific features from input photos by using data augmentation techniques to artificially enlarge the training dataset. ReLU-activated convolutional layers gradually capture features at different levels, from low to high, while max pooling layers minimise spatial dimensionality. After thecharacteristics are extracted, a fully-connected layers with a softmax activation function classify them into illness groups. A well chosen dataset containing labelled healthy and sick leaves is used to train and validate the model. Generalizability to untesteddata is ensured via performance evaluation on an independent testing set. The field of deep learning for agricultural applications benefits from this research. This strategy has the ability to rev- olutionise agricultural practices by enabling automated disease identification. This might result in early interventions, better crop health, and eventually sustainable agricultural production.

Existing methods for simulating missile dynamics and training often rely on three degrees of freedom (3DOF) motions, limiting their ability to represent the complex maneuvers and targeting strategies of modern missiles. This paper proposes integrating virtual reality (VR) technology with six degrees of freedom (6DOF) dynamics to enhance the accuracy and efficiency of missile simulation and training. By incorporating 6DOF dynamics, simulations can capture the full range of missile motion, providing a comprehensive understanding of their capabilities and limitations. Through VR, users can immerse themselves in realistic missile scenarios, interact with dynamic behaviors in real-time, and refine their targeting and interception skills. The integration of VR with 6DOF dynamics also facilitates real-time feedback and evaluation, enhancing training efficiency and proficiency in missile operations. This approach aims to revolutionize missile training by bridging the gap between traditional simulation methods and real-world dynamics.

This paper presents computer and color vision research focusing on human color perception in VR environments. A VR art gallery with digital twins of original artworks is created for this experiment. In this research, the field of colorimetry and the application of CMYK and RGB color models are applied. The interrelationships of the two colors models are applied to create a color modification of the VR art gallery environment using C# Script procedures. This color-edited VR environment works with a smooth change of color tone in a given time interval. At the same time, a sudden change in the color of the RGB environment is defined in this interval. This experiment aims to record a user's reaction embedded in a VR environment and the effect of color changes on human perception in a VR environment. The research uses lie detector sensors that record the physiological changes of the user embedded in VR. Five sensors are used to record the signal. An experiment on the influence of the user's color perception in a VR environment using lie detector sensors has never been conducted. This research defines the basic methodology for analyzing and evaluating the recorded signals from the lie detector. The presented text thus provides a basis for further research in the field of colors and human color vision in a VR environment and lays an objective basis for use in many scientific and commercial areas.

Fluorescence microscopic images of cells contain a large number of morphological features that are used as an unbiased source of quantitative information about cell status, through which researchers can extract quantitative information about cells and study the biological phenomena of cells through statistical and analytical analysis. As an important research object of phenotypic analysis, images have a great influence on the research results. Saturation artifacts present in the image result in a loss of grayscale information that does not reveal the true value of fluorescence intensity. From the perspective of data post-processing, we propose a two-stage cell image recovery model based on generative adversarial network to solve the problem of phenotypic feature loss caused by saturation artifacts. The model is capable of restoring large areas of missing phenotypic features. In the experiment, we adopt the strategy of progressive restore to improve the robustness of the training effect, and add the contextual attention structure to enhance the stability of the restore effect. We hope to use deep learning methods to mitigate the effects of saturation artifacts to reveal how chemical, genetic, and environmental factors affect cell state, providing an effective tool for studying the field of biological variability and improving image quality in analysis.

While cycling presents environmental benefits and promotes a healthy lifestyle, the risks associated with overtaking maneuvers by motorized vehicles pose significant hindrances for many potential cyclists. Large-scale analysis of overtaking maneuvers could inform traffic researches and city planners on how to reduce these risks by better understanding these maneuvers. Drawing from the fields of sensor-based cycling research and of LiDAR-based traffic data sets, this paper provides a step towards addressing these safety concerns by introducing the Salzburg Bicycle 3d (SaBi3d) data set, consisting of LiDAR point clouds capturing car-to-bicycle overtaking maneuvers. The data set, collected using a LiDAR-equipped bicycle, facilitates detailed analysis of a large quantity of overtaking maneuvers without the need for manual annotation through enabling automatic labeling by a neural network. Additionally, a benchmark result for 3d object detection using a competitive neural network is provided as a baseline for future research. The SaBi3d data set is structured identically to the nuScenes data set, and therefore offers compatibility with numerous existing object detection systems. This work provides valuable resources for future researchers to better understand cycling infrastructure and mitigate risks, thus promoting cycling as a viable mode of transportation.

Computational animation of spatiotemporal models for 3-D shape and motion is considered as core of dynamic real-time vision. This is based on sets of features evaluated from multi-focal image streams. Subjects are defined as special objects capable of sensing environmental parameters and of initiating own actions in combination with stored knowledge. Object/subject recognition and scene understanding are achieved on different levels. Multiple objects are tracked individually for perceiving their actual state (‘here and now’). Fast saccadic jumps in gaze direction allow flexible concentration on objects or parts of actual interest. By analyzing motion of relevant objects/subjects over a larger time scale on the level of state variables and documenting them in the ‘scene tree representation’ (computer graphics), the situation with respect to decision making is assessed. Various behavioral capabilities of subjects are represented on an abstract level for characterizing their potential behaviors. These are generated by stereotypical feed-forward and feedback control applications on a separate level close to the actuator hardware with corresponding methods. This dual representation on one level for decision making and one on the implementation level allows for flexibility and easy adaptation or extension. Results are shown for road vehicle guidance based on three cameras on a gaze-controlled platform.

The fusion of global contextual information with local cropped block details is crucial for segmenting ultra-high resolution images. In this study, we introduce a novel fusion mechanism termed Global-Local Deep Fusion (GL-Deep Fusion) based on an enhanced transformer architecture, which efficiently integrates global contextual information and local details. Specifically, we propose the Global-Local Synthesis Networks (GLSNet), a dual-branch network where one branch processes the entire original image, while the other branch handles cropped local patches as input. The feature fusion of different branches in GLSNet is achieved through GL-Deep Fusion, significantly enhancing the accuracy of ultra-high resolution image segmentation. Particularly effective is GLSNet in identifying tiny overlapping items. To optimize GPU memory utilization, we meticulously design a dual-branch architecture that proficiently leverages the features it extracts, seamlessly integrating them into the enhanced transformer framework of GL-Deep Fusion. Extensive experiments conducted on challenging benchmarks, including DeepGlobe and Vaihingen datasets, demonstrate that GLSNet achieves a new state-of-the-art performance in terms of GPU memory utilization and segmentation accuracy tradeoff.

This paper presents a novel segmentation algorithm specially developed for applications in 3D point clouds with high variability and noise, particularly suitable for heritage building 3D data. The method can be categorized within the segmentation procedures based on edge detection. In addition, it uses a graph-based topo-logical structure generated from the supervoxelization of the 3D point clouds, which is used to make the clo-sure of the edge points and to define the different segments. The algorithm provides a valuable tool for generating results that can be used in subsequent classification tasks and broader computer applications dealing with 3D point clouds. One of the characteristics of this segmentation method is that it is unsupervised, which makes it particularly advantageous for heritage applications where labelled data is scarce. It is also easily adaptable to different edge point detection and supervoxelization algorithms. Finally, the results show that the 3D data can be segmented into different architectural elements, which is important for further classification or recognition. Extensive testing on real data from historic buildings demonstrates the effectiveness of the method. The results show superior performance compared to three other segmentation methods, both globally and in the segmentation of planar and curved zones of historic buildings.

Sign language recognition technology can help people with hearing impairments to communicate with those who are hearing impaired. At present, with the rapid development of society, deep learning technology also provided certain technical support for sign language recognition work. In sign language recognition tasks, the use of traditional convolutional neural networks to extract spatio-temporal features from sign language videos suffers from insufficient feature extraction, resulting in low recognition rates. Nevertheless, video-based sign language datasets are very large and require a lot of computational resources for training and generalisation must be ensured, which poses a challenge for recognition. In this paper, we have presented a video-based sign language recognition method based on resnet and lstm. As the number of network layers increases, the ResNet network can effectively solve the granularity explosion problem and obtain better time series features. We use the Resnet convolutional network as the backbone model. At the initialisation stage, we obtain sign language features using ResNet; then, the learned feature space is used as the input of LSTM network to obtain long sequence features. The experimental results show that the accuracy of the above model is better than the mainstream model, and it can effectively extract the spatio-temporal features in sign language videos and improve the recognition rate of sign language actions.

Fire is a serious security threat that can lead to casualties, property damage, and environmental damage. However, despite the availability of object detection algorithms, challenges persist in detecting fires and smoke. These challenges include slow convergence speed, poor performance in detecting small targets, and high computational cost limiting deployments. In this paper, Fire smoke and human detection based on ConvNeXt and Mixed encoder (FCM-DETR), an end-to-end object detection algorithm based on Deformable DETR, is proposed. Firstly, we introduce ConvNeXt to take the place of Resnet, which greatly reduces the amount of computation and improves the ability to extract irregular flame features. Secondly, to effectively process multi-scale features, the original encoder is decoupled into two modules. Then Mixed encoder, an innovative encoder structure, is proposed, resulting in excellent performance of multi-scale fire and smoke features fusion. What can’t be overlooked is that the encoder block is compatible with any DETR-based models. Finally, the convergence speed is accelerated and the mean of average precision is improved by applying a novel loss function called Powerful IoU v2. The experimental results indicate that our model achieves the best detection accuracy compared to other models, with mAP reaching 66.7%, mAPs and Accuracyfire achieving impressive 50.2% and 98.05%, respectively.

Some of the barriers preventing Virtual Reality (VR) from being widely adopted are the cost and unfamiliarity of VR systems. Here we propose that in many cases the specialized controllers shipped with most VR head-mounted displays can be replaced by a regular smartphone, cutting the cost of the system, and allowing users to interact in VR using a device they are already familiar with. To achieve this, we developed SmartVR Pointer, an approach that uses smartphones as a replacement for the specialized controllers for two essential operations in VR: selection and navigation by teleporting. In SmartVR Pointer a camera mounted on the head-mounted display (HMD) is tilted downwards so that it points to where the user will naturally be holding their phone in front of them. SmartVR Pointer supports three selection modalities: tracker-based,gaze-based, and combined/hybrid. In the tracker-based SmartVR pointer selection we use image-based tracking to track a QR code displayed on the phone screen and them map the phone’s position to a pointer shown within the field of view of the camera in the Virtual Environment. In the gaze-based selection modality the user places the pointer using their gaze and taps on the phone for selection. The combined technique is a hybrid between head-gaze-based interaction in VR and smartphone-based Augmented Reality. It allows the user to control a VR pointer that behaves like a mouse pointer by moving their smartphone to select objects within the virtual environment, and to interact with the selected objects using the smartphone’s touch screen. The touchscreen is used for selection and dragging. SmartVR Pointer is simple and requires no calibration and no complex hardware assembly or disassembly. We demonstrate successful interactive applications of SmartVR Pointer in a VR environment with a demo where the user navigates in the virtual environment using teleportation points on the floor and then solves a Tetris-style key-and-lock challenge.

The event camera generates an event stream based on changes in brightness, retaining only the characteristics of moving objects, addresses the high power consumption associated with using high-frame-rate cameras for high-speed eye tracking tasks. However, the asynchronous incremental nature of event camera output has not been fully utilized, and there are also issues related to missing event datasets. Combining the temporal information encoding and state-preserving properties of spiking neural network (SNN) with event camera, a near-range eye tracking algorithm is proposed as well as a novel event-based dataset for validation and evaluation. According to experimental results, the proposed solution outperforms artificial neural network (ANN) algorithms, while computational time remains only 12.5\% of that of traditional SNN algorithms. Furthermore, the proposed algorithm allows for self-adjustment of time resolution, with a maximum achievable resolution of 0.081ms, enhancing tracking stability while maintaining accuracy.

Medical image segmentation is a critical step in various healthcare applications, aiding in diagnosis, treatment planning, and disease monitoring. In this study, we investigate the efficacy of a segmentation approach based on the 2D wavelet transform. Leveraging a dataset comprising 10 diverse medical images, we evaluate the performance of our segmentation method using three key metrics: accuracy, precision, and recall. Our findings demonstrate that the proposed approach enhances segmentation accuracy, offering promising results compared to existing methods. By harnessing the multi-resolution feature extraction capabilities of the 2D wavelet transform, our method achieves improved delineation of medical image structures, paving the way for more accurate and efficient healthcare interventions.

Real-time object detection remains a pivotal topic in the realm of computer vision. Balancing the accuracy and speed of object detectors poses a formidable challenge for both academic researchers and industry practitioners. While recent transformer-based models have showncase the prowess of the attention mechanism, delivering remarkable performance improvements over CNNs, their computational demands can hinder their effectiveness in real-time detection tasks. In this paper, we elect to use YOLOX as our robust starting point and introduce a series of effective enhancements, culminating in a new high-performance detector named YOLOAX. To further exploit the power of the attention mechanism, we devise multi-dimensional attention-based modules that activate CNNs, emphasizing the most pertinent regions and bolstering the capacity to learn the most informative image representations from feature maps. Moreover, we introduce a novel leading label assignment strategy called STA, along with a groundbreaking loss function named GEIOU Loss, to further refine our detector's performance. We provide extensive ablation studies on the COCO and PASCAL VOC 2012 detection datasets to validate our proposed methods. Remarkably, our YOLOAX is trained solely on the MS-COCO dataset from scratch, without leveraging any prior knowledge, which achieves an impressive 54.2% AP on the COCO 2017 test set while maintaining a real-time speed of 72.3 fps, surpassing YOLOX by a significant margin of 3.0% AP. Our source code and pre-trained models are openly accessible at https://github.com/KejianXu/yoloax.

In response to the burgeoning global demand for premium agricultural products, particularly within the competitive nut market, this paper introduces an innovative methodology aimed at enhancing the grading process for almonds and their shells. Leveraging state-of-the-art Deep Convolutional Neural Networks (CNNs), specifically the AlmondNet-20 architecture, our study achieves exceptional accuracy exceeding 99%, facilitated by the utilization of a 20-layer CNN model. To bolster robustness in differentiating between almonds and shells, data augmentation techniques are employed, ensuring the reliability and accuracy of our classification system. Our model, meticulously trained over 1000 epochs, demonstrates remarkable performance, boasting an accuracy rate of 99% alongside a minimal loss function of 0.0567. Rigorous evaluation through test datasets further validates the efficacy of our approach, revealing impeccable precision, recall, and F1-score metrics for almond detection. Beyond its technical prowess, this advanced classification system offers tangible benefits to both industry experts and non-specialists alike, ensuring globally reliable almond classification. The application of deep learning algorithms, as showcased in our study, not only enhances grading accuracy but also presents opportunities for product patents, thereby contributing to the economic value of our nation. Through the adoption of cutting-edge technologies such as the AlmondNet-20 model, we pave the way for future advancements in agricultural product classification, ultimately enriching global trade and economic prosperity.

Within environmental perception, automatic navigation and object detection, computer vision is a crucial and demanding field with many applications in modern industries, such as multi-target long-term visual tracking in automated production, defect detection, and driverless robotic vehicles with sensing ability. The performance of computer vision has greatly improved recently thanks to developments in deep learning algorithms and hardware computing capabilities, which has spawned the creation of a large number of related applications. This paper presents the results of a detailed review of over 50 papers published over the course of two decades (1999–2024), with a primary focus on the technical advancement of computer vision. To elucidate the foundational principles, an examination of typical visual algorithms based on traditional correlation filtering was initially conducted. Subsequently, a comprehensive overview of the most recent advancements in deep learning-based computer vision techniques was compiled. Furthermore, a comparative analysis between conventional and novel algorithms was undertaken to discuss the future trends and directions of computer vision. Lastly, the feasibility of employing visual SLAM (Simultaneous Localization and Mapping) algorithms in the context of autonomous vehicles with sensing capability was explored. Furthermore, we explored a thorough and efficient architecture for utilizing these techniques in autonomous robotic vehicles into unmanned green and low-carbon smart factories, underlining novel advances as well as potential prospects for future research.

The existing segmentation-based scene text detection methods mostly need complicated post-processing, and the post-processing operation is separated from the training process, which greatly reduces the detection performance. The previous method, DBNet successfully simplified post-processing and integrated the post-processing into a segmentation network. However, the training process of the model took a long time for 1200 epochs and the sensitivity to texts of various scales was lacking, leading to some text instances being missed. Considering the above two problems, we design the text detection Network with Binarization of Hyperbolic Tangent(HTBNet). First of all, we propose Binarization of Hyperbolic Tangent (HTB), optimized along with which, the segmentation network can expedite the initial convergent speed by reducing the amount of epochs from 1200 to 600. Because features of different channels in the same scale feature map focus on the information of different regions in the image, to better represent the important features of all objects in the image, we devise the Multi-Scale Channel Attention(MSCA). Meanwhile considering that multi-scale objects in the image cannot be simultaneously detected, we propose a novel module named Fused Module with Channel and Spatial(FMCS), which can fuse the multi-scale feature maps from channel and spatial dimension. Finally we adopt cross entropy as the loss function, which measures the difference between predicted values and ground truths. The experimental results show that HTBNet compared with lightweight models has achieved competitive performance and speed on Total-Text(F-measure:86.0%, FPS:30) and MSRA-TD500 (F-measure:87.5%, FPS:30).

Over the last few decades, advancements in astrophysics have been closely linked to the development of powerful machine-learning models that can accurately classify celestial bodies. At the same time, however, many astronomical datasets are filled with new features collected by increasingly powerful telescopes. These features can cause overfitting, clouding predictive abilities, and dampening the ability of many models to classify images. Therefore, motivation to design more efficient models has skyrocketed—aiming to optimize for lower run times and high accuracies, even with fewer provided features. In our project, we seek to optimize a convolutional neural network model using a technique known as wavelet analysis. This technique allows us to pick the key features of an astronomical image and accentuate niche details, saving time and boosting accuracy. We applied it to the MiraBest Dataset, a dataset compiled from the FIRST sky survey using a virtual telescope. In the end, after training our neural network on the original images and the five filters (approximation, horizontal, vertical, diagonal, and combined), we found that with fewer features and less overfitting, the vertical Daubechies-family wavelet filter outperformed the original runs with the unaltered images by over 10%. Our findings suggest that wavelet analysis can help harvest the most valuable features in images of celestial bodies–leading to enhanced predictions in astronomical applications and perhaps bolstering modern astrophysical theory.

In high-risk railway construction, personal protective equipment monitoring is critical but challenging due to small and frequently obstructed targets. We propose YOLO-EA, an innovative model that enhances safety measure detection by integrating ECA into its backbone's convolutional layers, improving discernment of minuscule objects like hardhats. YOLO-EA further refines target recognition under occlusion by replacing GIoU with EIoU loss. YOLO-EA's effectiveness was empirically substantiated using a dataset derived from real-world railway construction site surveillance footage. It outperforms YOLOv5, achieving 98.9% precision and 94.7% recall, up 2.5% and 0.5% respectively, while maintaining real-time performance at 70.774 fps. This highly efficient and precise YOLO-EA holds great promise for practical application in intricate construction scenarios, enforcing stringent safety compliance during complex railway construction projects.

Machine learning and computer vision have proven to be valuable tools for farmers to streamline their resource utilization to lead to more sustainable and efficient agricultural production. These techniques have been applied to strawberry cultivation in the past with limited success. To build on this past work, in this study two separate sets of strawberry images, along with their associated diseases, were collected and subjected to, resizing, and augmentation. Subsequently, a combined dataset consisting of 9 classes was utilized to fine-tune three distinct pre-trained models: Vision Transformer (ViT), MobileNetV2, and ResNet18. To address the imbalanced class distribution in the dataset, each class was assigned weights to ensure nearly equal impact during the training process. To enhance the outcomes, new images were generated by removing backgrounds, reducing noise, and flipping them. The performances of ViT, MobileNetV2, and ResNet18 were compared after being selected. Customization specific to the task was applied to all three algorithms, and their performances were assessed. Throughout this experiment, none of the layers were frozen, ensuring all layers remained active during training. Attention heads were incorporated into the first 5 and last 5 layers of MobileNetV2 and ResNet18, while the architecture of ViT was modified. The results indicated accuracy factors of 98.4%, 98.1%, and 97.9% for ViT, MobileNetV2, and ResNet18, respectively. Despite the data being imbalanced, the precision, which indicates the proportion of correctly identified positive instances among all predicted positive instances, approached nearly 99% with the ViT. MobileNetV2 and ResNet18 demonstrated similar results. Overall, the analysis revealed that the Vision Transformer model exhibited superior performance in strawberry ripeness and disease classification. The inclusion of attention heads in the early layers of ResNet18 and MobileNet18, along with the inherent attention mechanism in ViT, improved the accuracy of image identification. These findings offer the potential for farmers to enhance strawberry cultivation through passive camera monitoring alone, promoting the health and well-being of the population.

Breast cancer stands as a predominant health concern for women globally. As mammography is the primary screening tool for breast cancer detection, improving the detection of breast cancer at screening could save more lives. This mammography re-view paper provides a systematic review of computer-aided techniques during a specific time frame for the segmentation and classification of microcalcification, evaluating image processing, machine learning, and deep learning techniques. The systematic review is meticulously carried out, adhering closely to the preferred reporting items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. This article focuses on mammographic breast cancer detection approaches based on automated systems, discussed chronologically from 1970 through 2023. This article encompasses the breadth of artificial intelligence-based methods from the most primitive to the most sophisticated models. Image processing and machine learning-based methods are compre-hensively reviewed. The evaluation of a deep learning architecture based on self-extracted features for classification tasks demonstrated outclass performance. Large-scale datasets required for a broader and in-depth analysis of novel methods for breast cancer detection are also discussed in this article. This research work is aligned with the United Nations' sustainability de-velopment goals.

Laser-targeted weeding methods further enhance the sustainable development of green agriculture, with one key technology being the improvement of weed localization accuracy. Here, we propose an improved YOLOv8 instance segmentation based on bidirectional feature fusion and deformable convolution (BFFDC-YOLOv8-seg) to address the challenges of insufficient weed localization accuracy in complex environments with resource-limited laser weeding devices. Initially, by training on extensive datasets of plant images, the most appropriate model scale and training weights are determined, facilitating the development of a lightweight network. Subsequently, the introduction of the Bidirectional Feature Pyramid Network (BiFPN) during feature fusion effectively prevents the omission of weeds. Lastly, the use of Dynamic Snake Convolution (DSConv) to replace some convolutional kernels enhances flexibility, benefiting the segmentation of weeds with elongated stems and irregular edges. Experimental results indicate that the BFFDC-YOLOv8-seg model achieves a 4.9% increase in precision, an 8.1% increase in recall rate, and a 2.8% increase in mAP50 value to 98.8% on a vegetable weed dataset compared to the original model. It also shows improved mAP50 over other typical segmentation models such as Mask R-CNN, YOLOv5-seg, and YOLOv7-seg by 10.8%, 13.4%, and 1.8%, respectively. Furthermore, the model achieves a detection speed of 24.8 FPS on the Jetson Orin nano standalone device, with a model size of 6.8MB that balances between size and accuracy. The model meets the requirements for real-time precise weed segmentation, suitable for complex vegetable field environments and resource-limited laser weeding devices.

In the field of visualization, understanding users’ analytical process is important to determine the effectiveness of a designed visualization application. To understand the users’ analytical process, numerous studies have been performed to capture and analyze user interactions. Although many studies have emphasized the importance of analyzing user interactions to understand users’ analytical reasoning processes, few have successfully linked these interactions to users’ reasoning processes. This paper introduces an approach that bridges this gap by correlating semantic user interactions with analysis decisions through an interactive wire transaction analysis system and a visual state transition matrix. With the designed analysis system, interactive analysis can be performed to evaluate financial fraud in wire transactions. It also allows the mapping of captured user interactions and analytical decisions back onto the visualization, revealing users’ distinct results. The visual state transition matrix further aids to help understanding users’ analytical flows, revealing their decision-making processes. Classification machine learning algorithms are applied to assess the effectiveness of our approach in understanding analysts’ strategies by connecting them to their decisions. From the study, we observed an average of 72% accuracy in clearly classifying the semantic user interactions. For classifying individual decisions, we observed an average of 70% classification accuracy. This emphasizes the importance of capturing semantic user interactions to understand users’ analytical processes. Overall, it is determined that the proposed approach not only enhances the understanding of analytical behaviors but also offers a robust approach for evaluating user interactions in visualization tools.

Outliers seriously affect the accuracy in geometric model fitting. Previous works in coping with outliers involve threshold selection and scale estimation. However most scale estimators suppose the inlier distribution as a Gaussian model, which usually poorly meets the cases in geometric model fitting. Outliers, considered as the points with big residuals to all the true models, share common items with big values in quantized residual preferences, thus making the outliers gather away from the inliers in quantized residual preference space. In this paper we make use of this outlier consensus in quantized residual preference space by extending the usage of energy minimization to combine the model error and the spatial smoothness in quantized residual preference space for outlier detection. The energy minimization based outlier detection process follows an alternate sampling and labeling framework. After the outlier detection, an ordinary energy minimization method is employed to optimize the inlier labels, which also follows the framework of alternate sampling and labeling. The experimental results in this study show that the energy minimization based outlier detection method can detect most of the outliers in the data. Furthermore, the proposed energy minimization based inlier segmentation process segments the inliers into different models quite accurately. Overall, the performance of the proposed method is better than most of the state-of-the-art methods.

In this paper, we present a method for estimating GPS signals from visual information captured by a monocular camera mounted on a fixed-wing Unmanned Aerial Vehicle. The main challenge in visual odometry from aerial images is the computation of scale due to irregularities in the elevation of the terrain. That is, it is not possible to accurately convert from pixels in the image to meters in space, and the error accumulates. The contribution of this work is to reduce the accumulated error by comparing the images from the camera with satellite images. The algorithm has been tested in real-time experiments, and the results show that it is possible to eliminate the accumulated error using satellite images.

We introduce a novel microscopic image dataset augmented with segmentation and detection labels specifically designed for microplastic analysis in sewage environments. Recognizing the increasing concern over microplastics — particles of synthetic polymers smaller than 5mm — and their detrimental effects on marine ecosystems and human health, our research focuses on enhancing detection and analytical methodologies through advanced computer vision and deep learning techniques. The dataset comprises high-resolution microscopic images of microplastics collected from sewage, meticulously labeled for both segmentation and detection tasks, aiming to facilitate accurate and efficient identification and quantification of microplastic pollution. In addition to dataset development, we present example deep learning models optimized for segmentation and detection of microplastics within complex sewage samples. The models demonstrate significant potential in automating the analysis of microplastic contamination, offering a scalable solution to environmental monitoring challenges. Furthermore, we ensure the accessibility and reproducibility of our research by making the dataset and model codes publicly available, accompanied by detailed documentation on GitHub and LabelBox. The dataset and example deep learning models are publicly available at the following GitHub link. https://anonymous.4open.science/r/Microplastics-in-Sewage-1BEF

This paper investigates the efficacy of YOLOv8 variants for vehicle detection and license plate detection within smart parking applications, emphasizing performance under varying ambient lighting conditions. The proposed system is to seize full video frames, extracts regions of interest containing vehicles, and feeds them into separate, pre-trained YOLOv8 models – one dedicated to vehicle detection and another for license plate detection. Four YOLOv8 variants, nano, small, medium, and large, are evaluated. As a pre-processing step, the images are processed with the help of OpenCV and Pillow libraries to adjust the luminosity and increase the images’ DPI so that they would be easy to perceive by the Tesseract OCR engine. Sixteen potential combinations arise from pairing the four YOLOv8 models for vehicle and license plate detection tasks. To identify the most suitable combinations, we employ the Multi-Criteria Decision-Making method, specifically Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) analysis. This analysis considers 4 critical metrics: Precision, mean average precision at 95% Intersection of Union threshold, Recall and Total Inference Time. The objective is to achieve an optimal balance between high accuracy and real-time processing. Following the selection of optimal YOLOv8 combinations through TOPSIS analysis, we assess their performance under varying ambient light intensity (measured in lux). This evaluation aims to identify the most robust model combinations that ensure accurate vehicle and license plate recognition across the diverse lighting conditions encountered in real-world environment.

We propose a novel unsupervised semantic segmentation method for fast and accurate flood area detection utilizing color images acquired from Unmanned Aerial Vehicles (UAVs). To the best of our knowledge, this is the first fully unsupervised method for flood area segmentation in color images captured by UAVs, without the need of pre-disaster images. The proposed framework addresses the problem of flood segmentation based on parameter-free calculated masks and unsupervised image analysis techniques. First, a fully unsupervised algorithm gradually excludes areas classified as non-flood utilizing calculated masks over each component of the LAB colorspace, as well an RGB vegetation index and the detected edges of the original image. Unsupervised image analysis techniques, such as distance transform, are then applied, producing a probability map for the location of flooded areas. Finally, flood detection is obtained by applying the hysteresis thresholding segmentation. The proposed method is tested and compared with variations, and other supervised methods in two public datasets, consisting of 953 color images in total, yielding high-performance results, with 87.4% and 80.9% overall accuracy and F1-Score, respectively. The results and computational efficiency of the proposed method show that it is suitable for on board data execution and decision-making during UAVs flight.

High-power laser facilities necessitate predicting incremental damage to final optics to identify evolving damage trends. In this study, we propose a surface damage detection method utilizing image segmentation employing ResNet-18, and a damage area estimation network employing U-Net++. Paired sets of online and offline images of optics obtained from a large laser facility is used to train the network. The trends of varying damage could be identified by incorporating additional experimental parameters. A key advantage of the proposed method is that the network can be trained end-to-end on small samples, eliminating the need for manual labeling or feature extraction. The software developed based on these models can facilitate the daily inspection and maintenance of optics in large laser facilities.

In Indonesia, types of meat are still identified manually due to increasing prices for beef and buf-falo ahead of Idul Fitri. Traders mix beef with pork to thwart the beef scam. Haar wavelet and GLCM, GLCM with angles of 0°, 45°, 90°, and 135°, as well as matrices using contrast, correlation, energy, homogeneity, and entropy, are used in feature extraction. The following are the results of meat image classification testing using k-NN, Haar wavelet, and GLCM: The k-NN algorithm shows superiority in identifying fresh (99%), frozen (99%), and rotten (96%) meat texture images with the most fantastic accuracy results in every situation. Meanwhile, GLCM routinely provides good results, especially regarding the texture image of fresh meat (183.21) and rotten meat (115.79). However, despite delivering less throughput than k-NN and GLCM, Haar wavelets are still helpful, especially when dealing with fresh meat texture images (89,96).

Visualising textual content could be helpful to professionals as well as amateurs across several fields. However, training a text-to-image generator in the mainstream domain requires large amounts of paired text-image and data, which is too expensive to collect since labeling millions of images and videos can be tiresome. GANs like StackGAN and StyleGAN can be considered as solutions to generate images from text. But the images generated may be of low accuracy and resolution, and the entire processing can be highly time-consuming. Moreover, image generation is a notion that is still being researched. Hence, the process of developing a Video Generation model necessitates substantial research. Despite the need for such a model, modern technology has lagged behind the solutions to this problem. This proposal suggests combining two methods, Text modification for Action Definition (TexAD) and SeQuential Image Generation for Video Synthesis (SQIGen). The proposed solution synthesises a sequence of images from textual information feeds and combines these images to create a video. TexAD uses Natural Language Processing and Deep Learning techniques to process, classify and modify text data. SQIGen is an extension of the VQGAN+CLIP neural network architecture that generates a sequence of images from the modified text data

Camouflage evaluation typically involves human visual search and detection experiments that are time-consuming and expensive. Hence, there is a need for models that compute camouflage effectiveness from digital imagery. Convolutional neural networks are a powerful tool for automatic object detection and recognition. We investigated whether such a network (YOLO) can also provide a measure of camouflage effectiveness that is related to human perception. To this end, human performance measures of camouflage effectiveness such as detection time and target conspicuity were obtained in observer experiments and compared with the performance of the (standard, i.e. pre-trained) YOLO-V4-tiny algorithm. YOLO provides the probability that a detected object is correctly classified, and this is adopted as our measure of camouflage effectiveness. We compared the YOLO-predicted classification probability for a person in camouflage clothing moving through rural and urban scenes to human detection performance. Overall, camouflage effectiveness predicted by YOLO correlates with human observer performance. At close distances, YOLO’s classification performance appears sensitive to high-contrast shape-breaking elements of a camouflage pattern. This suggests that YOLO may be adapted to assess camouflage effectiveness for a wide range of applications, such as evaluating or optimizing one’s signature and predicting optimal hiding locations in a given environment. Further research is required to fully establish YOLO’s limitations and applicability for this purpose.

Generalized zero-shot learning (GZSL) aims to simultaneously recognize both seen classes and unseen classes by training only on seen class samples and auxiliary semantic descriptions. Recent state-of-the-art methods infer unseen classes based on semantic information or synthesize unseen classes using generative models based on semantic information, all of which rely on the correct alignment of visual-semantic features. However, they often overlook the inconsistency between original visual features and semantic attributes. Additionally, due to the existence of cross-modal dataset biases, the visual features extracted and synthesized by the model may also mismatch with some semantic features, which could hinder the model from properly aligning visual-semantic features. To address this issue, this paper proposes a GZSL framework that enhances the consistency of visual-semantic features using self-distillation and disentanglement network (SDDN). The aim is to utilize self-distillation and disentanglement network to obtain semantically consistent refined visual features and non-redundant semantic features to enhance the consistency of visual-semantic features. Firstly, SDDN utilizes self-distillation technology to refine the extracted and synthesized visual features of the model. Subsequently, the visual-semantic features are then disentangled and aligned using a disentanglement network to enhance the consistency of the visual-semantic features. Finally, the consistent visual-semantic features are fused to jointly train a GZSL classifier. Extensive experiments demonstrate that the proposed method achieves more competitive results on four challenging benchmark datasets (AWA2, CUB, FLO, and SUN).

Few-shot object detection (FSOD) aims to address the challenge of requiring a substantial amount of annotations for training in conventional object detection, which is very labor-intensive. However, existing few-shot methods achieve high precision with the sacrifice of time-consuming for exhaustive fine-tuning, or take poor performance in novel-class adaptation. We presume the major reason is that the valuable correlation feature among different categories is insufficiently exploited, hindering the generalization of knowledge from base to novel categories for object detection. In this paper, we propose Few-Shot object detection via Correlation-RPN and Transformer Encoder-Decoder (CRTED), a novel training network to learn object-relevant features of inter-class correlation and intra-class compactness while suppressing object-agnostic features in the background with limited annotated samples. And we also introduce a 4-way tuple-contrast training strategy to positively activate the training progress of our object detector. Experiments over two few-shot benchmarks (Pascal VOC, MS-COCO) demonstrate that, our proposed CRTED without further fine-tuning can achieve comparable performance with current state-of-the-art fine-tuned works. The codes and pre-trained models will be released.

Open-vocabulary learning has recently gained prominence as a means to enable image segmentation for arbitrary categories based on textual descriptions. This advancement has extended the applicability of segmentation systems to a broader range of generally purpose scenarios. However, current methods often revolve around specialized architectures and parameters tailored to specific segmentation tasks, resulting in a fragmented landscape of segmentation models. In response to these challenges, we introduce OVAMTSeg, a versatile framework designed for Open-Vocabulary and Multitask Image Segmentation. OVAMTSeg harnesses adaptive prompt learning to empower the model to capture category-sensitive concepts, enhancing its robustness across diverse multi-task and scenario contexts. Text prompts are employed to effectively capture semantic and contextual features of the text, while cross-attention and cross-modal interactions enable the fusion of image and text features. Furthermore, a transformer-based decoder is incorporated for dense prediction. Extensive experimental results underscore the effectiveness of OVAMTSeg, showcasing its state-of-the-art performance and superior generalization capabilities across three segmentation tasks. Notable achievements include a 47.5 mIoU in referring expression segmentation, 51.6 mIoU on Pascal-VOC with four unseen classes, 46.6 mIoU on Pascal-Context in zero-shot segmentation, 65.9 mIoU on Pascal-5i, and 35.7 mIoU on COCO-20i datasets for one-shot segmentation.

Facial recognition systems are widely used systems around the globe. Their applications in real world scenarios like law enforcement, surveillance, ID card identification, information security, access control and many more has made them very important in today’s world. These systems have shown promising results and excellent performance; however, these systems need tremendous number of facial images for training before they are implemented in the real-world scenarios. The problem arises when we face lack of training images due to which the performance of such systems decreases exponentially. The problem becomes even worse when there is only one image per subject for training our facial recognition system, the probe image might contain variation like illumination, expression and disguise which are hard to be accurately recognized. Generative models have achieved great success for handling the lack of training data by generating synthetic data that resembles the original data. We have created a new method by fine tuning the basic Conditional Generative Adversarial Network (CGAN) to generate synthetic facial images provided only single image per subject. These synthetic facial images contain six variations that are smile, anger, stare, illumination and disguise. We increased the size of the dataset and then applied a convolutional neural network for facial recognition, which improved the accuracy of our system from 0.76 to 0.99.

The state-of-the-art smart city has been calling for an economic but efficient energy management over large-scale network, especially for the electric power system. It is a critical issue to monitor, analyze and control electric loads of all users in system. In this paper, we employ the popular computer vision techniques of AI to design a non-intrusive load monitoring method for smart electric energy management. First of all, we utilize both signal transforms (including wavelet transform and discrete Fourier transform) and Gramian Angular Field (GAF) methods to map one-dimensional current signals onto two-dimensional color feature images. Second, we propose to recognize all electric loads from color feature images using a deep neural network with multi-scale feature extraction and attention mechanism. Third, we design our method as a cloud-based, non-intrusive monitoring of all users, thereby saving energy cost during electric power system control. Experimental results on both public and our private datasets have demonstrated our method achieves superior performances than its peers, and thus supports efficient energy management over large-scale Internet of Things.

The task of weakly supervised point cloud semantic segmentation has received widespread attention and also has been widely used in autonomous driving, robotics, and modern industries. Due to the high dimensionality and large volume of point cloud data, many technical difficulties appeal to weakly supervised semantic processing, especially segmentation. However, weakly-supervised schemes usually provide only partial labelling information of the underlying point cloud data and thus need to effectively extract local features, geometric information, and contextual relationships only using these limited labelled data for supervised learning. To improve weakly-supervised semantic segmentation, we propose a novel segmentation network through the boundary-based feature aggregation based on a K-NN algorithm with down-sampling operation, and also introduce the smoothness loss and Siamese loss for effective segmentation. The experiments on public datasets demonstrate that our presented segmentation method exceeds most of the existing fully supervised and weakly-supervised methods in terms of mIoU. Specifically, our network has high segmentation accuracy on the labels of objects with similar geometrical structures, such as ceiling, wall, floor, chair and table, reaching 91.2%, 98.8%, 83.3%, 75.3% and 80.2%, respectively. Extensive experiments also illustrate its robustness, effectiveness, and generalization of the proposed weakly supervised segmentation network.

This paper introduces PatchFusion, an innovative approach for nonrigid tracking and reconstruction of deformable objects using a single RGB-D sensor. Existing methods face challenges in accurately capturing the rapid deformations of soft and flexible objects, thereby limiting their utility in diverse scenarios. Our approach overcomes this challenge by employing a dynamic patch-based framework that adapts to rapid inter-frame motions. Firstly, patch-wise rigid transformation fields for non-overlapping patches are solved via Iterative Closest Point (ICP) by incorporating geometric features as additional similarity constraints, thereby enhancing robustness and accuracy. Secondly, deformation optimization based on a nonrigid solver is applied to refine the coarse transformation fields. In order to enable simultaneous tracking and reconstruction of deformable objects, the patch-based rigid solver is designed to run in parallel with the nonrigid solver, serving as a plug-and-play module requiring minimal modifications for integration while enabling real-time performance. Following a comprehensive evaluation, PatchFusion showcases superior performance in effectively dealing with rapid inter-frame deformations when compared to existing techniques, rendering it a promising solution with broad applicability across domains such as robotics, computer vision, and human-computer interaction.

This research presents the creation of a Virtual Reality experience for the Balzi Rossi museum, commemorating the centenary of Prince Albert I Grimaldi's archaeological work at the site. The project, a collaboration between the University of Genoa's 3D Lab Factory, Eurodrone, and the Balzi Rossi archaeological museum, aims to preserve and convey the site's heritage through advanced VR technology. Initially employing Unreal Engine 4, the team used photogrammetry for 3D reconstruction of the "Caviglione" and "Florestano" caves, notable for their ancient rock engravings. This first VR model, despite hardware limitations, provided an interactive exploration of the site. The subsequent development phase utilized Unreal Engine 5, incorporating Nanite technology for enhanced visual fidelity. This advancement resulted in a more detailed and immersive VR experience, presenting the Balzi Rossi cliff across different historical periods, including the Würm glaciation. Key to this phase was optimizing the VR experience for performance, focusing on stable frame rates and minimizing motion sickness, and integrating a multi-lingual interface for broader accessibility. Since November 2023, the VR setup at the Balzi Rossi Museum has been an educational and interactive feature, enabling visitors to explore the site's history virtually. This research demonstrates the potential of VR in cultural heritage preservation and education, providing a foundation for future technological improvements in the field.

The segmentation of abnormal regions is vital in smart manufacturing. However, the existing segmentation system for detecting sauce-packet leakage on intelligent sensors encounters an issue of imaging blurring caused by uneven illumination. This issue adversely affects segmentation performance, thereby impeding the rapid production of industrial assembly lines. To alleviate this issue, we propose the two-stage Illumination-aware Sauce-packet Leakage Segmentation (ISLS) method for intelligent sensors. The ISLS comprises two main stages: Illumination-aware region enhancement and leakage region segmentation. In the first stage, YOLO-Fastestv2 is employed to capture the Region of Interest (ROI), which reduces redundancy computations. Additionally, we propose an image enhancement to relieve the impact of uneven illumination, enhancing the texture details of ROI. In the second stage, we propose a novel feature extraction network. Specifically, we propose the Multi-scale Feature Fusion Module (MFFM) and the Sequential Self-Attention Mechanism (SSAM) to capture discriminative representations of leakage. The multi-level features are fused by MFFM with a small number of parameters, which capture leakage semantics at different scales. The SSAM realizes the enhancement of valid features and the suppression of invalid features by adaptive weighting of spatial and channel dimensions. Furthermore, we generated a self-built dataset of sauce-packets, including 606 images with various leakage areas. Comprehensive experiments demonstrate that our ISLS method shows better results than several state-of-the-art methods, with additional performance analyses deployed on intelligent sensors to affirm the effectiveness of our proposed method. Our code is available at https://github.com/LSJ5106/SauceDetect.

Abstract: Rationale. Object tracking has significance in many applications ranging from control of unmanned vehicles to autonomous monitoring of specific situations and events, especially when providing safety for patients with certain adverse conditions such as epileptic seizures. Conventional tracking methods face many challenges, such as the need for dedicated attached devices or tags, influence by high image noise, complex object movements, and intensive com-putational requirements. We have developed earlier computationally efficient algorithms for global optical flow reconstruction of group velocities that provide means for convulsive seizure detection and have potential applications in fall and apnea detection. Here, we address the chal-lenge of using the same calculated group velocities for object tracking in parallel. Methods. We propose a novel optical flow-based method for object tracking. It utilizes real-time image se-quences from the camera and directly reconstructs global motion-group parameters of the con-tent. These parameters can steer a rectangular region of interest surrounding the moving object to follow the target. The method successfully applies to multi-spectral data, further improving its effectiveness. Results. Experimental results on simulated tests and complex real-world data demonstrate the method's capabilities. The proposed optical flow reconstruction can provide accurate, robust, and faster results compared to current state-of-the-art approaches.

Cloud-induced atmospheric extinction and occlusion have a major effect on the effectiveness and quality of telescope observations. Real-time cloud cover distribution and long-term statistical data are essential for astronomical siting and telescope operations. At ground-based astronomical telescope sites, cloud cover distribution is currently analyzed using manual observation methods. However, the main disadvantages of manual observation methods are human subjective, heavy workloads and poor real-time performance. Therefore, a real-time automatic cloud images classification method is desperately needed. This paper presents a new cloud identification method, which is named PSO+XGBoost model by combining eXtreme Gradient Boosting (XGBoost) and Particle Swarm Optimization (PSO). The entire cloud image is divided into 37 sub-regions in order to more precisely identify the distribution of the clouds. 19 features are then extracted, including the sky background, star density, lighting conditions, and subregion grayscale values. The experimental results have shown that the overall classification accuracy of 96.91% and our model is able to outperform several state-of-the-art baseline methods. Our approach achieves high accuracy in comparison with the manual observation methods. Moreover, this method meets telescope real-time scheduling requirements.

The research of object classification and part segmentation is a hot topic in computer vision. A considerable number of studies have been carried out about deep learning on 3D point clouds. However,it is challenging to achieve effective feature learning due to sparsity of point clouds. Recently, a variety of Transformers have been adopted to improve point cloud processing and display great potential. Nevertheless, large numbers of Transformer layers tend to incur huge computational and memory costs. PointNet++ is one of the most influential neural architectures for point cloud understanding. Although the accuracy of PointNet++ has been largely surpassed by recent networks, this does not mean that PointNet++ has no potential.Thus, this paper offer two major contributions that significantly improve PointNet++ performance. Firstly, we introduce a novel contextual feature extraction (CFE) block that significantly enhances the feature extraction capabilities of PointNet++ networks. Secondly, to further enhance feature fusion, we seamlessly integrate a mutual learning (ML) block into the network architecture. By embedding these two innovative blocks within each layer of the network, we not only enrich the network's functionality but also impart it with greater robustness and adaptability. The specific experiments were conducted on the S3DIS (6-fold cross-validation) and Modelnet40 datasets with 86.5% and 92.7% accuracy, respectively, which proved that our model is comparable or even better than most existing methods for classification and segmentation tasks.

Accurate classification of objects in 3D point clouds is a significant problem in several applications, such as autonomous navigation and augmented/virtual reality scenarios, which has become a research hot spot. In this paper, we presented a deep learning strategy for 3D object classification in augmented reality. The proposed approach is a combination of the GRU and LSTM. LSTM networks learn longer dependencies well, but due to the number of gates, it takes longer to train; on the other hand, GRU networks have a weaker performance than LSTM, but their training speed is much higher than GRU, which is The speed is due to its fewer gates. The proposed approach used the combination of speed and accuracy of these two networks. The proposed approach achieved an accuracy of 0.99 in the 4,499,0641 points dataset, which includes eight classes (unlabeled, man-made terrain, natural terrain, high vegetation, low vegetation, buildings, hardscape, scanning artifacts, cars). Meanwhile, the traditional machine learning approaches could achieve a maximum accuracy of 0.9489 in the best case.

As technology and multimedia production have advanced, there has been a significant rise in attacks on digital media, resulting in duplicated, fraudulent, and altered data and the infringement of copyright laws. This paper presents a robust and secure digital image watermarking technique that has been implemented in the spatial domain and exploits the erratic and chaotic behaviour of the powerful elementary cellular automata rule-30. The crucial characteristics of the watermarking system, i.e., imperceptibility, capacity, and robustness, have been perfectly balanced by the suggested blind watermarking technique. In this approach, prior to embedding, the grayscale watermark image is downsized to its two Most Significant Bits (MSBs). Then, the 2-MSBs watermark is encrypted using an ECA rule-30 so as to level up the security attribute of the system. Then, the host image is scrambled using ECA rule-30 to distribute the watermark pixels throughout the host image and thus achieve the highest robustness against geometrical attacks. Finally, the encrypted watermark data is embedded into the scrambled host image using the ECA rule-30-based embedding key. The proposed method performs better in terms of imperceptibility, capacity, and robustness when compared to several systems with similar competencies. The simulation's findings demonstrate strong imperceptibility as evaluated by the Peak Signal-to-Noise Ratio (PSNR), which has an average value of 58.3735 dB and a high payload. The experimental outcomes, observed across a diverse range of standardized attack scenarios, unequivocally establish the ascendancy of the proposed algorithm over competing methodologies in the realm of image watermarking.

Image stitching is a crucial aspect of image processing. However, factors like perspective and environment often lead to irregular shapes in stitched images. Cropping or completion methods typically result in substantial loss of information. This paper proposes a method for rectifying irregularly images into rectangles using deformable meshes and residual networks. The method utilizes a convolutional neural network to quantify rigid structures of images. Choosing the most suitable mesh structure based on the extraction results, offering options such as triangular, rectangular, and hexagonal. Subsequently, the irregularly image, predefined mesh structure, and predicted mesh structure are input into a wide residual neural network for regression. The loss function comprises local and global, aimed at minimizing the loss of image information within the mesh and global target. This method not only significantly reduces information loss during rectification but also adapting to different images with various rigid structures. Validation on the DIR-D dataset shows this method outperforms state-of-the-art methods in image rectification.

In the era of Artificial Intelligence (AI), comprehending and responding to non-verbal communication is increasingly vital. This research extends AI's reach in bridging communication gaps, notably benefiting American Sign Language (ASL) and Taiwan Sign Language (TSL) communities. It focuses on employing various AI models, especially the Hierarchical Vision Transformer with Shifted Windows (Swin), for recognizing diverse sign language datasets. The study assesses Swin architecture's adaptability to different sign languages, aiming to create a universal platform for Unvoiced communities. Utilizing deep learning and transformer technologies, hybrid application prototypes have been developed for ASL-to-English translations and vice versa, with plans to expand this to multiple sign languages. The Swin models, trained on varied dataset sizes, show considerable accuracy, indicating their flexibility and effectiveness. This research underscores major advancements in sign language recognition and underlines a commitment to inclusive communication in the digital era. Future work will focus on enhancing these models and broadening their scope to include more sign languages, integrating multimodality and Large Language Models (LLMs), thereby fostering global inclusivity.

Artistic image completion is fundamental for the preservation and restoration of invaluable art paintings and it has experienced significant progress through the implementation of deep learning methodologies. Despite these advancements, challenges persist, even with sophisticated approaches like Generative Adversarial Networks (GANs), particularly in achieving optimal results for high-resolution paintings. Small-scale texture synthesis and the inference of missing information from distant contexts present persistent issues, leading to distortions in lines and unnatural colors, especially in art paintings with complicated structures and textures. Concurrently, patch-based image synthesis has evolved by incorporating global optimization on the image pyramid to enhance structural coherence and details. However, methods relying on gradient-based synthesis encounter obstacles related to directionality, inconsistency, and the heavy computational burdens associated with solving the Poisson equation in non-integrable gradient fields. This paper introduces a groundbreaking approach, integrating Weighted Similarity-Confidence Laplacian Synthesis, to comprehensively address these challenges and advance the field of artistic image completion. This proposal addresses challenges not only in high-resolution artistic image completion but also makes a significant contribution to the broader field of patch-based synthesis by utilizing the Laplacian pyramid for enhanced edge-aware correspondence search. Experimental result confirms the effectiveness of our approach, offering promising outcomes for the preservation and restoration of art paintings with complicated details and irregular missing regions.

A human eye has about 120 million rod cells and 6 million cone cells. This huge number of light sensing cells inside a human eye will continuously produce a huge quantity of visual signals which flow into a human brain for daily processing. However, the real-time processing of these visual signals does not cause any fatigue to a human brain. This fact tells us the truth which is to say that human-like vision processes do not rely on complicated formulas to compute depth, displacement, and colors, etc. On the other hand, a human eye is like a PTZ camera. Here, PTZ stands for pan, tilt and zoom. We all know that in computer vision, each set of PTZ parameters (i.e., coefficients of pan, tilt and zoom) requires a dedicated calibration to determine a camera’s projection matrix. Since there is an infinite number of PTZ parameters which could be produced by a human eye, it is unlikely that a human brain stores an infinite number of calibration matrices for each human eye. Therefore, it is an interesting question for us to answer, which is to say whether simpler formulas of computing depth and displacement exist or not. Moreover, these formulas must be calibration friendly (i.e., easy process on the fly or on the go). In this paper, we disclose an important discovery of a new solution to 3D projection in a human-like binocular vision system. The purpose of doing 3D projection in binocular vision is to undertake forward and inverse transformations (or mappings) between coordinates in 2D digital images and coordinates in a 3D analogue scene. The formulas underlying the new solution are accurate, easily computable, easily tunable (i.e., to be calibrated on the fly or on the go) and could be easily implemented by a neural system (i.e., a network of neurons). Experimental results have validated the discovered formulas.

Object detection is complex and involved diverse requirements for different applications. In critical application such as visual impaired navigation guide and real-time surveillance for instance, identifying a particular object of instance is required and this involved a complex approach for realization. Literatures were reviewed, starting with the application of deep learning techniques for object detection; their gaps were identified and addressed using real-time object detection models literature review. From the review gaps were also detected and addressed using literature review on occlusion detection techniques. Object of instance detection in clustered scene with similar object of interest was identified as an open gap not addressed in the existing literature; even through it is vital for many applications. This research recommended an Occlusion Based Object of Instance Detection (OBOID) technique which used the spatial information to identify instant object of interest in clustered scene with many similarly object of interest. Limitation of the recommended OBOID is that it requires only system where position and distance of object is necessary to inform other decision.

In the past few decades, the transmission of data over an unsecure channel resulted in an increased rate of hacking. Therefore, the requirement to make multimedia data more secure is increasing day by day. Numerous algorithms are developed to improve efficiency and robustness. In this article, a novel and secure image encryption algorithm is presented. It is based on a modified chaotic logistic map (CLM) that provides the advantage of having less computational time to encrypt an input image. The encryption algorithm is based on Shannon’s idea of using substitution-permutation and one-time-pad network to achieve ideal secrecy. The CLM is used for substitution and permutation to improve randomness and increase dependency on the encryption key. Various statistical tests are conducted, such as Key Space Analysis, Complexity Analysis, Sensitivity Analysis, Strict Avalanche Criteria (SAC), Histogram Analysis, Entropy Analysis, Mean of Absolute Deviation (MAD) Analysis, Correlation Analysis, Contrast Analysis, and Homogeneity to give a comparative analysis of the proposed algorithm and verify its security. As a result of various statistical tests, it is evident that the proposed algorithm is more efficient and robust as compared to previous ones.

In recent years, researches on combining wavelet decomposition and convolutional neural network (CNN) together to classify hyperspectral images (HSI) have emerged, and some effective classification models have been proposed and achieved good classification results. However, there are two problems for most of the proposed models. One is the heavy training parameters and the other is that there is no distinction between the classification effectiveness of low frequency and high frequency features after wavelet decomposition. In this paper, a new light-weighted HSI classification model (LLFWCNN) is proposed, which performs multi-layer wavelet decomposition for HSI after dimensionality reduction, and only arranges the low frequency features in a specific stack mode, then classifies them through a well-designed convolutional neural network. Compared with other classification models, the number of parameters is only 2.5% of that of other models. And only the low frequency features after wavelet decomposition are used, while the high frequency features are abandoned. The results showed that compared with the state-of-the-art classification models, LLFWCNN could obtain the same or even better classification results with fewer network parameters and proved that the low frequency features after wavelet decomposition provide LLFWCNN with more favorable information for HSI classification as well.

Infrared small target detection (ISTD) plays a crucial role in both civilian and military applications. The detection of small targets against dense cluttered backgrounds remains a challenging task, requiring the collaboration of false alarm source elimination and target detection. Existing approaches primarily focus on modeling targets while often neglecting false alarm sources. To address this limitation, we propose a Target and False Alarm Collaborative Detection Network to leverage the information provided by false alarm sources and the background. Firstly, we introduce a False Alarm Source Estimation Block (FEB) that estimates potential interferences present in the background by extracting features on multiple scales and employing stepwise upsampling for feature fusion. Subsequently, we propose a framework that employs multiple FEBs to eliminate false alarm sources across multiple scales. Finally, a Target Segmentation Block (TSB) is introduced to accurately segment the targets to produce the final detection result. Experiments on public datasets demonstrate that, compared to other methods, our model achieves high accuracy in segmenting targets while being able to extract false alarm sources which can be used for further studies.

With the continuous development of deep learning, the application of object detection based on deep neural network in the coal mine has been expanding. Simultaneously, as the production applications demand higher recognition accuracy, most research chooses to enlarge the depth and parameters of the network to improve accuracy. However, due to the limited computing resources in the coal mining face, it is challenging to meet the computation demands of a large number of hardware resources. Therefore, this paper proposes a lightweight object detection algorithm designed specifically for coal mining face, referred to as CM-YOLOv8. The algorithm introduces adaptive predefined anchor boxes tailored to the coal mining face dataset to enhance the detection performance of various targets. Simultaneously, a pruning method based on the L1 norm is designed, significantly compressing the model's computation and parameter volume without compromising accuracy. The proposed algorithm is validated on the coal mining dataset DsLMF+, achieving a compression rate of 40% on the model volume with less than a 1% drop in accuracy. Comparative analysis with other existing algorithms demonstrates its efficiency and practicality in coal mining scenarios. The experiments confirm that CM-YOLOv8 significantly reduces the model's computational requirements and volume while maintaining high accuracy.

In emergency situations, every second counts for an ambulance navigating through traffic. Efficient use of traffic light systems can play a crucial role in minimizing response time. This paper introduces a novel automated Real-Time Ambulance in an Emergency Detector (RTAIAED). The proposed system uses special Lookout Stations (LS) suitably positioned at a certain distance from each involved traffic light (TL), to obtain timely and safe transitions to green lights as the Ambulance in an Emergency (AIAE) approaches. The RTAIAED is particularly pertinent on one-way roads, addressing the challenge of regulating the sequence of traffic-lights-signals so as to minimize the time needed to safely grant the green signal to the AIAE. The proposed solution leverages a part-based model made of elementary detectors for video analysis, specifically realized with a customized YOLOv8 model, and for audio analysis, thanks to an additional neural network based on Mel Frequency Cepstral Coefficients (MFCCs). This way, the RTAIAED ensures the accurate and robust identification of an AIAE heading towards a traffic light in time to ensure a green light thanks to the strategic positioning of the LS detectors. Extensive experiments demonstrate the robustness of the approach and its reliable application in real-world scenarios thanks to its predictions in real-time, showcasing the ability to detect AIAEs even in challenging conditions such as noisy environments, nighttime, or adverse weather conditions, provided a suitable-quality camera is appropriately positioned. The proposed system can also find application in traffic flow management.

When it comes to financial transactions and counterfeit detection systems, banknote classification is crucial. In this paper, we propose an approach for the classification of Kazakhstan banknote images integrating Learning Vector Quantization (LVQ) with Statistical Texture Feature Extraction using the Gray Level Co-occurrence matrix (GLCM). Our methodology demonstrates effectiveness in accurately classifying banknote images, as evidenced by experimental results. The comprehensive testing scenarios show promising outcomes, with the combination of GLCM and Color Histogram under the LVQ algorithm achieving a high accuracy of 94.87 percent at distance 1 and 90°. These findings indicate the robustness and practical viability of the proposed method for authenticating banknotes.

Image stitching is a crucial aspect of image processing. However, factors like perspective and environment often lead to irregular shapes in stitched images. Cropping or completion methods typically result in substantial loss of information. This paper proposes a method for rectifying irregularly images into rectangles using deformable meshes and residual networks. The method utilizes a convolutional neural network to quantify rigid structures of images. Choosing the most suitable mesh structure based on the extraction results, offering options such as triangular, rectangular, and hexagonal. Subsequently, the irregularly image, predefined mesh structure, and predicted mesh structure are input into a wide residual neural network for regression. The loss function comprises local and global, aimed at minimizing the loss of image information within the mesh and global target. This method not only significantly reduces information loss during rectification but also adapting to different images with various rigid structures. Validation on the DIR-D dataset shows this method outperforms state-of-the-art methods in image rectification.

This study presents a novel approach for automating the body condition scoring of dairy cows, leveraging advancements in 3D imaging technology and deep neural networks. The primary objective was to design and implement a system capable of accurately and efficiently assessing the body condition of dairy cows, a critical metric in livestock management for optimizing health and productivity. To achieve this, a 3D camera was employed to capture detailed point cloud data, reconstructing the three-dimensional morphology of individual cows. The obtained data were then fed into a deep neural network, specifically tailored for the task of ranking body condition. The neural network was trained on a diverse dataset of annotated body condition score representing varying degrees of body condition, ensuring robust performance across different physiological states. The results demonstrate the efficacy of the proposed system in automatically and objectively scoring the body condition of dairy cows. The automated process not only expedites the assessment but also reduces the subjectivity associated with manual scoring methods. This innovative approach holds promise for improving the efficiency of dairy farm management by providing timely and accurate body condition assessments. The integration of 3D imaging and deep learning techniques paves the way for future advancements in precision livestock farming, contributing to enhanced animal welfare, optimized production, and sustainable agriculture practices.

Reconstructing a 3D model of a scene from a set of multiple views poses a significant challenge in the field of computer vision. The advent of NeRF has marked a major breakthrough in this domain. However, there is a need to extend the capabilities of the NeRF model to enable editing tasks such as relighting and deformation, as well as to enhance its training efficiency. Neural reflectance fields introduce the concept of the reflectance equation into the NeRF framework to achieve relighting of the NeRF model. We leverage the TensoRF approach, which incorporates tensor decomposition and employs multiple tensors to store features, to expedite the training process of the NeRF model. Our novel method, called StensorR, combines the reflectance equation and tensor decomposition within the radiation field model framework. Differing from previous approaches, we employ a single tensor to store scene features and render the surface color of our scene using a simplified reflectance equation. This approach accelerates model convergence and enables relighting of the NeRF model. Experimental results demonstrate that our method achieves a 50% faster convergence rate compared to existing relighting radiation field models, while successfully enabling relighting and improving the quality of synthesized images from new viewpoints.

Despite UAV multi-target detection exhibits considerable developmental potential worldwide, it suffers distinct challenges compared with traditional tasks in this field. These challenges include insufficient feature extraction capabilities for small targets, limited capabilities for multi-dimensional feature fusion, as well as constraints on hardware computation parameters. Especially in mission scenarios such as disaster detection, these challenges will be further amplified. Consequently, this paper introduces SSE-YOLO, an innovative YOLO framework algorithm specially designed to address these challenges. To enhance the model's feature extraction capability, we employ the SPDConv module to replace the original Conv in the backbone section, utilizing depth-separable convolution instead of traditional convolution pooling. Concurrently, we eliminate the SPPF module at the bottom and address a new Separate Kernel Attention Pyramid Pooling (SKAPP) module, substantially enhancing the feature fusion capability at the model's core. Moreover, to address the challenge of multi-dimensional feature fusion, we replace the Concat module of the neck and head with E-BiFPN, transmitting feature information from the backbone to the lower network through four CBS blocks, which effectively resolves the issue of lost contextual information in the model. Meanwhile, SSE-YOLO undergoes ablation experiments on the VisDrone2019 dataset to evaluate its effectiveness against alternative methods, and experimental results illustrate the model's exceptional precision in detecting UAV targets. In comparison to models with comparable experimental accuracy, SSE-YOLO requires remarkably fewer parameters. On the VisDrone2019-DET-test-dev dataset, SSE-YOLO enhances mAP by 17.3%, with a 42.5% reduction in the parameter amounts. Therefore, the proposed method effectively tackles the challenge of reconciling low parameters and high accuracy, presenting a novel pathway for deep learning-based UAV multi-target detection.

Street view images can help us better understand the city environment and potential characteristics. With the development of computer vision and deep learning, the technology of semantic segmentation algorithms has become more mature. However, DeeplabV3+, which is commonly used in semantic segmentation, has shortcomings such as a large number of parameters, high requirements for computing resources, and easy loss of detailed information. Therefore, this paper proposes LM-DeeplabV3+, which aims to greatly reduce parameters and computations of the model while ensuring the segmentation accuracy. Firstly, the lightweight network MobileNetV2 is selected as the backbone network, and the ECA attention mechanism is introduced after MobileNetV2 extracts shallow features to improve the ability of feature representation; secondly, the ASPP module is improved, and on its basis, the EPSA attention mechanism is introduced to achieve cross-dimensional channel attention and important feature interaction; thirdly, a loss function named CL loss is designed to balance the training offset of multiple categories and better indicate the segmentation quality. This paper conducted experimental verification on the Cityspaces dataset, and the results showed that the mIoU reached 74.9%, which was an improvement of 3.56% compared to DeeplabV3+; the mPA reached 83.01%, which was an improvement of 2.53% compared to DeeplabV3+.

We propose Context-aware Feature Transformer Network (CaFTNet), a novel network for human pose estimation. To address the issue of limited modeling of global dependencies in convolutional neural networks, we design Transformerneck to strengthen the expressive power of features. Transformerneck directly substitutes the 3×3 convolution in bottleneck of HRNet with Contextual Transformer (CoT) block, while reducing the complexity of the network. Specifically, CoT first produces keys with static contextual information through 3×3 convolution. Then, relying on the query and contextualization keys, the dynamic contexts are generated through two concatenated 1×1 convolutions. Static and dynamic contexts are eventually fused as an output. Additionally, for the multi-scale networks, in order to further refine the features of the fusion output, we propose an Attention Feature Aggregation Module(AFAM). Technically, given an intermediate input, AFAM successively deduces attention maps along channel and spatial dimensions. Then, Adaptive refinement module(ARM) is exploited to activate the obtained attention maps. Finally, the input undergoes adaptive feature refinement through multiplication with the activated attention maps. Through the above studies, our lightweight network provides a powerful clue for detection of keypoints. Experiments are implemented on the COCO and MPII datasets. The model achieves 76.2 AP on the COCO val2017. Compared to other methods with the CNN as the backbone, CaFTNet reduces the number of parameters by 72.9 %. On the MPII, our method uses only 60.7% of the number of parameters, acquiring semblable results to other methods with the CNN as the backbone.

Global pose refinement is a significant challenge within Simultaneous Localization and Mapping (SLAM) frameworks. For LIDAR-based SLAM systems, pose refinement is integral to correcting drift caused by the successive registration of 3D point clouds collected by the sensor. A divergence between the actual and calculated platform path characterizes this error. In response to this challenge, we propose a linear, parameter-free model, which uses a closed circuit for global trajectory corrections. Our model maps rotations to quaternions and uses Spherical Linear Interpolation (SLERP) for transitions between them. The intervals are established from the constraint set by the Least Squares (LS) method on rotation closure and are proportional to the circuit's size. Translations are globally adjusted in a distinct linear phase. Additionally, we suggest a coarse-to-fine pairwise registration method, integrating Fast Global Registration and Generalized ICP with multiscale sampling and filtering. The proposed approach is tested on three varied datasets of point clouds, including Mobile Laser Scanners and Terrestrial Laser Scanners. These diverse datasets affirm the model's effectiveness in 3D pose estimation with substantial pose differences and efficient pose optimization in larger circuits.

Haze always shrouds satellite images, obscuring valuable geographic information for military surveillance, natural calamity surveillance and mineral resource exploration. Satellite image dehazing (SID) provides the possibility for better applications of satellite images. Most of the existing dehazing methods are tailored for natural images and are not very effective for satellite images with non-homogeneous haze since the semantic structure information and inconsistent attenuation are not fully considered. To tackle this problem, this study proposes a hierarchical semantic-guided contextual structure-aware network (SCSNet) for satellite image dehazing. Specifically, a hybrid CNN-transformer architecture integrated with a hierarchical semantic guidance (HSG) module is presented to learn semantic structure information by synergetically complementing local representation from non-local features. Furthermore, a cross-layer fusion (CLF) module is specially designed to replace the traditional skip connection during the feature decoding stage, so as to reinforce the attention to the spatial regions and feature channels with more serious attenuation. The results on the SateHaze1k, RS-Haze, and RSID datasets demonstrate that the proposed SCSNet can achieve effective dehazing and outperforms existing state-of-the-art methods.

Leveraging the open-world understanding capacity of large-scale visual-language pre-trained models has become a hot-spot in point cloud classification. Recent approaches rely on transferable visual-language pre-trained models, classifying point clouds by projecting them into 2D images and evaluating consistency with textual prompts. These methods benefit from the robust open-world understanding capabilities of visual-language pre-trained models and require no additional training. However, they face several challenges summarized as prompt ambiguity, image domain gap, view weights confusion, and feature deviation. In response to these challenges, we propose PointBLIP, a zero-training point cloud classification network based on the recently introduced BLIP-2 visual-language model. PointBLIP is adept at processing similarities between multi-images and multi-prompts. We separately introduce a novel method for point cloud zero-shot and few-shot classification, which involves comparing multiple features to achieve effective classification. Simultaneously, we enhance the input data quality for both the image and text sides of PointBLIP. In point cloud zero-shot classification tasks, we outperform state-of-the-art methods on three benchmark datasets. For few-shot classification tasks, to the best of our knowledge, we present the first zero-training few-shot point cloud method, surpassing previous works under the same conditions and showcasing comparable performance to full-training methods.

In recent years, there has been a growing interest in the prediction of facial age due to its diverse applications in fields such as security, entertainment, and healthcare. The current scourge of underaged voting in Nigeria is a problem and this research delves into the realm of facial age prediction by employing four well-known convolutional neural network (CNN) architectures, namely VGG16, ResNet50, Mobile Net, and VGG19 to predict chronological ages from facial pictures of person. The objective is to achieve precise age estimation from facial images, utilizing two datasets: UTKFace and CASIA African Facial Datasets. The results of this investigation are noteworthy. Specifically, the VGG16 model which demonstrated remarkable performance, yielding a Mean Absolute Error (MAE) of 1.76 when applied to the UTK-Face dataset. Additionally, when utilizing Mobile-Net, an unprecedented MAE of 1.10 was achieved for the Casia Africa Face Dataset. Notably, this marks the first instance of employing the dataset for facial age detection with CNNs, and this approach outperformed previous works, yielding the lowest MAE among all the studies reviewed.

Deep learning heavily relies on statistical correlations to drive artificial intelligence (AI) innovations, particularly in computer vision applications like autonomous driving and robotics. However, despite providing a solid foundation for deep learning, these statistical correlations can be vulnerable to unforeseen and uncontrolled factors. The lack of prior knowledge guidance can result in spurious correlations, introducing confounding factors and affecting the model's robustness. To address this challenge, recent research efforts have focused on integrating causal theory into deep learning methodologies. By modelling the inherent and unbiased causal structure, causal theory can potentially mitigate the impact of spurious correlations effectively. Hence, this paper explores the basics of causal methodologies in image classification.

The automated transcription of handwritten characters into a legible output is a multi-faceted process with diverse applications. In this paper, a novel approach to optical character recognition (OCR) for handwritten digits is proposed that, in certain components, exceeds current architectures in terms of accuracy, effectiveness, adjustability, temporal efficiency, and/or computational simplicity. This model succeeds in the adoption and enhancement of deprecated or obsolete algorithms across eight steps of image pre-processing—normalization, grayscaling, thresholding/binarization, noise removal, skew-correction, skeletonization/thinning, line separation, and character segmentation. The aforementioned model is evaluated with the use of a Convolutional Neural Network (CNN) leveraging the Balanced eMNIST dataset for training and testing. By suggesting contour-based feature extraction methods as alternatives to pixel-by-pixel iteration, the proposed paradigm demonstrates its capacity to serve as a suitable alternative to current, commonly used algorithms and computational techniques for textual image classification.

Accurate geometric modeling of blood vessels lumen given their 3D images is a basis for vessel quantification in support of cardiovascular diseases diagnosis, treatment and monitoring. Unlike other approaches, which assume circular or elliptical vessel cross-section, the proposed method employs parametric B-splines combined with the image formation system equations to accurately localize the highly curved lumen boundaries. No image segmentation is involved which might reduce the localization accuracy in effect of spatial discretization. It is shown that the model parameters can be reliably identified by a feedforward neural network which, driven by the cross-section images, predicts the parameter values many times faster than a reference least-squares (LS) model fitting algorithm. Two example applications: modeling the lower extremities artery-vein complexes visualized in steady-state contrast-enhanced magnetic resonance images (MRI) and the coronary arteries pictured in computed tomography angiograms (CTA), are presented and discussed to demonstrate the method potential. Besides application to medical diagnosis, blood-flow simulation and vessel-phantom design, it can serve as a tool for automated annotation of image datasets required to train machine-learning algorithms.

Person Re-identification is the task of recognizing comparable subjects across a network of nonoverlapping cameras. This is typically achieved by extracting from the source image a vector of characteristic features of the specific person captured by the camera. Learning a good set of robust, invariant and discriminative features is a complex task, often leveraging contrastive learning. In this article, we explore a different approach, learning the representation of an individual as the conditioning information required to generate images of the specific person starting from random noise. In this way we decouple the identity of the individual from any other information relative to a specific instance (pose, background, etc.), allowing interesting transformations from one identity to another. As generative models, we use the recent diffusion models that have already proven their sensibility to conditioning in many different contexts. The results presented in this article serve as a proof-of-concept. While our current performance on common benchmarks is lower than state-of-the-art techniques, the approach is intriguing and rich of innovative insights, suggesting a wide range of potential improvements along various lines of investigation.

Neural radiance fields (NeRF) leverage a neural representation to encode scenes, obtaining photo-realistic rendering of novel views. However, NeRF has notable limitations. A significant drawback is that it does not capture surface geometry and only renders the object surface colors. Furthermore, the training of NeRF is exceedingly time-consuming. We propose Depth-NeRF as a solution to these issues. Specifically, our approach employs a fast depth completion algorithm to denoise and complete the depth maps generated by RGB-D cameras. These improved depth maps guide the sampling points of NeRF to be distributed closer to the scene's surface, benefiting from dense depth information. Furthermore, we have optimized the network structure of NeRF and integrated depth information to constrain the optimization process, ensuring that the termination distribution of the ray is consistent with the scene’s geometry. Compared to NeRF, our method accelerates the training speed by 18% and significantly reduces the RMSE between the rendered scene depth and the ground truth depth, which indicates that our method can better capture the geometric information of the scene. With these improvements, we can train the NeRF model more efficiently and achieve more accurate rendering results.