Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

A nonlocal feature self-similarity based tensor completion method for video recovery

Published: 02 July 2024 Publication History

Abstract

The nuclear norm-based tensor completion method effectively recovers missing multidimensional data in videos by minimizing the truncated nuclear norm. However, the conventional thresholding approach might overly punish larger singular values, which leads to loss of faithful information. In order to overcome this challenge, an improved model of the truncated nuclear norm-based tensor completion is proposed. This improved model is designed to incorporate information related to the prior rank and ensure the preservation of essential singular values to improve the approximation of the matrix rank. First, a video tensor is decomposed into matrices along different modes, and these matrices are divided into similar block matrices by using the K-means++ clustering method, which utilizes the non-local similarity to reduce the influence of noise in the video. Afterwards, a ranking algorithm based on noise matrix analysis is used, which automatically gets the truncated threshold by iterative optimization. Additionally, we propose back-projection method to balance local and global optimization. Finally, to evaluate our proposal, extensive experimental evaluations have been carried out and show that our approach outperforms a lot of recent tensor completion techniques in terms of quality metrics and visual impact.

References

[1]
Xiang H., Zou Q., Nawaz M.A., Huang X., Zhang F., Yu H., Deep learning for image inpainting: A survey, Pattern Recog. 134 (2023),.
[2]
Nie X., Jing W., Cui C., Zhang C.J., Zhu L., Yin Y., Joint multi-view hashing for large-scale near-duplicate video retrieval, IEEE Trans. Knowl. Data Eng. 32 (10) (2020) 1951–1965,.
[3]
Huang M., Ma S., Lai L., Robust low-rank matrix completion via an alternating manifold proximal gradient continuation method, IEEE Trans. Signal Process. 69 (2021) 2639–2652,.
[4]
Liu Y., Long Z., Huang H., Zhu C., Low CP rank and tucker rank tensor completion for estimating missing components in image data, IEEE Trans. Circuits Syst. Video Technol. 30 (4) (2020) 944–954,.
[5]
Dai C., Liu X., Xu H., Yang L.T., Deen J., Hybrid deep model for human behavior understanding on industrial Internet of Video Things, IEEE Trans Ind. Informat. (2021) 1,.
[6]
Long Z., Zhu C., Liu J., Liu Y., Bayesian low rank tensor ring for image recovery, IEEE Trans. Image Process. 30 (2021) 3568–3580,.
[7]
Wang J., Huang T., Zhao X., Jiang T., Ng M.K., Multi-dimensional visual data completion via low-rank tensor representation under coupled transform, IEEE Trans. Image Process. 30 (2021) 3581–3596,.
[8]
Li B., Zhao X., Wang J., Chen Y., Jiang T., Liu J., Tensor completion via collaborative sparse and low-rank transforms, IEEE Trans. Comput. Imaging. 7 (2021) 1289–1303,.
[9]
Liu Y., Long Z., Zhu C., Image completion using low tensor tree rank and total variation minimization, IEEE Trans. Multimedia 21 (2) (2019) 338–350,.
[10]
Liu Y., Long Z., Huang H., Zhu C., Low CP rank and tucker rank tensor completion for estimating missing components in image data, IEEE Trans. Circuits Syst. Video Technol. 30 (4) (2020) 944–954,.
[11]
Zhang Z., Aeron S., Exact tensor completion using t-SVD, IEEE Trans. Signal Process. 65 (6) (2017) 1511–1526,.
[12]
Du S., Xiao Q., Shi Y., Cucchiara R., Ma Y., Unifying tensor factorization and tensor nuclear norm approaches for low-rank tensor completion, Neurocomputing 458 (2021) 204–218,.
[13]
Wang J., Huang T., Zhao X., Jiang T., Ng M.K., Multi-dimensional visual data completion via low-rank tensor representation under coupled transform, IEEE Trans. Image Process. 30 (2021) 3581–3596,.
[14]
Shen L., Yan J., Sun X., Li B., Pan Z., Wavelet-based self-attention GAN with collaborative feature fusion for image inpainting, IEEE Trans. Emerg. Top. Comput. Intell. 7 (6) (2023) 1651–1664,.
[15]
Zhu L., Han Y., Xi X., Li L., Yan B., Completion of metal-damaged traces based on deep learning in sinogram domain for metal artifacts reduction in CT images, Sensors 21 (24) (2021) 8164,.
[16]
Fan Y.-R., Huang T.-Z., Hyperspectral image restoration via superpixel segmentation of smooth band, Neurocomputing 455 (2021) 340–352,.
[17]
He J., Zheng X., Gao P., Zhou Y., Low-rank tensor completion based on tensor train rank with partially overlapped sub-blocks, Signal Process. 190 (2022),.
[18]
Wu Q., Xu A., A biased deep tensor factorization network for tensor completion, 2021, CoRR abs/2105.09629, arXiv:2105.09629.
[19]
Fonał K., Zdunek R., Fast hierarchical tucker decomposition with single-mode preservation and tensor subspace analysis for feature extraction from augmented multimodal data, Neurocomputing 445 (2021) 231–243,.
[20]
Liu Y., Liu J., Zhu C., Low-rank tensor train coefficient array estimation for tensor-on-tensor regression, IEEE Trans. Neural Netw. Learn. Syst. 31 (12) (2020) 5402–5411,.
[21]
X.L. C. Dai, A Tucker Decomposition Based on Adaptive Genetic Algorithm for Efficient Deep Model Compression, in: IEEE 22nd International Conference on High Performance Computing and Communications, Yanuca Island, Cuvu,Fiji, 2020, pp. 507–512, https://doi.org/10.1109/HPCC-SmartCity-DSS50907.2020.00062.
[22]
Ko C.-Y., Batselier K., Daniel L., Yu W., Wong N., Fast and accurate tensor completion with total variation regularized tensor trains, IEEE Trans. Image Process. 29 (2020) 6918–6931,.
[23]
Liu J., Musialski P., Wonka P., Ye J., Tensor completion for estimating missing values in visual data, IEEE Trans. Pattern Anal. Mach. Intell. 35 (1) (2015) 208–220,.
[24]
Li X., Zhao X., Jiang T., Zheng Y., Ji T., Huang T., Low-rank tensor completion via combined non-local self-similarity and low-rank regularization, Neurocomputing 367 (2019) 1–12,.
[25]
Buades A., Coll B., Morel J.-M., Sbert C., Self-similarity driven color demosaicking, IEEE Trans. Image Process. 18 (6) (2009) 1192–1202,.
[26]
Dabov K., Foi A., Katkovnik V., Egiazarian K.O., Image denoising by sparse 3-D transform-domain collaborative filtering, IEEE Trans. Image Process. 16 (8) (2007) 2080–2095,.
[27]
Xie M., Liu X., Yang X., A nonlocal self-similarity-based weighted tensor low-rank decomposition for multichannel image completion with mixture noise, IEEE Trans. Neural Netw. Learn. Syst. (2022) 1–15,.
[28]
Li X.-T., Zhao X.-L., Jiang T.-X., Zheng Y.-B., Ji T.-Y., Huang T.-Z., Low-rank tensor completion via combined non-local self-similarity and low-rank regularization, Neurocomputing 367 (2019) 1–12,.
[29]
Ren X., Yang W., Cheng W.-H., Liu J., LR3M: Robust low-light enhancement via low-rank regularized retinex model, IEEE Trans. Image Process. 29 (2020) 5862–5876,.
[30]
Chen X., Yang J., Sun L., A nonconvex low-rank tensor completion model for spatiotemporal traffic data imputation, 2020, CoRR abs/2003.10271, arXiv:2003.10271.
[31]
Gao Y., Yang L.T., Yang J., Zheng D., Zhao Y., Jointly low-rank tensor completion for estimating missing spatiotemporal values in logistics systems, IEEE Trans. Ind. Inform. 19 (2) (2023) 1814–1822,.
[32]
Wang Y., Yang W., Li D., Zhang J.Q., A novel time-frequency model, analysis and parameter estimation approach: Towards multiple close and crossed chirp modes, Signal Process. 201 (2022),.
[33]
Zhou P., Lu C., Lin Z., Zhang C., Tensor factorization for low-rank tensor completion, IEEE Trans. Image Process. 27 (3) (2018) 1152–1163,.
[34]
Miao J., Kou K.I., Liu W., Low-rank quaternion tensor completion for recovering color videos and images, Pattern Recognit. 107 (2020),.
[35]
Xu Y., Hao R., Yin W., Su Z., Parallel matrix factorization for low-rank tensor completion, 2013, arXiv preprint arXiv:1312.1254.
[36]
Li X., Zhang H., Wang R., Nie F., Multiview clustering: A scalable and parameter-free bipartite graph fusion method, IEEE Trans. Pattern Anal. Mach. Intell. 44 (1) (2022) 330–344,.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Neurocomputing
Neurocomputing  Volume 580, Issue C
May 2024
318 pages

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 02 July 2024

Author Tags

  1. Nuclear norm-based tensor completion
  2. Nonlocal similarity
  3. Video recovery
  4. Truncated nuclear norm minimization
  5. Rank determination

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 0
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 13 Sep 2024

Other Metrics

Citations

View Options

View options

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media