research-article

OraclePoints: A Hybrid Neural Representation for Oracle Character

Authors:

Yahong HanAuthors Info & Claims

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Pages 7901 - 7911

https://doi.org/10.1145/3581783.3612534

Published: 27 October 2023 Publication History

Abstract

Oracle Bone Inscriptions (OBI) are ancient hieroglyphs originated in China and are considered one of the most famous writing systems in the world. Up to now, thousands of OBIs have been discovered, which require deciphering by experts to understand their contents. Experts typically need to restore, classify, and compare each character with previous inscriptions. Although existing research can assist with one of these operations, their performance falls short of practical requirements. In this work, we propose the OraclePoints framework, which represents OBI images as hybrid neural representations comprising features of images and point sets. The image representation provides inscription appearance and character structure, while the point representation makes it easy and effective to distinguish characters and noises. In addition, we demonstrate that OraclePoints can be easily integrated with existing models in a plug-and-play manner. Comprehensive experiments demonstrate that the proposed hybrid neural representation framework supports a range of OBI tasks, including character image retrieval, recognition, and denoising. It is also demonstrated that OraclePoints is helpful for deciphering OBIs by linking ancient characters to modern Chinese characters. Our codes are available at https://ddghjikle.github.io/.

References

[1]

Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Süsstrunk. 2012. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE transactions on pattern analysis and machine intelligence 34, 11 (2012), 2274--2282.

Digital Library

[2]

Xiang Chang, Fei Chao, Changjing Shang, and Qiang Shen. 2022. Sundial-GAN: A Cascade Generative Adversarial Networks Framework for Deciphering Oracle Bone Inscriptions. In Proceedings of the 30th ACM International Conference on Multimedia. 1195--1203.

Digital Library

[3]

Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao. 2021. Pre-trained image processing transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12299--12310.

[4]

Xinyang Chen, Sinan Wang, Mingsheng Long, and Jianmin Wang. 2019. Transferability vs. discriminability: Batch spectral penalization for adversarial domain adaptation. In International conference on machine learning. PMLR, 1081--1090.

[5]

Yang Chi, Fausto Giunchiglia, Daqian Shi, Xiaolei Diao, Chuntao Li, and Hao Xu. 2022. ZiNet: Linking Chinese Characters Spanning Three Thousand Years. In Findings of the Association for Computational Linguistics: ACL 2022. 3061--3070.

[6]

Shuhao Cui, Shuhui Wang, Junbao Zhuo, Chi Su, Qingming Huang, and Qi Tian. 2020. Gradually vanishing bridge for adversarial domain adaptation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 12455--12464.

[7]

Aram Danielyan, Vladimir Katkovnik, and Karen Egiazarian. 2011. BM3D frames and variational image deblurring. IEEE Transactions on image processing 21, 4 (2011), 1715--1728.

Digital Library

[8]

Shu Feng. 2019. A novel variational model for noise robust document image binarization. Neurocomputing 325 (2019), 288--302.

[9]

Rowan K Flad. 2008. Divination and power: a multiregional view of the development of oracle bone divination in early China. Current Anthropology 49, 3 (2008), 403--437.

[10]

Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In International conference on machine learning. PMLR, 1180--1189.

[11]

S Gu. 2016. Identification of oracle-bone script fonts based on topological registration. Computer & Digital Engineering 10 (2016), 029.

[12]

Jun Guo, Changhu Wang, Edgar Roman-Rangel, Hongyang Chao, and Yong Rui. 2015. Building hierarchical representations for oracle character and sketch recognition. IEEE Transactions on Image Processing 25, 1 (2015), 104--118.

Digital Library

[13]

Shi Guo, Zifei Yan, Kai Zhang, Wangmeng Zuo, and Lei Zhang. 2019. Toward convolutional blind denoising of real photographs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 1712--1722.

[14]

Jiantang Han. 2012. Chinese characters. Cambridge University Press.

[15]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[16]

Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7132--7141.

[17]

Zixuan Huang and Yin Li. 2020. Interpretable and accurate fine-grained recognition via region grouping. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8662--8672.

[18]

Zhi-Kai Huang, Zhi-Hong Li, Han Huang, Zhi-Biao Li, and Ling-Ying Hou. 2016. Comparison of different image denoising algorithms for Chinese calligraphy images. Neurocomputing 188 (2016), 102--112.

[19]

Hee Jae Jun, Byungsoo Ko, Youngjoon Kim, Insik Kim, and Jongtack Kim. 2019. Combination of multiple global descriptors for image retrieval. arXiv preprint arXiv:1903.10663 (2019).

[20]

David N Keightley. 1997. Graphs, words, and meanings: Three reference works for Shang oracle-bone studies, with an excursus on the religious role of the day or sun.

[21]

Yang Liu, Zhenyue Qin, Saeed Anwar, Pan Ji, Dongwoo Kim, Sabrina Caldwell, and Tom Gedeon. 2021. Invertible denoising network: A light solution for real noise removal. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 13365--13374.

[22]

Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. 2015. Learning transferable features with deep adaptation networks. In International conference on machine learning. PMLR, 97--105.

[23]

Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. 2018. Conditional adversarial domain adaptation. Advances in neural information processing systems 31 (2018).

[24]

Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017).

[25]

Xu Ma, Yuqian Zhou, Huan Wang, Can Qin, Bin Sun, Chang Liu, and Yun Fu. 2023. Image as Set of Points. In The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=awnvqZja69

[26]

Reyhaneh Neshatavar, Mohsen Yavartanoo, Sanghyun Son, and Kyoung Mu Lee. 2022. CVF-SID: Cyclic multi-variate function for self-supervised image denoising by disentangling noise from image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 17583--17591.

[27]

Hao Ren, Ziqiang Zheng, and Hong Lu. 2022. Energy-guided feature fusion for zero-shot sketch-based image retrieval. Neural Processing Letters (2022), 1--10.

[28]

Hao Ren, Ziqiang Zheng, Yang Wu, Hong Lu, Yang Yang, Ying Shan, and Sai-Kit Yeung. 2023. ACNet: Approaching-and-Centralizing Network for Zero-Shot Sketch-Based Image Retrieval. IEEE Transactions on Circuits and Systems for Video Technology (2023).

[29]

Xiaofeng Ren and Jitendra Malik. 2003. Learning a classification model for segmentation. In Computer Vision, IEEE International Conference on, Vol. 2. IEEE Computer Society, 10--10.

[30]

Yuming Shen, Li Liu, Fumin Shen, and Ling Shao. 2018. Zero-shot sketch-image hashing. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3598--3607.

[31]

Daqian Shi, Xiaolei Diao, Lida Shi, Hao Tang, Yang Chi, Chuntao Li, and Hao Xu. 2022. CharFormer: A Glyph Fusion based Attentive Framework for Highprecision Character Image Denoising. In Proceedings of the 30th ACM International Conference on Multimedia. 1147--1155.

Digital Library

[32]

Daqian Shi, Xiaolei Diao, Hao Tang, Xiaomin Li, Hao Xing, and Hao Xu. 2022. RCRN: Real-world Character Image Restoration Network via Skeleton Extraction. In Proceedings of the 30th ACM International Conference on Multimedia. 1177--1185.

Digital Library

[33]

Karen Simonyan and AndrewZisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[34]

Baochen Sun and Kate Saenko. 2016. Deep coral: Correlation alignment for deep domain adaptation. In Computer Vision-ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15--16, 2016, Proceedings, Part III 14. Springer, 443--450.

[35]

Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. 2014. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014).

[36]

Mei Wang, Weihong Deng, and Cheng-Lin Liu. 2022. Unsupervised Structure-Texture Separation Network for Oracle Character Recognition. IEEE Transactions on Image Processing 31 (2022), 3137--3150.

[37]

Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li. 2022. Uformer: A general u-shaped transformer for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 17683--17693.

[38]

Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas Breuel, Jan Kautz, and Xiaolong Wang. 2022. Groupvit: Semantic segmentation emerges from text supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 18134--18144.

[39]

Hong Xuan, Richard Souvenir, and Robert Pless. 2018. Deep randomized ensembles for metric learning. In Proceedings of the European conference on computer vision (ECCV). 723--734.

Digital Library

[40]

Lu Xuzheng, Cai Hengjin, and Lin Li. 2020. Recognition of Oracle radical based on the capsule network. CAAI transactions on intelligent systems 15, 2 (2020), 243--254.

[41]

Jiulong Zhang, Mingtao Guo, and Jianping Fan. 2020. A novel generative adversarial net for calligraphic tablet images denoising. Multimedia Tools and Applications 79 (2020), 119--140.

Digital Library

[42]

Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. 2017. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing 26, 7 (2017), 3142--3155.

Digital Library

[43]

Yaping Zhang, Shuai Nie, Shan Liang, and Wenju Liu. 2021. Robust text image recognition via adversarial sequence-to-sequence domain adaptation. IEEE Transactions on Image Processing 30 (2021), 3922--3933.

[44]

Yi-Kang Zhang, Heng Zhang, Yong-Ge Liu, Qing Yang, and Cheng-Lin Liu. 2019. Oracle character recognition by nearest neighbor classification with deep metric learning. In 2019 International Conference on Document Analysis and Recognition (ICDAR). IEEE, 309--314.

Cited By

Hu ZCheung YZhang YZhang PTang PGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Component-Level Oracle Bone Inscription RetrievalProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658116(647-656)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658116

Index Terms

OraclePoints: A Hybrid Neural Representation for Oracle Character
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
  2. Computer graphics
    1. Image manipulation
      1. Image processing

Recommendations

Dynamic Dataset Augmentation for Deep Learning-based Oracle Bone Inscriptions Recognition
Oracle bone inscriptions (OBIs) are a kind of hieroglyph, used about 3,600 years ago for divination and the recording of events. The characters on these OBIs are of great interest because they are precursors to the modern Chinese characters widely used ...
Deciphering Ancient Chinese Oracle Bone Inscriptions Using Case-Based Reasoning
Case-Based Reasoning Research and Development
Abstract
Ancient oracle bone inscriptions (OBIs) are important Chinese cultural artefacts, which are difficult and time-consuming to decipher even by the most expert paleographers and, as a result, a large proportion of excavated OBIs remain unidentified. ...
Gujarati handwritten numeral optical character reorganization through neural network

This paper deals with an optical character recognition (OCR) system for handwritten Gujarati numbers. One may find so much of work for Indian languages like Hindi, Kannada, Tamil, Bangala, Malayalam, Gurumukhi etc, but Gujarati is a language for which ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

October 2023

9913 pages

ISBN:9798400701085

DOI:10.1145/3581783

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada & MBZUAI, UAE
,
Tao Mei
HiDream.ai, China
,
Rita Cucchiara
University of Modena and Reggio Emilia, Italy
,
Program Chairs:
Marco Bertini
University of Florence, Italy
,
Diana Patricia Tobon Vallejo
Unversidad de Medellin, Colombia
,
Pradeep K. Atrey
University at Albany, State University of New York, USA
,
M. Shamim Hossain
M. Shamim Hossain (King Saud University, KSA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
National Social Science Foundation of China
Ancient Characters and Chinese Civilization Inheritance and Development Project

Conference

MM '23

Sponsor:

SIGMM

MM '23: The 31st ACM International Conference on Multimedia

October 29 - November 3, 2023

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 995 of 4,171 submissions, 24%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
117
Total Downloads

Downloads (Last 12 months)117
Downloads (Last 6 weeks)7

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hu ZCheung YZhang YZhang PTang PGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Component-Level Oracle Bone Inscription RetrievalProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658116(647-656)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658116

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents