research-article

Protecting Intellectual Property of Large Language Model-Based Code Generation APIs via Watermarks

Authors:

Chaozheng Wang,

Cuiyun GaoAuthors Info & Claims

CCS '23: Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security

Pages 2336 - 2350

https://doi.org/10.1145/3576915.3623120

Published: 21 November 2023 Publication History

Abstract

The rise of large language model-based code generation (LLCG) has enabled various commercial services and APIs. Training LLCG models is often expensive and time-consuming, and the training data are often large-scale and even inaccessible to the public. As a result, the risk of intellectual property (IP) theft over the LLCG models (e.g., via imitation attacks) has been a serious concern. In this paper, we propose the first watermark (WM) technique to protect LLCG APIs from remote imitation attacks. Our proposed technique is based on replacing tokens in an LLCG output with their "synonyms" available in the programming language. A WM is thus defined as the stealthily tweaked distribution among token synonyms in LLCG outputs. We design six WM schemes (instantiated into over 30 WM passes) which rely on conceptually distinct token synonyms available in programming languages. Moreover, to check the IP of a suspicious model (decide if it is stolen from our protected LLCG API), we propose a statistical tests-based procedure that can directly check a remote, suspicious LLCG API.

We evaluate our WM technique on LLCG models fine-tuned from two popular large language models, CodeT5 and CodeBERT. The evaluation shows that our approach is effective in both WM injection and IP check. The inserted WMs do not undermine the usage of normal users (i.e., high fidelity) and incur negligible extra cost. Moreover, our injected WMs exhibit high stealthiness and robustness against powerful attackers; even if they know all WM schemes, they can hardly remove WMs without largely undermining the accuracy of their stolen models.

References

[1]

[n. d.]. billing for AI21 LAB. https://studio.ai21.com/pricing.

[2]

[n. d.]. billing for aixcoder. shorturl.at/evxzA.

[3]

[n. d.]. billing for amazon p2. https://aws.amazon.com/ec2/instance-types/p2/.

[4]

[n. d.]. billing for GitHub Copilot. https://docs.github.com/en/billing/managing-billing-for-github-copilot/about-billing-for-github-copilot.

[5]

[n. d.]. Codex. https://openai.com/blog/openai-codex/.

[6]

[n. d.]. Copilot. https://github.com/features/copilot.

[7]

[n. d.]. deeptabnine. shorturl.at/bS049.

[8]

[n. d.]. Google Bard steal. https://www.theinformation.com/articles/alphabets-google-and-deepmind-pause-grudges-join-forces-to-chase-openai.

[9]

[n. d.]. JensenShannon divergence. shorturl.at/hAGK2.

[10]

[n. d.]. jurassic. https://www.ai21.com/studio.

[11]

[n. d.]. OpenAI Watermarking prototype. https://scottaaronson.blog/?p=6823.

[12]

[n. d.]. pycode-similar. https://pypi.org/project/pycode-similar/.

[13]

[n. d.]. Pygment. https://pygments.org/.

[14]

[n. d.]. PyPI Website. https://pypi.org/.

[15]

[n. d.]. Semgrep. https://semgrep.dev/.

[16]

[n. d.]. Tabnine. https://www.tabnine.com/.

[17]

[n. d.]. Tabnine-with-Copilot. https://www.tabnine.com/blog/tabnine-vs-github-copilot/.

[18]

[n. d.]. Top ten Python Package. https://interviewbit.com/blog/python-libraries/.

[19]

[n. d.]. Top twenty six Python Package. https://mygreatlearning.com/blog/open-source-python-libraries/.

[20]

Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. 2018. Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In 27th USENIX Security Symposium (USENIX Security. 1615--1631.

[21]

Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2021. Unified pre-training for program understanding and generation. arXiv preprint arXiv:2103.06333 (2021).

[22]

Artifact. 2022. ToSyn. https://sites.google.com/view/tosyn.

[23]

Patrick Bareiß, Beatriz Souza, Marcelo d'Amorim, and Michael Pradel. 2022. Code Generation Tools (Almost) for Free? A Study of Few-Shot, Pre-Trained Language Models on Code. arXiv preprint arXiv:2206.01335 (2022).

[24]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877--1901.

[25]

Max Brunsfeld. [n. d.]. Tree-sitter. https://github.com/tree-sitter/tree-sitter.

[26]

Huili Chen, Bita Darvish Rouhani, Cheng Fu, Jishen Zhao, and Farinaz Koushan-far. 2019. DeepMarks: A Secure Fingerprinting Framework for Digital Rights Management of Deep Learning Models (ICMR '19). New York, NY, USA. https: //doi.org/10.1145/3323873.3325042

Digital Library

[27]

Jialuo Chen, Jingyi Wang, Tinglan Peng, Youcheng Sun, Peng Cheng, Shouling Ji, Xingjun Ma, Bo Li, and Dawn Song. 2022. Copy, Right? A Testing Framework for Copyright Protection of Deep Learning Models. In 43rd IEEE Symposium on Security and Privacy, SP 2022, San Francisco, CA, USA, May 22-26, 2022. IEEE, 824--841.

[28]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021).

[29]

Xuxi Chen, Tianlong Chen, Zhenyu Zhang, and Zhangyang Wang. 2021. You are caught stealing my winning lottery ticket! Making a lottery ticket claim its ownership. Advances in Neural Information Processing Systems 34 (2021), 1780--1791.

[30]

Yufei Chen, Chao Shen, Cong Wang, and Yang Zhang. 2022. Teacher Model Fingerprinting Attacks Against Transfer Learning. In 31st USENIX Security Symposium (USENIX Security 22).

[31]

Yufei Chen, Chao Shen, Cong Wang, and Yang Zhang. 2022. Teacher model finger-printing attacks against transfer learning. In 31st USENIX Security Symposium (USENIX Security 22). 3593--3610.

[32]

Christian Collberg, Clark Thomborson, and Douglas Low. 1998. On the limits of software watermarking. Technical Report. Citeseer.

[33]

Bita Darvish Rouhani, Huili Chen, and Farinaz Koushanfar. 2019. Deepsigns: An end-to-end watermarking framework for ownership protection of deep neural networks. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems. 485--497.

Digital Library

[34]

Robert I Davidson and Nathan Myhrvold. 1996. Method and system for generating and auditing a signature for a computer program. US Patent 5,559,884.

[35]

Ayan Dey, Sukriti Bhattacharya, and Nabendu Chaki. 2019. Software watermarking: Progress and challenges. INAE Letters 4, 1 (2019), 65--75.

[36]

EleutherAI. [n. d.]. GPT-J. https://huggingface.co/EleutherAI/gpt-j-6B.

[37]

Lixin Fan, Kam Woh Ng, Chee Seng Chan, and Qiang Yang. 2021. Deepip: Deep neural network intellectual property protection with passports. IEEE Transactions on Pattern Analysis & Machine Intelligence 01 (2021), 1--1.

[38]

Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In EMNLP Findings.

[39]

Liu Fenlin, Lu Bin, and Luo Xiangyang. 2006. A chaos-based robust software watermarking, information security practice and experience. (2006).

[40]

Python Software Foundation. [n. d.]. black. https://github.com/psf/black.

[41]

Jia Guo and Miodrag Potkonjak. 2018. Watermarking deep neural networks for embedded systems. In Proceedings of the International Conference on Computer-Aided Design, ICCAD 2018, San Diego, CA, USA, November 05-08, 2018, Iris Bahar (Ed.). ACM, 133.

Digital Library

[42]

James Hamilton and Sebastian Danicic. 2011. A survey of static software watermarking. In 2011 World Congress on Internet Security (WorldCIS-2011). IEEE, 100--107.

[43]

Hanieh Hashemi, Yongqin Wang, and Murali Annavaram. 2021. DarKnight: An Accelerated Framework for Privacy and Integrity Preserving Deep Learning Using Trusted Hardware. In MICRO '21: 54th Annual IEEE/ACM International Symposium on Microarchitecture, Virtual Event, Greece, October 18-22, 2021. ACM, 212--224.

[44]

Xuanli He, Qiongkai Xu, Lingjuan Lyu, Fangzhao Wu, and Chenguang Wang. 2022. Protecting Intellectual Property of Language Generation APIs with Lexical Watermark. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22 - March 1, 2022. AAAI Press, 10758--10766.

[45]

Xuanli He, Qiongkai Xu, Yi Zeng, Lingjuan Lyu, Fangzhao Wu, Jiwei Li, and Ruoxi Jia. 2022. CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks. arXiv preprint arXiv:2209.08773 (2022).

[46]

Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the Knowl-edge in a Neural Network. CoRR abs/1503.02531 (2015). arXiv:1503.02531 http://arxiv.org/abs/1503.02531

[47]

Jiahui Hou, Huiqi Liu, Yunxin Liu, Yu Wang, Peng-Jun Wan, and Xiang-Yang Li. 2021. Model Protection: Real-time Privacy-preserving Inference Service for Model Privacy at the Edge. IEEE Transactions on Dependable and Secure Computing (2021).

[48]

Xing Hu, Ling Liang, Shuangchen Li, Lei Deng, Pengfei Zuo, Yu Ji, Xinfeng Xie, Yufei Ding, Chang Liu, Timothy Sherwood, and Yuan Xie. 2020. DeepSniffer: A DNN Model Extraction Framework Based on Learning Architectural Hints. In ASPLOS '20: Architectural Support for Programming Languages and Operating Systems, Lausanne, Switzerland, March 16-20, 2020. ACM, 385--399.

[49]

Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, and Luke Zettlemoyer. 2018. Mapping language to code in programmatic context. arXiv preprint arXiv:1808.09588 (2018).

[50]

Hengrui Jia, Christopher A Choquette-Choo, Varun Chandrasekaran, and Nicolas Papernot. 2021. Entangled watermarks as a defense against model extraction. In 30th USENIX Security Symposium (USENIX Security 21). 1937--1954.

[51]

Yin Ke-xin, Yin Ke, and Zhu Jian-qi. 2009. A robust dynamic software watermarking. In 2009 International Conference on Information Technology and Computer Science, Vol. 1. IEEE, 15--18.

[52]

John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. A watermark for large language models. arXiv preprint arXiv:2301.10226 (2023).

[53]

Kalpesh Krishna, Gaurav Singh Tomar, Ankur P Parikh, Nicolas Papernot, and Mohit Iyyer. 2019. Thieves on sesame street model extraction of bert-based apis. arXiv preprint arXiv:1910.12366 (2019).

[54]

Klemens Lagler, Michael Schindelegger, Johannes Böhm, Hana Krásná, and Tobias Nilsson. 2013. GPT2: Empirical slant delay model for radio space geodetic techniques. Geophysical research letters 40, 6 (2013), 1069--1073.

[55]

Erwan Le Merrer, Patrick Perez, and Gilles Trédan. 2020. Adversarial frontier stitching for remote neural network watermarking. Neural Computing and Applications 32, 13 (2020), 9233--9244.

[56]

Zongjie Li, Pingchuan Ma, Huaijin Wang, Shuai Wang, Qiyi Tang, Sen Nie, and Shi Wu. 2022. Unleashing the Power of Compiler Intermediate Representation to Enhance Neural Program Embeddings. In 44th IEEE/ACM 44th International Conference on Software Engineering, ICSE 2022, Pittsburgh, PA, USA, May 25-27, 2022. ACM, 2253--2265.

Digital Library

[57]

Zongjie Li, Chaozheng Wang, Pingchuan Ma, Chaowei Liu, Shuai Wang, Daoyuan Wu, and Cuiyun Gao. 2023. On the Feasibility of Specialized Ability Extracting for Large Language Code Models. arXiv preprint arXiv:2303.03012 (2023).

[58]

Zheng Li and Yang Zhang. 2021. Membership Leakage in Label-Only Exposures. In CCS '21: 2021 ACM SIGSAC Conference on Computer and Communications Security, Virtual Event, Republic of Korea, November 15 - 19, 2021. ACM, 880--895.

[59]

Jian Han Lim, Chee Seng Chan, Kam Woh Ng, Lixin Fan, and Qiang Yang. 2022. Protect, show, attend and tell: Empowering image captioning models with ownership protection. Pattern Recognition 122 (2022), 108285.

Digital Library

[60]

Aiwei Liu, Leyi Pan, Xuming Hu, Shu'ang Li, Lijie Wen, Irwin King, and Philip S Yu. 2023. A Private Watermark for Large Language Models. arXiv preprint arXiv:2307.16230 (2023).

[61]

Hanwen Liu, Zhenyu Weng, and Yuesheng Zhu. 2021. Watermarking Deep Neural Networks with Greedy Residuals. In ICML. 6978--6988.

[62]

Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al. 2021. Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv preprint arXiv:2102.04664 (2021).

[63]

Fan Mo, Hamed Haddadi, Kleomenis Katevas, Eduard Marin, Diego Perino, and Nicolas Kourtellis. 2021. PPFL: privacy-preserving federated learning with trusted execution environments. In MobiSys '21: The 19th Annual International Conference on Mobile Systems, Applications, and Services, Virtual Event, Wisconsin, USA, 24 June - 2 July, 2021. ACM, 94--108.

Digital Library

[64]

Fan Mo, Ali Shahin Shamsabadi, Kleomenis Katevas, Soteris Demetriou, Ilias Leontiadis, Andrea Cavallaro, and Hamed Haddadi. 2020. DarkneTZ: towards model privacy at the edge using trusted execution environments. In MobiSys '20: The 18th Annual International Conference on Mobile Systems, Applications, and Services, Toronto, Ontario, Canada, June 15-19, 2020. ACM, 161--174.

Digital Library

[65]

Jasvir Nagra, Clark Thomborson, and Christian Collberg. 2002. A functional taxonomy for software watermarking. In ACSC, Vol. 4. 177--186.

[66]

OpenAI. [n. d.]. ChatGPT. https://openai.com/blog/chatgpt/.

[67]

Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. 2019. Knockoff Nets: Stealing Functionality of Black-Box Models. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16--20, 2019. Computer Vision Foundation / IEEE, 4954--4963.

[68]

Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. 2019. Knockoff Nets: Stealing Functionality of Black-Box Models. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. Computer Vision Foundation / IEEE, 4954--4963.

[69]

Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. 2020. Prediction Poison-ing: Towards Defenses Against DNN Model Stealing Attacks. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.

[70]

Soham Pal, Yash Gupta, Aditya Shukla, Aditya Kanade, Shirish K. Shevade, and Vinod Ganapathy. 2019. A framework for the extraction of Deep Neural Networks by leveraging public data. CoRR abs/1905.09165 (2019).

[71]

Jens Palsberg, Sowmya Krishnaswamy, Minseok Kwon, Di Ma, Qiuyun Shao, and Yi Zhang. 2000. Experience with software watermarking. In Proceedings 16th Annual Computer Security Applications Conference (ACSAC'00). IEEE, 308--316.

[72]

Kishore Papineni, Salim Roukos, Todd Ward, and Wei jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. 311--318.

[73]

Md Rizwan Parvez, Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2021. Retrieval Augmented Code Generation and Summarization. arXiv preprint arXiv:2108.11601 (2021).

[74]

Python. 2021. The Python Language Reference. https://docs.python.org/3/ reference/index.html.

[75]

Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun. ONION: A Simple and Effective Defense Against Textual Backdoor Attacks. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.

[76]

Gang Qu and Miodrag Potkonjak. 1998. Analysis of watermarking techniques for graph coloring problem. In Proceedings of the 1998 IEEE/ACM international conference on Computer-aided design. 190--193.

Digital Library

[77]

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.

[78]

Adnan Siraj Rakin, Md Hafizul Islam Chowdhuryy, Fan Yao, and Deliang Fan. 2022. DeepSteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memories. In 43rd IEEE Symposium on Security and Privacy, SP 2022, San Francisco, CA, USA, May 22-26, 2022. IEEE, 1157--1174.

[79]

Alexander Schlögl and Rainer Böhme. 2020. eNNclave: Offline Inference with Model Confidentiality. In AISec@CCS 2020: Proceedings of the 13th ACM Workshop on Artificial Intelligence and Security, Virtual Event, USA, 13 November 2020. ACM, 93--104.

[80]

Tianxiang Shen, Ji Qi, Jianyu Jiang, Xian Wang, Siyuan Wen, Xusheng Chen, Shixiong Zhao, Sen Wang, Li Chen, Xiapu Luo, et al. 2022. SOTER: Guarding Black-box Inference for General Neural Networks at the Edge. In 2022 USENIX Annual Technical Conference (USENIX ATC 22). 723--738.

[81]

Youren Shen, Hongliang Tian, Yu Chen, Kang Chen, Runji Wang, Yi Xu, Yubin Xia, and Shoumeng Yan. 2020. Occlum: Secure and Efficient Multitasking Inside a Single Enclave of Intel SGX. In ASPLOS '20: Architectural Support for Programming Languages and Operating Systems, Lausanne, Switzerland, March 16-20, 2020. ACM, 955--970.

[82]

Julien P Stern, Gael Hachez, Francois Koeune, and Jean-Jacques Quisquater. 1999. Robust object watermarking: Application to code. In International Workshop on Information Hiding. Springer, 368--378.

[83]

Zhichuang Sun, Ruimin Sun, Long Lu, and Somesh Jha. 2020. ShadowNet: A Secure and Efficient System for On-device Model Inference. CoRR abs/2011.05905 (2020).

[84]

Sebastian Szyller, Buse Gul Atli, Samuel Marchal, and N Asokan. 2021. Dawn: Dynamic adversarial watermarking of neural networks. In Proceedings of the 29th ACM International Conference on Multimedia. 4417--4425.

Digital Library

[85]

Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. 2020. Efficient transformers: A survey. ACM Computing Surveys (CSUR) (2020).

[86]

Florian Tramèr and Dan Boneh. 2019. Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware. In ICLR.

[87]

Eric Wallace, Mitchell Stern, and Dawn Song. 2020. Imitation attacks and defenses for black-box machine translation systems. arXiv preprint arXiv:2004.15015 (2020).

[88]

Bolun Wang, Yuanshun Yao, Bimal Viswanath, Haitao Zheng, and Ben Y. Zhao. 2018. With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning. In 27th USENIX Security Symposium, USENIX Security 2018, Baltimore, MD, USA, August 15-17, 2018. USENIX Association, 1281--1297.

[89]

Chaozheng Wang, Yuanhang Yang, Cuiyun Gao, Yun Peng, Hongyu Zhang, and Michael R Lyu. 2022. No more fine-tuning? an experimental evaluation of prompt tuning in code intelligence. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 382--394.

Digital Library

[90]

Jialai Wang, Han Qiu, Yi Rong, Hengkai Ye, Qi Li, Zongpeng Li, and Chao Zhang. 2022. BET: black-box efficient testing for convolutional neural networks. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis. 164--175.

Digital Library

[91]

Tianhao Wang and Florian Kerschbaum. 2019. Robust and undetectable white-box watermarks for deep neural networks. arXiv preprint arXiv:1910.14268 1, 2 (2019).

[92]

Tianhao Wang and Florian Kerschbaum. 2021. Riga: Covert and robust white-box watermarking of deep neural networks. In Proceedings of the Web Conference 2021. 993--1004.

Digital Library

[93]

Yue Wang, Weishi Wang, Shafiq Joty, and Steven CH Hoi. 2021. Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. arXiv preprint arXiv:2109.00859 (2021).

[94]

Bolin Wei, Ge Li, Xin Xia, Zhiyi Fu, and Zhi Jin. [n. d.]. Code Generation as a Dual Task of Code Summarization. CoRR abs/1910.05923 ([n. d.]).

[95]

Wiki. [n. d.]. Salt. https://en.wikipedia.org/wiki/Salt_(cryptography).

[96]

Yecheng Xiang, Yidi Wang, Hyunjong Choi, Mohsen Karimi, and Hyoseung Kim. 2021. AegisDNN: Dependable and Timely Execution of DNN Tasks with SGX. In 42nd IEEE Real-Time Systems Symposium, RTSS 2021, Dortmund, Germany, December 7-10, 2021. IEEE, 68--81.

[97]

Qiongkai Xu, Xuanli He, Lingjuan Lyu, Lizhen Qu, and Gholamreza Haffari. 2021. Beyond model extraction: Imitation attack for black-box nlp apis. arXiv preprint arXiv:2108.13873 (2021).

[98]

Qiongkai Xu, Xuanli He, Lingjuan Lyu, Lizhen Qu, and Gholamreza Haffari. 2022. Student Surpasses Teacher: Imitation Attack for Black-Box NLP APIs. In Proceedings of the 29th International Conference on Computational Linguistics. International Committee on Computational Linguistics, Gyeongju, Republic of Korea, 2849--2860. https://aclanthology.org/2022.coling-1.251

[99]

Sangwon Yu, Jongyoon Song, Heeseung Kim, Seongmin Lee, Woo-Jong Ryu, and Sungroh Yoon. 2022. Rare Tokens Degenerate All Tokens: Improving Neural Text Generation via Adaptive Gradient Gating for Rare Token Embeddings. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 29--45.

[100]

Jie Zhang, Dongdong Chen, Jing Liao, Weiming Zhang, Gang Hua, and Nenghai Yu. 2020. Passport-aware normalization for deep model protection. Advances in Neural Information Processing Systems 33 (2020), 22619--22628.

[101]

Jialong Zhang, Zhongshu Gu, Jiyong Jang, Hui Wu, Marc Ph Stoecklin, Heqing Huang, and Ian Molloy. 2018. Protecting intellectual property of deep neural networks with watermarking. In Proceedings of the 2018 on Asia Conference on Computer and Communications Security. 159--172.

Digital Library

[102]

Jialong Zhang, Zhongshu Gu, Jiyong Jang, Hui Wu, Marc Ph. Stoecklin, Heqing Huang, and Ian M. Molloy. 2018. Protecting Intellectual Property of Deep Neural Networks with Watermarking. In Proceedings of the 2018 on Asia Conference on Computer and Communications Security, AsiaCCS 2018, Incheon, Republic of Korea, June 04-08, 2018, Jong Kim, Gail-Joon Ahn, Seungjoo Kim, Yongdae Kim, Javier López, and Taesoo Kim (Eds.). ACM, 159--172.

Digital Library

[103]

William Zhu, Clark Thomborson, and Fei-Yue Wang. 2005. A survey of software watermarking. In International Conference on Intelligence and Security Informatics. Springer, 454--458.

Digital Library

[104]

Frank F. Xu, Uri Alon, Graham Neubig, and Vincent Josua Hellendoorn. 2022. A Systematic Evaluation of Large Language Models of Code. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3520312. 3534862

Digital Library

Cited By

Pang KQi TWu CBai MJiang MHuang Y(2025)ModelShield: Adaptive and Robust Watermark Against Model Extraction AttackIEEE Transactions on Information Forensics and Security10.1109/TIFS.2025.353069120(1767-1782)Online publication date: 2025
https://doi.org/10.1109/TIFS.2025.3530691
Ding WAbdel-Basset MAli AMoustafa N(2025)Large language models for cyber resilience: A comprehensive review, challenges, and future perspectivesApplied Soft Computing10.1016/j.asoc.2024.112663170(112663)Online publication date: Feb-2025
https://doi.org/10.1016/j.asoc.2024.112663
Bui MBoffa MValentim RNavarro JChen FBao XHouidi ZRossi D(2024)A Systematic Comparison of Large Language Models Performance for Intrusion DetectionProceedings of the ACM on Networking10.1145/36963792:CoNEXT4(1-23)Online publication date: 25-Nov-2024
https://dl.acm.org/doi/10.1145/3696379
Show More Cited By

Index Terms

Protecting Intellectual Property of Large Language Model-Based Code Generation APIs via Watermarks
1. Security and privacy
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory

Recommendations

Watermark Embedding Mechanism Using Modulus-Based for Intellectual Property Protection on Image Data
EC-WEB '02: Proceedings of the Third International Conference on E-Commerce and Web Technologies

In this paper, an intellectual property protection mechanism realized on a watermarking scheme is proposed. The embedding technique we adopted in this paper is based on the modular operation. The modulus is a threshold value which determines how the ...
An efficient wavelet-tree-based watermarking method

This paper proposes a blind watermarking scheme based on wavelet tree quantization for copyright protection. In such a quantization scheme, there exists a large significant difference while embedding a watermark bit 1 and a watermark bit 0; it then does ...
Adversarial Text Purification: A Large Language Model Approach for Defense
Advances in Knowledge Discovery and Data Mining
Abstract
Adversarial purification is a defense mechanism for safeguarding classifiers against adversarial attacks without knowing the type of attacks or training of the classifier. These techniques characterize and eliminate adversarial perturbations from ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CCS '23: Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security

November 2023

3722 pages

ISBN:9798400700507

DOI:10.1145/3576915

General Chairs:
Weizhi Meng
Technical University of Denmark
,
Christian D. Jensen
Technical University of Denmark
,
Program Chairs:
Cas Cremers
CISPA Helmholtz Center for Information Security
,
Engin Kirda
Khoury College of Computer Sciences

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 November 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CCS '23

Sponsor:

SIGSAC

CCS '23: ACM SIGSAC Conference on Computer and Communications Security

November 26 - 30, 2023

Copenhagen, Denmark

Acceptance Rates

Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

Upcoming Conference

CCS '25

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 13 - 17, 2025

Taipei , Taiwan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
1,355
Total Downloads

Downloads (Last 12 months)829
Downloads (Last 6 weeks)52

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Pang KQi TWu CBai MJiang MHuang Y(2025)ModelShield: Adaptive and Robust Watermark Against Model Extraction AttackIEEE Transactions on Information Forensics and Security10.1109/TIFS.2025.353069120(1767-1782)Online publication date: 2025
https://doi.org/10.1109/TIFS.2025.3530691
Ding WAbdel-Basset MAli AMoustafa N(2025)Large language models for cyber resilience: A comprehensive review, challenges, and future perspectivesApplied Soft Computing10.1016/j.asoc.2024.112663170(112663)Online publication date: Feb-2025
https://doi.org/10.1016/j.asoc.2024.112663
Bui MBoffa MValentim RNavarro JChen FBao XHouidi ZRossi D(2024)A Systematic Comparison of Large Language Models Performance for Intrusion DetectionProceedings of the ACM on Networking10.1145/36963792:CoNEXT4(1-23)Online publication date: 25-Nov-2024
https://dl.acm.org/doi/10.1145/3696379
Li ZWang CMa PLiu CWang SWu DGao CLiu YRoychoudhury APaiva AAbreu RStorey M(2024)On Extracting Specialized Code Abilities from Large Language Models: A Feasibility StudyProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639091(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639091
Li ZQiu WMa PLi YLi YHe SJiang BWang SGu W(2024)Poster Abstract: On the Accuracy and Robustness of Large Language Models in Chinese Industrial Scenarios2024 23rd ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)10.1109/IPSN61024.2024.00042(283-284)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1109/IPSN61024.2024.00042
Yao YDuan JXu KCai YSun ZZhang Y(2024)A survey on large language model (LLM) security and privacy: The Good, The Bad, and The UglyHigh-Confidence Computing10.1016/j.hcc.2024.1002114:2(100211)Online publication date: Jun-2024
https://doi.org/10.1016/j.hcc.2024.100211
Wang ZChu ZDoan TNi SYang MZhang W(2024)History, development, and principles of large language models: an introductory surveyAI and Ethics10.1007/s43681-024-00583-7Online publication date: 14-Oct-2024
https://doi.org/10.1007/s43681-024-00583-7

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten