short-paper

Building Privacy-Preserving and Secure Geospatial Artificial Intelligence Foundation Models (Vision Paper)

Authors:

Krzysztof JanowiczAuthors Info & Claims

SIGSPATIAL '23: Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems

Article No.: 41, Pages 1 - 4

https://doi.org/10.1145/3589132.3625611

Published: 22 December 2023 Publication History

Abstract

In recent years we have seen substantial advances in foundation models for artificial intelligence, including language, vision, and multimodal models. Recent studies have highlighted the potential of using foundation models in geospatial artificial intelligence, known as GeoAI Foundation Models, for geographic question answering, remote sensing image understanding, map generation, and location-based services, among others. However, the development and application of GeoAI foundation models can pose serious privacy and security risks, which have not been fully discussed or addressed to date. This paper introduces the potential privacy and security risks throughout the lifecycle of GeoAI foundation models and proposes a comprehensive blueprint for research directions and preventative and control strategies. Through this vision paper, we hope to draw the attention of researchers and policymakers in geospatial domains to these privacy and security risks inherent in GeoAI foundation models and advocate for the development of privacy-preserving and secure GeoAI foundation models.

References

[1]

Yuntao Bai et al. "Constitutional AI: Harmlessness from AI Feedback". In: arXiv preprint arXiv:2212.08073 (2022).

[2]

Yuntao Bai et al. "Training a helpful and harmless assistant with reinforcement learning from human feedback". In: arXiv preprint arXiv:2204.05862 (2022).

[3]

Rishi Bommasani et al. "On the opportunities and risks of foundation models". In: arXiv preprint arXiv:2108.07258 (2021).

[4]

Tom Brown et al. "Language models are few-shot learners". In: Advances in neural information processing systems 33 (2020), pp. 1877--1901.

[5]

Nicholas Carlini et al. "Extracting Training Data from Large Language Models." In: USENIX Security Symposium. Vol. 6. 2021.

[6]

Mark Chen et al. "Evaluating large language models trained on code". In: arXiv preprint arXiv:2107.03374 (2021).

[7]

Haixing Dai et al. "AD-AutoGPT: An Autonomous GPT for Alzheimer's Disease Infodemiology". In: arXiv preprint arXiv:2306.10095 (2023).

[8]

Paolo Fraccaro et al. HLS Foundation. Aug. 2023. url: https://github.com/nasa-impact/hls-foundation-os.

[9]

Rohit Girdhar et al. "Imagebind: One embedding space to bind them all". In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, pp. 15180--15190.

[10]

Niv Haim et al. "Reconstructing training data from trained neural networks". In: arXiv preprint arXiv:2206.07758 (2022).

[11]

Krzysztof Janowicz. "Philosophical foundations of geoai: Exploring sustainability, diversity, and bias in geoai and spatial data science". In: arXiv preprint arXiv:2304.06508 (2023).

[12]

Krzysztof Janowicz et al. "GeoAI: spatially explicit artificial intelligence techniques for geographic knowledge discovery and beyond". In: International Journal of Geographical Information Science 34.4 (2020), pp. 625--636.

[13]

Yuhao Kang et al. "The ethics of AI-Generated maps: A study of DALLE 2 and implications for cartography". In: arXiv preprint arXiv:2304.10743 (2023).

[14]

Carsten Keßler et al. "A geoprivacy manifesto". In: Transactions in GIS 22.1 (2018), pp. 3--19.

[15]

Alexander Kirillov et al. "Segment anything". In: arXiv preprint arXiv:2304.02643 (2023).

[16]

Junnan Li et al. "Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models". In: arXiv preprint arXiv:2301.12597 (2023).

[17]

Zhenlong Li et al. "Autonomous GIS: the next-generation AI-powered GIS". In: arXiv preprint arXiv:2305.06453 (2023).

[18]

Zilong Liu et al. "Geoparsing: Solved or Biased? An Evaluation of Geographic Biases in Geoparsing". In: AGILE: GIScience Series 3 (2022), p. 9.

[19]

Jiaying Lu et al. "Evaluation and Mitigation of Agnosia in Multimodal Large Language Models". In: arXiv preprint arXiv:2309.04041 (2023).

[20]

Gengchen Mai et al. "CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representations". In: arXiv preprint arXiv:2305.01118 (2023).

[21]

Gengchen Mai et al. "On the opportunities and challenges of foundation models for geospatial artificial intelligence". In: arXiv preprint arXiv:2304.06798 (2023).

[22]

Gengchen Mai et al. "Towards a foundation model for geospatial artificial intelligence (vision paper)". In: Proceedings of the 30th International Conference on Advances in Geographic Information Systems. 2022, pp. 1--4.

[23]

Justus Mattern et al. "Membership Inference Attacks against Language Models via Neighbourhood Comparison". In: arXiv preprint arXiv:2305.18462 (2023).

[24]

OpenAI. GPT-4 Technical Report. 2023. arXiv: 2303.08774 [cs.CL].

[25]

Fábio Perez et al. "Ignore Previous Prompt: Attack Techniques For Language Models". In: arXiv preprint arXiv:2211.09527 (2022).

[26]

Jinmeng Rao et al. "A privacy-preserving framework for location recommendation using decentralized collaborative machine learning". In: Transactions in GIS 25.3 (2021), pp. 1153--1175.

[27]

Jinmeng Rao et al. "CATS: Conditional Adversarial Trajectory Synthesis for Privacy-Preserving Trajectory Data Publication Using Deep Learning Approaches". In: International Journal of Geographical Information Science 37 (2023), pp. 1--30.

[28]

Jinmeng Rao et al. "LSTM-TrajGAN: A Deep Learning Approach to Trajectory Privacy Protection". In: 11th International Conference on Geographic Information Science (GIScience 2021)-Part I. Schloss Dagstuhl-Leibniz-Zentrum für Informatik. 2020.

[29]

Timo Schick et al. "Toolformer: Language models can teach themselves to use tools". In: arXiv preprint arXiv:2302.04761 (2023).

[30]

Yongliang Shen et al. "Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface". In: arXiv preprint arXiv:2303.17580 (2023).

[31]

Jiawen Shi et al. "BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT". In: arXiv preprint arXiv:2304.12298 (2023).

[32]

Hugo Touvron et al. "Llama: Open and efficient foundation language models". In: arXiv preprint arXiv:2302.13971 (2023).

[33]

Dongjie Wang et al. "Towards automated urban planning: When generative and chatgpt-like ai meets urban planning". In: arXiv preprint arXiv:2304.03892 (2023).

[34]

Jiashu Xu et al. "Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models". In: arXiv preprint arXiv:2305.14710 (2023).

[35]

Sixing Yu et al. "Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Models". In: arXiv preprint arXiv:2305.11414 (2023).

[36]

Weichen Yu et al. "Bag of tricks for training data extraction from language models". In: arXiv preprint arXiv:2302.04460 (2023).

[37]

Jianyi Zhang et al. "Towards Building the Federated GPT: Federated Instruction Tuning". In: arXiv preprint arXiv:2305.05644 (2023).

[38]

Jielu Zhang et al. "Text2Seg: Remote Sensing Image Semantic Segmentation via Text-Guided Visual Foundation Models". In: arXiv preprint arXiv:2304.10597 (2023).

[39]

Xuezhou Zhang et al. "Adaptive reward-poisoning attacks against reinforcement learning". In: International Conference on Machine Learning. PMLR. 2020, pp. 11225--11234.

Cited By

Mai GHuang WSun JSong SMishra DLiu NGao SLiu TCong GHu YCundy CLi ZZhu RLao N(2024)On the Opportunities and Challenges of Foundation Models for GeoAI (Vision Paper)ACM Transactions on Spatial Algorithms and Systems10.1145/365307010:2(1-46)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3653070
Guo DYu YGe SGao SMai GChen H(2024)SpatialScene2Vec: A self-supervised contrastive representation learning method for spatial scene similarity evaluationInternational Journal of Applied Earth Observation and Geoinformation10.1016/j.jag.2024.103743128(103743)Online publication date: Apr-2024
https://doi.org/10.1016/j.jag.2024.103743

Index Terms

Building Privacy-Preserving and Secure Geospatial Artificial Intelligence Foundation Models (Vision Paper)
1. Computing methodologies
  1. Artificial intelligence
2. Security and privacy

Recommendations

Towards a foundation model for geospatial artificial intelligence (vision paper)
SIGSPATIAL '22: Proceedings of the 30th International Conference on Advances in Geographic Information Systems

Large pre-trained models, also known as foundation models (FMs), are trained in a task-agnostic manner on large-scale data and can be adapted to a wide range of downstream tasks by fine tuning, few-shot, or even zero-shot learning. Despite their ...
Privacy-preserving artificial intelligence in healthcare: Techniques and applications
Abstract
There has been an increasing interest in translating artificial intelligence (AI) research into clinically-validated applications to improve the performance, capacity, and efficacy of healthcare services. Despite substantial research worldwide, ...
Highlights
- A comprehensive overview of AI healthcare privacy concerns is presented.
- Vulnerabilities across the AI healthcare pipeline are highlighted.
- A taxonomy of various privacy preserving techniques is provided.
- Limitations of privacy-...
Revisiting unpredictability-based RFID privacy models
ACNS'10: Proceedings of the 8th international conference on Applied cryptography and network security

Recently, there have been several attempts in establishing formal RFID privacy models in the literature. These models mainly fall into two categories: one based on the notion of indistinguishability of two RFID tags, denoted as ind-privacy, and the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGSPATIAL '23: Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems

November 2023

686 pages

ISBN:9798400701689

DOI:10.1145/3589132

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSPATIAL: ACM Special Interest Group on Spatial Information

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 December 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

National Science Foundation

Conference

SIGSPATIAL '23

Sponsor:

SIGSPATIAL

SIGSPATIAL '23: 31st ACM International Conference on Advances in Geographic Information Systems

November 13 - 16, 2023

Hamburg, Germany

Acceptance Rates

Overall Acceptance Rate 220 of 1,116 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
188
Total Downloads

Downloads (Last 12 months)188
Downloads (Last 6 weeks)19

Reflects downloads up to 11 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mai GHuang WSun JSong SMishra DLiu NGao SLiu TCong GHu YCundy CLi ZZhu RLao N(2024)On the Opportunities and Challenges of Foundation Models for GeoAI (Vision Paper)ACM Transactions on Spatial Algorithms and Systems10.1145/365307010:2(1-46)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3653070
Guo DYu YGe SGao SMai GChen H(2024)SpatialScene2Vec: A self-supervised contrastive representation learning method for spatial scene similarity evaluationInternational Journal of Applied Earth Observation and Geoinformation10.1016/j.jag.2024.103743128(103743)Online publication date: Apr-2024
https://doi.org/10.1016/j.jag.2024.103743

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents