Materiality and Risk in the Age of Pervasive AI Sensors

Matthew Stewart Harvard University matthew˙stewart@g.harvard.edu , Emanuel Moss emanuel.moss@intel.com Intel Labs , Pete Warden Stanford University , Brian Plancher bplancher@barnard.edu Barnard College, Columbia University , Susan Kennedy Santa Clara University , Mona Sloane University of Virginia and Vijay Janapa Reddi Harvard University

(2024; 18 January 2024)

Abstract.

Artificial intelligence systems connected to sensor-laden devices are becoming pervasive, which has significant implications for a range of AI risks, including to privacy, the environment, autonomy, and more. There is therefore a growing need for increased accountability around the responsible development and deployment of these technologies. In this paper, we provide a comprehensive analysis of the evolution of sensors, the risks they pose by virtue of their material existence in the world, and the impacts of ubiquitous sensing and on-device AI. We propose incorporating sensors into risk management frameworks and call for more responsible sensor and system design paradigms that address risks of such systems. To do so, we trace the evolution of sensors from analog devices to intelligent, networked systems capable of real-time data analysis and decision-making at the extreme edge of the network. We show that the proliferation of sensors is driven by calculative models that prioritize data collection and cost reduction and produce risks that emerge around privacy, surveillance, waste, and power dynamics. We then analyze these risks, highlighting issues of validity, safety, security, accountability, interpretability, and bias. We surface sensor-related risks not commonly captured in existing approaches to AI risk management, using a materiality lens that reveals how physical sensor properties shape data and algorithmic models. We conclude by advocating for increased attention to the materiality of algorithmic systems, and of on-device AI sensors in particular, and highlight the need for development of a responsible sensor design paradigm that empowers users and communities and leads to a future of increased fairness, accountability and transparency.

TinyML, ML Sensors, Embedded Systems, Sensors, Responsible AI

^†^†copyright: acmcopyright^†^†journalyear: 2024^†^†doi: XXXXXXX.XXXXXXX^†^†price: 15.00^†^†isbn: 978-1-4503-XXXX-X/18/06^†^†conference: ACM Conference on Fairness, Accountability, and Transparency; June 03–06, 2024; Rio de Janeiro, Brazil^†^†ccs: Applied computing → Law, social and behavioral sciences → Sociology^†^†ccs: Human-centered computing → Human computer interaction (HCI)^†^†ccs: Human-centered computing → Ubiquitous and mobile devices

1. Introduction

Over the past decade, several overlapping, multidisciplinary communities of research and development have emerged to analyze and address the implications of artificial intelligence (AI) systems operating across society, particularly through engineering, governance, critical academic, and advocacy perspectives.¹¹1See especially: https://www.fatml.org/; https://facctconference.org/; https://www.aies-conference.com/2023/; https://eaamo.org/ This work has focused crucial attention on how datasets, algorithms, and machine learning systems deployed at scale produce significant impacts for human rights, equity, already-disadvantaged populations and communities, and society at large (metcalf2021algorithmic, ; smuha2021beyond, ; shelby2023sociotechnical, ; weidinger2021ethical, ; selbst2021institutional, ; birhane2021algorithmic, ; barocas2016big, ; noble2018algorithms, ; eubanks2018automating, ). Furthermore, this work has expanded the scope of investigations of these impacts beyond a relatively narrow focus on, e.g., bias in algorithms (danks2017algorithmic, ; mitchell2021algorithmic, ) or datasets (mehrabi2021survey, ; hazirbas2021towards, ) to the situated interactions of complex computational systems in society (ehsan2022algorithmic, ; metcalf2021algorithmic, ). Only recently, this scope has also broadened to include the material dimensions of AI systems, particularly around the environmental impacts of computation in terms of carbon-intensive energy consumption and the geographic footprint of server infrastructure (dodge2022measuring, ; bender2021dangers, ; weidinger2022taxonomy, ), as well as e-waste (bridges2023geographies, ; kidd_energy_2023, ). However, relatively scant attention has been paid to the materiality of AI technologies in terms of the hardware, particularly in terms of the sensors that interact with the physical environment and which are crucial for data gathering (i.e., the sensing of physical phenomena to produce data) and enacting algorithmic inferences (e.g., AI-enabled Internet-of-Things (IoT) devices).

Indeed, there are significant material dimensions to problems of fairness, accountability, and transparency that can be addressed by understanding how ubiquitous, algorithmically-enabled sensors produce significant risks around bias, equity, privacy, accountability, transparency, and consent. Materiality, as has often been noted, is a complex term to define given the role it plays alongside social practices and social institutions in shaping the significance of sociotechnical systems (like artificial intelligence) (law1995notes, ; pinch2008technology, ; lievrouw2006introduction, ; miller2020materiality, ). Following Lievrouw, who focuses on materiality as ”the physical character and existence of objects and artifacts that makes them useful and usable for certain purposes under particular conditions” (lievrouw2014materiality, , p. 25), we draw attention to the material design of sensors, how their design affects the ways they transform physical phenomena into data, and how their material integration into algorithmic systems shapes their overall impacts.

To illustrate how the materiality of sensors is deeply implicated in topics of fairness, accountability and transparency, consider the way physical properties change the performance of sensors for varying skin tones (roth2009looking, ; galdino2001standardizing, ; guo2023calibrating, ). For example, the physical properties of the charge-coupled devices (CCDs) inside digital cameras, a paradigmatic sensor, and the algorithms that process their outputs, contribute to how skin tones are rendered in digital files (roth2009looking, ), which in turn contribute to computer vision applications like face detection and facial recognition (buolamwini2018gender, ). Indeed, how sensors are calibrated may impact the downstream use of the data they generate in ways that call for deep analysis by those concerned with algorithmic fairness.

Adopting a material lens also makes it possible to apprehend the risks that adhere not to just a single sensor, device, or AI system, but instead emerge from their widespread adoption. Cameras, microphones, and other sensors are increasingly common in home appliances, public infrastructure, automobiles, offices, and factories. For example, there is a small camera in the K-Supreme Plus Smart coffee maker,²²2https://www.keurig.com/content/k-supreme-plus-smart-coffee-maker that detects the type of pod being inserted in order to adjust the water temperature for optimal brewing. This proliferation of sensors into the public, professional, and intimate dimensions of our daily routines enables unprecedented data mining into once private or anonymous moments and behaviors—from when you open your refrigerator, to how long you spend in the shower, to the route you take to work. This has led to the near future being described as the age of “Data Analytics Anywhere and Everywhere” (elmqvist2023data, ). The near-ubiquitous presence of sensors also gives rise to significant concerns. Surveillance technologies are more likely to be deployed in already over-surveilled communities and professions (fernback2013sousveillance, ), and the sensors in mobile devices (such as cameras, microphones, gyroscopes, GPS antennae, and inertial measurement units) and home electronics are often opaque and unaccountable in ways that make it difficult for anyone to understand when and what information they might be collecting and transmitting.

In this paper, we highlight the materiality of risk production by paying close attention to how sensors are incorporated into AI systems. We show how sensors can be analyzed in the context of the material and digital infrastructure of AI technologies to better understand the role they play in producing risks to individuals, communities, society, and the environment. Sensors contribute to the risks posed by computational systems according to their capabilities, which have evolved significantly over the past half-century, and their ubiquity, which has been spurred by economic calculative models that have also shifted over time. To illustrate this, we lay out the ‘evolutionary history’ of sensors, mapping the risks they pose as components of AI systems onto their evolving capabilities. We present the calculative models that have contributed to their ubiquity, and we deploy the AI risk management framework (RMF) developed by NIST (nist_artificial_2023, )–a general-purpose, domain-agnostic, adaptable framework that will likely shape the majority of AI risk management practices in the U.S., particularly in the government sector (noauthor_executive_2023, ), to highlight the ways that sensors contribute to the overall risk framework of AI systems. Additionally, we raise a call for the development of sensor design paradigms, like those that take advantage of Tiny Machine Learning (TinyML)—i.e., machine learning approaches tailored to resource-constrained devices like sensors—and on-board processing, to address many of the risks posed by the convergence of ubiquitous sensing and AI technologies. Throughout, we urge expanded attention to the materiality of AI systems. We encourage researchers to include the technical properties of sensors and their socially-situated activity analyses of AI systems, and we encourage those developing AI governance mechanisms to include sensors within their remit.

2. Sensors Everywhere

Sensors are devices that quantify environmental aspects such as light, heat, and pressure. They are designed to monitor phenomena within and beyond our human perception, such as imagery, sound, or chemical composition. They transform physical phenomena into numerical representations, adding to the ”avalanche” of numbers (hacking2015biopower, ) that make it possible for the world — people, commodities, communities, and nature — to be represented computationally, for behaviors to be analyzed, and for those who have the power to do so to act on it. Crucially, they are the foundation for many of the datasets that undergird powerful AI technologies. Digital cameras are the sensors that produced datasets that enabled image recognition (krizhevsky2012imagenet, ). Speech recognition would be impossible without the datasets generated using digital microphones (ardila2019common, ). And many more AI applications, from autonomous vehicles to personal athletic trackers, rely on sensor data as well. In this section, we layout how sensors have evolved over time and the calculative models developers and manufacturers have worked from that have spurred this evolution to set the stage for a sensor-saturated world. In doing so, we identify how successive stages of sensor development have added to the risks produced by sensors as components of AI systems, which we discuss in the following section (Section 3).

2.1. The Evolution of Sensing

As technology has advanced, sensors have become smaller, more efficient, more sensitive, and more connected, allowing them to be embedded in various environments and objects, thus providing real-time data and insights that were previously unattainable. This evolution has not only extended our ability to understand and interact with the world around us but also paved the way for innovations that continue to shape our future. Sensors have evolved from simple analog devices into ‘intelligent’ systems capable of analyzing data and making decisions at the edge. The advent of wireless sensor networks in the late 1990s marked an important milestone, allowing remote collection and analysis of sensor data (buratti2009overview, ). However, sensors were still reliant on external processing power. The subsequent rise of smartphones and IoT devices brought sensors out of isolation, interconnecting them through the internet (mainetti2011evolution, ). For the first time, distributed networks of sensors could coordinate to achieve larger goals. Yet even as billions of sensors flooded the environment with data, their capabilities remained confined to data collection rather than interpretation.

This changed with the emergence of Edge AI sensors and TinyML sensors, which move machine learning out of data centers and closer to the sensor devices themselves. So-called Edge AI sensors communicate with nearby processors–either separate components within the same device that contains the sensor, or a mobile phone or computer wirelessly connected to the sensor device–where machine learning models run locally. TinyML sensors incorporate machine learning models developed from the field of TinyML³³3https://www.tinyml.org/. These models enable the sensors to perform machine learning tasks directly on their microcontrollers, which are typically integral components of the physical sensor hardware. This allows for real-time data processing and analysis at the sensor level, without needing to transmit data to a separate computing device. Furthermore, TinyML sensors usually limit the information shared from the sensor device to only application-specific instructions, e.g., whether or not a person is standing in front of the device, rather than an entire data stream that could then be put to many purposes off-device (warden2019tinyml, ; warden23MLSensors, ). Rather than indiscriminately streaming all collected data to cloud servers, Edge AI and TinyML sensors interpret their environment, surface insights, and make decisions not in distant server centers, but at the edge, i.e. on the device or a nearby connected device.

Below, we list successive stages of sensor development with definitions relevant for analyzing the risks they contribute to AI systems, and we provide common examples of each in Figure 1. Each of the stages of sensor development we identify build on earlier properties of sensors while adding additional properties. Accordingly, when thinking about the risks sensors contribute to algorithmic systems we see many of those risks accumulate from one stage to the next. We also offer a comparison of sensor types in Table 1 based on their connectivity and the location where algorithmic processing occurs, as these factors are highly significant for the later discussion of risk in subsection 3.1.

•

Traditional Sensor. A device capable of generating data from or about the physical world. Data produced by the device may need post-processing on- or off-device to be useful. Data may be stored on- or off-device. Device does not require an internet connection to function and typically has no internet connection capability. Data must be purposefully transferred off-device and stored using specific protocols such as SPI or I²C. No statistical inferencing happens on-device.
•

IoT Sensor. A device capable of generating data from or about the physical world, that transmits that data in realtime or near-realtime using internet transfer protocols. IoT sensors may or may not require active internet connection to generate or store data. No statistical inferencing happens on-device, and rarely even happens in the cloud. Data transmitted over the internet is (almost) always stored in a centralized location. This type of device acts primarily as an interface to cloud-based data storage systems.
•

AIoT Sensor. A device capable of generating data from or about the physical world, that transmits data in realtime or near-realtime using internet transfer protocols, and is designed to be used in conjunction with AI/ML inferencing techniques in the cloud (and importantly not on or near the device). As such, this type of device requires an active internet connection to operate at full functionality. Importantly, full datastreams generated by the sensor are (temporarily) stored in centralized locations (i.e., cloud or server device). This type of devices acts primarily as an interface to cloud-based decision-making systems powered by AI.
•

Edge AI Sensor. A device capable of generating data from or about the physical world and processing that data uses machine learning techniques through a combination of on-device and near-device edge processing (e.g., the Apple Watch offloads some computations to a user’s cell phone). This data may be transmitted off device, and possibly over the internet, in real or near-realtime, in its raw or processed form. Transmitted data may or may not be stored in a centralized location. This type of device extends the cloud-based intelligence of AIoT sensors to the edge.
•

TinyML Sensor. A device capable of processing data from or about the physical world using machine learning inferencing techniques entirely on-device, often transmitting the minimal information necessary to accomplish a predetermined task. These devices are typically self-contained and store minimal or no sensed data.

Table 1. Comparison of different sensor types at successive stages of development.

Characteristic	Traditional Sensor	IoT Sensor	AIoT Sensor	Edge AI Sensor	TinyML Sensor
Timeframe	1970s-1990s	1990s-2010s	2010s	Late 2010s-Present	Present
Wireless connectivity	No	Yes	Yes	Optional	Optional
On-device processing	No	Minimal	Minimal	Partial	Yes
ML capabilities	No	No	Yes	Yes	Yes
Cloud reliance	None	High	High	Optional	Low/None

Refer to caption — Figure 1. Timeline of sensor evolution: from passive analog detectors to intelligent IoT and ML-enabled systems.

2.2. Calculative Models of Sensor Development

The wide adoption and evolutionary transformation of sensors can be explained in part by their sheer utility for gathering data about the physical world. And, each stage of sensor development adds capabilities that lead to new products and services. But, the proliferation of sensors cannot be explained entirely by the technical needs they satisfy. Instead, the integration of sensors into consumer devices, industrial machinery, and civil infrastructure—like any other product—is motivated more by dynamics of economic calculation rather than by technical features.

Economic markets are not independent agential entities that are external to social and material life, but rather are collectively organized tools that facilitate the calculation of the value of goods (callon2005peripheral, ; muniesa2007introduction, ; callon2007market, ). Calculation bridges quantitative and qualitative aspects in an effort to make goods tradeable. The practices and cultures of calculation (hansen2021model, ) differ by market, but always evolve around certain sets of calculative models, i.e., repeatable ways of calculating an object or good. These calculative models combine cultural knowledges and assumptions about people and consumption (for example, about convenience in the home, stipulating that people will prefer to operate their lightswitch with voice commands over physically switching on lights) with determinations of costs of manufacture, actual and projected volume of sales, and cumulative profits in the context of the many possible additional factors that affect price, costs, and earnings (besedovsky_financialization_2018, ; mackenzie_engine_2008, ). Calculative models are based on certain elements, notably shared knowledges, conditions, and trends. For sensors, these notably are:

•

Cost Reduction in Sensor Manufacturing. The dramatic decrease in the cost of sensor production over the years has been a key driver. This reduction is due to advancements in manufacturing technologies, economies of scale, and the development of more more affordable materials. As sensors become cheaper to produce, they become more accessible for a wider range of applications.
•

Economic Models of IoT and Sensor Networks. IoT has created new business models where sensors play a crucial role. Notably, they facilitate the rendering of data into a calculable unit that can be traded, for example in the context of predictive maintenance, energy management, and consumer behavior analysis.
•

Subscription-Based and Service-Oriented Models. A shift towards subscription-based and service-oriented models as calculative units has also influenced the adoption of sensors. Companies are increasingly offering sensors as part of a service package, where the initial cost is low, but users pay a recurring fee for data analysis, cloud storage, and other associated services.
•

Advancements in On-Device AI. The emergence of Edge AI and TinyML sensors, where machine learning models runs at the edge and enhance efficiency by removing the need for cloud services, has transformed sensors into devices capable of real-time data analysis. This advancement has expanded the calculative unit of sensors, making them more valuable and attractive for a wide range of applications where an internet connectivity is impractical or latency restrictions are severe.

As a result of these calculative models, even mundane appliances have increasing numbers of sensors built into them. As such, there has been an exponential growth in the market size of deployed smart IoT devices, with a total 14.3 billion devices now in use globally in 2022 and a projected 29 billion devices by 2027 (iot_market, ). Due to this explosion, sensors stand ready to infiltrate nearly every moment of our daily lives. Most importantly, these efforts and new capabilities shape the effects sensor devices have on the world, as will be discussed below. Examining the materiality and technical development of machine learning-enabled sensors through the lens of calculation can help explain the growing proliferation of sensors.

Even more importantly, tending to calculative models and their elements can also help map out the relationship between the proliferation of AI-connected sensors and risk: as sensors proliferate, connect to AI services, and embed machine learning capabilities on-device, they shape and reshape how algorithmic harms may occur. This has important implications for AI risk management given the many ways different sensors interact with social worlds and the physical environment, how data is shared between and across devices containing different types of sensors, and how mitigations can be implemented with respect to the material embedding of AI systems on devices.

3. Applying Existing Risk Management Frameworks for a Sensor-Saturated World

Increased attention to the harms and risks of AI systems has led to the development of several risk management frameworks for AI (see, e.g., (saif_trustworthy_2020, ; floridi_capai_2022, )) that are variously oriented toward specific business purposes or regulatory conformity. In the U.S., the National Institute of Standards and Technology (NIST) has developed an AI Risk Management Framework (RMF) that is designed as a general purpose framework, applicable across many domains (nist_artificial_2023, ). Given that NIST’s AI RMF is customizable and not domain-specific, we focus here on that framework as the locus of our analysis, rather than other frameworks produced by governmental organizations, like the OECD (oecd_advancing_2023, ), or consultancy groups, e.g., (baquero_derisking_2020, ; deloitte_ai_2018, ). In that spirit, we examine the role sensors play in producing risk as part of algorithmic systems using the NIST AI RMF trustworthiness characteristics as our categories of analysis. We examine the trustworthiness characteristics of sensors themselves, across their successive stages of development, incorporating machine learning models and inference as part of sensors’ functionality where appropriate. This allows us to identify potential limitations of existing frameworks and to outline areas for future work.

3.1. Sensor-focused Risk Management

The evolution of sensing technology has led to changes in the associated risks. Although we categorize these stages with distinct labels, they actually form a continuous spectrum of gradual shifts in data collection and processing, which in turn create various risks for AI systems. In Table 2, we outline the specific concerns that arise at each stage of sensor development in relation to the characteristics of trustworthy systems. We use a color scale, shading from yellow to red, to visually represent our assessment of the relative risk posed by each sensor type for each of the sensor characteristics.

Table 2. Comparison of trustworthiness characteristics across different sensor types. Color scale ranges from green to red indicating our interpretation of the relative risk within each trustworthiness characteristic (row) posed by each type of sensor (column).

Trustworthiness Characteristic	Traditional Sensor	IoT Sensor	AIoT Sensor	Edge AI Sensor	TinyML Sensor
Valid and Reliable	Basic sensors can be validated through calibration and testing against known inputs and conditions. Reliability depends on mechanical integrity.	Can be validated by testing connectivity, networking, and data transmission capabilities. Reliability depends on power and connectivity.	All of the reliability concerns relevant to IoT sensors remain here, with the addition of ML model robustness. ML models are difficult to validate, but performance is typically validated using validation and test datasets.	Validating compressed ML models is more challenging due to their limited representational capacity. Reliability depends on model robustness and on-device monitoring, updates can be done using federated learning.	Similar issues to Edge AI sensors, but lack of connectivity can prevent approaches like federated learning. Even more compressed models have reduced representational capacity. Reliability depends on stability of deployed model.
Safe	Safety risks mainly from sensor failures or measurement errors. Can be made safe by redundancy and following installation guidelines.	Network connectivity introduces additional safety risks associated with loss of connectivity, higher latencies, or denial of service.	The unpredictability of ML models makes holistic safety validation of AI systems difficult. Techniques like simulations and sandboxing may be necessary.	On-device analytics and limited resources create potential for unanticipated failures. Rigorous functional safety testing is required.	Safety risks from unexpected model behaviors, but reduced risk due to limited data transmission and connectivity. Constrained resources limit safeguards.
Secure and Resilient	Generally secure due to limited attack surface. Resilient to environmental changes with proper installation.	Increased attack surface from networking expands security risks like spoofing. Resilience requires cyber protections.	Increased attack surface due to ML models vulnerable to data poisoning and adversarial attacks. Existing issues of IoT sensors remain. Robust cyber protections is essential.	Reduced attack surface due to less data transmission. However, tampering risks exist due to edge processing capabilities. Security protections for on-device analytics needed.	Heavily constrained resources make strong protections difficult. Limited to no data transmission and connectivity significantly reduces attack surface. Model extraction risks heightened.
Accountable and Transparent	Operation is observable but limited transparency into sensor internals. Accountability depends on application.	Networking expands accountability risks. Transparency improved by logging, documentation.	Cloud ML model opacity hurts transparency. More parties contribute to the AI system, making accountability more difficult.	More device diversity within AI system complicates accountability and transparency further.	Self-contained system and lack of connectivity fosters greater accountability. Transparency concerns remain.
Explainable and Interpretable	Sensor outputs straightforward to interpret based on specifications. Limited explainability of internal processing.	Network connectivity can complicate interpretability as data is transmitted and interpreted externally. Non-ML methods typically used which offer more interpretability.	The opacity of ML models limits attempts at interpretability and explainability. Explainability methods can shed some light on decision-making within ML models.	Models more lightweight and interpretable. However, less data availability hinders attempts at assessment. Computationally intensive techniques not as feasible at the edge.	Similar issue to Edge AI sensors, but even fewer resources available on-device for interpretability analysis.
Privacy-Enhanced	Generally limited privacy risks. Individual sensors collect minimal data.	Network connectivity enables broader personal data collection. Encryption, access controls key for privacy.	ML enables large-scale data aggregation increasing privacy risks. Raw data is often transmitted to and stored in the cloud.	On-device processing mitigates some privacy risks relative to cloud. Raw data may still be transmitted.	Heavily constrained resources can reduce data sharing risks. Raw data is inaccessible off-device.
Fair with Harmful Bias Managed	Limited risks of unfairness or bias since no decision making. But sensor placement could disproportionately affect certain groups.	Biases can emerge from IoT system deployment and uses. Disproportionate impacts on groups can be mitigated through equitable sensor deployment and data usage.	Same concerns as IoT devices, but ML models introduce algorithmic biases. Diverse and representative data training sets needed for AI model training. Extensive bias testing and mitigation needed.	On-device analytics could perpetuate biases if not continuously monitored and managed. Compressed ML models tend to exacerbate existing algorithmic biases.	Similar concerns to Edge AI sensors. The more constrained resources on TinyML sensors may make post-deployment bias testing and mitigation difficult.

3.1.1. Valid and Reliable:

All sensors must be properly calibrated to return valid and reliable data about the physical world. Traditional sensors are often calibrated before they leave the factory and provide users a means of recalibrating the sensor to a known value (e.g., boiling water for a digital thermometer). Edge AI and TinyML sensors also raise validity concerns, as their calibration often cannot be remotely maintained once deployed. For networked sensors—IoT and AIoT sensors—more sophisticated computational techniques may be needed to calibrate already-deployed sensors over a wide area without direct access to the device itself or a reliable reference measurement to calibrate sensor readings (delaine2019situ, ). The reliability of networked sensors can also be tracked, as they are capable of communicating when they are in a failure mode, which is not the case for Traditional, Edge AI, and TinyML sensors that often lack a stable network connection. All types of sensors contribute significantly to the validity and reliability of the AI systems in which they are embedded (jacobs2021measurement, ), as they may be properly calibrated for some settings and use cases despite being deployed more broadly. Oxygen-saturation sensors, for example, must be properly calibrated to a patient’s skin tone to work properly (guo2023calibrating, ).

3.1.2. Safe:

Safety, as it pertains to sensors, is largely focused on the risk of failure for mission-critical scenarios. Traditional sensors were deployed as components of complex systems (leveson2016engineering, ), but this risk is partially mitigated as sensors become more connected to AI systems that enable self-testing (rausand2014reliability, ). For AIoT, Edge AI, and TinyML sensors, additional risks are introduced by the possibility that machine learning models may behave in unexpected ways. These risks are exacerbated for Edge AI and TinyML sensors due to the limited resources available on-device, which may lead to lead to unexpected hardware issues. However, the reduce reliance on the cloud and connectivity in these systems reduces concerns related to latency and connectivity loss. This makes software safety engineering increasingly relevant, in addition to mechanical safety engineering, as machine learning is more closely embedded—first as part of networks that the sensors are connected to (for AIoT sensors), and then within the sensor itself (for Edge AI and TinyML sensors).

3.1.3. Secure and Resilient:

The risks related to security and resiliency for sensors ebb and flow across their development. Traditional sensors are protected by the need to have physical access to the sensor for anyone to tamper with it, which is also true of TinyML sensors that do not have network access. Connected (IoT, AIoT, and some Edge AI) sensors are vulnerable to attack over networks (atlam2020iot, ). These attacks may target the devices themselves (such as in denial-of-service attacks) or target the sensitive data stored on-device or being transmitted (e.g., on-path attacks). Traditional sensors are often designed to be resilient under anticipated environmental conditions. However, other sensor types necessitate extra software and hardware safeguards to guard against tampering. When sensors are integrated into AI systems, they not only inherit the security and resiliency risks associated with the AI system but also introduce new vulnerabilities. For instance, the physical architecture of a TinyML sensor model may be susceptible to extraction (shabir2023toward, ). Similarly, AIoT and EdgeAI sensors are potential targets for model poisoning attacks (nguyen2020poisoning, ; tolpegin2020data, ). This underscores the need for robust security measures in these advanced sensor applications to maintain their integrity and resilience.

3.1.4. Accountable and Transparent:

The operation of traditional sensors is commonly made transparent through datasheets, which have been broadly adopted across industry and cover everything from electrical components to chemical ingredients,⁴⁴4https://www.datasheets.com/ and have been the inspiration for better documentation of datasets (gebru2021datasheets, ) and ML models (mitchell2019model, ). Traditional (and, increasingly, IoT) sensors are also subject to stringent industry standards (trappey2017review, ; anton1995standard, ; busch2011standards, ) that are still emerging for ML algorithmic components. Accountability and transparency are both more complicated for other types of sensors. Questions of ownership—of both device and data—and responsibility for data breaches become clouded by the ”many hands” that may be involved in producing and maintaining networked sensors and IoT devices (cooper2022accountability, ; smith2018alexa, ).

3.1.5. Explainable and Interpretable:

Traditional sensors’ outputs are both explainable and transparent, given appropriate documentation through a standard datasheet. IoT and AIoT sensors, due to the opacity of data flows (nissenbaum2020privacy, ; burrell2016machine, ), are less explainable and interpretable. Sensors that comprise components of AI systems inherit many of the explainability and transparency challenges of AI systems writ large, and TinyML sensors may be even more opaque than most other sensors due to the fact that not only are explainable models rarely deployed onto the edge, but also that additional black-boxing may be required to compress the models onto limited edge devices (rudin2022interpretable, ).

3.1.6. Privacy-Enhanced:

Like the risks related to security and resiliency, risks related to privacy similarly ebb and flow across sensor types. Traditional sensors introduce limited privacy risks, as data collection is often limited to sensed data and a minimal amount of necessary metadata, and data access is limited to those entities with direct physical access to the sensor itself. Network connectivity raises the privacy risks for IoT, AIoT, and some Edge AI sensors. Large-scale aggregation of data is possible for systems using such sensors, in which data schema allow for records from many devices, along with a great deal of metadata, to be stored on centralized servers. EdgeAI offers the opportunity to implement federated learning (kairouz2021advances, ), which may mitigate some privacy risks, and TinyML sensors’ heavily constrained resources and connectivity mitigate privacy risks from superfluous data collection and aggregation even further. Specifically, since inferencing is done on-device in TinyML sensors, raw data need not be transmitted off-device.

3.1.7. Fair with Harmful Bias Managed:

Managing the risks of bias for sensors is a problem that is closely linked to validity and reliability (see above), in terms of calibration and subgroup validity. This problem is heightened as machine learning techniques are incorporated into the processing of sensor data, and particularly heightened for TinyML sensors, which must be carefully documented because sensor characteristics and machine learning models are often co-developed and unalterable. Bias testing of code embedded in TinyML sensors is additionally difficult to test and mitigate post-development due to on-device resource constraints. Furthermore, existing biases tend to be exacerbated in compressed models (hooker2020characterising, ), making it more difficult to address these concerns in Edge AI and TinyML sensors.

3.2. The Implications of Materiality for Risk Management Frameworks

While the NIST AI RMF (nist_artificial_2023, ) offers key trustworthiness characteristics for developers and regulators to assess algorithmic systems for, a materiality lens suggests there are additional dimensions of trustworthiness and risk in the context of ML-integrated sensors that are not currently captured by this (or most other) risk management frameworks which we discuss below. It is also difficult to attribute responsibility for sensors to discrete parties, categorically, because they might be owned by consumer, vendors, private companies, various levels of government (cities, counties, states or provinces, nations, all within a single geographic area), or other organizations (e.g. home-owners associations (ellickson_cities_1982, ) or business improvement districts (briffault_government_1999, )) (metcalf_relationship_2022, ; cooper_accountability_2022, ). That said, it is nevertheless possible to inventory the potential risks a sensor-saturated world pose for individuals and society-at-large, beyond those already addressed in risk management frameworks like the NIST AI RMF.

Below, we highlight additional risks that should be taken into consideration when evaluating the trustworthiness of an algorithmic system, particularly those that leverage data produced by sensors for algorithmic decision making. These risks become relevant as algorithmic techniques are incorporated into sensor-based system, i.e., from the AIoT stage and beyond (see Table 1). These risks can be mitigated at the design, evaluation, deployment, and monitoring stages, and are additional vectors for producing more trustworthy systems and for engaging in more responsible design.

3.2.1. (Proprietary) Data Profusion

The calculative models that underlie a sensor-saturated world tend toward reducing the unit-costs of sensors and ensures their omnipresence. The material devices are becoming cheaper to produce and any excess unit-costs are often rationalized by the calculative models discussed above. This has the result that the amount of data collected, which can be used for providing services (for free or on a subscription plan) as well as sold on a secondary market through data brokers (anthes_data_2015, ), is always maximized. Such a model ensures that the sheer scale of data collected will be extraordinary. Given the shift in the calculative models attached to sensors, particularly the perceived need to recoup per-unit costs, there will be increasing pressures for private- and public-sector organizations to keep data proprietary.

Data profusion brought by sensors has the potential to amplify the divergence between data quantity and quality and its associated risks. Calculative models are oriented towards the deployment of low-cost sensors which, unlike industry grade sensors, are more susceptible to factors that will result in incomplete or inaccurate data (teh2020sensor, ). Sensor data quality can also suffer due to variations in hardware manufacturing, sensor drift, or the state of the battery—as sensor data tends to becomes less reliable as the battery nears the end of its lifespan (ye2016detecting, ). Critiques of ”big data” practices underscore the failure of datasets to offer an objective, accurate, and comprehensive portrayal of real-world phenomena (d2023data, ; o2017weapons, ; crawford2013hidden, ). Given that policy-making decisions and resource allocation often rely on quantifiable information, the invisibility or inaccurate portrayal of specific individuals or phenomena within datasets contributes to their marginalization. An increase in dirty data generated in a sensor-saturated world risks perpetuating these issues. Moreover, data profusion propelled by sensors might exacerbate the risks associated with poor data quality as it will create an overwhelming demand for data cleaning. This is a time-consuming process which requires a domain expert to be done effectively and cannot be easily scaled to meet demand (ridzuan2019review, ).

3.2.2. Privacy & Surveillance

Although the NIST AI RMF addresses privacy and advocates for practices such as control over data and data minimization methods, the materiality of sensors and the scale of their deployment present a unique set of challenges with respect to privacy that are worthy of explicit attention.

The physical characteristics of sensors in terms of their small size allows them to be inconspicuously integrated into one’s surroundings in novel and unexpected ways. This presents a formidable obstacle to safeguarding privacy through the mechanisms of notice and consent. Individuals who are unaware of sensors may be involuntarily subjected to data collection and algorithmic processing. But even if individuals were notified of sensors, the abundance of these devices would render the practice of consent untenable. The time and attention required to provide informed consent in every case would exceed an individual’s finite capacities (scott2019trouble, ; froomkin2019big, ).

The NIST AI RMF notes how AI systems can present new privacy risks such as enabling inferences that jeopardize de-identification efforts. This risk will become especially salient in a sensor saturated world, as an increase in the volume and breadth of data is positively correlated with the ability to draw such inferences. Moreover, the process of sensor fusion, where data from multiple sensors is combined, enables inferences that would otherwise not be possible from a single data stream (elmenreich2002introduction, ). The expanding potential to draw inferences and cross-reference data not only contributes to re-identification risks of de-identified data (sweeney2000simple, ), but also threatens an individual’s ability to exercise control over their data in at least two respects. First, the potential to draw inferences means that individuals may not fully understand either the fact of data collection (i.e., the fact that data was being collected by a sensor at all) or the implications of data collection, casting doubt on the informed nature of their consent. Second, an individual’s decision to opt out of data collection may be rendered futile, as the decisions of others to permit data collection can enable inferences that inadvertently implicate those who seek to abstain, eroding the notion of individual agency in data sharing (barocas2014big, ; ding2019survey, ).

3.2.3. Power

As sensors become interwoven into the fabric of everyday life, they may fade into the background of conscious experience. Nevertheless, these devices are capable of exerting powerful influences over individuals’ choices and behavior. AI sensor-based systems can subtly or overtly exert power, whether through persuasion or coercion (verbeek2009ambient, ). Activity trackers, for instance, employ a subtle approach by using data-driven feedback to encourage users to exercise more. In contrast, a seat belt sensor that locks the car ignition until the driver buckles up represents a more coercive form of influence. The risk lies in the potential for a sensor-saturated world to provide designers with more opportunities to exert power over individuals, fueled by an abundance of data that enhances their effectiveness. Their shrinking size and increasing processing power also provides designers with material affordances to integrate sensors into devices that are ever more intimately linked to individuals daily routines (nafus2016quantified, )

Another significant risk pertains to the data generated by sensors rather than the sensors themselves. While personal data is often considered a natural resource or a mere byproduct of daily living, this perspective has faced criticism for obscuring the predatory nature of personal data extraction and commodification (couldry2019data, ; thatcher2016data, ). The term ’data colonialism’ has emerged to describe how corporations appropriate personal data, contributing to the exploitation of individuals (couldry2019data, ). The concern is that the proliferation of sensors may further normalize and rationalize this extraction of personal data, reinforcing the core principles of data colonialism and solidifying its unjust power dynamics centered around domination and subjugation in the digital realm.

3.2.4. Waste & Energy Inefficiency

One element of the calculative models and the materiality of sensors focuses on the drastic cost reduction of sensor manufacturing, where the cheap production of sensors spurs their widespread application and adoption. But there are also environmental costs associated with manufacturing sensors that must be taken into account. With respect to computing technologies, concerns about environmental costs and carbon emissions have been widely discussed (dhar2020carbon, ; wu2022sustainable, ; van2021sustainable, ; lannelongue2021green, ). These concerns have typically been framed in terms of operational activities associated with product use, such as the energy consumption required for training and running inference on an ML model. For example, training a single AI model for NLP can emit as much carbon as five cars (strubell2019energy, ).

As industry efforts to increase energy efficiency have successfully reduced the carbon output associated with operational activities, the leading factor behind carbon emissions now lies in the manufacturing and production of hardware, including environmental costs tied to the underlying supply chain and end-of-life processing (gupta2021chasing, ). The operation of battery-powered sensors, even at a massive scale, can be expected to consume significantly less energy than other computing technologies. And while the overall carbon footprint of such devices may be net negative (Prakash23SustainableTinyML, ), this assumes that these devices are deployed in the market towards sustainable aims and ignores other potential environmental risks such as freshwater usage, eutrophication potential, and emissions of volatile organic compounds. Regardless, these devices will likely still lead to the generation of substantial amounts of electronic waste. Sensor components such as batteries and circuit boards might not be easily recyclable or biodegradable, leading to accumulation in landfills or, worse yet, improper disposal that poses environmental hazards. Furthermore, the production process for sensor components often involves the use of rare earth elements and other critical materials, which have their own environmental and ethical implications. The mining and processing of these materials can lead to habitat destruction, water pollution, and other ecological impacts. In this respect, the widespread use of sensors may create environmental risks that mirror those associated with single-use plastics, ignoring their long-term environmental threats due to their immediate benefits and functionalities.

3.2.5. Attitudes and Practices

Research on users’ perceptions of IoT devices reveals a prevalent ’I want it anyway’ attitude, where the benefits offered by these devices are perceived to outweigh risks including privacy violations and security breaches (wang2018want, ). However, as these devices become pervasive, users may reassess their willingness to accept associated risks. For instance, one study suggests that users are less inclined to embrace data collection if they perceive it as excessive within the broader context of their lives: ”Smart home technology was often viewed as a further invasion of, or threat to, privacy in a society where already too much personal information is collected and stored” (balta2013social, , p. 369). Notably, individuals’ attitudes towards AI sensor-based systems appear to be sensitive to the scale of their deployment.

The tendency of users to embrace smart devices despite the risks is often framed as conditional on users’ perceptions of utility (page2018internet, ; li2021motivations, ; zheng2018user, ). However, calculative models favor the widespread deployment of sensors from the perspective of providers, independent of considerations about their utility for users. A related, albeit more nuanced, concern involves the possibility that users’ views on ’utility’ will shift as sensors become integrated across every facet of daily life. Smart devices are often presented as an opportunity to increase efficiency and liberate users from the demands to perform certain tasks and other forms of labor. But this kind of liberation promised by technologies is not unanimously regarded as beneficial. Some scholars have argued that increased automation will result in the loss of certain routines and habits that subsequently leads to a loss of meaning (borgmann1984technology, ; strong1995crazy, ) or contribute to the de-skilling of individuals (vallor2015moral, ). This perspective has been echoed by study participants who voice concerns about smart devices leading to laziness, complacency, and reduced self-sufficiency (balta2013social, ).

Lastly, it is worth considering how the pervasiveness of sensors may contribute to paranoia and an ever-present feeling of being ‘watched’. This feeling, abstract though it might be, is nevertheless latent in concerns that constant surveillance might limit freedom of movement, association, or expression, which are fundamental human rights that could be put at risk by ubiquitous sensor-focused AI systems. This phenomenon is often framed in terms of a ‘chilling effect’, where the mere presence of digital surveillance can cause individuals to modify or refrain from certain kinds of activities for fear of potential repercussions or adverse consequences. These chilling effects can have a detrimental impact on individuals, owing to the self-imposed limitations on their freedom and their ability to exercise their rights, as well as on society, by undermining the conditions for public discourse that is needed for a well-functioning democracy (townend2017freedom, ; penney2016chilling, ; penney2017internet, ; buchi2022chilling, ).

Importantly, the presence of a chilling effect is based on users’ perceptions which may or may not align with reality (laudrain2019smart, ). In other words, even if certain speech or behaviors would not actually result in an adverse consequence, users may nonetheless exhibit self-imposed constraints on their speech and behavior. As a result, users’ suspicions about data collection practices, whether well-founded or not, can contribute to a chilling effect. The concern is that a sensor-saturated world would not only increase the number of devices that may give rise to suspicion but also that the interconnectedness of sensors, and the inferences that can be drawn from cross-referencing data, will increasingly obscure the ramifications of data collection practices from users’ perspective.

4. Discussion

The materiality of sensors—i.e., the role their technical and physical properties play in the impacts they produce–gives rise to new dimensions of risk that are not fully captured by existing approaches to responsible AI development or in existing risk management frameworks. Further work is necessary to support efforts toward the responsible design, development, and deployment of sensors, and to produce more comprehensive frameworks for managing risk that can better addresses the risks posed by algorithmic techniques incorporated into sensor-based systems from the AIoT stage and beyond. Additionally, the limitations of this paper inevitably create opportunities for further work refining and filling in gaps in the above.

4.1. Call to Action

In the context of responsible AI, the need for interdisciplinary, collaborative efforts is well established. However, what has not been widely acknowledged is the growing need for collaborative efforts that focus on AI sensor-based systems as distinct from, or as special cases of, AI systems more broadly construed. The latter has brought important developments such as proposals for Model Cards for Model Reporting and Datasheets for Datasets (gebru2021datasheets, ; mitchell2019model, ). But only recently has there been parallel developments with respect to the former, such as the introduction of the ‘datasheet’ concept for sensors embedded with AI capabilities which, importantly, addresses both the software and hardware components of these systems (stewart23MLSensorDatasheets, ).

4.1.1. Responsible Sensor Architectures

To mitigate the risks outlined in Section 3.1, as well as the concerns outlined in Section 3.2, further progress must be made in the development of fundamentally responsible sensor architectures. A recent example of such an approach is the Machine Learning Sensor paradigm (warden23MLSensors, ). This approach suggests that sensors should process all data internally and transmit only abstracted, high-level data through a streamlined interface. This adheres to the principle of data minimization, ensuring that raw data remains exclusively accessible to the onboard sensor processor. This architecture not only enhances system self-containment, thereby improving auditability and accessibility, but also empowers users by maintaining control over their raw data, reducing the likelihood of unwarranted data exploitation by commercial and governmental entities. The proliferation of such a paradigm or the development of alternatives will be critical in ensuring that responsibility is a core design principle for future sensor systems. While substantial additional work needs to be done to adequately address the risks posed by AI sensor-based systems, this approach provides a strong starting point. Parallel efforts can also use the lens of calculative models to intervene in how sensor-focused AI is underwritten financially. Taxation and increased regulatory scrutiny of such devices can shift the development logics away from integrating sensors that facilitate unconstrained data collection in every device.

4.1.2. Activating Stakeholder Communities

The potential risks of deploying AI sensors at scale highlight, in particular, the critical role of communities in collaboratively spearheading a multifaceted approach to responsible sensor deployment. The algorithmic fairness, transparency, and accountability community, composed of technical experts like data scientists, engineers, and AI ethicists, alongside non-technical experts such as policy makers, social scientists, legal scholars, and affected communities, must work together to address these challenges. A range of different stakeholders, all of whom are experts in various dimensions of risk discussed above, should focus on collaborating to create sensing paradigms that are more responsible, aiming to alleviate the negative impacts stemming from the calculative models prevalent in today’s sensor ecosystem. They should also work on scoping transparency in ways that are relevant to the lived experience of interacting with sensors in order to promote the creation of transparent systems that make it easy for users to understand how their data is being used and for what purpose. Incorporating methodologies like the Machine Learning Technology Readiness Levels (MLTRL) framework can provide a structured approach to ensuring these systems are robust, reliable, and responsible from development through deployment (lavin2022technology, ).

This work can be paired with a focus on interpreting the societal impact of these technologies, advocating for the rights of affected communities, and helping to draft robust regulatory frameworks that govern the ethical use of sensor data. Sociotechnical approaches like this can also play a vital role in AI design (baxter2011socio, ; bauer2009designing, ), particularly in the public sector (leslie2019understanding, ; abbas2021socio, ), as well as fostering public awareness and education, ensuring that the implications of sensor technology are widely understood. Additionally, dialogues can be facilitated among various stakeholders, contributing to the creation of inclusive guidelines and best practices. This ensures that the deployment of sensor technologies considers a broad spectrum of perspectives and needs, balancing technological advancement with societal wellbeing (ada_lovelace_institute_algorithmic_2022, ; metcalf_relationship_2022, ).

4.2. Limitations

This paper is intended to draw attention to the dimensions of risk contributed to AI systems by sensors’ technical properties and, by extension, to the materiality of algorithmic systems in general. This attention is highlighted by intersecting the set of concerns germane to sensors with the dimensions of risk already present in risk management frameworks. The analysis presented in this paper is limited, however, by the fact that only a single risk management framework, namely the NIST AI RMF (nist_artificial_2023, ), is closely analyzed. The paper is further limited in terms of the additional risks posed by sensors discussed in 3.2. While the calculative models discussed in 2.2 are oriented towards producing a profusion of sensors, the scale of sensor deployment envisioned in this paper remains a future scenario for which empirical research is not currently available. Consequently, some of the additional risks identified, such as users’ attitudes and practices in a sensor-saturated world, remain speculative in nature. Moreover, these additional risks were identified through a multidisciplinary workshop that sought to enumerate as many types of risk as possible for sensors, but was necessarily constrained by the particular individuals and forms of expertise represented at that workshop, and any such list will therefore always be provisional and incomplete.

5. Conclusion

Sensors, as components of AI systems, require thoughtful design and governance to ensure the effects they produce in the world are responsible, fair, and transparent. Crucially, the responsible development of sensors, and how they are integrated into AI systems, is a crucial node in ensuring trustworthy systems. The trustworthiness characteristics outlined by the NIST AI RMF, for example, ought to apply to sensors as much as they do to any other components of algorithmic systems. To that end, this paper highlights how sensors present an opportunity to study, evaluate, and mitigate existing trustworthiness characteristics accounted for in risk management frameworks, as typified by the NIST AI RMF, and additional vectors of harm that are not currently captured risk management practices. Indeed, bringing the materiality of AI systems into sharper focus is a crucial practice for algorithmic accountability, in general. Adopting a materiality lens allows for the technical capabilities of sensors, which have evolved significantly over the past half-century, as well as their ubiquity, to receive explicit consideration as part of the risk management of AI systems. Given the proliferation of sensors driven by calculative models, this paper advocates for increased emphasis on responsible design and governance of sensing-enabled AI systems. It specifically calls for urgent attention to be directed towards developing responsible sensor architectures and regulatory frameworks concerning sensor development and associated data usage. While some recent work provides a commendable starting point, there is still much work to be done in this evolving field.

Acknowledgements.

We gratefully acknowledge Kasia Chmielinski, Sara Newman, and Matt Taylor from the Data Nutrition Project for providing helpful insights related to dataset transparency and ML model cards that helped shape the sensor 2.0 paradigm. We would also like to thank the attendees of the Harvard Radcliffe Exploratory seminar on ”MLsensors: safeguarding user privacy in the era of sensor intelligence” including Evgeni Gousev, Massimo Banzi, Sally Ward-Foxton, and Marco Zennaro. The vigorous discussion and feedback during the seminar helped evolve many of the ideas discussed herein.

References

[1] Executive Order 14110. Federal Register, 88(210):75191–75226, October 2023.
[2] ‌state of iot – spring 2023. Technical report, IoT Analytics, 2023.
[3] Roba Abbas, Jeremy Pitt, and Katina Michael. Socio-technical design for public interest technology. IEEE Transactions on Technology and Society, 2(2):55–61, 2021.
[4] Gary Anthes. Data brokers are watching you. Communications of the ACM, 58(1):28–30, January 2015.
[5] James J Anton and Dennis A Yao. Standard-setting consortia, antitrust, and high-technology industries. Antitrust LJ, 64:247, 1995.
[6] Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M Tyers, and Gregor Weber. Common voice: A massively-multilingual speech corpus. arXiv preprint arXiv:1912.06670, 2019.
[7] Hany F Atlam and Gary B Wills. Iot security, privacy, safety and ethics. Digital twin technologies and smart cities, pages 123–149, 2020.
[8] Nazmiye Balta-Ozkan, Rosemary Davidson, Martha Bicket, and Lorraine Whitmarsh. Social barriers to the adoption of smart homes. Energy policy, 63:363–374, 2013.
[9] Juan Aristi Baquero, Roger Burkhardt, Arvind Govindarajan, and Thomas Wallace. Derisking AI: Risk management in AI development. McKinsey, August 2020.
[10] Solon Barocas and Helen Nissenbaum. Big data’s end run around procedural privacy protections. Communications of the ACM, 57(11):31–33, 2014.
[11] Solon Barocas and Andrew D Selbst. Big data’s disparate impact. California law review, pages 671–732, 2016.
[12] Johannes M Bauer and Paulien M Herder. Designing socio-technical systems. In Philosophy of technology and engineering sciences, pages 601–630. Elsevier, 2009.
[13] Gordon Baxter and Ian Sommerville. Socio-technical systems: From design methods to systems engineering. Interacting with computers, 23(1):4–17, 2011.
[14] Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623, 2021.
[15] Natalia Besedovsky. Financialization as calculative practice: the rise of structured finance and the cultural and calculative transformation of credit rating agencies. Socio-Economic Review, 16(1):61–84, January 2018.
[16] Abeba Birhane. Algorithmic injustice: a relational ethics approach. Patterns, 2(2), 2021.
[17] Albert Borgmann. Technology and the character of contemporary life: A philosophical inquiry. University of Chicago Press, 1984.
[18] Lauren Bridges, Ingrid Burrington, Ann Chen, Zane Griffin Talley Cooper, Margaret Macherera, Jasper Mangwana, Vusumuzi Maphosa, and David Zezai. Geographies of digital wasting: Electronic waste from mine to discard and back again. 2023.
[19] Richard Briffault. A Government for Our Time? Business Improvement Districts and Urban Governance. Columbia Law Review, 99(2):365, March 1999.
[20] Moritz Büchi, Noemi Festic, and Michael Latzer. The chilling effects of digital dataveillance: A theoretical model and an empirical research agenda. Big Data & Society, 9(1):20539517211065368, 2022.
[21] Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pages 77–91. PMLR, 2018.
[22] Chiara Buratti, Andrea Conti, Davide Dardari, and Roberto Verdone. An overview on wireless sensor networks technology and evolution. Sensors, 9(9):6869–6896, 2009.
[23] Jenna Burrell. How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big data & society, 3(1):2053951715622512, 2016.
[24] Lawrence Busch. Standards: Recipes for reality. MIT Press: Cambridge, MA, 2011.
[25] Michel Callon, Yuval Millo, Fabian Muniesa, et al. Market devices. Technical report, 2007.
[26] Michel Callon and Fabian Muniesa. Peripheral vision: Economic markets as calculative collective devices. Organization studies, 26(8):1229–1250, 2005.
[27] A Feder Cooper, Emanuel Moss, Benjamin Laufer, and Helen Nissenbaum. Accountability in an algorithmic society: relationality, responsibility, and robustness in machine learning. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 864–876, 2022.
[28] A. Feder Cooper, Emanuel Moss, Benjamin Laufer, and Helen Nissenbaum. Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 864–876, Seoul Republic of Korea, June 2022. ACM.
[29] Nick Couldry and Ulises A Mejias. Data colonialism: Rethinking big data’s relation to the contemporary subject. Television & New Media, 20(4):336–349, 2019.
[30] Kate Crawford. The hidden biases in big data. Harvard business review, 1(4), 2013.
[31] David Danks and Alex John London. Algorithmic bias in autonomous systems. In Ijcai, volume 17, pages 4691–4697, 2017.
[32] Florentin Delaine, Bérengère Lebental, and Hervé Rivano. In situ calibration algorithms for environmental sensor networks: A review. IEEE Sensors Journal, 19(15):5968–5978, 2019.
[33] Deloitte. AI and risk management. Technical report, Deloitte, 2018.
[34] Payal Dhar. The carbon impact of artificial intelligence. Nat. Mach. Intell., 2(8):423–425, 2020.
[35] Catherine D’ignazio and Lauren F Klein. Data feminism. MIT press, 2023.
[36] Wenxiu Ding, Xuyang Jing, Zheng Yan, and Laurence T Yang. A survey on data fusion in internet of things: Towards secure and privacy-preserving fusion. Information Fusion, 51:129–144, 2019.
[37] Jesse Dodge, Taylor Prewitt, Remi Tachet des Combes, Erika Odmark, Roy Schwartz, Emma Strubell, Alexandra Sasha Luccioni, Noah A Smith, Nicole DeCario, and Will Buchanan. Measuring the carbon intensity of ai in cloud instances. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1877–1894, 2022.
[38] Upol Ehsan, Ranjit Singh, Jacob Metcalf, and Mark Riedl. The algorithmic imprint. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1305–1317, 2022.
[39] Robert C. Ellickson. Cities and Homeowners Associations. University of Pennsylvania Law Review, 130(6):1519, June 1982.
[40] Wilfried Elmenreich. An introduction to sensor fusion. Vienna University of Technology, Austria, 502:1–28, 2002.
[41] Niklas Elmqvist. Data analytics anywhere and everywhere. Communications of the ACM, 66(12):52–63, 2023.
[42] Virginia Eubanks. Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press, 2018.
[43] Jan Fernback. Sousveillance: Communities of resistance to the surveillance environment. Telematics and Informatics, 30(1):11–21, 2013.
[44] Luciano Floridi, Matthias Holweg, Mariarosaria Taddeo, Javier Amaya Silva, Jakob Mökander, and Wen Yuni. capAI: A procedure for conducting conformity assessment of AI systems in line with the EU Artificial Intelligene Act. Technical report, March 2022.
[45] A Michael Froomkin. Big data: Destroyer of informed consent. Yale JL & Tech., 21:27, 2019.
[46] Gregory M Galdino, James E Vogel, and Craig A Vander Kolk. Standardizing digital photography: it’s not all in the eye of the beholder. Plastic and reconstructive surgery, 108(5):1334–1344, 2001.
[47] Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. Datasheets for datasets. Communications of the ACM, 64(12):86–92, 2021.
[48] Cheng-Yan Guo, Wen-Yao Huang, Hao-Ching Chang, and Tung-Li Hsieh. Calibrating oxygen saturation measurements for different skin colors using the individual typology angle. IEEE Sensors Journal, 2023.
[49] Udit Gupta, Young Geun Kim, Sylvia Lee, Jordan Tse, Hsien-Hsin S Lee, Gu-Yeon Wei, David Brooks, and Carole-Jean Wu. Chasing carbon: The elusive environmental footprint of computing. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pages 854–867. IEEE, 2021.
[50] Ian Hacking. Biopower and the avalanche of printed numbers. Biopower: Foucault and beyond, 65, 2015.
[51] Kristian Bondo Hansen. Model talk: Calculative cultures in quantitative finance. Science, Technology, & Human Values, 46(3):600–627, 2021.
[52] Caner Hazirbas, Joanna Bitton, Brian Dolhansky, Jacqueline Pan, Albert Gordo, and Cristian Canton Ferrer. Towards measuring fairness in ai: the casual conversations dataset. IEEE Transactions on Biometrics, Behavior, and Identity Science, 4(3):324–332, 2021.
[53] Sara Hooker, Nyalleng Moorosi, Gregory Clark, Samy Bengio, and Emily Denton. Characterising bias in compressed models, 2020.
[54] Ada Lovelace Institute. Algorithmic Impact Assessment: A case study in healthcare. Technical report, Ada Lovelace Institute, 2022.
[55] Abigail Z Jacobs and Hanna Wallach. Measurement and fairness. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 375–385, 2021.
[56] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1–2):1–210, 2021.
[57] Mark Kidd. Energy and e-waste: The AI tsunamis, October 2023.
[58] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
[59] Loïc Lannelongue, Jason Grealey, and Michael Inouye. Green algorithms: quantifying the carbon footprint of computation. Advanced science, 8(12):2100707, 2021.
[60] Arthur PB Laudrain. Smart-city technologies, government surveillance and privacy: Assessing the potential for chilling effects and existing safeguards in the echr. 2019.
[61] Alexander Lavin, Ciarán M Gilligan-Lee, Alessya Visnjic, Siddha Ganju, Dava Newman, Sujoy Ganguly, Danny Lange, Atílím Güneş Baydin, Amit Sharma, Adam Gibson, et al. Technology readiness levels for machine learning systems. Nature Communications, 13(1):6039, 2022.
[62] John Law and Annemarie Mol. Notes on materiality and sociality. The sociological review, 43(2):274–294, 1995.
[63] David Leslie. Understanding artificial intelligence ethics and safety. arXiv preprint arXiv:1906.05684, 2019.
[64] Nancy G Leveson. Engineering a safer world: Systems thinking applied to safety. The MIT Press, 2016.
[65] Wenda Li, Tan Yigitcanlar, Isil Erol, and Aaron Liu. Motivations, barriers and risks of smart home adoption: From systematic literature review to conceptual framework. Energy Research & Social Science, 80:102211, 2021.
[66] Leah A Lievrouw. Materiality and media in communication and technology studies: An unfinished project. Media technologies: Essays on communication, materiality, and society, pages 21–51, 2014.
[67] Leah A Lievrouw and Sonia Livingstone. Introduction to the updated student edition. Handbook of new media: Social shaping and social consequences of ICTs, pages 1–14, 2006.
[68] Donald MacKenzie. An Engine, Not a Camera: How Financial Models Shape Markets. The MIT Press, Cambridge, Mass., first edition edition, August 2008.
[69] Luca Mainetti, Luigi Patrono, and Antonio Vilei. Evolution of wireless sensor networks towards the internet of things: A survey. In SoftCOM 2011, 19th international conference on software, telecommunications and computer networks, pages 1–6. IEEE, 2011.
[70] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6):1–35, 2021.
[71] Jacob Metcalf, Emanuel Moss, Ranjit Singh, Emnet Tafese, and Elizabeth Anne Watkins. A relationship and not a thing: A relational approach to algorithmic accountability and assessment documentation. arXiv preprint, page 19, March 2022.
[72] Jacob Metcalf, Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, and Madeleine Clare Elish. Algorithmic impact assessments and accountability: The co-construction of impacts. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 735–746, 2021.
[73] Daniel Miller. Materiality. Duke University Press, 2020.
[74] Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency, pages 220–229, 2019.
[75] Shira Mitchell, Eric Potash, Solon Barocas, Alexander D’Amour, and Kristian Lum. Algorithmic fairness: Choices, assumptions, and definitions. Annual Review of Statistics and Its Application, 8:141–163, 2021.
[76] Fabian Muniesa, Yuval Millo, and Michel Callon. An introduction to market devices. The sociological review, 55(2_suppl):1–12, 2007.
[77] Dawn Nafus. Quantified: Biosensing technologies in everyday life. MIT Press, 2016.
[78] Thien Duc Nguyen, Phillip Rieger, Markus Miettinen, and Ahmad-Reza Sadeghi. Poisoning attacks on federated learning-based iot intrusion detection system. In Proc. Workshop Decentralized IoT Syst. Secur.(DISS), pages 1–7, 2020.
[79] Helen Nissenbaum. Privacy in context: Technology, policy, and the integrity of social life. Stanford University Press: Stanford, CA, 2020.
[80] NIST. Artificial Intelligence Risk Management Framework (AI RMF 1.0). Technical Report NIST AI 100-1, National Institute of Standards and Technology (U.S.), Gaithersburg, MD, January 2023.
[81] Safiya Umoja Noble. Algorithms of oppression: data discrimination in the age of Google. New York University Press, 2018.
[82] OECD. Advancing accountability in AI: Governing and managing risks throughout the lifecycle for trustworthy AI. OECD Digital Economy Papers 349, February 2023. Series: OECD Digital Economy Papers Volume: 349.
[83] Cathy O’neil. Weapons of math destruction: How big data increases inequality and threatens democracy. Crown, 2017.
[84] Xinru Page, Paritosh Bahirat, Muhammad I Safi, Bart P Knijnenburg, and Pamela Wisniewski. The internet of what? understanding differences in perceptions and adoption for the internet of things. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(4):1–22, 2018.
[85] Jon Penney. Internet surveillance, regulation, and chilling effects online: A comparative case study. Regulation, and Chilling Effects Online: A Comparative Case Study (May 27, 2017), 6(2), 2017.
[86] Jonathon W Penney. Chilling effects: Online surveillance and wikipedia use. Berkeley Tech. LJ, 31:117, 2016.
[87] Trevor Pinch. Technology and institutions: Living in a material world. Theory and society, 37:461–483, 2008.
[88] Shvetank Prakash, Matthew Stewart, Colby Banbury, Mark Mazumder, Pete Warden, Brian Plancher, and Vijay Janapa Reddi. Is tinyml sustainable? assessing the environmental impacts of machine learning on microcontrollers. Communications of the ACM, 66(11):68–77, Nov 2023.
[89] Marvin Rausand. Reliability of safety-critical systems: theory and applications. John Wiley & Sons, 2014.
[90] Fakhitah Ridzuan and Wan Mohd Nazmee Wan Zainon. A review on data cleansing methods for big data. Procedia Computer Science, 161:731–738, 2019.
[91] Lorna Roth. Looking at shirley, the ultimate norm: Colour balance, image technologies, and cognitive equity. Canadian Journal of Communication, 34(1):111–136, 2009.
[92] Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Semenova, and Chudi Zhong. Interpretable machine learning: Fundamental principles and 10 grand challenges. Statistic Surveys, 16:1–85, 2022.
[93] Irfan Saif and Beena Ammanath. ‘Trustworthy AI’ is a framework to help manage unique risk, March 2020.
[94] E Scott. The trouble with informed consent in smart cities. IAAP, Feb, 28, 2019.
[95] Andrew D Selbst. An institutional view of algorithmic impact. Harvard Journal of Law & Technology, 35(1), 2021.
[96] Muhammad Yasir Shabir, Gianluca Torta, Andrea Basso, and Ferruccio Damiani. Toward secure tinyml on a standardized ai architecture. In Device-Edge-Cloud Continuum: Paradigms, Architectures and Applications, pages 121–139. Springer, 2023.
[97] Renee Shelby, Shalaleh Rismani, Kathryn Henne, AJung Moon, Negar Rostamzadeh, Paul Nicholas, N’Mah Yilla-Akbari, Jess Gallegos, Andrew Smart, Emilio Garcia, et al. Sociotechnical harms of algorithmic systems: Scoping a taxonomy for harm reduction. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, pages 723–741, 2023.
[98] Anne Logsdon Smith. Alexa, who owns my pillow talk: Contracting, collateralizing, and monetizing consumer privacy through voice-captured personal data. Cath. UJL & Tech, 27:187, 2018.
[99] Nathalie A Smuha. Beyond the individual: governing ai’s societal harm. Internet Policy Review, 10(3), 2021.
[100] Matthew Stewart, Pete Warden, Yasmine Omri, Shvetank Prakash, Joao Santos, Shawn Hymel, Benjamin Brown, Jim MacArthur, Nat Jeffries, Brian Plancher, and Vijay Janapa Reddi. Datasheets for machine learning sensors, 2023.
[101] David Strong. Crazy mountains: Learning from wilderness to weigh technology. SUNY Press, 1995.
[102] Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and policy considerations for deep learning in nlp. arXiv preprint arXiv:1906.02243, 2019.
[103] Latanya Sweeney. Simple demographics often identify people uniquely. Health (San Francisco), 671(2000):1–34, 2000.
[104] Hui Yie Teh, Andreas W Kempa-Liehr, and Kevin I-Kai Wang. Sensor data quality: A systematic review. Journal of Big Data, 7(1):1–49, 2020.
[105] Jim Thatcher, David O’Sullivan, and Dillon Mahmoudi. Data colonialism through accumulation by dispossession: New metaphors for daily data. Environment and Planning D: Society and Space, 34(6):990–1006, 2016.
[106] Vale Tolpegin, Stacey Truex, Mehmet Emre Gursoy, and Ling Liu. Data poisoning attacks against federated learning systems. In Computer Security–ESORICS 2020: 25th European Symposium on Research in Computer Security, ESORICS 2020, Guildford, UK, September 14–18, 2020, Proceedings, Part I 25, pages 480–501. Springer, 2020.
[107] Judith Townend. Freedom of expression and the chilling effect. Routledge companion to media and human rights, pages 73–82, 2017.
[108] Amy JC Trappey, Charles V Trappey, Usharani Hareesh Govindarajan, Allen C Chuang, and John J Sun. A review of essential standards and patent landscapes for the internet of things: A key enabler for industry 4.0. Advanced Engineering Informatics, 33:208–229, 2017.
[109] Shannon Vallor. Moral deskilling and upskilling in a new machine age: Reflections on the ambiguous future of character. Philosophy & Technology, 28:107–124, 2015.
[110] Aimee Van Wynsberghe. Sustainable ai: Ai for sustainability and the sustainability of ai. AI and Ethics, 1(3):213–218, 2021.
[111] Peter-Paul Verbeek. Ambient intelligence and persuasive technology: The blurring boundaries between human and technology. Nanoethics, 3(3):231–242, 2009.
[112] Xuequn Wang, Tanya Jane McGill, and Jane E Klobas. I want it anyway: Consumer perceptions of smart home devices. Journal of Computer Information Systems, 2018.
[113] Pete Warden and Daniel Situnayake. Tinyml: Machine learning with tensorflow lite on arduino and ultra-low-power microcontrollers. O’Reilly Media, 2019.
[114] Pete Warden, Matthew Stewart, Brian Plancher, Sachin Katti, and Vijay Janapa Reddi. Machine learning sensors: A design paradigm for the future of intelligent sensors. Communications of the ACM, 66(11):25–28, Nov 2023.
[115] Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359, 2021.
[116] Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, et al. Taxonomy of risks posed by language models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 214–229, 2022.
[117] Carole-Jean Wu, Ramya Raghavendra, Udit Gupta, Bilge Acun, Newsha Ardalani, Kiwan Maeng, Gloria Chang, Fiona Aga, Jinshi Huang, Charles Bai, et al. Sustainable ai: Environmental implications, challenges and opportunities. Proceedings of Machine Learning and Systems, 4:795–813, 2022.
[118] Juan Ye, Graeme Stevenson, and Simon Dobson. Detecting abnormal events on binary sensors in smart home environments. Pervasive and Mobile Computing, 33:32–49, 2016.
[119] Serena Zheng, Noah Apthorpe, Marshini Chetty, and Nick Feamster. User perceptions of smart home iot privacy. Proceedings of the ACM on human-computer interaction, 2(CSCW):1–20, 2018.