Towards a Framework for Openness in Foundation Models: Proceedings from the Columbia Convening on Openness in Artificial Intelligence
Authors:
Adrien Basdevant,
Camille François,
Victor Storchan,
Kevin Bankston,
Ayah Bdeir,
Brian Behlendorf,
Merouane Debbah,
Sayash Kapoor,
Yann LeCun,
Mark Surman,
Helen King-Turvey,
Nathan Lambert,
Stefano Maffulli,
Nik Marda,
Govind Shivkumar,
Justine Tunney
Abstract:
Over the past year, there has been a robust debate about the benefits and risks of open sourcing foundation models. However, this discussion has often taken place at a high level of generality or with a narrow focus on specific technical attributes. In part, this is because defining open source for foundation models has proven tricky, given its significant differences from traditional software dev…
▽ More
Over the past year, there has been a robust debate about the benefits and risks of open sourcing foundation models. However, this discussion has often taken place at a high level of generality or with a narrow focus on specific technical attributes. In part, this is because defining open source for foundation models has proven tricky, given its significant differences from traditional software development. In order to inform more practical and nuanced decisions about opening AI systems, including foundation models, this paper presents a framework for grappling with openness across the AI stack. It summarizes previous work on this topic, analyzes the various potential reasons to pursue openness, and outlines how openness varies in different parts of the AI stack, both at the model and at the system level. In doing so, its authors hope to provide a common descriptive framework to deepen a nuanced and rigorous understanding of openness in AI and enable further work around definitions of openness and safety in AI.
△ Less
Submitted 17 May, 2024;
originally announced May 2024.
On the Societal Impact of Open Foundation Models
Authors:
Sayash Kapoor,
Rishi Bommasani,
Kevin Klyman,
Shayne Longpre,
Ashwin Ramaswami,
Peter Cihon,
Aspen Hopkins,
Kevin Bankston,
Stella Biderman,
Miranda Bogen,
Rumman Chowdhury,
Alex Engler,
Peter Henderson,
Yacine Jernite,
Seth Lazar,
Stefano Maffulli,
Alondra Nelson,
Joelle Pineau,
Aviya Skowron,
Dawn Song,
Victor Storchan,
Daniel Zhang,
Daniel E. Ho,
Percy Liang,
Arvind Narayanan
Abstract:
Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to bo…
▽ More
Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to both their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate. Overall, our work helps support a more grounded assessment of the societal impact of open foundation models by outlining what research is needed to empirically validate their theoretical benefits and risks.
△ Less
Submitted 27 February, 2024;
originally announced March 2024.
