Evaluation of large language models for discovery of gene set function
Authors:
Mengzhou Hu,
Sahar Alkhairy,
Ingoo Lee,
Rudolf T. Pillich,
Dylan Fong,
Kevin Smith,
Robin Bachelder,
Trey Ideker,
Dexter Pratt
Abstract:
Gene set analysis is a mainstay of functional genomics, but it relies on curated databases of gene functions that are incomplete. Here we evaluate five Large Language Models (LLMs) for their ability to discover the common biological functions represented by a gene set, substantiated by supporting rationale, citations and a confidence assessment. Benchmarking against canonical gene sets from the Ge…
▽ More
Gene set analysis is a mainstay of functional genomics, but it relies on curated databases of gene functions that are incomplete. Here we evaluate five Large Language Models (LLMs) for their ability to discover the common biological functions represented by a gene set, substantiated by supporting rationale, citations and a confidence assessment. Benchmarking against canonical gene sets from the Gene Ontology, GPT-4 confidently recovered the curated name or a more general concept (73% of cases), while benchmarking against random gene sets correctly yielded zero confidence. Gemini-Pro and Mixtral-Instruct showed ability in naming but were falsely confident for random sets, whereas Llama2-70b had poor performance overall. In gene sets derived from 'omics data, GPT-4 identified novel functions not reported by classical functional enrichment (32% of cases), which independent review indicated were largely verifiable and not hallucinations. The ability to rapidly synthesize common gene functions positions LLMs as valuable 'omics assistants.
△ Less
Submitted 1 April, 2024; v1 submitted 7 September, 2023;
originally announced September 2023.
Drugst.One -- A plug-and-play solution for online systems medicine and network-based drug repurposing
Authors:
Andreas Maier,
Michael Hartung,
Mark Abovsky,
Klaudia Adamowicz,
Gary D. Bader,
Sylvie Baier,
David B. Blumenthal,
Jing Chen,
Maria L. Elkjaer,
Carlos Garcia-Hernandez,
Mohamed Helmy,
Markus Hoffmann,
Igor Jurisica,
Max Kotlyar,
Olga Lazareva,
Hagai Levi,
Markus List,
Sebastian Lobentanzer,
Joseph Loscalzo,
Noel Malod-Dognin,
Quirin Manz,
Julian Matschinske,
Miles Mee,
Mhaned Oubounyt,
Alexander R. Pico
, et al. (14 additional authors not shown)
Abstract:
In recent decades, the development of new drugs has become increasingly expensive and inefficient, and the molecular mechanisms of most pharmaceuticals remain poorly understood. In response, computational systems and network medicine tools have emerged to identify potential drug repurposing candidates. However, these tools often require complex installation and lack intuitive visual network mining…
▽ More
In recent decades, the development of new drugs has become increasingly expensive and inefficient, and the molecular mechanisms of most pharmaceuticals remain poorly understood. In response, computational systems and network medicine tools have emerged to identify potential drug repurposing candidates. However, these tools often require complex installation and lack intuitive visual network mining capabilities. To tackle these challenges, we introduce Drugst.One, a platform that assists specialized computational medicine tools in becoming user-friendly, web-based utilities for drug repurposing. With just three lines of code, Drugst.One turns any systems biology software into an interactive web tool for modeling and analyzing complex protein-drug-disease networks. Demonstrating its broad adaptability, Drugst.One has been successfully integrated with 21 computational systems medicine tools. Available at https://drugst.one, Drugst.One has significant potential for streamlining the drug discovery process, allowing researchers to focus on essential aspects of pharmaceutical treatment research.
△ Less
Submitted 4 July, 2023; v1 submitted 24 May, 2023;
originally announced May 2023.
