Computer Science > Computation and Language
[Submitted on 21 Aug 2023 (v1), revised 9 Oct 2023 (this version, v3), latest version 26 Aug 2024 (v6)]
Title:PlatoLM: Teaching LLMs via a Socratic Questioning User Simulator
View PDFAbstract:The unparalleled performance of closed-sourced ChatGPT has sparked efforts towards its democratization, with notable strides made by leveraging real user and ChatGPT conversations, as evidenced by Vicuna. However, due to challenges in gathering conversations involving human participation, current endeavors like Baize and UltraChat aim to automatically generate conversational data. They primarily rely on ChatGPT conducting roleplay to simulate human behaviors based on instructions rather than genuine learning from humans, resulting in limited scope, diminished diversity, and an absence of genuine multi-round conversational dynamics. To address the above issues, we target human questions extracted from genuine human-machine conversations as a learning goal and train a user simulator called `Socratic' to produce a high-quality human-centric synthetic conversation dataset. Subsequently, this dataset was used to train our assistant model, named `PlatoLM'. Experimentally, PlatoLM outpaces baseline models in both Vicuna-Bench and MT-Bench by pairwise comparison when considering equivalent training set sizes, and manual evaluation also shows that our model is highly competitive. Impressively, when fine-tuned with the latest LLaMA 2 model, PlatoLM achieves the SOTA performance among 7B models (including LLaMA-2-7B-chat and Vicuna-7B) in MT-Bench benchmark and in Alpaca-Eval benchmark, it ranks second among 7B models, even beating some larger scale models (including LLaMA-2-13B-chat and GPT-3.5). Further in-depth analysis demonstrates the scalability and transferability of our approach. The code is available at this https URL.
Submission history
From: Chuyi Kong [view email][v1] Mon, 21 Aug 2023 06:51:56 UTC (685 KB)
[v2] Wed, 23 Aug 2023 14:33:53 UTC (685 KB)
[v3] Mon, 9 Oct 2023 15:39:21 UTC (2,347 KB)
[v4] Thu, 12 Oct 2023 08:50:19 UTC (2,346 KB)
[v5] Mon, 27 May 2024 16:32:16 UTC (3,222 KB)
[v6] Mon, 26 Aug 2024 08:52:44 UTC (3,132 KB)
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.