ChainStream: A Stream-based LLM Agent Framework for Continuous Context Sensing and Sharing

This paper introduces ChainStream, an LLM-based framework for building and serving context-aware AI agents. Driven by the goal to enable context awareness of LLM agents and flexible information sharing between them, we adopt a stream-based design, in which the agents are responsible for producing and transforming different types of streams, including the low-level sensing signals and high-level semantic events. The streams can be shared between different agents at the system level, so that developers can build new features upon existing streams. Richer features and higher levels of intelligence can be obtained by agents collectively transforming the streams. ChainStream offers an easy-to-use programming interface to facilitate agent development and a runtime system that supports high-performance scalable agent serving. The system design is inspired by microkernel and dataflow computation. We demonstrate the feasibility and usefulness of ChainStream with several use cases in personal assistant, smart home, and business intelligence. The code is open-sourced at https://github.com/MobileLLM/ChainStream.