research-article

Get real: realism metrics for robust limit order book market simulations

Authors:

Svitlana Vyetrenko,

Mahmoud Mahfouz,

Danial Dervovic,

Manuela Veloso,

Tucker BalchAuthors Info & Claims

ICAIF '20: Proceedings of the First ACM International Conference on AI in Finance

Article No.: 2, Pages 1 - 8

https://doi.org/10.1145/3383455.3422561

Published: 07 October 2021 Publication History

Abstract

Market simulation is an increasingly important method for evaluating and training trading strategies and testing "what if" scenarios. The extent to which results from these simulations can be trusted depends on how realistic the environment is for the strategies being tested. As a step towards providing benchmarks for realistic simulated markets, we enumerate measurable stylized facts of limit order book (LOB) markets across multiple asset classes from the literature. We apply these metrics to data from real markets and compare the results to data originating from simulated markets. We illustrate their use in five different simulated market configurations: The first (market replay) is frequently used in practice to evaluate trading strategies; the other four are interactive agent based simulation (IABS) configurations which combine zero intelligence agents, and agents with limited strategic behavior. These simulated agents rely on an internal "oracle" that provides a fundamental value for the asset. In traditional IABS methods the fundamental originates from a mean reverting random walk. We show that markets exhibit more realistic behavior when the fundamental arises from historical market data. We further experimentally illustrate the effectiveness of IABS techniques as opposed to market replay.

References

[1]

Frédéric Abergel, Marouane Anane, Anirban Chakraborti, Aymen Jedidi, and Ioane Muni Toke. 2016. Limit order books. Cambridge University Press.

[2]

Robert Almgren and Neil Chriss. 1999. Optimal execution of portfolio transactions. Journal of Risk (1999).

[3]

Robert Almgren, Chee Thum, Emmanuel Hauptmann, and Hong Li. 2005. Direct Estimation of Equity Market Impact. RISK 18 (04 2005).

[4]

Thierry Ane and Helyette Geman. 2000. Order Flow, Transaction Clock, and Normality of Asset Returns. The Journal of Finance 55, 5 (2000), 2259--2284.

[5]

Tucker Hybinette Balch, Mahmoud Mahfouz, Joshua Lockhart, Maria Hybinette, and David Byrd. 2019. How to Evaluate Trading Strategies: Single Agent Market Replay or Multiple Agent Interactive Simulation? arXiv preprint arXiv:1906.12010 (2019).

[6]

Giuseppe Ballocchi, Michel M. Dacorogna, Carl M. Hopman, Ulrich A. Muller, and Richard B. Olsen. 1999. The intraday multivariate structure of the Eurofutures markets. Journal of Empirical Finance 6, 5 (December 1999), 479--513.

[7]

Eric Bonabeau. 2002. Agent-Based Modeling: Methods And Techniques for Simulating Human Systems. Proceedings of the National Academy of Sciences of the United States of America (06 2002).

[8]

Jean-Philippe Bouchaud. 2010. Price impact. Encyclopedia of quantitative finance (2010).

[9]

Jean-Philippe Bouchaud, Julius Bonart, Jonathan Donier, and Martin Gould. 2018. Trades, quotes and prices: financial markets under the microscope. Cambridge University Press, Cambridge.

[10]

Jean-Philippe Bouchaud, Yuval Gefen, Marc Potters, and Matthieu Wyart. 2003. Fluctuations and response in financial markets: the subtle nature of random price changes. Quantitative Finance (07 2003).

[11]

Jean-Philippe Bouchaud, Marc Mezard, and Marc Potters. 2002. Statistical properties of stock order books: Empirical results and models. Quantitative Finance 2 (04 2002), 301.

[12]

Olivier Brandouy, Angelo Corelli, Iryna Veryzhenko, and Roger Waldeck. 2012. A re-examination of the "zero is enough" hypothesis in the emergence of financial stylized facts. Journal of Economic Interaction and Coordination 7 (10 2012), 223--248.

[13]

H Buehler, L Gonon, J Teichmann, and B Wood. 2018. Deep Hedging. arXiv:1802.03042 (2018).

[14]

David Byrd. 2019. Explaining Agent-Based Financial Market Simulation. arXiv preprint arXiv:1909.11650 (2019).

[15]

David Byrd, Maria Hybinette, and Tucker Hybinette Balch. 2019. ABIDES: Towards High-Fidelity Market Simulation for AI Research. arXiv preprint arXiv:1904.12066 (2019).

[16]

CFTC. 2010. Findings regarding the market events of May 6, 2010. Report of the staffs of the CFTC and SEC to the Joint Advisory Committee on Emerging Regulatory Issues. (2010).

[17]

Tanmoy Chakraborty and Michael Kearns. 2011. Market making and mean reversion. In Proceedings of the 12th ACM conference on Electronic commerce. ACM, 307--314.

Digital Library

[18]

Rama Cont. 2001. Empirical properties of asset returns: stylized facts and statistical issues. (2001).

[19]

Wei Cui and Anthony Brabazon. 2012. An agent-based modeling approach to study price impact. In 2012 IEEE Conference on Computational Intelligence for Financial Engineering and Economics. 1--8.

[20]

Michael M. Dacorogna, Ulrich Alfons Müller, Robert J. Nagler, Richard B. Olsen, and Olivier V. Pictet. 1993. A geographical model for the daily and weekly seasonal volatility in the foreign exchange market.

[21]

Vincent Darley and Alexander V. Outkin. 2007. Nasdaq Market Simulation: Insights on a Major Market from the Science of Complex Adaptive Systems.

Digital Library

[22]

J Farmer, Paolo Patelli, and Ilija Zovko. 2005. The Predictive Power of Zero Intelligence in Financial Markets. Proceedings of the National Academy of Sciences of the United States of America 102 (03 2005), 2254--9.

[23]

Jim Gatheral. 2010. No-Dynamic-Arbitrage and Market Impact. Quantitative Finance (2010).

[24]

Dhananjay K Gode and Shyam Sunder. 1993. Allocative efficiency of markets with zero-intelligence traders: Market as a partial substitute for individual rationality. Journal of political economy 101, 1 (1993), 119--137.

[25]

Jakob Grazzini, Matteo Richiardi, and Efthymios Tsionas. 2015. Bayesian Estimation of Agent-Based Models. Journal of Economic Dynamics and Control (11 2015).

[26]

Junling Hu and Michael P. Wellman. 1998. Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm. In Proceedings of the Fifteenth International Conference on Machine Learning (ICML '98).

[27]

Bruce I. Jacobs, Kenneth N. Levy, and Harry M. Markowitz. 2004. Financial Market Simulation. The Journal of Portfolio Management 30, 5 (2004), 142--152. arXiv:https://jpm.pm-research.com/content/30/5/142.full.pdf

[28]

Bruce I Jacobs, Kenneth N Levy, and Harry M Markowitz. 2004. Financial market simulation. The Journal of Portfolio Management 30, 5 (2004), 142--152.

[29]

Bruce I Jacobs, Kenneth N Levy, and Harry M Markowitz. 2010. Simulating security markets in dynamic and equilibrium modes. Financial Analysts Journal 66, 5 (2010), 42--53.

[30]

Albert S Kyle. 1985. Continuous auctions and insider trading. Econometrica: Journal of the Econometric Society (1985), 1315--1335.

[31]

Sandrine Leal and Mauro Napoletano. 2016. Market Stability vs. Market Resilience: Regulatory Policies Experiments in an Agent Based Model with Low- and High-Frequency Trading. Journal of Economic Behavior and Organization (12 2016).

[32]

B. LeBaron. 2001. A builder's guide to agent-based financial markets. Quantitative Finance 1, 2 (2001), 254--261.

[33]

Blake LeBaron and Ryuichi Yamamoto. 2007. Long-Memory in an Order-Driven Market.

[34]

Moshe Levy, Haim Levy, and Sorin Solomon. 1994. A microscopic model of the stock market: cycles, booms, and crashes. Economics Letters 45, 1 (1994), 103--111.

[35]

Junyi Li, Xintong Wang, Yaoyang Lin, Arunesh Sinha, and Michael P. Wellman. 2019. Generating Realistic Stock Market Order Streams. https://openreview.net/forum?id=rke41hC5Km

[36]

Li Li and Zuliu Hu. 1998. Responses of the Stock Market to Macroeconomic Announcements Across Economic States. IMF Working Paper (1998).

[37]

Andrew Lo. 2004. Reconciling Efficient Markets with Behavioral Finance: The Adaptive Markets Hypothesis. Journal of Investment Consulting 7 (01 2004).

[38]

David Lu. 2017. Agent Inspired Trading Using Recurrent Reinforcement Learning and LSTM Neural Networks. arXiv:1707.07338 (07 2017).

[39]

C Macal and M North. 2010. Tutorial on agent-based modeling and simulation. Journal of Simulation (2010).

[40]

Y. Nevmyvaka, Y. Feng, and M. Kearns. 2006. Reinforcement Learning for Optimized Trade Execution. In Proceedings of the 23rd International Conference on Machine Learning. Pittsburg, PA.

[41]

Mark Paddrik, Roy Hayes, Andrew Todd, Steve Yang, Peter Beling, and William Scherer. 2012. An agent based model of the E-Mini S&P 500 applied to Flash Crash analysis. In 2012 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr). IEEE, 1--8.

[42]

Imon Palit, Steve Phelps, and Wing Lon Ng. 2012. Can a Zero-Intelligence Plus Model Explain the Stylized Facts of Financial Time Series Data? 653--660.

[43]

N. Ruiz, S. Schulter, and M. Chandraker. 2019. Learning to Simulate. In Proceeding of the International Conference on Learning Representations. New Orleans, LA.

[44]

L. Julian Schvartzman and Michael P. Wellman. 2009. Stronger CDA Strategies Through Empirical Game-theoretic Analysis and Reinforcement Learning. In Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 1.

[45]

T. Spooner, J. Fearnley, R. Savani, and A. Koukorinis. 2018. Market Making via Reinforcement Learning. In Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems. Stockholm, Sweden, 434--442.

[46]

Thomas Spooner, John Fearnley, Rahul Savani, and Andreas Koukorinis. 2018. Market Making via Reinforcement Learning. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems.

Digital Library

[47]

Gabriele Tedeschi, Maria Recchioni, and Mauro Gallegati. 2013. A Calibration Procedure for Analyzing Stock Price Dynamics in an Agent-Based Framework. http://ssrn.com/abstract=2330739 (09 2013).

[48]

Svitlana Vyetrenko and Shaojie Xu. 2019. Risk-Sensitive Compact Decision Trees for Autonomous Execution in Presence of Simulated Market Response. In ICML 2019 Workshop on AI in Finance.

[49]

Elaine Wah and Michael P Wellman. 2013. Latency arbitrage, market fragmentation, and efficiency: a two-market model. In Proceedings of the fourteenth ACM conference on Electronic commerce. ACM, 855--872.

Digital Library

[50]

Elaine Wah, Mason Wright, and Michael P Wellman. 2017. Welfare effects of market making in continuous double auctions. Journal of Artificial Intelligence Research 59 (2017), 613--650.

Digital Library

[51]

Xintong Wang and Michael P Wellman. 2017. Spoofing the limit order book: An agent-based model. In Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems. International Foundation for Autonomous Agents and Multiagent Systems, 651--659.

[52]

Michael P Wellman. 2006. Methods for empirical game-theoretic analysis. In AAAI. 1552--1556.

[53]

S. Yang, M. Paddrik, R. Hayes, A. Todd, A. Kirilenko, P. Beling, and W. Scherer. 2012. Behavior based learning in identifying High Frequency Trading strategies. In 2012 IEEE Conference on Computational Intelligence for Financial Engineering Economics (CIFEr).

Cited By

Evans BGanesh SDastani MSichman JAlechina NDignum V(2024)Learning and Calibrating Heterogeneous Bounded Rational Market Behaviour with Multi-agent Reinforcement LearningProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3662904(534-543)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3662904
Brusatin SPadoan TColetta ADelli Gatti DGlielmo A(2024)Simulating the Economic Impact of Rationality through Reinforcement Learning and Agent-Based ModellingProceedings of the 5th ACM International Conference on AI in Finance10.1145/3677052.3698621(159-167)Online publication date: 14-Nov-2024
https://dl.acm.org/doi/10.1145/3677052.3698621
Yang ZZhong MYang P(2024)Evolutionary Dynamic Optimization-Based Calibration Framework for Agent-Based Financial Market Simulators2024 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC60901.2024.10612133(01-08)Online publication date: 30-Jun-2024
https://doi.org/10.1109/CEC60901.2024.10612133
Show More Cited By

Recommendations

Discovering Traders' Heterogeneous Behavior in High-Frequency Financial Data

This paper develops a utility-based heterogeneous agent model for empirically investigating intraday traders' behaviors. Two types of agents, which consist of fundamental traders and technical analysts, are considered in the proposed model. They differ ...
Next-generation securities market systems: an experimental investigation of quote-driven and order-driven trading
Special section: Strategic and competitive information systems

Several major securities markets including Nasdaq in the United States and the London Stock Exchange's SEAQ are organized as dealer markets that use computer screen displays of competitive dealer quotes to establish fair trade prices. To improve their ...
Volatility Uncertainty, Time Decay, and Option Bid-Ask Spreads in an Incomplete Market

This paper documents the fact that in options markets, the (percentage) implied volatility bid-ask spread increases at an increasing rate as the option’s maturity date approaches. To explain this stylized fact, this paper provides a market microstructure ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICAIF '20: Proceedings of the First ACM International Conference on AI in Finance

October 2020

422 pages

ISBN:9781450375849

DOI:10.1145/3383455

Conference Chair:
Tucker Balch
J.P. Morgan AI Research

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

ACM: Association for Computing Machinery

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICAIF '20

Sponsor:

ACM

ICAIF '20: ACM International Conference on AI in Finance

October 15 - 16, 2020

New York, New York

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
517
Total Downloads

Downloads (Last 12 months)107
Downloads (Last 6 weeks)7

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Evans BGanesh SDastani MSichman JAlechina NDignum V(2024)Learning and Calibrating Heterogeneous Bounded Rational Market Behaviour with Multi-agent Reinforcement LearningProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3662904(534-543)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3662904
Brusatin SPadoan TColetta ADelli Gatti DGlielmo A(2024)Simulating the Economic Impact of Rationality through Reinforcement Learning and Agent-Based ModellingProceedings of the 5th ACM International Conference on AI in Finance10.1145/3677052.3698621(159-167)Online publication date: 14-Nov-2024
https://dl.acm.org/doi/10.1145/3677052.3698621
Yang ZZhong MYang P(2024)Evolutionary Dynamic Optimization-Based Calibration Framework for Agent-Based Financial Market Simulators2024 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC60901.2024.10612133(01-08)Online publication date: 30-Jun-2024
https://doi.org/10.1109/CEC60901.2024.10612133
Shi ZCartlidge J(2024)Neural stochastic agent‐based limit order book simulation with neural point process and diffusion probabilistic modelInternational Journal of Intelligent Systems in Accounting and Finance Management10.1002/isaf.155331:2Online publication date: 5-Apr-2024
https://dl.acm.org/doi/10.1002/isaf.1553
Mounjid OLehalle C(2023)Improving reinforcement learning algorithms: Towards optimal learning rate policiesMathematical Finance10.1111/mafi.1237834:2(588-621)Online publication date: 26-Feb-2023
https://doi.org/10.1111/mafi.12378
Zhu HVyetrenko SDwarakanath KBalch TGrundl SByrd D(2023)Once burned, twice shy? The effect of stock market bubbles on traders that learn by experience2023 Winter Simulation Conference (WSC)10.1109/WSC60868.2023.10408617(291-302)Online publication date: 10-Dec-2023
https://doi.org/10.1109/WSC60868.2023.10408617
Hultin HHult HProutiere ASamama STarighati A(2023)A generative model of a limit order book using recurrent neural networksQuantitative Finance10.1080/14697688.2023.220558323:6(931-958)Online publication date: 24-May-2023
https://doi.org/10.1080/14697688.2023.2205583
Cont RCucuringu MGlukhov VPrenzel F(2023)Analysis and modeling of client order flow in limit order marketsQuantitative Finance10.1080/14697688.2022.215028223:2(187-205)Online publication date: 16-Jan-2023
https://doi.org/10.1080/14697688.2022.2150282
Shi ZCartlidge JZhang ARangwala H(2022)State Dependent Parallel Neural Hawkes Process for Limit Order Book Event Stream Prediction and SimulationProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3534678.3539462(1607-1615)Online publication date: 14-Aug-2022
https://dl.acm.org/doi/10.1145/3534678.3539462
Mizuta T(2022)Artificial Intelligence (AI) for Financial Markets: A Good AI for Designing Better Financial Markets and a Bad AI for Manipulating MarketsDigital Designs for Money, Markets, and Social Dilemmas10.1007/978-981-19-0937-5_13(305-329)Online publication date: 17-Feb-2022
https://doi.org/10.1007/978-981-19-0937-5_13
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten