Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Quantum Finance and Fuzzy Reinforcement Learning-Based Multi-agent Trading System

  • Published:
International Journal of Fuzzy Systems Aims and scope Submit manuscript

Abstract

In a volatile stock market, an investor’s long-term goal involves determining the most effective buying, selling strategies, and money management techniques in order to maximize profits. This paper introduces a multi-agent trading system to achieve this goal, termed QF-FRL, based on quantum finance and fuzzy reinforcement learning (QF-FRL). The system comprises two agents: (1) The trading agent, constructed using the Deep Deterministic Policy Gradient (DDPG) and Twin Delayed Deep Deterministic Policy Gradient (TD3). This agent employs a Denoising Auto Encoder (DAE) to extract stock representations from historical time series data. The trading agent initially employed the DDPG model, which was subsequently supplanted by the TD3 model. It integrates traditional financial technology indicators, like moving averages, with modern deep reinforcement learning technology to generate buying and selling signals for determining the optimal strategy. (2) The risk control agent, founded on quantum finance and an adaptive network-based fuzzy inference system. This agent merges the QPL indicator with a fuzzy risk control method to ascertain transaction amounts. Furthermore, a genetic algorithm is utilized to optimize the parameters of the fuzzy system, aiming to enhance profits and ensure accuracy in transactions at specific amounts. The experiments in this study involved selecting nine stocks and testing them against seven competing quantitative trading models. Upon comparing the profit rate, trading frequency, Sharpe ratio, and average return of each stock, eight stocks within the QF-FRL system achieved the highest returns and a greater number of transactions. Additionally, the QF-FRL system has also attained the highest average return and the second highest average Sharpe ratio. The results indicate that QF-FRL outperforms competing models, yielding higher profits and being particularly suitable for long-term investment. Moreover, it exhibits more favorable risk-adjusted returns and a notable degree of robustness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Azhikodan, A.R., Bhat, A.G.K., Jadhav, M.V.: Stock trading bot using deep reinforcement learning. In: Lecture Notes in Networks and Systems, pp. 41–49. Springer, Singapore (2019)

  2. Baostock: http://baostock.com. Accessed 26 Feb 2024

  3. Bekiros, S.D.: Heterogeneous trading strategies with adaptive fuzzy actor-critic reinforcement learning: a behavioral approach. J. Econ. Dyn. Control 34(6), 1153–1170 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bengio, Y., Vincent, P., Larochelle, H., Manzagol, P.A.: Extracting and composing robust features with denoising autoencoders. In: Machine Learning, Proceedings of the Twenty-Fifth International Conference (ICML 2008), Helsinki, Finland, 5–9 June 2008

  5. Carta, S., Ferreira, A., Podda, A.S., Reforgiato Recupero, D., Sanna, A.: Multi-DQN: An ensemble of deep Q-learning agents for stock market forecasting. Expert Syst. Appl. 164, 113820 (2021)

    Article  Google Scholar 

  6. Chang, P.C., Liao, T.W., Lin, J.J., Fan, C.Y.: A dynamic threshold decision system for stock trading signal detection. Appl. Soft Comput. 11(5), 3998–4010 (2011)

    Article  MATH  Google Scholar 

  7. Cheng, Y., Xu, B., Lian, Z., Shi, Z., Shi, P.: Adaptive learning control of switched strict-feedback nonlinear systems with dead zone using NN and DOB. IEEE Trans. Neural Netw. Learn. Syst. 34(5), 2503–2512 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  8. Chourmouziadis, K., Chourmouziadou, D.K., Chatzoglou, P.D.: Embedding four medium-term technical indicators to an intelligent stock trading fuzzy system for predicting: a portfolio management approach. Comput. Econ. 57(4), 1183–1216 (2020)

    Article  Google Scholar 

  9. Dempster, M.A.H., Leemans, V.: An automated FX trading system using adaptive reinforcement learning. Expert Syst. Appl. 30, 543–552 (2006)

    Article  MATH  Google Scholar 

  10. Fujimoto, S., Hoof, H.V., Meger, D.: Addressing function approximation error in actor-critic methods. ICML 2018, 1582–1591 (2018)

    MATH  Google Scholar 

  11. Gong, X., Yu, C., Min, L., Ge, Z.: Regret theory-based fuzzy multi-objective portfolio selection model involving DEA cross-efficiency and higher moments. Appl. Soft Comput. 100, Article 106958 (2021)

  12. Gupta, P., Mehlawat, M.K., Khan, A.Z.: Multi-period portfolio optimization using coherent fuzzy numbers in a credibilistic environment. Expert Syst. Appl. 167, Article 114135 (2021)

  13. Huang, S., Miao, Y., Hsiao, Y.: Novel deep reinforcement algorithm with adaptive sampling strategy for continuous portfolio optimization. IEEE Access 9, 77371–77385 (2021)

    Article  Google Scholar 

  14. Huang, Q., Yang, J., Feng, X., Liew, A.W.C., Li, X.: Automated trading point forecasting based on bicluster mining and fuzzy inference. IEEE Trans. Fuzzy Syst. 28(2), 259–272 (2020)

    Article  MATH  Google Scholar 

  15. Kirkpatrick, C.D., Dahlquist, J.: Technical Analysis: The Complete Resource for Financial Market Technicians, 2nd edn. Pearson, New Jersey (2011)

    MATH  Google Scholar 

  16. Lee, R.S.T.: Quantum Finance: Intelligent Forecast and Trading Systems. Springer, Singapore (2020)

    Book  MATH  Google Scholar 

  17. Li, N., Li, X., Peng, J., Xu, Z.Q.: Stochastic linear quadratic optimal control problem: a reinforcement learning method. IEEE Trans. Autom. Control 67, 5009–5016 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  18. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., et al.: Continuous control with deep reinforcement learning. arXiV Preprint (2016). arXiv:1509.02971

  19. Lo, A.W., Mamaysky, H., Wang, J.: Foundations of technical analysis: computational algorithms, statistical inference, and empirical implementation. J. Financ. 55(4), 1705–1770 (2000)

    Article  MATH  Google Scholar 

  20. Luo, L., Chen, X.: Integrating piecewise linear representation and weighted support vector machine for stock trading signal prediction. Appl. Soft Comput. 13(2), 806–816 (2013)

    Article  MATH  Google Scholar 

  21. Mashayekhi, Z., Omrani, H.: An integrated multi-objective Markowitz–DEA cross-efficiency model with fuzzy returns for portfolio selection problem. Appl. Soft Comput. 38, 1–9 (2016)

    Article  MATH  Google Scholar 

  22. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)

    Article  Google Scholar 

  23. Moody, J.E., Saffell, M.: Learning to trade via direct reinforcement. IEEE Trans. Neural Netw. 12(4), 875–889 (2001)

    Article  MATH  Google Scholar 

  24. Neuneier, R.: Enhancing Q-learning for optimal asset allocation. Adv. Neural. Inf. Process. Syst. 10(1), 936–942 (1998)

    MATH  Google Scholar 

  25. de Oliveira, F.A., Nobre, C.N., Zárate, L.E.: Applying Artificial Neural Networks to prediction of stock price and improvement of the directional prediction index—case study of PETR4, Petrobras, Brazil. Expert Syst. Appl. 40(18), 7596–7606 (2013)

    Article  MATH  Google Scholar 

  26. Pendharkar, P., Cusatis, P.: Trading financial indices with reinforcement learning agents. Expert Syst. Appl. 103, 1–13 (2018)

    Article  MATH  Google Scholar 

  27. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back propagating errors. Nature 323(6088), 533–536 (1986)

    Article  MATH  Google Scholar 

  28. Silver, D., Lever, G., Heess, N., Degris, T., Riedmiller, M.: Deterministic policy gradient algorithms. In: International Conference on Machine Learning. PMLR (2014)

  29. Skeepers, T., van Zyl, T.L., Paskaramoorthy, A.: MA-FDRNN: Multi-asset fuzzy deep recurrent neural network reinforcement learning for portfolio management. In: 2021 8th International Conference on Soft Computing & Machine Intelligence (ISCMI), pp. 32–37 (2021)

  30. Tsaur, R.C., Chiu, C.L., Huang, Y.Y.: Guaranteed rate of return for excess investment in a fuzzy portfolio analysis. Int. J. Fuzzy Syst. 23, 94–106 (2021)

    Article  MATH  Google Scholar 

  31. Wang, C., Sandas, P., Beling, P.: Improving pairs trading strategies via reinforcement learning. In: 2021 International Conference on Applied Artificial Intelligence (ICAPAI) (2021)

  32. Wang, J., Zhang, Y., Tang, K., Wu, J., Xiong, Z.: AlphaStock: a buying-winners-and-selling-losers investment strategy using interpretable deep reinforcement attention networks. CoRR (2019). arXiv:1908.02646

  33. Yang, X.Y., Liu, W.L., Chen, S.D., Zhang, Y.: A multi-period fuzzy mean minimax risk portfolio model with investor’s risk attitude. Soft. Comput. 25, 2949–2963 (2021)

    Article  MATH  Google Scholar 

  34. Zadeh, L.A.: Fuzzy sets. Inf. Control 8(3), 338–353 (1965)

    Article  MATH  Google Scholar 

  35. Zhang, Y.Y., Li, X., Guo, S.N.: Portfolio selection problems with Markowitz’s mean-variance framework: a review of literature. Fuzzy Optim. Decis. Making 17(2), 125–158 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  36. Zhang, Y., Liu, W., Yang, X.: An automatic trading system for fuzzy portfolio optimization problem with sell orders. Expert Syst. Appl. 187, 115822 (2022)

    Article  MATH  Google Scholar 

Download references

Acknowledgements

This paper was supported in part by the Guangdong Provincial Key Laboratory IRADS (2022B1212010006, R0400001-22), Key Laboratory for Artificial Intelligence and Multi-Model Data Processing of Department of Education of Guangdong Province and Guangdong Province F1 project grant on Curriculum Development and Teaching Enhancement on Quantum Finance course UICR0400050-21CTL.

Author information

Authors and Affiliations

  1. Guangdong Provincial Key Laboratory of Interdisciplinary Research and Application for Data Science, Beijing Normal University-Hong Kong Baptist University United International College, Zhuhai, Guangdong Province, China

    Chi Cheng, Bingshen Chen, Ziting Xiao & Raymond S. T. Lee

Authors
  1. Chi Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bingshen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ziting Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Raymond S. T. Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Dr Raymond Lee: Supervision, project administration, funding acquisition, reviewing and editing. Chi Cheng: Conceptualization, methodology, reviewing and editing. Bingshen Chen: Conceptualization, methodology, writing—original draft preparation, formal analysis, data visualization, validation. Ziting Xiao: Investigation, literature review, software design, implementation.

Corresponding author

Correspondence to Raymond S. T. Lee.

Ethics declarations

Conflict of interest

The authors have not disclosed any competing interests.

Ethical Approval

This article does not contain any studies conducted by any of the authors on human participants or animals.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, C., Chen, B., Xiao, Z. et al. Quantum Finance and Fuzzy Reinforcement Learning-Based Multi-agent Trading System. Int. J. Fuzzy Syst. 26, 2224–2245 (2024). https://doi.org/10.1007/s40815-024-01731-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40815-024-01731-1

Keywords

Search

Navigation