Real-time strategy (RTS) games have captivated players for decades with their heady mix of strategic planning, quick reflexes, and complex decision-making; from early classics like Dune II and Command & Conquer to modern hits like Starcraft II and Age of Empires IV, RTS games task players with gathering resources, building armies, and outmaneuvering opponents in battles that blend brains and speed.
Image credit: Anastassiya Bezhekeneva/Shutterstock
Artificial intelligence (AI) has emerged as a transformative force across the video game industry in recent years. AI promises to reshape game experiences by enabling more realistic and reactive non-player characters, adaptive gameplay mechanics, and automated game testing. As one of video gaming's crown jewels, the RTS genre is primed for an AI revolution that could fundamentally alter how these iconic strategy games are designed and played.
The Current State of AI in RTS Games
AI has long played an integral role in RTS games by controlling opponent factions and armies. Early RTS AI systems relied on simple rule-based logic and cheating to pose a challenge to human players. These AI opponents would receive extra resources or have perfect knowledge of the game map to compensate for their tactical and strategic deficits. While serviceable at the time, these systems broke the illusion of facing another strategic mind like the human player's.
Modern RTS AI has come a long way from its rudimentary beginnings thanks to techniques like machine learning and Monte Carlo tree search. Sophisticated modern systems can mimic human behavior by analyzing replays of fundamental player strategies. Machine learning algorithms train neural networks to choose actions by assessing the game state based on thousands of playthroughs. At the same time, Monte Carlo tree search simulates possible futures to identify optimal moves. The integration of these techniques has led to AI opponents like DeepMind's AlphaStar defeating top professional StarCraft II players in 2019.
However, despite these advances, RTS game AI still has limitations, including computational expense, behavior predictability, and a dependence on human demonstration data. Even AlphaStar relies on heavy computing resources that are not feasible in consumer gaming rigs and exhibits some inhuman behaviors that professional players could exploit. To achieve the next level in RTS AI and unlock its transformative potential, research must continue to push the frontiers of efficient training frameworks, human-like gameplaying, and autonomous learning.
Self-Learning Through Auto-Play
An up-and-coming area in modern RTS AI is autonomous self-learning through auto-play. Traditional RTS game AI trains on datasets of human replays before deployment. This means developers must curate massive libraries of professional player strategies for each new game and update constantly with the evolving metagame. The process is time-consuming and costly, even for large studios.
Auto-play flips this paradigm by having AI agents learn entirely through self-play without any human data required. The agents start tabula rasa, only knowing the game's basic rules. They then repeatedly compete against versions of themselves in simulated matches, each time inventing new strategies and learning which decisions lead to better outcomes. While still an emerging subfield, auto-play presents an immense opportunity to overcome data dependence and produce extremely tailored AI for specific games.
Open-source projects like Limeoat's Orpheus self-play framework highlight the early promise of auto-play for RTS games. After only two weeks of self-directed learning, Orpheus developed novel tactics miming professional StarCraft II strategies. Auto-play could enable indie developers to implement capable AI with minimal effort. If perfected and optimized, auto-play may one day produce RTS opponents that train themselves to match different player skill levels and styles, adapting on the fly in a way pre-scripted AI never could.
Game Design with Procedural Generation
Beyond enhancing in-game AI opponents, RTS game creation itself stands to be transformed through AI-enabled procedural generation. RTS titles have long relied on human designers hand-crafting myriad units, buildings, maps, campaigns, and multiplayer modes to deliver fresh experiences over years of gameplay. This dependence on manual content creation poses a bottleneck that houses with bigger budgets can better afford to overcome.
Modern AI research seeks to automate game design elements through procedural generation - algorithmically creating content programmatically rather than directly by a human. Procedural generation via AI promises to increase development efficiency, enable dynamic content tailored to individual players, and unlock new possibilities like infinite maps and campaigns.
Machine learning offers data-driven approaches to procedural generation by training models on datasets of existing game content rated by humans. The models then generate new content optimized for the features humans prefer. For instance, this approach enabled AI researcher Alex Acks to procedurally generate small but playable StarCraft maps that exhibited creativity exceeding rule-based algorithms. Other novel research has explored using generative adversarial networks to develop balanced RTS game units with only high-level guidance on relationships between factions.
Meanwhile, search-based algorithms offer an approach based on game theory and simulations rather than big datasets. Search-based procedural generation can automatically tune content like unit stats for optimal balance by codifying game design requirements and leveraging Monte Carlo planning methods. This technique does not rely on existing developer content, enabling a one-click setup even for indie studios. Like their game Cannon Brawl, startups like Clockwork Games utilize these algorithms to build commercially viable RTS titles with infinite replayability.
Between the data-hungry power of machine learning and the efficient search-based methods, AI-powered procedural generation can alleviate intensive manual work for developers big and small in the competitive and hit-driven RTS industry. Single creators could now feasibly complete what once required large teams to craft content endlessly. Moreover, by outsourcing repetitive design tasks to AI, human developers gain the capacity to focus creativity on the critical high-level decisions that shape core gameplay and net player retention.
The Promises and Perils of Automated Playtesting
Playtesting marks another integral yet laborious aspect of RTS development that stands to gain through AI assistance. Playtesting iteratively evaluates game builds to catch bugs, analyze balance, collect feedback, and verify that design changes have the intended effect. As RTS titles feature vast possibility spaces, playtesting at scale poses an intractable problem. Hundreds to thousands of hours across a diverse player base is ideal yet impossible to coordinate manually by studios.
AI again offers a procedural solution, this time by autonomously playing games. Automated playtesting systems can simulate thousands of matches between AI agents running 24/7 to provide test coverage unattainable through manual methods alone. Microsoft recently unveiled one such automated playtesting framework tailored for RTS titles based on the Forge deep learning platform. Forge was leveraged to playtest Halo Wars 2 by gathering win rates, session times, and unit usage metrics across procedurally generated maps and game modes. Compared to standard playtesting, Forge delivered an estimated tenfold efficiency gain for Ensemble Studios.
Looking further ahead, automated playtesting and procedural generation constitute an autonomous game refinement loop. AI could design content like units or terrain, instantly simulate games with said content at a massive scale, evaluate balance metrics, and then reiterate the content accordingly ad infinitum. This framework could continuously auto-balance RTS titles based on evolving metagames after launch, alleviating intense developer workload and keeping multi-faction dynamics in check.
However, fully automated playtesting AI poses risks beyond the boons. Lacking human judgment, these systems optimize solely for metrics indicative of balance and testing thoroughness. This creates the potential for gameplay exploitation if the metrics prove incomplete, resulting in overpowered strategies overlooked during testing. Moreover, automated playtesting in a proprietary closed box prevents community participation that made eSports titans like StarCraft possible. Lastly, the automated frameworks could accelerate developer layoffs if applied carelessly without retraining initiatives.
Recommendations for Responsible AI Integration
As with most disruptive technologies, AI and automation create risks despite their astounding potential. Game studios must embrace ethical guidelines for implementation to enhance rather than replace the masterful craft of human developers. Several leading recommendations are as follows:
- Prioritize AI assistance over outright replacement of developer jobs through retraining programs for new oversight and quality evaluation roles.
- Maintain ultimate human oversight over automated systems to catch errors in corner cases.
- Preserve avenues for community feedback through open testing incentives and AI transparency.
- Couple procedural generation techniques with curation tools for human creators to organize and refine content.
- Allow AI augmentation of manual processes before fully automating pipelines to build trust.
- Incentivize research into innovative AI frameworks that expand possibilities for gameplay rather than solely optimize for cost reduction.
AI promises to unlock new horizons for game creators and players in the RTS genre's coming renaissance through responsible implementation centered on human collaboration and oversight.
The Future of RTS
RTS stands poised on the precipice of an AI-driven transformation after over two decades of innovation within constraints. The entire gaming landscape appears headed toward increasing AI influence as the technology proliferates, spreading beyond historical testbeds like chess and Go. Modern techniques offer the building blocks to overcome longstanding game development barriers through procedural generation and playtesting automation. However, these techniques require thoughtful integration with human creativity. Rather than AI displacing the ingenious RTS developers who birthed iconic series, machines, and humans must combine talents to craft a novel, balanced, and endlessly enjoyable game experiences.
RTS games stand ready to reach their most significant potential through this harmonious synergy of human creativity and AI capability. Both burgeoning indie studios and veteran houses can exponentially increase productivity and focus innovation where it matters most - at the bleeding edge of strategic gameplay formats. Titles can launch with diverse content and entire engines to continually expand and balance that content based on how players push the meta. Single-player campaigns can feature intelligent enemies that evolve unpredictably using self-directed learning without restrictions on data dependence. Multiplayer can remain fresh forever by regularly injecting new units from automated pipelines. Moreover, RTS may come closer to realizing the supreme dream game developers and players share - creating a brilliant, evenly-matched artificial opponent.
AI will shape video games for decades, but through responsible collaboration, it need not sideline the human element. Instead, an egalitarian intelligence revolution may germinate a renaissance within foundational yet underserved genres like RTS. Power now lies in both humans and machines - it remains up to both parties to envision and execute a future of augmented creativity heralding ever more compelling virtual battlefields. After years of laying the technological foundation, the time has come to architect a new age of strategy games harnessing semantic design and silicon speed. Players stand ready to reap the rewards.
References and Further Reading:
Weber, B., Mateas, M., & Jhala, A. (n.d.). Building Human-Level AI for Real-Time Strategy Games. https://cdn.aaai.org/ocs/4209/4209-17783-1-PB.pdf
Robertson, G., & Watson, I. (2014). A Review of Real-Time Strategy Game AI. AI Magazine, 35(4), 75. https://doi.org/10.1609/aimag.v35i4.2478
Sethy, H., Patel, A., & Padmanabhan, V. (2015). Real Time Strategy Games: A Reinforcement Learning Approach. Procedia Computer Science, 54, 257–264. https://doi.org/10.1016/j.procs.2015.06.030
Oh, I.-S., Cho, H., & Kim, K.-J. (2017). Playing real-time strategy games by imitating human players' micromanagement skills based on spatial analysis. Expert Systems with Applications, 71, 192–205. https://doi.org/10.1016/j.eswa.2016.11.026