Microsoft’s Xbox AI Era Starts with a Model That Can Generate Gameplay

WHAM-BlogHeroFeature

WHAM-BlogHeroFeature

Microsoft's latest innovation, the Muse AI model, marks the beginning of a new era of AI-powered gaming. This innovation in generative AI has the potential to fundamentally alter game creation by supporting designers in creating game worlds and interactions according to player input. Muse, while not designed to replace human creativity, has the potential to substantially streamline game development and improve the preservation of historical games.

Developed by Microsoft Research, Muse is a groundbreaking AI model that understands the 3-D game world and physics, allowing it to build consistent and interactive gameplay. Muse was trained on seven years of human gaming data from Bleeding Edge and has learned from over one billion image action plays. This extensive data set allows the model to produce game images and landscapes dynamically, responding to player actions in real-time.

Example gameplay sequences generated by Muse (based on WHAM-1.6B) demonstrate that our model can generate complex gameplay sequences that are consistent over several minutes.
expand image
Credit: Microsoft
Example gameplay sequences generated by Muse.

Despite its promise, the technology remains in its early stages. Muse currently generates gameplay visuals at a resolution of 300x180 pixels, which is better than earlier AI gaming models but falls short of modern gaming standards. However, Microsoft is committed to improving its capabilities, with the goal of producing higher-resolution output and more advanced AI-driven game interactions.

One of Muse's most exciting prospects is its ability to help reserve classic games. Microsoft Gaming CEO Phil Spencer envisions a future in which AI can evaluate gameplay footage and data from older titles, allowing them to be modified for new hardware without relying on their original engines. This could create new potential for revitalizing beloved games on numerous platforms.

In this example, the user is loading a visual as an initial prompt to the model, here a single promotional image for the game Bleeding Edge. They use Muse to generate multiple potential continuations from this starting point.
expand image
Credit: Microsoft
Muse can generate multiple potential continuations from a starting point.

Additionally, Microsoft is exploring how Muse can help developers prototype and integrate AI-powered features into existing games. Early experimental AI experiences will soon be available on Copilot Labs for users and creators to test. However, Microsoft emphasizes that AI will assist developers instead of replacing them.

As the gaming industry struggles with concerns over AI's impact on jobs, Microsoft argues that Muse is designed to boost creativity instead of automating content creation. Microsoft plans to achieve a balance between technological innovation and human artistry by offering gaming studios control over how AI is integrated into their creative processes.