a16z: Opportunities for Generative AI in the Gaming Sector

Authors: James Gwertzman & Jack Soslow

Compiled by: Alpha Rabbit

A16Z recently wrote an interesting article discussing the opportunities they see in the intersection of generative AI and gaming. After translating, the author added some annotations to certain parts. The article is mainly divided into two parts: the first part includes A16Z's observations and predictions about generative AI in the gaming sector; the second part includes A16Z's assessment of the market ecosystem in the gaming + generative AI space.

Introduction

What is the connection between the gaming sector and generative artificial intelligence?

In the field of game design, there exists an impossible triangle: typically, only two of the three aspects—cost, quality, or speed—can be achieved. Now, designers can use these AIGC tools to create high-quality images in just a few hours without spending a lot of manual production time. What is truly transformative is that anyone who learns a few simple tools can gain this creative capability.

These tools can create endless variations in a rapidly iterative manner, and once trained, the entire process is real-time, meaning the results are almost instantly available.

Since the advent of real-time 3D technology, no other technology has had such a potential impact on gaming (with real-time 3D software, entire virtual worlds can be digitally rendered at a faster pace, providing users with a more engaging and immersive experience).

What is the development direction of generative AI? How will it change gaming?

First, generative AI is a category of machine learning where computers can generate original new content based on user input/prompts. Currently, the most mature applications of this technology are primarily in the fields of text and images, but similar advancements (applications of generative AI technology) are seen in almost all creative fields, covering animation, sound effects, music, and even the creation of original virtual characters with complete personalities.

Of course, artificial intelligence is not new in gaming. Even early games, like Atari's "Pong," had computer-controlled opponents battling players.

However, the virtual opponents in those computers are not the same as the generative artificial intelligence we discuss today. These computer opponents were merely scripted programs meticulously designed by game developers; they simulate an AI opponent but cannot learn or iterate, performing at the same level as the engineers who programmed them.

So, what are the underlying technological changes in the combination of generative AI and gaming?

Microprocessor speeds are faster, cloud computing and various computational capabilities are stronger, with the potential to build large neural networks that can recognize patterns and representations in highly complex domains.

Part One: Some Assumptions and Industry Observations

Some assumptions:

First, let's explore some assumptions that the remainder of the article is based on:

1. The number of successful research projects in general artificial intelligence will continue to grow, leading to more effective technologies. The above image shows the number of academic papers published monthly on machine learning or artificial intelligence in arXiv. As shown in the image, the number of papers is growing exponentially, with no signs of slowing down. This data only includes published papers; many studies have not been publicly published but are directly applied to open-source models or product development, leading to explosive innovation.

2. Among all entertainment categories, gaming will be the field most significantly impacted by generative artificial intelligence.

In terms of the types of assets involved (2D art, 3D art, sound effects, music, etc.), gaming is the most complex category in entertainment, and it is also the most interactive, emphasizing real-time experiences. This creates a very high barrier to entry for new game developers and results in high costs for producing a truly AAA game. These existing barriers and cost issues create significant opportunities for disruptive innovation by generative AI in the gaming sector (as shown in the image below):

For example, games like "Red Dead Redemption 2" are among the most expensive games ever made, with production costs nearing $500 million. "Red Dead Redemption" is also one of the games with the best visual effects on the market, taking nearly eight years to produce, featuring over 1,000 game characters (each with their own personality and dedicated voice actors), a game world nearly 30 square miles in size, over 100 missions across six chapters, and nearly 60 hours of music created by over 100 musicians. The production of all content in this game is immense.

So, if we compare "Red Dead Redemption 2" with "Microsoft Flight Simulator," the latter is even more massive… because players of Microsoft Flight Simulator can fly around the entire Earth, covering all 1.97 million square miles. So how did Microsoft create such a large game? Mainly through artificial intelligence, as Microsoft collaborated with blackshark.ai to train AI to generate an infinitely realistic 3D world from 2D satellite images.

What is blackshark.ai?

blackshark.ai is a company that uses machine learning technology to extract global infrastructure data from satellite and aerial images, creating digital twin scenarios based on current geographic data. These results can be used for visualization, simulation, mapping, mixed reality environments, and other enterprise solutions, with the technology's cloud computing capabilities allowing for real-time data updates.

This is just one example; without the use of AI technology, it would be virtually impossible to produce "Microsoft Flight Simulator." Additionally, the game's success is also due to the fact that these models can continuously improve over time. For example, the "highway cloverleaf overpass" model can be enhanced by running the entire construction process through AI, allowing for immediate improvements to all highway interchanges across the game's entire Earth.

3. Every asset involved in game production will have a generative AI model.

So far, 2D image generators like Stable Diffusion or MidJourney have captured much of the current excitement around generative AI due to their ability to generate eye-catching images. Now, generative AI models have emerged for almost all assets in games, from 3D models to character animations, dialogues, and music. (The next article will discuss the specific market ecosystem of companies.)

4. Content costs will continue to decline, and in some cases, the cost of content will drop to zero.

When we talk to game developers trying to integrate generative AI into their production processes, the biggest excitement lies in the significant reduction of time and costs for game production. One developer told us that the time to generate a concept image dropped from three weeks to one hour. We believe similar "cost reduction and efficiency improvement" can be achieved throughout the game production process.

It is worth noting that there is no danger of artists being replaced, meaning artists no longer need to do all the work themselves: artists and designers can set the initial creative direction and delegate most of the time-consuming and technical execution work to AI. In this regard, it is similar to early hand-drawn animation, where highly skilled "drawing experts" outline the animation, and then relatively lower-cost artists complete the time-consuming tasks of coloring and filling in lines, but we are discussing applications in the game creation field.

5. We are still in the early stages of this industry transformation, and there are many aspects that need improvement.

Despite the recent excitement, we are still just at the starting line. There is still a lot of work to be done in understanding how to truly apply this new technology in conjunction with the gaming sector, and there will be huge opportunities for companies that enter this new field quickly.

Part Two: Predictions for the Future

Given the above assumptions, this article makes predictions and extrapolations about how the gaming industry will be transformed.

1. Learning how to effectively apply generative artificial intelligence will become a market skill.

There are already pioneers who can apply generative AI more effectively than others. To make the best use of this new technology, one needs to understand various tools and techniques and know how to combine them. We predict that effectively applying generative AI will itself become a highly valuable skill, as it can combine the creative vision of artists with the technical capabilities of programmers.
Chris Anderson famously said, "Every abundance creates a new scarcity." As content becomes increasingly abundant, those who understand how to collaborate most efficiently with AI tools will be the most sought after.
For example, using generative AI for art generation will also bring some challenges, including:
Maintaining coherence: It is necessary to modify or edit various assets in the game, which means that AI tools need to be able to replicate (digital) assets with the same signals so that we can modify and challenge them. This can be tricky, as the same prompt may yield completely different results.
Maintaining stylistic consistency: All artworks in a single game need to maintain a consistent style, which means that AI tools need to be trained or linked to the established style of the artist/designer.

2. The lowering of barriers to game development will lead to more experimentation and creative exploration.

We may soon enter a new "golden age" of game development, where lower barriers to entry will lead to more innovative and creative games. This is not only because lower production costs reduce the risks for game developers, but also because these tools represent the ability to create high-quality content for a broader audience.

3. AI-assisted "micro game studios" will gradually rise.

With the tools and services of generative AI, more viable commercial games may be produced by small "micro studios" with only one or two employees. Of course, small indie game studios are already common; the popular game "Among Us" was created by a studio called Innersloth with only five employees, and the scale of games that these small studios can create will grow.

4. The number of games released each year will increase.

The success of Unity and Roblox shows that providing powerful creative tools leads to more games being built. Generative AI will further lower barriers and create more games. The industry has already faced discovery challenges—over 10,000 games were added to Steam just last year—putting greater pressure on discovery. However, we will also see…

5. New game types will be invented.

New game types will be invented, like the previously mentioned "Microsoft Flight Simulator," but entirely new game types that combine real-time content generation will emerge.
For example, Spellbrush's RPG game Arrowmancer features AI-generated characters and nearly limitless new gameplay. Other game developers are using AI to allow players to create their own avatars in the game: automatically generating avatar images based on player descriptions. Note that from the user experience perspective, allowing players to generate content through AI can give them a greater sense of ownership.

6. Value will accrue to AI tools specific to certain industries, rather than just foundational models.

The hype surrounding foundational models like Stable Diffusion and Midjourney is generating extremely inflated valuations, but as new research continues to emerge, new models will appear and iterate with the advancement of new technologies. Based on the website search traffic of the three popular generative AI models (Dall-E, Midjourney, and Stable Diffusion), each new model has specific focal points around it.

Another approach is to build toolkits that meet industry-specific (vertical industry) needs, focusing on the generative AI demands of specific industries, gaining deep insights into specific audiences, and integrating with existing production environments (Unity or Unreal).

A typical example is Runway, which provides AI-assisted tools for video creators, such as video editing, green screen removal, inpainting, and motion tracking. Such tools can increase new application scenarios over time. We have yet to see game tools like Runway emerge, but this is a promising area.

7. Upcoming legal challenges.

A commonality among all these generative AI models is that they are trained on vast datasets of content, often created from internet data. For example, "Stable Diffusion" was trained on over 5 billion images/titles collected from the web. Currently, these models claim to operate under the copyright principle of "fair use," but this argument has yet to be clearly tested in law. Clearly, the upcoming legal challenges could reshape the landscape of generative AI.
Large film companies may leverage their copyright advantages to establish proprietary models and seek competitive advantages. For instance, Microsoft has many studios under its umbrella, especially after acquiring Activision Blizzard.

8. At least for now, unlike in the arts, generative AI may not bring about a huge transformation in programming.

Software engineering is another major cost source in game development, but generating code with AI models requires more testing and validation. Therefore, the productivity gains from code generation are lower than those from generating creative assets. We believe that coding tools like Copilot may provide moderate performance improvements for engineers, but in the short term, the changes will not be as significant as those in the content domain.

Part Three: Some Recommendations

1. Start exploring generative artificial intelligence:

It will take some time to figure out how to fully leverage the power of the upcoming generative AI revolution. Companies that start developing their business early will have an advantage. Several studios are conducting internal experimental projects to explore how these technologies impact game production.

2. Look for opportunities in market gaps:

Currently, many parts of the entire field are very crowded, such as animation, voice, and dialogue, but many areas remain wide open. We encourage entrepreneurs interested in this field to focus on still undeveloped areas, such as the "gaming + generative AI track."