Faced with tectonic changes in consumer consumption and epic competition from new and traditional rivals, media and entertainment (M&E) companies are eagerly casting AI/ML in key roles they expect will soon lead to business stardom.

Advances in AI programs, high-performance cloud infrastructure, and accelerated services are helping industry firms power data-driven transformation on screen and off.  Smart technology is already reshaping  content production, personalization, talent casting and other key support functions. Disney, global advertising and marketing giant WPP and scores of other M&E companies see AI as huge game-changer. Many are putting it at the heart of their business strategy

“There is no part of the ecosystem that AI doesn’t touch or benefit in some way,” says Andy Beach, CTO for Media and Entertainment at Microsoft.

A preview of coming attractions for every industry

As an “industry of industries, [M&E] “is an encapsulation of all other verticals,” says Simon Crownshaw, a former Disney executive who now heads worldwide strategy in the sector for Microsoft. This sprawling sector is also the largest in the world ($2.5 trillion) and in the U.S. ($422 billion). “We’ll see many things before almost anyone else,” Crownshaw says. “AI enables us to look at major problems like resources and solve them in creative and different ways.” These factors make M&E a valuable bellwether.

This in-depth coverage looks at use cases transforming the industry, creative use of AI at WPP , the crucial role of optimized cloud infrastructure, and what’s next for AI growth in M&E. Let’s start with the business drivers.

Media, film and TV, Over the Top (OTT) internet services, music, gaming and esports will invest heavily in AI/ML in 2023. Grandview Research expects the combined global sales of the technology in the industry to hit $13.7 billion, up from $10.7 billion in 2022, led by streaming video and audio companies.

Individual company goals differ, but common themes span diverse M&E firms: slashing content production time and costs;  innovating profitable new revenue streams; attracting and retaining new customers; enhancing user experience (UX) by delivering hyper-personalized content anytime, anywhere to any device; and building a profitable path to AR/VR.

 

Behind AI’s rapid rise in the M&E charts

For M&E firms, AI and data science offer a powerful way to fight for attention, time and revenue. Not  just against each other, but also with free or low-cost content from formidable new frenemies like social media, users and independent creators.

COVID accelerated changes in viewer consumption habits and the convergence of film, TV and games. As the pandemic morphed into inflation, M&E firms faced intense pressures to cut costs, boost profits and expand or protect market share. Many concluded that success — and perhaps survival — depended on their ability to create more content quickly and more cheaply.

Have visual effects become too good? 

Ironically, the industry is partially a victim of its own success. Over the last decade, content firms became increasingly adept at releasing flashy productions, juiced by crowd-pleasing visual effects (VFX). Ultimately, that raised the bar on quality, says Rick Champagne, M&E industry lead for NVIDIA.  “As a result, many companies today are straining to keep up with voracious consumer demand for more and more high-quality content.”

Fans, for instance, have criticized Marvel for the production and dramatic quality of its latest “Phase 4” films.

A talent force-multiplier

Firms also face a global shortage of sophisticated artistic talent. “Schools aren’t turning out artists that can operate at that level fast enough,” says Champagne, “so everyone’s competing for the same resources.” Ditto for workers with AI and data science skills, a core challenge that transcends industries.

Against this backdrop, AI is a perfect force multiplier. To cope with demand, M&E leaders are turning to AI to accelerate routine parts of content production such as aging, de-aging and cosmetics, or removing wireframes from models to help speed up production.

The huge upsides of AI are reflected in a recent Deloitte study. The consultancy found M&E leaders surveyed recognized the criticality of AI “far more” than respondents in other industries. Some 72% “strongly” believe the technology will be key to competitiveness over the next five years.

Computer generated imagery (CGI) made its film debut in 1957 with a purple spiral expanding out of an eyeball in the opening credits of Alfred Hitchcock’s thriller “Vertigo.” Industry creators today continue to innovate increasingly sophisticated on-screen VFX to dazzle mass audiences. With AI/ML, media and entertainment firms have both broadened and narrowed the focus of their technology efforts.

Current use of intelligent tools extends behind the scenes to less glamorous but important tasks: ensuring compliance with copyright standards, automating residual payment forecasting and other HR, legal and administrative activities that improve efficiency and reduce time and costs. While still targeting audiences of billions, industry companies are also racing to develop AI that helps more precisely deliver personalized user experiences to billions of “audiences of one.”

Yet content remains the essential core of M&E companies. So, it makes sense that many firms in the sector are focusing their early AI efforts on improving the speed, cost and ease of how it’s created, managed, distributed and consumed. A sampling:

Hello DALL-E and friends

The massive worldwide buzz around GPT-4, DALL-E 2 and others resonates especially loudly — both positively and negatively — within M&E companies. Businesses see a chance to create content much faster and at lower costs. Accenture estimates that across all industries, as much as 40% of all working hours will be supported or augmented by language-based AI.

Using generative AI for script- and copywriting offers intriguing possibilities. Beyond scheduling, and managing budgets, AI can automate tasks such as including script breakdown, storyboarding and shot-lists generation. Many artists and writers worry about displacement and job loss. No surprise, then, that concerns with AI as a “plagiarism machine” feature prominently in the Hollywood writer’s strike.

Yet big, open questions remain around copyright, data privacy, accuracy, bias and other serious issues. For many M&E companies, these concerns have so far limited the use of generative AI in large-scale commercial production to research, generating ideas and concepts, and other “safer” activities.

To address the need for Generative AI models with ethically licensed content, Shutterstock and Getty Images have partnered with NVIDIA. The latter also has announced NVIDIA AI Foundations, a set of cloud services. They enable businesses to build, refine and operate custom generative AI models trained with their own proprietary data and created for unique, domain-specific tasks.

 

AI FOR VOICE – For now, AI is proving useful for creating key components of computer-generated content, such as voice and 3D graphic facial animations. A recent high-profile example: AI gave voice to Val Kilmer in the update of “Top Gun“– even though the actor had lost the ability to speak years before because of throat cancer.

AI-GENERATED CHARACTER ANIMATION. Creators can quickly and easily generate character animation using deep learning technology as part of a custom 3D workflow.

NVIDIA Omniverse Audio2Face, for example, lets artists generate realistic facial expressions and emotions from a real time or recorded audio source. The beta reference application simplifies animation of a 3D character to match any voice-over track for film, games, digital assistants and more. The app can be used for interactive real-time applications or as a traditional facial animation authoring tool.

Upcoming updates include Mandarin support, overall facial animation and lip-sync quality improvement across multi-languages. See it in action here.

The Omniverse Audio2Gesture (A2G) neural network  is trained to generate body motion derived entirely from an audio source. Various animation styles and options available let creators animate full or upper bodies. Audio2Gesture provides a high-quality and efficient solution to generating body motion for characters in heavy dialogue scenarios.   

 

REAL-TIME ML CLOTHING. Current high-fidelity simulation of clothing is slow. New software uses a proprietary algorithm to train a machine learning system that can replicate the original’s quality and run at 150+ frames per second. This capability accelerates cloth, muscle and skin simulations, allowing animators to work with full resolution, complex characters instead of low-resolution set ups.

Credit: Digital Domain

 

DIGITAL HUMANS.  More studios and advertising firms are starting to use AI to create complete digital humans. These synthetic people can be easily changed and adapted for a wide variety of target audiences.  With a few quick commands, a character’s age, ethnicity, hair, language and other physical characteristics can be changed, speeding up production.

 

VIRTUAL PRODUCTION SETS. Beyond these individual human components, AI is also proving useful for entire composing scenes and virtual production sets. Learn how.  To overcome tricky and complex problems with lighting on “The Mandalorian,” for example, Disney designed an ingenious, AI-powered circular “blue screen” virtual set.

 

AI-ACCELERATED DENOISING. Much of today’s content production work, such as adding and cleaning up shots, is done in the later stages. This kind of AI-powered content creation has become commonplace across the industry. “Denoising” produces clearer renders to allow an artist to make faster creative decisions.  The capability can also be used for video, as well as for other routine tasks such as sharpening, painting, re-colorization and adjusting lighting — all done faster, with improved final quality.

Credit: NVIDIA

Going a step further, AI-powered specialty software from Adobe, Autodesk, MARZ, Topaz and the DaVinci neural engine lets creators digitally age and de-age characters, derive 3D maps and much more. NVIDIA’s Champagne says some studios are creating custom solutions based on their own content archives.

 

HYPER-PERSONALIZATION. Analyzing data ­– such as box office, sweeps and readership — has long been an important focus of M&E. Content providers have become increasingly sophisticated in tracking how consumers use their platforms. Many already employ ML to suggest content based on an individual’s habits and preferences. Gaining better, more granular understanding of customers is essential to the industry’s current massive push to identify and deliver highly personalized content. More interaction provides more data for AI-enabled algorithms. That means better, on-target recommendations and playlists for content delivered to a preferred mobile device, PC, smart TV, digital media player or game console.

AI is already using large data sets gathered from multiple platforms to understand interest and acceptance of proposed content, Grandview Research notes. Video, for example, can be created based on exact, customized suggestions and observations of user behaviors like browsing history, interest and pausing or rewinding videos.

Half of the media sector’s revenue comes from advertising, according to the Boston Consulting Group. For an inside peek at the industry’s AI-powered present and future, spend a few minutes with Perry Nightingale. He’s Senior Vice President for Creative AI at WPP, the London-based global giant and the world’s largest marketing services company by billings ($60B). He’s proud to show the firm’s striking, cutting-edge commercials crafted with the latest generative AI.

There’s the recent Coca-Cola spot that magically brings a gallery of paintings to life. In a similar vein, a Nestlé ad expands the world around Johannes Vermeer’s famous painting, “The Milkmaid.” Here’s a new Nike promotion in which a 2017 simulated Serena Williams plays a digital tennis match against 1999 Serena Williams.

Credit:  Coca-Cola

Designing tomorrow with AI

The success is by design.  WPP has invested in specialist AI company Satalia. Three years ago, Nightingale was charged by his boss with exploring and developing ways that the world’s largest, marketing and communications firm could leverage Generative AI for competitive advantage. As an artist, self-taught programmer and Oxford lecturer on the ethics of generative AI, he embraced the challenge.

With an early effort, using dual-screen Microsoft Surface devices to interface with NVIDIA GAUGAN, Nightingale visualized a key goal: democratizing creativity by developing a futuristic, cloud-based, AI-powered, super workstation and workflows for the tens of thousands of creatives in Ogilvy, Grey, Wunderman Thompson and other agencies in the WPP network. Or “a giant creative brain in the cloud,” as he describes it.

Centralized collaboration and storage powered by AI would provide an important foundation, with several advantages over current manual methods. Like many M&E firms, WPP needs new ways to produce high-quality content for clients quickly and cost-effectively. AI helps overcome the budget, time and talent constraints plaguing the industry today, Nightingale says.

“Moving our heavy production workflow into the cloud and AI allows us to make a staggeringly higher volume of work and this increase in assets almost always improves relevancy and therefore performance,” says Nightingale. “A single ad campaign can be tens of millions of dollars, so the numbers are huge.”

Beyond operational efficiencies, optimized and accelerated cloud infrastructure running Generative AI also opens new creative possibilities. Of the Coke spot, he says “That sort of piece, the effects, wouldn’t have been possible ten years ago without a huge amount of painstaking manual work.”

Commercials for you

Generative AI also brings WPP a new ability to finely target ads for specific geographic and demographic audiences. A car commercial, for instance, can be shot once, then quickly customized with AI to show different locations. “People in Montana hate ads with palm trees. Ideally, we’d try and avoid showing ads in Montana with palm trees,” Nightingale explains. He slides a few software controls and presto, mountains. This capability will make it possible to quickly create “local” spots for an unlimited number of markets.

Similarly, AI can make possible “shoot once/use many” production for talent. To illustrate, he shows a TV ad made for India featuring a well-known Bollywood star promoting a small local business. A few adjustments and some smart voice cloning and the actor is promoting another small business and their product. The process can be repeated again and again, providing big, personalized star power on a small budget — this work was a campaign by confectionary giant Mondelez to say thank you to small businesses for their support during the pandemic and won a coveted Titanium Lion Award in advertising.

Like the feature-film industry, WPP is researching the use of synthetic digital humans. “We know that the more we relate to people in adverts, the more we are likely to purchase — it’s common sense that if you have a large family, it makes sense for you to see one in your car ad,” Nightingale continues. “’Virtual casting’ of digitally generated people in the cloud allows us to do that far more easily and cheaply but we’re doing a lot of research to ensure the public is okay with that and we know there will be regulation around disclosure, etc. Ultimately, these tools are very useful when used in the right way, but we need to be responsible, and we’ve seen backlash towards brands that have deployed synthetic humans to increase diversity for example.”

He adds: “Ultimately, generative AI allows you to conceivably reach every single person who watches, say the Super Bowl, with a different ad. It’s probably not worth it. But you can create one for every state in the U.S, three or four different ages of people driving the car, with each of the colors, trims and a couple of different lifestyle choices.”

New technology, same mission

Large-scale production using AI at WPP is still “a couple of years” away. Nightingale is upbeat about the many positive ways the technology can transform WPP’s creativity and business. The Mondelez small business campaign is a great example of how this can scale. Ultimately, Nightingale believes that a cloud-based workstation could be an interface that unlocks the power of AI technology, much like the iPhone did for 3G cellular.

In the meantime, Nightingale continues work on a diverse range of AI-related projects, including supply chain simulation, rendering, combining creative workflows and functions and working out ROI metrics. He’s also focused on several important external issues: ensuring the fairness and privacy of datasets, researching public perception of “deep fake” people, EU regulations around disclosure around the use of synthetic humans, and easing artist anxieties so they can see AI as a powerful creative helper, not a threat.

“It’s a marathon,” says Nightingale. Wherever that AI race leads, he says, the ultimate goal for WPP remains the same: “Connecting you to the products you want to buy. Our role is to connect people with the right products, and we believe AI will be transformational.”

Like other industries, business and technology leaders in M&E are discovering that developing, running and scaling innovative AI workloads requires faster and more powerful processing, storage, networking, software and virtual machines. “It’s a heavier lift” than conventional computing, notes Microsoft’s Andy Beach, made heavier by complex production workflows and processes.

Fortunately, many firms in the sector recognize the crucial role played by accelerated and optimized infrastructure. They see heavy-duty, “purpose-built” as central to creating sustainable production flows, data sets ­and in building advanced analytics, tools, workloads and applications.

 

An “intelligent cloud” lets media and entertainment companies use AI to create content quickly and collaboratively, then maximize management, monetization and personalized distribution to millions of customers. Source: Microsoft

 

M&E companies are under intense pressure to “do more with less,” explains Microsoft’s John Lee, AI Platforms & Infrastructure Principal Lead for Microsoft Azure. Business and IT leaders don’t want to pay for expensive technology that will sit idle. Both factors, says Lee, have supercharged adoption of pre-trained models and on-demand infrastructure-as-a-service (IaaS) for AI.

Choosing the right cloud-based foundations and tuning lets companies focus IT and resources on creative work and maximizing content assets instead of building out infrastructure. It’s especially important in later stages of content production. It’s not unusual for a production company to need 50,000 more GPUs to complete rendering, or new shots to meet unmovable film release dates. In such situations, Lee says, the ability to quickly add computing firepower without planning six to nine months ahead is a huge advantage that’s unavailable with on-premise infrastructure.

“The world is just beginning to envision how to combine cognitive services to transform industry-based workflows,” says Jon Weisner, Microsoft Azure HPC AI Global Blackbelt, M&E lead. “The security, accessibility, scalability and performance of the foundation these services will run on matters. The depth and breadth of Azure High Performance Compute infrastructure, connected by high throughput and low latency InfiniBand networking, becomes critically important as companies develop sophisticated AI solutions.” As “the world’s AI supercomputer”, Weisner says, Azure will play a crucial role in enabling development of next generation AI-powered applications.

Coming attractions

M&E companies have begun nothing less than an AI-driven automation of content and supporting functions. It’s an advance some believe could be as transformative as the internet, television and printing press, and shows no signs of slowing.

By 2027, overall AI investment in the sector worldwide is forecast to reach nearly $35 billion, with demand for services in North America steadily increasing through 2030. All told, McKinsey estimates traditional and advanced AI could add $448 billion, or 17.2% of sales, of yearly value to M&E companies. Major consultancies are rushing to build out generative AI services. Among global business executives surveyed, 98% across a wide range of industries agree: AI foundation models will play an important role in their organization’s strategies over the next three to five years.

New possibilities for creativity and profit

Advocates are excited by a sky’s-the-limit mix of creative and operational possibilities: tours of museums and historical sites given by autonomous avatars who are artists and gladiators, for example. Instead of making a film every five years, Lee says, directors like James Cameron could use AI to craft a compelling visual pitch in an hour and make two. Vaunted democratization of AI will help a new universe of small companies and individuals create high-impact work and get a bigger piece of the pie.

Yet, Microsoft’s Lee reminds, “we’re still in the very early days.” Innovative use cases continue to emerge. Complex and crucial questions await ­– including copyright and plagiarism, data and customer privacy, worker and company acceptance, and government regulation. But it’s already clear: M&E players who carefully choose models, infrastructures and platforms will profitably deliver new differentiated, one-on-one experiences never before possible or dreamed.

As showbiz people in the early 20^th century used to say: “You ain’t seen nothing yet.”