PATENT DROP: Big Tech’s Battle Royale
Discover more from PATENT DROP
PATENT DROP: Big Tech’s Battle Royale
Breaking down tech firms’ gaming sector ambitions.
Happy Thursday and welcome to Patent Drop!
Today, we’re taking a look at how tech firms are getting their heads in the game. Sony and Meta both want to track your facial movements in VR; Intel wants to minmax its use of AI to render graphics; and EA wants to make its titles more accessible to people with colorblindness.
Let’s get into it.
#1. Give ‘em a smile
As AR and VR slowly but surely gain traction in gaming, tech firms want to know where your head’s at: Sony and Meta both filed patent applications for tech that tracks facial expressions while in an artificial reality device.
Let’s start with Meta. The company is seeking to patent a method for facial expression tracking that, in some examples, doesn’t require “cameras or complex image data processing.” Instead, Meta’s tech uses an illuminator and a photon detector.
As the name suggests, the headset’s illuminator lights up a user’s face, and the photon detector then picks up the ways in which that light is reflected off of it. The information from these two devices are then sent to a processor, which determines the look on a user’s face using an algorithm that can “reconstruct facial expressions.”
Along with obviating the need for an inward-facing camera system in an AR or VR headset, Meta’s said its tech “provides a low cost, low computational overhead facial recognition system.”
Sony, meanwhile, is working on tracking user facial movements in an artificial reality game using barometric pressure sensors. Using headset mounted sensors, this tech captures “pressure variances” from the motion of a user’s facial features, including eye movements, breathing patterns, blinking or speaking. These variances are then analyzed by a processor to detect and evaluate “motion metrics,” or a user’s facial expressions, and are used to “fine tune the user interactions.”
Sony said this kind of tracking helps monitor user engagement metrics with AR and VR games, to “gauge user interest or lack thereof” in the content being shown. Don’t let them catch you yawning.
“Gauging interest level of the users to the content of the different applications will assist the developers and content providers to provide content that is useful and/or interesting to the user,” Sony said in its filing.
As we’ve seen in several editions of Patent Drop, tech companies seem very interested in making tech that can track users' faces, whether it be to make video calls less awkward with simulated eye contact or make rendering easier for VR headsets.
What Meta and Sony’s patent filings have in common, however, is that they employ sensors rather than cameras to track your facial movement, potentially preserving privacy in a way that camera-based facial tracking methods don’t, Jake Maymar, VP of innovation at The Glimpse Group, told me. Ditching cameras also creates a better financial picture for the companies. Given the surfeit of patents for camera-based methods of expression tracking that already exist, sensors provide a workaround for these companies to get their hands on unique facial tracking tech, he said.
“It makes perfect sense. They're just trying to figure out the workaround of ‘How do I get facial capture without using cameras?’” Maymar said. “Because cameras have been patented and explored; they're expensive and also processor intensive. There’s also privacy and security issues with that.”
Sony and Meta are hardly newbies in the extended reality gaming space. Sony has long been an industry leader in console gaming with the PlayStation, and has its own headset offering with PS VR. Meta, of course, has its line of Quest VR and mixed-reality headsets to support its lofty metaverse goals, which supports a handful of games, including Among Us and Beat Saber.
As for facial tracking itself, there are a few potential reasons that Sony and Meta may be interested. In their filings, both companies touch on the unique engagement metrics that can be deciphered from viewing a user’s expression in-game, such as checking their emotional responses to certain content. But another potential use, Maymar said, is how these expressions could enhance the VR experience.
Maymar gave the example of going on an in-game VR quest with friends: “Let’s say you're in this scary corridor, and you can see your friends’ faces and they can see your face. You can see those expressions — that excitement — not only through gestures, but their actual face. It feels like a bonding moment, because it feels like you’re there.”
At this point, extended reality experiences represent a small fraction of the gaming sector, said Maymar. Part of the reason is because it can be a solitary experience, devoid of connection or emotion. But adding in a way to convey emotions in gameplay, such as through facial expressions, could be a key to growing user interest.
But until that happens, Maymar said, “XR is probably not going to really have a big impact on gaming.”
#2. Intel’s A-eye of the beholder
Intel’s tired of doing cost-benefit analysis. It’s making AI do it instead.
It may sound a little meta, but Intel is working on tech that uses AI to decide whether or not to use AI to regenerate video graphics. Here’s how it works: Neural networks are fed reference images of graphics, such as frames of a video game, that have been rendered without the use of AI. Based on the quality of those images, this tech then predicts a “visual quality score” of how well an AI could reproduce those graphics.
Intel argues in its filing that this tech essentially does the job of a quality assessment tester to “maximize quality while minimizing performance budget,” and can help decide what Intel called the “cutoff point” for when to stop training a GPU’s machine learning-based renderer.
The current industry standard for testing the visual quality of graphics is to use human opinion scoring, Intel notes, which can be a “time consuming and expensive” approach as it involves a large group of people to rate an even larger set of photos and video sequences. The company said this tech automates the process with a “high correlation to a subjective human opinion score of quality.”
“To determine whether such AI-based rendering models are able to improve the rendering performance while maintaining the image or video quality, a measure of how good the quality of the image/video is versus the analytic solution may be utilized,” Intel said in its filing. “Such a measure of image/video quality should correlate with a subjective human observer’s opinion.”
One additional note: Intel mentioned that this tech can be integrated with several gaming devices, including a server-based gaming platform, or handheld, mobile, and online gaming consoles.
Intel’s patent isn’t the first time we’ve seen this in gaming QA testing. Just last month, Google patented tech that allows developers to use AI to test their games, claiming that quality testing teams can’t “scale with the complexity of modern games, leading to delayed launches and lower quality products.” While the focus of Google’s tech differs from Intel’s, the basic point is the same: AI is being used to replace inefficient and typically human-run processes.
Intel, however, is not exactly the shining star of the gaming PC sector. Its latest earnings were grim, with the company seeing revenues for its client computing sector, which includes gaming computers, fall 38% as demand continues to tumble. The company’s results echo the industry-wide turmoil that started last year and has continued into 2023, with worldwide shipments falling 30% in the first quarter, according to Gartner.
Nvidia far outpaces Intel in the gaming PC market, said Romeo Alvarez, director and research analyst at William O’Neil. Nvidia currently hoards 82% of the market, leaving Intel neck-and-neck with AMD at 9% each. Intel didn’t enter into the space until 2020, Alvarez said, and its gaming offerings really started to pick up in 2022 when it started shipping its Arc graphics card. Despite the gaming sector’s woes last year, the company managed to nearly double its market share since the end of 2021.
“Intel’s (stock) has been an underperformer for a couple of years now,” said Alvarez. “They’re definitely becoming more competitive. There is upside there. But I think we still need more confirmation that the market is going to normalize.”
By adding more AI offerings as this patent suggests, the company may be itching to stay competitive. But as rivals race ahead with AI development, Intel is still lagging behind.
“In gaming PCs, they are gaining ground already,” said Alvarez. “In AI applications, I think it remains to be seen whether they can be a competitor. They're definitely moving in the right direction, but I still think by far Nvidia is the king there.”
#3. EA’s accessibility quest
Electronic Arts wants to make sure that everyone can enjoy its releases.
The company is seeking to patent a “color blindness diagnostic system,” which automatically determines whether or not a user needs color blindness accessibility settings turned on within the narrative course of a game.
First, this system determines a user’s “dichromatic visual deficiency type,” (a.k.a. type of color blindness, with a test based on “virtual color blindness indication objects,” or those with colors that are visually distinct from one another. This test may be built into EA games through a task or objective that prompts the user to interact with one of these objects. Then, based on that interaction, the game can determine if a user is colorblind and what kind of color blindness they have.
EA said that this allows colorblind users to not only avoid having to immediately modify their settings at the start, but also helps them find color-based cues throughout the game.
“Typically, the settings of a virtual environment, including those for color blind accessibility, are unavailable during the introduction of a user interactive narrative,” EA said in the filing. “Due to this limitation, color blind users of a video game often miss key visual cues for navigating and progressing through the user interactive narrative.”
Color can often be an important visual indicator in video game design. Think of the pulsing red outline that appears around the screen as a character is about to get knocked out in a fight, or how a certain color may indicate an enemy team member’s uniform. Without intuitive accessibility settings like EA is working on, these details represent a frustrating unintentional barrier to enjoyment for colorblind users.
“Modern virtual environments, such as video games, often presume a user does not require modifications for color blind accessibility,” EA noted in its filing.
EA’s patent isn’t the only move by a gaming company to make gaming more accessible. The pandemic gave way to a massive boom in the number of gamers. With that rise came a growing interest in accessibility features, with indie developers and high-budget studios alike upping their game.
For example, the Last of Us Part II, published by Sony in 2020, was widely regarded as a milestone for accessible gaming. More recently, God of War Ragnarök debuted in November with more than 70 accessibility settings, winning the AAA Excellence and Best Deaf/Hard-of-Hearing accessibility categories at the Game Accessibility Conference Awards this year. And Microsoft said its upcoming Forza Motorsport release will be the most accessible title of the franchise yet, with additions like Blind Driving Assist for low- or no-vision players, text-to-speech and fully remappable controls.
With the momentum around accessible gaming from developers of all sizes, the tech in EA’s patent may be a step in trying to keep pace with the rising tide.
Extra Drops
We’re not done yet!
Anduril wants to make the drone version of a Transformer. The company is seeking to patent an unmanned aerial vehicle with retractable wings that deploy after at least “one pre-set condition,” such as a certain velocity, acceleration or elevation, is met.
Mastercard wants to give your plastic a Web 3.0 flare. The company is seeking to patent a system for associating custom card designs with NFTs as a way of performing dual verification.
Spotify wants to stay on the beat. The company wants to patent “cadence-based playlists management,” which essentially automatically creates playlists with music of different tempo ranges to support users’ “repetitive motion activity.”
What else is new?
Montana Governor Greg Gianforte banned TikTok in the state on Wednesday, making it the first state to do so amid tension between the company and Congress.
Alibaba will spin off its cloud services business into a separate publicly-traded company, CEO Daniel Zhang announced at the company’s earnings call.
Calling all Android owners — the digital landscape and ways to use your device are evolving. Luckily, our friends at Android Intelligence deliver three clever tips directly to your inbox every Friday. As Google continues R&D on AI, this no-cost newsletter will keep you in the loop, turbo-charge your productivity, and teach you to take ultimate control of your devices. Join 50k+ Android savants — sign up for no cost here. *
*Partner
Have any comments, tips or suggestions? Drop us a line! Email at admin@patentdrop.xyz or shoot us a DM on Twitter @patentdrop.
 | A guest post by
|