PATENT DROP: Apple breaks the sound barrier

Plus: Google’s abuse detector; Ford’s face ID

Happy Thursday and welcome to Patent Drop! 

This morning, we’re checking out tech from Apple to help developers curate their soundscapes; a tool from Google that aims to stop abusive content in its tracks; and a filing from Ford that show its interest in making your biometrics your new car key. 

But first: Today’s sponsor is actually another great newsletter, because we could all use a little more insight in our inboxes. Chartr sends you gorgeous visuals that help deepen your understanding of the biggest trends shaping the world; sign up right here. 

And a quick programming note: The next edition of Patent Drop will be published on Tuesday, June 20, in observance of Juneteenth. 

Alrighty folks, let's take a look. 

#1. Apple sounds off

In the wake of Apple debuting its long-awaited headset at WWDC last week, the company may be looking to give developers more control over extended reality soundscapes. 

Apple is seeking to patent an API for developers of “simulated reality” applications that can “determine a total calibration gain” of sound in-headset experiences. Fundamentally, this allows developers to apply Apple’s concepts of 3D and spatial audio to their own simulated reality experiences. 

This system lets developers control the sound of a specific virtual object, and adjusts based on the end-user’s distance from that virtual object. The developer controls what Apple calls the “total calibration gain,” or essentially a combination of factors that determine the volume an end-user hears, and adjusts this “to reflect the expected real world sound level desired by the author of the app.”

Though this sounds a bit technical, think of it like this: If you’re wearing a headset, and a virtual object falls from a virtual ledge and shatters 20 feet away, this system makes it sound that way. Or, if the developer wants a virtual bee to sound like it’s flying quickly by your ear, this system will simulate that sound, no matter what volume the end-user has set. 

While this tech could apply to apps developed for Apple’s new headset, which includes it’s Spatial Audio system, the company noted in its filing that this system allows developers to accommodate the playback experiences of different types of simulated reality devices, allowing the playback to stay “faithful to the desires of the author,” no matter the hardware. 

“Different hardware devices … have different playback sensitivity,” Apple noted in its patent. “As such, the same digital audio asset will be experienced with different playback loudness across different hardware devices.” 

Photo via the U.S. Patent and Trademark Office.

No matter how good visuals are, good sound design is crucial in making any extended reality experience engaging, said Jake Maymar, VP of Innovation at The Glimpse Group. “75% of a presentation is sound,” he said. “If you watch a really great movie without sound, it just feels empty. But if you have okay visuals and amazing sound, it's fantastic.” 

Apple has long understood the importance of good sound design, said Maymar, and that priority is evident in the way the company is marketing its new Vision Pro headset. Rather than trying to sell the device with a focus on gaming as many other VR and AR companies have, Apple is marketing the headset as an immersive “home theater replacement,” said Maymar, with spatial audio as a core feature. Apple getting a patent on the system used for this sound design could be its plan to preemptively ward off copycats before it’s released to consumers. 

“The best sound you can experience is with this headset,” said Maymar. “It makes sound exist in space where it would actually, physically exist in the space. That's why I think they're patenting the sound design, because it's essentially their secret sauce.” 

While Apple is quite new to the so-called simulated reality market, Vision Pro is an impressive first offering, said Maymar. The company has solved the common distortions issues caused by “passthrough” in AR, or seeing through the visuals of the headset, he noted, that plague other VR and AR companies. 

“You’re going to have a seamless experience, even though mathematically (the headset) is doing all these crazy calculations, it makes it look like you're just looking through it,” said Maymar. “That's the beauty of Apple, they make these deceptively easy to use experiences.” 

The Vision Pro isn’t cheap – with the first version running at $3499 – but a less expensive, second generation version of the product could reportedly hit the shelves as soon as the end of 2025, Bloomberg’s Mark Gurman reported. Since the Vision Pro is likely first in a long line of AR devices for Apple, it’s a good idea that the API in this patent filing is designed to help developers give the same sound experience to every listener, in any device. 

#2. Google’s AI spam detector 

Google is searching for new ways to fight spammy query results. 

The company is seeking to patent a method for identifying abusive user accounts “based on playlists.” This system, which would likely be built into YouTube,  calculates a “playlist score” based on features associated with a user’s playlist. A playlist’s features include its content, view count, “quality score,” and the activity level of the user that created it. This system also calculates a “channel score” based on the user’s average playlist score. 

After making these calculations, if the system determines the user is abusive, their playlists are subsequently “demoted,” meaning their rank is lowered in search results and their reach is limited. 

Google’s method relies on a “playlist classifier” and a “channel classifier,” or an AI model trained to make these calculations based on information that indicates “bad” and “good” playlists, such as those created by suspended channels versus those with a high “quality score.” The system is also continuously trained to recalculate playlist scores based on “recent trends in keywords and abuse patterns.” 

This tech allows Google to filter results to control the reach of “spam playlists” on platforms like YouTube, which can include content that is “unrelated, misleading, repetitive, racy, pornographic, infringing, and/or “clickbait,” but often use popular keywords in their descriptions to gain popularity and perform better in search queries. 

“Existing approaches for controlling spam playlists are ineffective due to the sheer number of spam playlists,” Google said in its filing. “Manual user reporting of individual spam playlists simply cannot address the thousands of new spam playlists that are automatically generated by abusive users on their respective user channels each day.” 

Photo via the U.S. Patent and Trademark Office.

This patent is the latest of several that show Google’s interest in slowing the spread of low-quality content. The company filed a patent application for a machine learning-based system that essentially predicts how viral a piece of content may go, aiming to create a safety net to catch dangerous content before it goes live. The company also has sought to patent tech that could  steer users away from misinformation or potentially dangerous content, which uses machine learning to track and predict user behavior. 

Taken together, these patent filings could help the company keep the wrong content from going viral on YouTube and beyond. And using its strengths in AI to combat harmful content this only makes sense, given that the tech can more quickly identify disparate anomalies across the platform than a team of content moderators on their own. 

But at the end of the day, all roads lead back to its highly-lucrative ad business. Both Google and YouTube saw slight dips in their ad revenue in the latest quarter amid a slowdown in the sector. Bad content surfacing at the top of search queries likely won’t make its advertising partners happy, potentially leading them to favor rivals like Meta or TikTok. Curtailing the reach of that content is likely a high priority. 

Google has faced a number of claims over the years that it hasn’t been so diligent about catching bad content. Both YouTube and the search engine have faced allegations that the platforms have amplified fake news, hate speech, and disinformation. The company also announced last week that it reversed its election integrity policy, which took down content claiming fraud in the 2020 election. YouTube claimed that the policy had the “unintended effect of curtailing political speech without meaningfully reducing the risk of violence or other real-world harm.” 

YouTube said that it will “remain vigilant as the election unfolds,” and have more details about its approach for the 2024 election in the coming months. AI-based systems like those in Google’s recent patents may be part of its plan behind the scenes. 

---

SPONSORED BY THE CHARTR

It Takes a Great Newsletter to Know a Great Newsletter

You’d go to a podiatrist recommended by your primary care provider, right? So take it from newsletter folks: Chartr’s visual newsletter is one of the best out there. 

They’re a lot like us, but with a different specialty: gorgeous visual illustrations that provide deeper understanding of the trends shaping the world. 

With their free newsletter, you’ll become your social group’s new go-to authority on not just the latest headlines, but the broader forces driving them. 400,000+ professionals already receive Chartr’s snackable charts and easy-to-recall insights as their morning dose of top-notch small talk fodder. 

Join Chartr for a deeper understanding of our world – via beautiful email newsletter – right here. 

---

#3. Ford’s biometric car key

The days of digging through your purse, backpack or pockets to find your car keys may soon be over: Ford wants to make your car a keyless experience.

The company is seeking to patent a method for “enhanced biometric authorization.” Ford’s filing notes two ways that it may authorize a user to access a car without a key: from outside of the car and from within it. From the outside, this system may determine a user is authorized to access a vehicle as they walk up to it, unlocking the doors automatically. Once inside, the system uses cameras and sensors to perform facial recognition to determine if the vehicle can accept the user’s commands to operate it. 

The automaker’s system also updates a user’s biometric print if a “trigger,” or the “presence of a variable characteristic,” is seen in a user. Ford said that this is because “biometric data for the user can change over time, e.g., due to weight gain or loss, age, hair growth or loss, etc., which can reduce the accuracy of biometric authorization for the user.” 

Ford noted that these updates happen in “real-time or near real-time” and “without input from the user,” allowing the system to “maintain user data that corresponds to current user biometric data, which can reduce the likelihood that an authorized user will fail biometric authorization, or an unauthorized user will pass biometric authorization.” 

While Ford used vehicles as its example throughout the patent, the company noted that this system can be applied to other structures, including buildings, garages, and gated areas. 

Photos courtesy of the U.S. Patent and Trademark Office.

This isn’t the first time Ford has taken an interest in faces: The company filed a patent for a “vehicle vision system” with a similar concept in mid-2020. Ford also isn’t the first car manufacturer to show an interest in replacing physical car keys with biometrics. Hyundai debuted a fingerprint-activated vehicle back in 2018 that could unlock and start using touch ID, and Mercedes introduced fingerprint ID vehicles in 2022.

Adding facial recognition to the mix is the “natural evolution of biometrics and automobiles,” Aubrey Turner, executive advisor at security firm Ping Identity, told me. Plus, the use cases for biometrics stretch beyond just security, he said, with biometric user profiles having the capability to make things like payments or vehicle settings more convenient.

“Biometrics have a natural way of removing friction, whether it be for the security of the vehicle itself, or the usability of the vehicle,” said Turner. 

While biometrics can provide more security, automakers have to consider a number of factors before taking this route, said Turner. Because this type of data is highly sensitive, storing it safely should be of high priority, he said, whether it be through a cloud service or in the vehicle’s computer system itself. “Thieves have evolved as the anti-theft measures have evolved,” he said.

Another consideration is the accuracy and reliability of those biometrics, Turner added. Specifically for facial recognition, making sure that biometric sensors are accurate for people of “all skin tones, shades and looks” is key if these companies want to make this kind of security a widespread reality, said Turner. 

And of course, these companies should be prepared to mess up and right their wrongs before they grow more pronounced. “Mistakes will be made,” Turner said. “It's just inevitable that somebody's going to implement something badly.” 

---

Extra Drops

Some other fun patents we wanted to share. 

• Visa wants to make sure you are who you say you are (in the most outlandish way possible). The company wants to patent a method of identity verification that involves dispatching a drone to your location to take a photo of you. Why? Who knows. 

• EA wants to get its reactions right. The game maker wants to patent an AI-based system that predicts what a character’s facial expression should be based on their pose in the development process. 

• Ford wants to make sure your kid is home on time. The company is seeking a patent on tech to “enforce a curfew,” that reports whether or not a vehicle is within a geofenced area by a certain time at night. 

What else is new? 

• Twitter is facing a $250 million lawsuit from The National Music Publishers’ Association, which represents Universal, BMG and Sony, for copyright violation.

• AI systems like ChatGPT are still not as smart as your dog, as they are trained solely on language, Meta’s AI chief Yann LeCun said at the Viva Tech conference in Paris.

• Tesla is set to see some major EV competition: Bank of America predicts that the company’s share of the U.S. EV market could drop by 18% by 2026.

Have any comments, tips or suggestions? Drop us a line! Email at admin@patentdrop.xyz or shoot us a DM on Twitter @patentdrop.