Discover more from PATENT DROP
PATENT DROP: Apple’s bug in your ear
Plus: Baidu’s AI videomaker; Snap’s NFT gambit
Happy Monday and welcome to Patent Drop!
Today, we’re checking out tech from Apple that would track your health through your AirPods. Plus: Baidu wants to bring your portraits to life; and filings provide a peek at what Snap’s NFT operability could look like.
But first, let’s shed some light on today’s sponsor. GoSun is redefining the possibilities of off-the-grid living with lightweight, solar-powered outdoor products. Satisfying the needs of the $570B outdoor and disaster relief industry, GoSun’s lineup of RVs, trailers, cook stoves, coolers, battery chargers, and generators have partners (and investors) lining up. See why Top Chef, Discovery, Forbes, and Nat Geo made such a fuss.
Let’s dive in, shall we?
#1. Apple’s AirStethoscope
Apple wants your AirPods to do more than just play music… a lot more.
The company filed several patent applications for tech that monitors biometric data with in-ear sensors. For starters, the company wants to patent a method for “capturing cardiopulmonary signals” using an accelerometer, a device that measures vibration, mounted within a headphone. The accelerometer picks up the “subtle vibrations” created by breathing and heart rate, as well as the “acoustic waves produced by heart and lung sounds,” which in turn are used to measure your heart rate.
Apple said in its filing that current means of capturing audio measurements of health indicators “suffer from susceptibility to environmental noise or low-frequency motion artifacts, and have difficulty capturing both mechanical and acoustic signals simultaneously with high fidelity.”
Next up, the company filed to patent a method for “measuring and tracking ear characteristics.” This tech uses an in-ear or on-hear headphone to perform a “baseline measurement,” a hearing test that sets a standard and is used for future comparison. With this system, Apple can monitor your ears over time for hearing loss, as well as learn the characteristics of your ears so that if something deviates from the norm, you can know when to go to an audiologist before a condition is exacerbated.
TLDR; these patents allow Dr. Apple to get to know your ears really, really well. (Add that to the long list of bodily functions that Apple already tracks intimately.)
This isn’t the first time Apple has shown interest in getting into its users’ ears. The company reportedly patented in-ear biometric identity scanning, aiming to protect the user privacy of AirPods, early last year. The company also included in-ear devices in a reference image in a patent that became public earlier this year for heat-activated electrodes for better physiological monitoring.
The company has progressively added more and more monitoring capabilities to its Apple Watch, such as blood oxygen level, sleep monitoring and respiratory rate, and has upped the price point as a result, charging $799 for its top model. Apple may have the same strategy in mind for monitors in headphones: The more features they can add, the more they can charge.
Apple has spent years becoming a major player in the health and wellness sector, mainly due to its complete domination of the smartwatch market. The Apple Watch accounted for 30% of all smartwatch shipments last year and 56% of all revenue made in the smartwatch market globally in that timeframe, according to data from analytics firm Counterpoint.
While Apple is a leader in the market, there is still room to grow. McKinsey estimated that the personal wellness market was worth $1.5 trillion in 2021, and grows by 10% every year. Upgrading the AirPods’ capabilities could be part of the company’s plan to keep hold of its dominance.
#2. Baidu, the filmmaker
If you have a photo of yourself that you hate, Baidu may be able to turn it into a video of yourself that you like.
Baidu is seeking to patent tech which generates videos from a reference photo. To break it down: This tech takes an original reference portrait and performs “posture change processing” using computer vision and deep learning to create change images. This tech then strings those images together to create a video of that image subject making slight movements.
For example, if you give Baidu’s tech an image of yourself slouching in a chair, it may adjust your posture, or turn and tilt your head slightly. It would then compile those slightly changed images into a video that shows you sitting up, looking towards the camera and smiling. (Not creepy at all, right?)
“The reference image of the portrait and the obtained plurality of change images with different portrait postures may be taken as a plurality of video frame images of a portrait video to form a virtual portrait video,” Baidu said in the patent.
While the tech outlined in Baidu’s patent can only make you move a little bit, it’s another step towards making realistic videos using AI. If this patent is granted, claiming this method of AI video generation as its own could be a win for the company.
However, while Baidu has big ambitions, it also has stiff competition in the AI arms race. OpenAI, creator of the popular ChatGPT chatbot, is “leading in a lot of ways,” Jake Maymar, VP of Innovation at The Glimpse Group, told me, with Big Tech firms like Google, Microsoft and Meta all throwing their hats in the ring. In a supposed attempt to keep up, Baidu released its own ChatGPT competitor, Ernie Bot, a few weeks ago, but saw shares tumble briefly as the chatbot disappointed users.
“(Baidu) put out a couple of offerings already, and they didn’t hit as well (in quality),” Maymar said.
And while this patent and other video generation tech has great potential, it comes with some ethical issues. These issues, and quick development of AI, gained widespread attention after an open letter from the nonprofit Future of Life Institute called for a 6-month moratorium on training massive AI models in late March. The letter gained support from the likes of Elon Musk and Steve Wozniak.
Maymar noted that photo generation AI software like DALL-E and Midjourney have already advanced to the point where they can create a fake image of someone that looks entirely real. Just look at what happened last week when practically the whole internet fell for AI images of a swagged-out Pope Francis in a white puffer jacket. Realistic AI video is the next frontier, said Maymar. It could be dangerous if ethics aren’t called into question.
“Pretty soon we're going to get to videos where you cannot tell that they were generated by an AI. There really aren't a lot of guardrails up right now,” he said. “Unfortunately, there are bad apples that want to generate things that could harm, but you can also generate really amazing experiences and ideas.”
SPONSORED BY GOSUN
The Galaxy’s Greatest Energy Source…
…is fueling the Outdoor Industry’s greatest opportunity (literally).
As seen on Top Chef, Discovery, and Nat Geo: GoSun and its roster of lightweight, solar-powered RVs, solar cookstoves, generators, coolers and off-grid solar appliances are trailblazing the future of outdoor living and recreation.
Why NGOs and outdoor enthusiasts love GoSun:
No fuel, no mess–just clean, solar-powered energy sources
Weather disasters on the rise globally
Portable food, water, and shelter solutions
Allows greater access to off-the-grid lifestyles
Why investors love GoSun:
$570B TAM between outdoor recreation and disaster relief
Fresh off a $2.4M funding round
$5.87M in sales in 2021, up 50% from 2020
Partnerships with the American Red Cross and other NGOs
Recently released their all-electric boat
The future looks pretty bright for GoSun, but only days remain to become an investor. See the brief here.
#3. Snap’s NFT plan Crackles and Pops
While hype for NFTs has plummeted massively, patent filings show what a Snapchat NFT-verse could look like.
The company filed two patent applications for NFT integration on its platform. First, the company is seeking to patent NFT dynamics, or a system that allows a user to mint NFTs and allocate them to themselves or other users.
The filing also details Snap’s plans for “open-edition” and “one-of-a-kind” NFTs. Think of it like this: the original owner of an NFT creates an open-edition version (a.k.a. one with “no limit to how many times it can be minted,”) from a one-of-a-kind version, that allows the original owner to give out as many versions of the same NFT as they want, and edit the metadata of that version however they see fit.
One more note: The company also mentioned that the NFTs on its system could come in the form of augmented reality objects or filters, potentially playing into its larger AR ambitions.
While the first patent only briefly mentions fusing NFTs and AR, this next one spells out those plans in detail: The company wants to patent tech for “augmented reality experience control” through NFTs. In this system, an NFT of the AR filter offers “an interactive experience of a real-world environment where the objects that reside within the real world are enhanced by computer-generated perceptual information,” which is available exclusively to the owners of said NFT.
In June, The Financial Times reported that Snap was test-running allowing users to import NFTs as AR filters, though not much has been publicized of its blockchain exploits since. In the months following, Snap let go a big portion of its Web.3 team in a layoff affecting 20% of its staff, saying that the projects were “not a priority.”
But with three million AR lenses offered on Snap, NFT integration could be used to authenticate specific filters belonging to specific artists, and could be a major draw for creators, Jordan Gutt, Web 3.0 lead at The Glimpse Group, told me. And drawing in more lens creators to keep the platform fresh is key if Snap wants to keep up user growth momentum.
Still, it’s no secret that NFT and blockchain hype fell significantly last year, with the crypto crash putting a wet blanket on investor and consumer interest. But in the aftermath, blockchain companies have started to rebuild “smarter, better and easier to deploy because of it,” DJ Smith, co-founder and Chief Creative Officer at Glimpse, told me. Sales show that NFTs may be rebounding, surging to $2 billion in February, the highest they’ve been since last May.
“The hype may not be as plentiful as it was, but the technology is still here to stay,” Gutt added.
If NFT operability on Snap ever sees the light of day, these patents could provide a peek at how a potential integration might work.
We’re not done yet!
Microsoft wants to make sure the right people are on the right tasks. The company filed a patent application for tech that “identifies the most relevant person” to handle tasks based on a “triggered event,” by accessing the activity logs of people on a team.
Google wants to take on the tedious tasks. The company filed a patent for tech which automatically updates spreadsheets based on natural language processing and “knowledge monitoring,” which reduces the human error that comes from staring down a 1,000 row budget spreadsheet for 8 hours on end.
Apple wants to count your calories correctly. The company is seeking to patent tech which accounts for whether a user is riding an E-Bike or not, so as not to overestimate how much of a workout you’re actually getting on your commute home.
What else is new?
Amazon CEO Andy Jassy reportedly wants to double down on physical stores, blaming the pandemic for the company’s struggles in the grocery sector.
Apple CEO Tim Cook talked up the creative and communication benefits of a mixed reality headset in an interview with GQ, saying the ability to “overlay the physical world with things from the digital world could greatly enhance … people’s connection.”
Tesla reported more than 422,000 deliveries in the first quarter, representing a 36% year-over-year uptick.