PATENT DROP: Adobe’s AI DEI
Discover more from PATENT DROP
PATENT DROP: Adobe’s AI DEI
Plus: Dr. Nvidia checks in; Microsoft’s vocal backpack
Happy Monday and welcome to Patent Drop!
One thing before we get into it: Patent Drop is moving to a new home.
To ensure we are hitting your inbox uninterrupted, we have made the switch to a new email sender. That being said, today is our last issue on Substack.
Fear not! To ensure you continue getting this newsletter without disruption, please do the following:
Add team@patentdrop.thedailyupside.com to your Contacts or VIPs.
And starting with next week, if you don’t see the newsletter in your inbox on Thursday, search for team@patentdrop.thedailyupside.com and drag the newsletter to your Primary inbox if it’s in Spam or Promotions.
Now, on to the main event!
Today, a filing from Adobe for an AI-based diversity auditor; a patent from Nvidia for high-quality medical imaging; and a patent from Microsoft to take your digital assistants on the go.
Let's dive in.
Adobe’s AI auditor
Adobe wants to add some AI into diversity, equity, and inclusion. What could go wrong?
The company is seeking to patent a system for “diversity auditing” using computer vision. Essentially, this system uses facial detection and image classification to break down photos of employees and slot them into categories based on certain physical traits and characteristics.
Adobe’s system looks through several images and detects faces in each one, then classifies each face based on a predicted “sensitive attribute” relating to “protected classes of individuals,” such as race, age or gender. For example, Adobe noted, this system may classify images from a company’s website, then compare its predictions to a “comparison population.” (Adobe noted that this could mean census or employment data, but it could also potentially include internal data, such as a companywide diversity report.)
The system then calculates a “diversity score” for the set of images using machine learning by comparing the classified images to the comparison population. Finally, the system then “augments the set of images to increase diversity” using “additional retrieved images” until a certain threshold of diversity is met.
Adobe noted that conventional diversity auditing systems are time-consuming and “rely on manual identification of image attributes, and this manual approach does not scale to large image sets.”
Adobe’s intentions are seemingly in the right place, but potential problems exist with a system like this, said Mukul Dhankhar, founder at computer vision firm Mashgin. For starters, measuring diversity from photos is a difficult task. Can you really tell someone’s age, gender or ethnicity just by looking at them? How does this account for people of multiple races, or gender-nonconforming individuals? Those questions are left unanswered in the filing, Dhankhar noted.
“They talk about a single diversity score, but generally, diversity has multiple dimensions,” said Dhankhar. For example, he said, if you have a set of images that “represent multiple ethnicities, but they’re all male, would the model say that’s diverse? Or what if it takes in images of stereotypes? Those are some of the things that need to be taken into account.”
Adobe also has to be sure that the AI model itself was created without biases, as the biases of the person that built it can often translate to the outputs of the model itself unintentionally. “They don't go into detail on how they will make this model or algorithm unbiased itself,” Dhankhar said.
However, there are potential good use cases for this technology if it’s done right, said Dhankhar. For example, this system could be used for checking if the data that’s used to train AI models is diverse enough. And if expanded beyond images into textual data, the use cases stretch even further.
“In general, the idea of identifying data set bias is a good thing,” he said. “But unfortunately, this patent doesn't go into very many details.”
Dr. Nvidia takes a picture
Nvidia is making a little x-ray go a long way.
The company filed a patent application for “constructing medical images” using machine learning-based techniques. Nvidia’s tech essentially allows its users to obtain better medical images with less data to assist in a variety of medical contexts.
The company’s patent is quite complex, but Nvidia’s system first feeds a “medical item,” (i.e., a medical image), as well as data associated with the “spectral domain” of the item (meaning, what kind of medical imaging it is) to a machine learning model. The output of this model is, in fact, another AI model, which spits out “predicted values,” or predictions for more in-depth data about what exactly is in the image.
From this, a high-resolution medical image can be created, allowing for a deeper look and better diagnostic insights than what may have been possible with traditional imaging. These improved images could potentially mitigate the need for invasive measures such as biopsies.
Nvidia noted the medical images created using this system are “substantially more accurate” than other methods that use little training data. “Producing accurate images from sparsely sampled data is a difficult technical problem that exists in many applications,” Nvidia said.
This isn’t the first patent from Nvidia for medical-related AI tech. The company filed to patent tech that estimates heart rate and respiratory rate using neural networks with higher accuracy and lower latency using facial image and motion data. Nvidia also has a medical technology business that focuses on things like imaging, biopharma, and genomics, so a patent like this follows suit, especially given its tight grip on the AI industry.
Nvidia remains the top dog in AI as developers continue to snatch up its chips and use its data centers. The company’s shares have risen more than 200% so far this year, and after reporting blowout earnings and an optimistic outlook last week, the company hit an intraday record for its share price.
Nvidia reported sales of more than $13.5 billion for the quarter, up 101% year over year. The company also said it expects sales in the current quarter to skyrocket a whopping 170% to $16 billion, way above the $12.5 billion expected by Wall Street analysts.
The company’s largest business for the quarter was data centers, which “captures that AI opportunity,” Romeo Alvarez, director and research analyst at William O’Neil, said on Bloomberg News. “What we’re seeing is very high demand for these AI chips. The company is fully booked in terms of demand until next year. The problem right now isn’t demand for their offerings, but actually the supply.”
Because of its success in AI chips, the company likely has more leeway than others to experiment broadly with other AI technologies, whether that be chatbots, driverless vehicles, or in this case, medical imaging. In fact, the company’s success in the sector could buoy its other AI ventures – a rising tide does lift all boats, after all.
Microsoft straps AI to your back
Microsoft wants you to be able to bring AI with you wherever you go.
The company filed a patent application for an “artificial intelligence assisted wearable.” Microsoft’s filing pitches a “context-aware” wearable that can complete commands or answer questions while on the go. This is basically like if a smart speaker and a backpack had a baby.
Here’s how it works: The backpack is equipped with sensors such as a camera and microphone to be able to complete user commands. The backpack may receive a contextual voice command from a user with a non-explicit reference to the environment around them. The backpack would use sensor data from the camera and microphone to be able to complete said command. This backpack may be activated with a button on the strap by pressing, holding or double-tapping for different operations.
The company gave a number of scenarios of how someone would use this backpack, including giving the user directions, adding an event to their calendar, or price comparisons while shopping in person. Microsoft notes that this backpack would have access to user data to be able to complete commands with the most context possible.
Microsoft says the drawback of many conventional digital assistants is that they’re “physically stationary.” While they can control operations within a home such as lighting or thermostat, “the functionality and usefulness of such conventional digital assistants are limited to the home surroundings.”
When a digital assistant is portable, Microsoft notes, it generally isn’t aware of the surrounding environment.
The majority of Microsoft’s public patent filings detail inventions that are equipped with AI in some way, whether it be ad tech, digital assistants or just plain machine learning training techniques. It’s also representative of its broader AI goals as the company throws out AI-enabled products like 365Copilot and Bing Chat, its partnership with OpenAI, and plans to reportedly implement AI features into Windows updates.
While an AI-based backpack may seem outlandish, the patent could indicate an interest in AI wearables. The filing only details how this tech would be implemented into a backpack, but given that patents offer a limited view into how tech can actually be applied, the company may have other ideas in store. After all, Microsoft Research does have a wearable technology team — though the company’s only wearable offering is a smartwatch.
This could also be a case of the company trying to get ahead of predicted future trends. Though AI wearables development is more in the realm of smartwatches and AR glasses, Microsoft may want to be able to pull this off the proverbial shelf if the pendulum swings in that direction.
However, this patent also underscores the idea that AI development isn’t always so straight-laced. The average person doesn’t understand the inner workings of a neural network or an AI chip — they’re more likely to gain exposure to it from AI covers of Spongebob singing Billy Joel’s “Piano Man,” using ChatGPT to make cocktail recipes, or in Microsoft’s case, backpacks that talk back to you à la Dora the Explorer. (Whether or not consumers would want a talking backpack with a nanny cam is up for debate.)
It shows that AI is pervasive beyond tech circles, melding with every part of our lives in the same way the dawn of the internet did, gaining notoriety from both the offbeat and the serious.
Extra Drops
A few more before you go.
Sony wants to show you the highlights. The company is seeking to patent a system for “generating sports game highlight video(s)” based on “excitement of gameplay.”
Apple wants you to know when your phone slides under the car seat. The company filed a patent application for a system for locating a mobile device within a vehicle, like a hyper-specific version of Find My iPhone.
Yes, the Legoverse is still happening. The toymaker filed a patent for a method for “toy recognition” using computer vision for “toys-to-life applications.”
What else is new?
A former Goldman Sachs trader has been working to build an urban utopia near Silicon Valley for the past few years. The plan is backed by some of the biggest investors in tech, including Reid Hoffman, Marc Andreessen and Michael Moritz.
Google Flights will now tell you the cheapest time to book, using historical trend data to find out when it’s typically been least expensive to book a flight for a user’s requested dates.
DoorDash is launching an AI-powered voice ordering feature, which essentially makes DoorDash the middleman between the user and the restaurant when making a phone order.
Have any comments, tips or suggestions? Drop us a line! Email at admin@patentdrop.xyz or shoot us a DM on Twitter @patentdrop.
 | A guest post by
|