PATENT DROP: Pinterest digs in deep

Plus: Airbus’s eye in the sky; Adobe paints a picture

Happy Thursday and welcome to Patent Drop! 

Today, we’re checking out Pinterest’s plan to scour your email for more detail on your interests; Airbus’s tech to keep a watchful eye on the workers building its airplanes; and Adobe helping language models surpass their blind spots. 

Let's get into it.

#1. Pinterest reads into things

Pinterest wants to get to know you from more than just your scrolling habits. 

The company is seeking to patent a method for “generating personalized content” based on a user’s email data. The system would, with user authorization, go through an email that a user has connected to their Pinterest account to identify topics that they may be interested in. 

This system relies on a machine learning model that essentially makes your emails the dataset it learns from: It evaluates emails to identify new topics of interest, update existing topics, or “simply record the information as user data as the basis of making further enhancements or revisions to the user's preferences.” The system may decide the “strength and sentiment” of a user’s affinity for a topic based on how often it comes up in their inbox and how it relates to the user’s current Pinterest habits.

Based on the model’s findings, Pinterest will then serve you custom content, auto-generating boards and surfacing posts based on your indicated interests. For example, if you sign up for a newsletter about gardening, Pinterest’s AI may fill your boards with gardening tips and inspiration. If it stumbles on an email about travel bookings to Costa Rica, it may put outfit ideas or restaurant recommendations on your feed. 

Because people often use multiple social media platforms, Pinterest said it may be difficult to understand the full extent of a user’s preferences. “For example, one social networking service may have substantial user preference information in regard to a user's preference for travel, while another service may hold information regarding that same user's preference to hobbies.” 

“Each social networking service is in competition with one another for a finite amount of a user's attention,” Pinterest noted in its filing.  

Photo courtesy of the U.S. Patent and Trademark Office.

Pinterest’s main moneymaker is digital advertising, a business model that isn’t doing too hot amid an uncertain economy. The company recognizes this, too: In its most recent earnings call, CFO Todd Morgenfeld noted that “the ads market continues to be uncertain given the macroeconomic environment.” 

This system could help Pinterest get to know its users much more closely, and in turn, help the platform keep hold of user attention. The more monthly active users it has, the better it looks to advertisers, so finding efficient ways to appeal to its user base may be Pinterest’s plan to get more eyes on its ads.

Pinterest could have another lucrative use for your email data: shopping. The platform has boosted its attention to shopping tech in recent months. In February, CEO Bill Ready said that Pinterest intends to make “every pin shoppable.” 

In April, Axios reported that the company planned to invest heavily in shopping tech, including computer vision, machine learning and AI to personalize shopping and track user behavior. Chief Revenue Officer Bill Watkins told the publication at the time that the tech aims to “drive the best shopping experience fulfilled by merchants that are already on our platform.” This patent, filed in February, could provide a look at what’s next in Pinterest’s shopping strategy.

But Pinterest’s email-scraping ambitions may come with a few hinges. For one, email inboxes aren’t always as clean cut as this patent lays out. Pinterest has to make sure its tech doesn’t study spam, work and personal emails as it makes its predictions, especially since machine learning models are only as good as the data they’re trained on. 

And even though Pinterest hasn’t had any significant data breaches in recent years, the question remains of whether or not the platform’s users would be comfortable with handing over access to their potentially personal or sensitive data in the name of better pins. 

#2. The watchful eye of Airbus

Airbus wants to get up close and personal with the workers assembling its aircraft.  

The company is seeking to patent a system for “cognitive assistance” in the manual assembly of an aircraft. Here’s how it works: Sensors, such as cameras, EEGs and eye tracking tech, monitor physical and physiological data on a worker that’s assembling an aircraft. That data is continuously fed to a “cognitive model,” which makes real-time predictions of that person’s expected behavior during the aircraft assembly process.

This information, in turn, controls the actions of different machines in the assembly process. For example, if a worker takes an unexpected break, the system reacts to this change and automatically halts operations of the machines that they’re working with. This would help workers avoid any injuries caused by automation in manufacturing. 

By making industrial machines more reactive, Airbus said that its tech could help overcome fundamental challenges of human-machine collaboration in manufacturing environments, improving things like safety and assembly quality. This system also has the potential to majorly cut labor costs, as it can be applied to the highly specific and typically “personnel intensive” tasks that aircraft manufacturing demands.

“Control systems of today's collaborative robots are often unable to handle the prediction and incorporation of human behavior in complex assembly situations” Airbus said in its filing. “The highly demanding requirements on the quality of the assembled components pose another hurdle.” 

Photo courtesy of the U.S. Patent and Trademark Office.

Airbus’s patent is the latest of several filings from tech firms that aim to implement AI into a factory setting. Baidu sought to patent an AI model that could detect whether or not workers are wearing safety gear, and Airbus rival Boeing filed a patent application for an AR and AI-based system that automatically detects anomalies during aircraft inspections.

What set’s Airbus’s patent apart is that it essentially aims to automate the job of a factory foreman, said Rhonda Dibachi, CEO of manufacturing-as-a-service company HeyScottie. Rather than implementing AI to control just the machinery itself, this system does the job of top-down management by overseeing the interaction between the human worker and the machine. 

“It enables partial automation of assembly operations – that is a foreman’s job,” said Dibachi. “It is no longer just doing single operations, it's actually doing an assembly(line). This is lights-out management, which is a fascinating next step.” 

As skilled manufacturing workers become harder to find, embedding robotics further up the chain will only further reduce the heavy cost of labor, said Dibachi. While the startup costs of implementing a system like Airbus’s can be prohibitive, “everybody who is adapting robotics in any sense … doing it because of labor.”

But one thing this patent doesn’t explicitly note, Dibachi pointed out, is an override function. AI-based tools can go haywire without oversight. If a factory’s foreman is replaced by AI, employees should have the capability to hit the kill switch if something goes wrong. 

Another consideration: While tech like this has the potential to improve factory safety with the right precautions in place, the intense monitoring involved in Airbus’s system does involve employees giving up a certain level of privacy, bringing up the ever-consistent question of cost versus benefit involved with AI and data privacy. 

Whether or not employees will be comfortable with tech that tracks them closely depends on how Airbus presents it to them, said Dibachi. “If there’s an obvious safety link, worker adoption is going to be higher,” she said.

#3. Adobe catches blind spots 

Art doesn’t come easy to everyone. Adobe wants to help language-based AI paint a better picture. 

The company filed a patent application for what it calls a “visually guided” machine learning language model. Adobe’s system aims to help language-based machine learning models overcome the “limited visual intuition” they have trying to visibly comprehend what’s represented in text. 

Essentially, a machine learning model is trained to understand and visually interpret meaning from text. The model is trained with digital images and text associated with those images (i.e., photos and their captions). The model is trained in a way that causes “similar visual concepts to be clustered together,” allowing it to identify items that are in similar categories but may be described in different contexts.

Once the model is trained, it can be used for a variety of functions, including text classification, natural language understanding, digital content searches and text summarization. While this sounds technical, think of it like this: If you give Adobe’s trained model an image of the beach scattered in chairs and sand toys, then ask it to remove “beach accessories,” it’ll identify and remove those objects.

Adobe said its system bypasses a common issue that conventional language models face: when digital images are visually similar, but the text describing them is different. The company noted “systems that rely on conventional language models to support visual concepts may encounter inaccuracies and fail in some instances.” 

Photo courtesy of the U.S. Patent and Trademark Office.

Adobe has been working hard at making AI a focus of its suite of audio-visual tools. As the company noted in its filing, this new tech could support a host of AI-enabled photo and video editing. But securing patents on any visual-based AI tools could give the company a competitive advantage, both in the creative tools market and the general battle for AI dominance. 

The tech firm seemingly sees the benefit of scooping up visual AI patents. Most of Adobe’s latest published patent filings are for AI-based tools, including tech that improves machine learning for prediction and document rendering, and a method for efficiently training neural networks to better catch blurry or less-obvious features in photos. 

Outside of patent filings, the company has already released several AI-enabled features. Most recently, the company launched its own AI image generator called Firefly on a few of its apps, which it claims is “designed to be safe for commercial use” as it is only trained on content the company is licensed to use. One of its first Firefly-backed tools is Generative Fill, which can expand an image's borders or add in new objects and is expected to be available to all photoshop users later this year.

While the company’s Creative Cloud suite is widely seen as an industry standard, Adobe still has incentive to keep up with the rising tide of AI and stay on top. Canva, one of the tech firm’s only competitors, recently debuted its own suite of AI-powered tools, including generative design and copywriting assistants. 

Plus, while Adobe doesn’t directly compete with top players like Google and OpenAI, AI is occupying the brainspace of anyone that has a stake in the world of tech. Every tech firm worth its salt seems to be grasping for some way to implement AI, whether they need it or just fear being left behind.

---

Extra Drops

Some other fun patents we wanted to share. 

• Hey Google, what’s that smell? The company is seeking to patent tech that allows automated assistants like phones or smart speakers for “leveraging odor sensor(s)” to detect where a certain smell may be coming from in a user’s vicinity. 

• PayPal wants to gatekeep NFTs. The company is seeking to patent a system for automatically granting or restricting access to NFTs based on factors like location and identity to ensure security. 

• Intel wants to help you get the best view. The company is seeking to patent “low-latency generation of the immersive video” for spectators of e-sports events. 

What else is new? 

• A federal judge is slated to hear arguments today in the FTC’s lawsuit that aims to block Microsoft from acquiring Activision Blizzard, a nearly $69 billion merger that is planned to close next month.

• Stability AI is releasing a new AI model that it says can create better images with more compositional detail than its past models.

• Google has accused Microsoft of anticompetitive practices with its Azure unit, alleging that the company uses unfair licensing terms to “lock in clients” and control the market.

Have any comments, tips or suggestions? Drop us a line! Email at admin@patentdrop.xyz or shoot us a DM on Twitter @patentdrop.