Machine ‘unlearning’ helps filter out copyrighted, violent content

Even in the world of machine learning, forgetting things is just as challenging as it is for us humans. Especially for artificial intelligence programs trying to act like humans, they face a tough time dealing with things like copyrighted stuff and sensitive content.

Researchers at The University of Texas at Austin have developed a solution called “machine unlearning” to address the burning issue related to the rapidly growing use (and misuse) of AI. This method is designed specifically for image-based generative AI systems. It helps these systems get rid of copyrighted or violent images without forgetting everything else they’ve learned. They’ve explained their findings in a paper on the arXiv preprint server.

Machine unlearning in practice

Machine unlearning is a new way to deliberately forget certain data from a model to meet strict rules. But so far, it has mostly been used for certain types of models, leaving out others like generative ones. Companies working on self-driving cars use machine unlearning to get rid of old or irrelevant data. This helps them keep improving their driving algorithms and adjust to new situations.

Hospitals and health tech companies use the system to keep patient information safe. For example, if a patient wants to take their data out of studies or databases, machine unlearning makes sure it’s completely removed, following privacy rules.

Likewise, streaming services and online platforms use machine unlearning to update user profiles. This means they can change recommendations based on what users like right now, instead of using old information.

And in setups where machine learning happens across different devices, like the ones used by mobile device makers and app developers, machine unlearning is used to delete data from devices. This way, when a user removes data from their device, it’s also taken out of the overall learning model without having to start over from scratch.

Their experiments now show that their new method works well even without the data it is supposed to remember, which fits with rules about keeping data. According to the researchers, this is the first time anyone has really looked deeply into how to unlearn things from generative models that work with images, covering both theory and real-world tests.

How machine unlearning works

AI models are trained on large sets of data, some unwanted things gets in there too. In the past, the only choice was to start over and carefully take out the problematic stuff. But the researchers claim that their new method has offered a better solution.

“When these models are trained on vast datasets, it’s inevitable that some undesirable data creeps in. Previously, the only recourse was to start over, painstakingly removing problematic content. Our approach offers a more nuanced solution,” Professor Radu Marculescu from the Cockrell School of Engineering’s Chandra Family Department of Electrical and Computer Engineering, a key figure in this endeavor, said.

Generative AI relies a lot on internet data, which is huge but also full of copyrighted stuff, private info, and inappropriate content. This was evident in a recent legal fight between The New York Times and OpenAI over using articles without permission to train AI chatbots.

According to Guihong Li, a graduate research assistant on the project, adding protections against copyright issues and misuse of content in generative AI models is crucial for their commercial success.

The research has mainly looked at image-to-image models, which change input images based on context. The new machine unlearning algorithm allows these models to get rid of flagged content without needing to start all over again, with human oversight providing an extra layer of supervision.

While machine unlearning has mostly been used in classification models, using it in generative models, especially for image processing, is a new area, as pointed out by the researchers in their paper.