Ohio State researchers developing AI to make the internet more accessible

An artificial intelligence software currently in development called Mind2Web is looking to make the internet more accessible for people with disabilities by making complex online tasks doable with a simple text request.

This AI, created by Ohio State researchers, can help people complete over 2,000 online tasks, ranging from finding documents and booking hospital appointments to browsing comedy films on Netflix and following celebrities on social media. Boyuan Zheng, a Ph.D. student in computer science and co-author of the study on the tool’s development, said the software could also help people with visual impairments accomplish tasks in the digital world.

“It is kind of like a guide dog in their cyberspace,” Zheng said.

In addition to helping people with disabilities, Zheng said the AI can help those who are not very tech-savvy.

“With [Generation Z], we are kind of familiar and take for granted this kind of web search, this kind of software, but it’s not always that easy for everyone,” Zheng said.

Eventually, this technology could alter how people who regularly use the internet interact with it, Huan Sun, an associate professor of computer science and co-author of the study, said.

“Ideally, you can leverage this tool to automatically do a task for you,” Sun said. “In addition to that, we can sort of enhance the productivity for everybody, so you can save a lot of time from not doing these kinds of tedious tasks.”

The researchers started developing the AI by creating Mind2Web, the dataset that allows the software to perform tasks and learn new ones. Unlike other AI agents that were developed and trained by using synthetic websites, Mind2Web was created by downloading 137 real sites from the Internet, Zheng said.

MindAct, the AI agent itself, uses long language models — algorithms that are trained to detect and comprehend human language — to predict what actions should be done on a webpage as a person gives it a task, according to Sun.

“The large language model is the backbone model,” Sun said. “It can make the right predictions and take the actions until you arrive at the final step, which is to present to the user a list of options for you to choose.”

Although the AI is currently sitting at a success rate of 50%, Zheng is thrilled with the progress already made and what the future has in store for this project.

“I’m very excited,” Zheng said. “It’s like every day I wake up, I am very excited. I get a lot of exciting work to do, and we are so close to the real implementation [of our AI].”

Mind2Web is accessible for download now using this link.