Here’s an example of how artificial intelligence can still seriously lack behind some human attributes: tests have shown how natural language processing (NLP) systems can be tricked into misunderstanding text by merely swapping one word for a synonym.
A research team at MIT developed software, called TextFooler, which looked for words which were most crucial to an NLP classifier and replaced them. The team offered an example:
“The characters, cast in impossibly contrived situations,
are totally estranged from reality”, and
“The characters, cast in impossibly engineered circumstances, are fully
estranged from reality”
No problem for a human to decipher. Yet the results on the AIs were startling. For instance BERT, Google’s neural net, was worse by a factor of up to seven at identifying whether reviews on Yelp were positive or negative.
Douglas Heaven, writing a roundup of the study for MIT Technology Review, explained why the research was important. “We have seen many examples of adversarial attacks, most often with image recognition systems, where tiny alterations to the input can flummox an AI and make it misclassify what it sees,” Heaven wrote. “TextFooler shows that this style of attack also breaks NLP, the AI behind virtual assistants – such as Siri, Alexa and Google Home – as well as other language classifiers like spam filters and hate-speech detectors.”
This publication has explored various methods where AI technologies are outstripping human efforts, such as detecting breast cancer, playing StarCraft, and public debating. In other fields, resistance – however futile – remains. In December it was reported that human drivers were still overall beating AIs at drone racing, although the chief technology officer of the Drone Race League predicted that 2023 would be the year where AI took over.
The end goal for software such as TextFooler, the researchers hope, is to make NLP systems more robust.
Postscript: For those reading from outside the British Isles, China, and certain Commonwealth countries – to ‘snooker’ someone, deriving from the sport of the same name, is to ‘leave one in a difficult position.’ The US equivalent is ‘behind the eight-ball’, although that would have of course thrown the headline out.
Interested in hearing industry leaders discuss subjects like this? Attend the co-located 5G Expo, IoT Tech Expo, Blockchain Expo, AI & Big Data Expo, and Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London, and Amsterdam.
