We often hear about the benefits artificial intelligence (AI) can bring to medicine and healthcare through drug research, but could it also pose a threat?
Researchers from Collaborations Pharmaceuticals, a North Carolina-based drug discovery company, have published a paper that highlights the dangerous potential of AI and machine learning to discover biochemical weapons.
By simply tweaking a machine learning model called MegaSyn to reward instead of penalise predicted toxicity, their AI was able to generate 40,000 biochemical weapons in six hours.
Obvious in hindsight?
Worryingly, the researchers admitted to never having considered the risks of misuse involved in designing molecules.
“The thought had never previously struck us. We were vaguely aware of security concerns around work with pathogens or toxic chemicals, but that did not relate to us; we primarily operate in a virtual setting.
Our work is rooted in building machine learning models for therapeutic and toxic targets to better assist in the design of new molecules for drug discovery. We have spent decades using computers and AI to improve human health—not to degrade it,” the paper noted.
Even the company’s work on Ebola and neurotoxins did not alert them to the damage that could be caused by flipping their models to seek out rather than avoid toxicity.
From generation to synthesis
The barriers to misusing machine learning models like MegaSyn to design harmful molecules are lower than you might expect.
Plenty of open-source software has similar capabilities and the datasets that trained it are available to the public. What’s more, the 40,000 toxins were generated on a 2015 Apple Mac laptop.
Of these, hundreds were found that are more lethal than the nerve agent VX.
One of the most potent chemical warfare agents of the twentieth century, VX uses the same mechanism to paralyse the nervous system as the Novichok nerve agent used in the 2018 Salisbury poisonings.
Fortunately, actually synthesising these potential new bioweapons is far more of a challenge than generating them on a computer. The specific molecules that are needed to create VX, for example, are strictly regulated.
Dangers would only arise if a toxin was found that did not require any regulated substances. Whilst easy to figure out through another set of parameters, the researchers felt uncomfortable taking this extra step.
Before publication, Collaborations Pharmaceuticals presented their findings at the Spiez Laboratory, one of five labs in the world that is permanently certified by the Organisation for the Prohibition of Chemical Weapons (OPCW).
The researchers’ findings make an important case for the need to oversee AI models and fully consider the ramifications of utilising complex AI.
Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo. The next events in the series will be held in Santa Clara on 11-12 May 2022, Amsterdam on 20-21 September 2022, and London on 1-2 December 2022.
Explore other upcoming enterprise technology events and webinars powered by TechForge here.
