U.S. Army researchers have developed a new approach for training machine learning models that can better withstand dirty and deceptive data. Models trained under this method have greatly surpassed other state-of-the-art models in terms of robustness, scientists said.
Machines outperform humans in many data-processing tasks, but sometimes fall victim to obvious mistakes that humans can see a mile away.
Scientists at the U.S. Army Combat Capabilities Development Command’s Army Research Laboratory designed a new approach that makes it harder for adversaries to trick machine learning models.
“We were able to reduce model complexity by about a factor of 10 without affecting other performance metrics under benign conditions,” said Army scientist Dr. Ananthram Swami. “Under different adversarial attacks, our model performed two to eight times better.”
This was an extraordinary discovery, he said. Previously, numerous attempts to account for dirty and deceptive data have either failed or yet to face a thorough examination.
Within the domain of machine learning, one of the biggest hurdles is how machine learning algorithms can accurately interpret clean data, but avoid becoming susceptible to non-ideal inputs that are deceptive or have a low signal-to-noise ratio.
For the U.S. Army, such flaws in the algorithms could leave entire systems vulnerable to adversaries who can manipulate the data to produce wrong outputs.
According to Swami and fellow ARL scientist Gunjan Verma, their team’s approach tackles the fundamental idea behind the classification problem and how modern machine learning models numerically represent the labels assigned to different categories.
“The standard approach uses a vector with one digit being 1 and the rest being 0,” Swami said. “This is called one-hot encoding, and it serves the purpose of making different labels distinct from one another.
For instance, if a neural network took in a series of handwritten digits from 0 to 9 as inputs, a standard model would be trained to produce 10 output units with each unit corresponding to a specific digit. Then, once a single digit passes through the system as an input, the correct corresponding output would be assigned with the marker 1 while the rest would be assigned with a 0.
With this method, standard machine learning architectures categorize the outputs by labeling each digit with a distinct code word.
In contrast, the approach developed by Army scientists uses different denotations instead of just ones and zeroes.
“We make a small but important change; we assign the label for each digit, still a vector, to a specific pattern of 1’s and -1’s corresponding to that digit,” Swami said. “Compared to the standard approach, ours enables us to increase the separation or distance between the patterns representing the labels, which in turn increases resiliency and makes it harder for an adversary to cause misclassification.”
As a result of its design, models trained by this approach tends to be more conservative when generating probability estimates compared to standard models that remain confident even when the input has been perturbed.
In order to gauge whether the new approach led to more robust models than the standard, the researchers conducted a comprehensive series of experiments on benchmark dataset, such as the Modified National Institute of Standards and Technology database and the Canadian Institute for Advanced Research dataset.
The new approach was also tested against random noise images and on state-of-the-art adversarial manipulations of the aforementioned datasets. Metrics of performance included network complexity, accuracy under benign conditions and accuracy under attack.
At the conclusion of the evaluation, Verma and Swami found their approach improved model robustness significantly and often surpassed current state-of-the-art models by a large margin.
This new approach can operate using smaller architectures than conventional methods.
“Other methods that proved successful require very large architectures, which mandate larger training datasets, longer training times,” Verma said. “However, our method can work with significantly smaller architectures that can better adapt to the size, weight, and power constraints of military applications.”
In the future, the Army will rely on artificial intelligence and machine learning to ensure mission success. Next-generation combat vehicles will incorporate machine learning classifiers to interpret imagery obtained from its various sensors, and intelligent tactical networks will depend on different sensors to monitor the state of the system and autonomously perform behaviors.
Defense mechanisms to protect Army systems against deceptive data would serve a vital role in building resilience in machine learning algorithms situated in an adversarial environment, researchers said.
“We hope that with more robust machine learning models, we can increase resilience to adversarial manipulation, increase confidence in the results of the machine learning model, and increase the warfighter’s trust in autonomy,” Swami said.
Verma and Swami recently presented their new approach at the 33rd Annual Conference on Neural Information Processing Systems in Vancouver, British Columbia