Save time with unbiased, independent feedback on vendor solutions.
Watch weekly bite-sized webinars hosted by IANS Faculty.
AI has both weaknesses and unintended consequences, and it is impossible for security teams to guard against all of them. This piece explains measures to help businesses minimize the risk of AI use, abuse and accidents that could potentially impact reputation
Humans are already getting ChatGPT and its ilk to produce malware, phishing emails (see Figure 1) and methods to cause various types of trouble.
Another issue is oracles. The term comes from the smart contract world, where control over a data source (an oracle) can net a malicious actor an unethical or illegal edge over the competition. An example of this would be a bet over the temperature in
an area for a specific day. If one party has the weather station for the area in their backyard, it’s relatively trivial to hold a flame near it to raise the temperature artificially.
Breaking an AI model is another way to destroy the veracity of an AI system. If you can change the weighting of a choice or the patterns an AI is looking for, then you can change all the answers it provides, changing how it buys stocks or measures the
effectiveness of a medication. Such changes can include presenting biased training data, manually overriding the weighting of choices or tampering with the model in some other way.
One of the largest risks is understanding “why” the AI system selected a specific choice. AIs that are simply black boxes are useless to furthering actual solutions and may instead prolong problems. Additionally, they are wholly incapable
of being understood or human-verified.
Luckily, NIST has recently released an AI Risk Management Framework. It’s well documented
and includes many of the answers needed to be more confident in deploying and using an AI system.
The framework’s core functions include risk governance, context, identifying specific risks and understanding the impact of likely risks (see Figure 2).
The framework, along with the companion documents and videos, provides a solid foundation from which to build and deploy an AI risk management program.
Mitigating the human “simulation theory” risk, wherein a human writes a way for an AI to answer a question it normally wouldn’t, depends on good filtering of terms, phrases and having the AI check its own work. The AI system can be trained
to see these kinds of requests as a poisoning of the system, i.e., a pattern. As such, it can be trained to inspect its own work and reject anything that shouldn’t be transmitted to the requestor.
Monitoring for and removing poisoned or biased data sources, poor training data or bad methodologies is exactly the same process used when a database is constructed. Data normalization, data cleaning, data testing, etc., is a very mature art and science.
Performing commonsense testing of data sets to determine if the right answers are being given helps discover if anything has made it past the data cleansing systems.
Effectively, the same issues and prioritizations found within data-intensive systems today should be used with AI systems tomorrow.
The only real addition would be a comprehensive error-checking and error-notification process. If a user can click a single button to point out the answer is wrong, it will lead to understanding where the error came from much quicker than if it takes
weeks to figure out there was an error in the first place.
The hardest thing to find is a bias in a data set. It may seem fine at first glance, and even when closely examined. But there are always further layers of statistical theory, and fallacies, to ponder on.
AI systems are being planned for just about any use case you can imagine. They are also being used and abused. To mitigate risks that are fundamental to the technology:
Although reasonable efforts will be made to ensure the completeness and accuracy of the information contained in our blog posts, no liability can be accepted by IANS or our Faculty members for the results of any actions taken by individuals or firms in
connection with such information, opinions, or advice.
September 26, 2023
By IANS Faculty
Access key data sets from the 2023 edition of IANS and Artico Search’s Security Budget Benchmark Report. Gain valuable insights on security budget increases and the drivers behind them.
September 21, 2023
Learn why CISOs Need D&O Liability Insurance Coverage now more than ever along with guidance to help minimize potential cyber liability risk.
September 19, 2023
Discover the diversity of IANS Faculty's real-world expertise. Learn how our faculty members can help you solve your most challenging security issues.