Key Points
- A researcher found AWS Security Agent could be manipulated into testing websites outside its authorized scope by confusing its ownership verification process.
- The testing also revealed excessive privileges, command execution opportunities, exposure of sensitive data in findings, and more aggressive testing behavior than intended.
- IANS Faculty say organizations should treat AI pentesting agents as high-privilege non-human identities and enforce scope, authorization, and safety controls outside the model itself.
Researcher Manipulates AWS AI Pentester Beyond Authorized Targets
A researcher found that AWS Security Agent could be manipulated into conducting penetration tests against websites the user did not own/control, introducing new risks when AI-powered security tools are granted broad autonomy.
AWS Security Agent automates application security across design reviews, code reviews and on-demand penetration testing, requiring users to verify website ownership before launching tests.
Researcher Richard Fan found that he could confuse that verification process by manipulating DNS records, causing the agent to target websites outside of his control. The researcher was also able to execute commands within its environment and gain deeper access than intended.
During testing, he noticed the agent would occasionally choose more aggressive techniques than necessary when safer methods could have determined whether the vulnerability existed.
Fan also created a vulnerable application that exposed user passwords and found that the agent included the passwords in the findings rather than redacting them.
Fan concluded that website owners have limited ability to defend themselves from unauthorized AI-driven testing and remain heavily dependent on vendor safeguards to prevent misuse.
Big Picture
Fan’s findings show that familiar identity, privilege and containment failures become more consequential when wrapped in an autonomous agent that can select targets and take offensive action at machine speed.
If an AI pentesting agent can be tricked into expanding its scope, using excessive privileges or exposing secrets, the organization cannot assume the vendor will take full responsibility when something goes wrong, making the security of the tool itself a critical concern.
"We are very often turning these AI tools at trying to solve security problems, when people aren’t asking the obvious question: Is the AI tool itself secure?” Jake Williams, IANS Faculty.
Fan’s findings aren’t a reason to avoid AI pen testing, but a warning to govern agents like powerful non-human identities.
"Your pentest agent is the most overprivileged insider you have, and you should scope it accordingly. Treat it as a non-human identity and place the controls outside the model, where it cannot talk its way past them." George Gerchow, IANS Faculty.
More broadly, organizations shouldn’t rely on vendor assurances because unauthorized testing may still appear to be originating from their own cloud accounts and infrastructure.
"As a CISO, the question about the AWS Security Agent -- and really, any AI penetration testing agent -- is one of control. Before I delegate an offensive action to an agent my team can't fully supervise, how do I keep the blast radius bounded, and the targets authorized and observable?" Wolfgang Goerlich, IANS Faculty.
IANS Faculty Recommendations
- Continuously verify scope: Reconfirm ownership before every target and action and stop the agent if DNS changes, redirects or untrusted content push it outside the authorized environment.
- Treat the agent as a high-privilege identity: Use just-in-time credentials, block metadata access, remove blanket sudo and Docker socket access, restrict egress and maintain an emergency kill mechanism.
- Keep enforcement outside the model: Use environment-specific policy controls to block out-of-scope or destructive actions and require human approval for high-impact steps.
- Assume prompt injection will occur: Ask the vendor how the agent detects and mitigates malicious instructions embedded in files, databases and other post-exploitation data.
- Define accountability before deployment: Document customer and vendor responsibilities, review potential legal exposure and accept that harmful activity may be attributed to the organization’s own cloud environment.
Authors & Contributors
Nuria Diaz Munoz, Author - Security Reporter, IANS News
Wolfgang Goerlich, IANS Faculty
Although reasonable efforts will be made to ensure the completeness and accuracy of the information contained in our News & blog posts, no liability can be accepted by IANS or our Faculty members for the results of any actions taken by individuals or firms in connection with such information, opinions, or advice.