On 17 November 2026 we pointed an AI hacker at our own website. Shannon, an open-source AI penetration testing tool by Keygraph, ran against a copy of gotchaa-lab.com on our laptop, never the live site. Ten minutes later it had found one way in: an internal planning page that anyone with the link could read.
It's a small finding. We still missed it, and we build websites for a living.
What is AI penetration testing?
AI penetration testing uses AI agents, small programs that plan and act on their own, to attack your website the way a human tester would. They map the site, try attacks, and report the ones that worked. A penetration test, or pentest, is an agreed attempt to break in before a stranger does. (The phrase also gets used for testing chatbots, which is a different job.)
How Shannon works, in plain words
Shannon's GitHub page calls it "an AI pentester for web applications and APIs". The open-source version is free to download, but it runs on an AI model you pay for. First it reads your site's source code to spot likely weak points. Then it attacks the running site to see which are real.
Its rule is "No exploit, no report". A finding only makes the report if a working attack proved it, so you skip a scanner's long list of "possible issues".
It tests five kinds of attack: injection (sneaking commands into a form), XSS (planting a script on a page), SSRF (tricking the server into fetching something it shouldn't), broken login, and broken access rules. It has 52,000 GitHub stars as of 17 November 2026.
What it found on our site
We ran Shannon v1.9.0 on Claude Sonnet 5.5. Nine agents took turns: two to map the site, five to hunt for each attack type, one to try the access-rule attack, and one to write the report.
Our Shannon run on 17 November 2026, starting at 6:03 pm. Captured from our own dashboard.
The report found no login, script, injection or server-request problems. It proved one low-severity access gap: an internal planning page served to anyone, no login needed. It held no passwords, keys or customer data, and nothing could be changed through it. It was still never meant to be public, so we took it off the live site the same day.
Why so little? Our site has no logins or database, so most attacks had nothing to grab. That doesn't make our site fully safe: one 10-minute run covered five attack types, nothing more.
How this affects Malaysian businesses
The gap Shannon found needed no hacking skill: anyone with the link could read the page. Company sites collect pages like that, from old promos to forgotten test pages.
Now picture a shop or booking site with logins, orders and personal details, built by a small team or a web development partner. There, a tool like this tries to log in as someone else, read another customer's order, or slip commands into a search box. If personal data turns up, Malaysia's PDPA may also come into play.
Our take: a tool like Shannon is a cheap first pass, not a full pentest. It shows proof instead of guesses. Any team building a site with customer data should run one on a staging copy before launch.
Where AI penetration testing tools stop
| AI tool like Shannon | Human penetration test | |
|---|---|---|
| What it needs | Source code and a running copy | A written scope and permission |
| What it covers | Five common attack types | Whatever is in scope, including business logic |
| Speed | Minutes to about 1.5 hours | Set by the scope and the tester's time |
| Report | Proven findings only | Findings plus a tester's sign-off |
It has other limits too:
- Only on systems you own. Shannon's README says to run it only with ownership or written permission, and "Do not run Shannon against production systems." It creates users and changes data.
- It needs setup. Docker, the source code and a paid AI model account. That is your developer's job.
- The AI provider can stop it. Shannon's docs note that Anthropic and OpenAI apply real-time safeguards to cyber-security work. We explain how verification for that works in our Claude Mythos access post.
- Sites built fast with AI help need it more, for the reasons in vibe coding security risks.
When a customer or regulator wants a pentest report, you need a real one. Our penetration testing cost guide explains how to tell a real test from a scan.
What to do this week
- List every page on your site that is meant to be private.
- Open each one in a private browser window. If it loads without a login, it is public.
- Ask your developer when your site was last tested.
Curious how AI fits into building and checking your software? Let's chat. We'll give you an honest take, no sales pitch.
This article is for general information and does not constitute professional cybersecurity or legal advice.




