Gotchaa Lab
← Back to Blog
cybersecurityAIwebsite-securitypenetration-testingmalaysia

We Ran AI Penetration Testing on Our Own Site. It Found a Way In

17 November 2026·5 min read·By Gotchaa Lab
We Ran AI Penetration Testing on Our Own Site. It Found a Way In

TL;DR

  • AI penetration testing here means AI agents that try to break into your website the way a human tester would, and report only the attacks that worked.
  • We ran Shannon v1.9.0, an open-source tool by Keygraph, on a copy of our own site on 17 November 2026. Nine agents finished in 10 minutes 38 seconds.
  • It proved one low-severity gap: an internal planning page that anyone with the link could open. No passwords, keys or customer data were exposed, and we took the page off the live site the same day.
  • Our site has no logins or database, so there was little else to break. A shop or booking site with customer accounts gives a tool like this far more to test.
  • This week, open every page meant to be private in a private browser window. If it loads without a login, it is public.

On 17 November 2026 we pointed an AI hacker at our own website. Shannon, an open-source AI penetration testing tool by Keygraph, ran against a copy of gotchaa-lab.com on our laptop, never the live site. Ten minutes later it had found one way in: an internal planning page that anyone with the link could read.

It's a small finding. We still missed it, and we build websites for a living.

What is AI penetration testing?

AI penetration testing uses AI agents, small programs that plan and act on their own, to attack your website the way a human tester would. They map the site, try attacks, and report the ones that worked. A penetration test, or pentest, is an agreed attempt to break in before a stranger does. (The phrase also gets used for testing chatbots, which is a different job.)

How Shannon works, in plain words

Shannon's GitHub page calls it "an AI pentester for web applications and APIs". The open-source version is free to download, but it runs on an AI model you pay for. First it reads your site's source code to spot likely weak points. Then it attacks the running site to see which are real.

Its rule is "No exploit, no report". A finding only makes the report if a working attack proved it, so you skip a scanner's long list of "possible issues".

It tests five kinds of attack: injection (sneaking commands into a form), XSS (planting a script on a page), SSRF (tricking the server into fetching something it shouldn't), broken login, and broken access rules. It has 52,000 GitHub stars as of 17 November 2026.

What it found on our site

We ran Shannon v1.9.0 on Claude Sonnet 5.5. Nine agents took turns: two to map the site, five to hunt for each attack type, one to try the access-rule attack, and one to write the report.

Shannon's run dashboard showing Completed, a duration of 10 minutes 38 seconds, and the nine agents it ran Our Shannon run on 17 November 2026, starting at 6:03 pm. Captured from our own dashboard.

The report found no login, script, injection or server-request problems. It proved one low-severity access gap: an internal planning page served to anyone, no login needed. It held no passwords, keys or customer data, and nothing could be changed through it. It was still never meant to be public, so we took it off the live site the same day.

Why so little? Our site has no logins or database, so most attacks had nothing to grab. That doesn't make our site fully safe: one 10-minute run covered five attack types, nothing more.

How this affects Malaysian businesses

The gap Shannon found needed no hacking skill: anyone with the link could read the page. Company sites collect pages like that, from old promos to forgotten test pages.

Now picture a shop or booking site with logins, orders and personal details, built by a small team or a web development partner. There, a tool like this tries to log in as someone else, read another customer's order, or slip commands into a search box. If personal data turns up, Malaysia's PDPA may also come into play.

Our take: a tool like Shannon is a cheap first pass, not a full pentest. It shows proof instead of guesses. Any team building a site with customer data should run one on a staging copy before launch.

Where AI penetration testing tools stop

AI tool like ShannonHuman penetration test
What it needsSource code and a running copyA written scope and permission
What it coversFive common attack typesWhatever is in scope, including business logic
SpeedMinutes to about 1.5 hoursSet by the scope and the tester's time
ReportProven findings onlyFindings plus a tester's sign-off

It has other limits too:

  • Only on systems you own. Shannon's README says to run it only with ownership or written permission, and "Do not run Shannon against production systems." It creates users and changes data.
  • It needs setup. Docker, the source code and a paid AI model account. That is your developer's job.
  • The AI provider can stop it. Shannon's docs note that Anthropic and OpenAI apply real-time safeguards to cyber-security work. We explain how verification for that works in our Claude Mythos access post.
  • Sites built fast with AI help need it more, for the reasons in vibe coding security risks.

When a customer or regulator wants a pentest report, you need a real one. Our penetration testing cost guide explains how to tell a real test from a scan.

What to do this week

  1. List every page on your site that is meant to be private.
  2. Open each one in a private browser window. If it loads without a login, it is public.
  3. Ask your developer when your site was last tested.

Curious how AI fits into building and checking your software? Let's chat. We'll give you an honest take, no sales pitch.

This article is for general information and does not constitute professional cybersecurity or legal advice.

References

  1. Shannon by Keygraph, GitHub README
  2. Shannon AI providers documentation

Share this article

Frequently Asked Questions

What is AI penetration testing?
The phrase has two meanings. One is testing AI systems, like chatbots, for weaknesses. The other, which this post covers, is using AI agents to attack a website or app the way a human penetration tester would: map it, try attacks, and keep only the ones that actually work.
Is Shannon free to use?
The open-source version on GitHub is free under the AGPL-3.0 licence, but it runs on an AI model you pay for. Its README says a full run may incur AI API costs. Version 1.9.0, which we used, is the last release that accepts a Claude Code subscription login.
Can I run Shannon on any website?
No. Shannon's README says to run it only on applications you own or have explicit written permission to test, and never on production systems. It creates users, submits forms and changes data, so point it at a copy of your site.
Will AI replace penetration testers?
Not on what we saw. A tool like Shannon is a fast first pass on a handful of common attack types. A human tester still judges business logic, chains small issues together, and signs a report a client or regulator can rely on.
How long does an AI penetration test take?
Shannon's README says a full run can take roughly 1 to 1.5 hours. Our run on a small marketing site with no logins took 10 minutes 38 seconds. A larger app with accounts and forms takes longer.

Need help building this for your business?

We help Malaysian companies turn ideas like these into working software. Free consultation, no obligation.