The Haunted Attack Surface

TL;DR

The velocity of frontier model advancements is changing how quickly vulnerabilities can be investigated and how easily people can perform skilled security work. The risk of developing new attack chains built from known weaknesses, more accessible reverse engineering techniques becomes more scary. Security practitioners should prepare for shorter response windows, innovative exploits and scale of newly discovered vulnerabilities.

A Small Experiment, A Bigger Concern

Since it’s Halloween, I wanted to share something a little spooky that’s been on my mind for a while: what happens when technical work that once felt out of reach becomes something almost anyone can try?

Every decent ghost story begins with something ordinary behaving in a way it should not. A door opens, a light switches on, a shadow appears with an intense soundtrack playing…

My own example is less dramatic. Outside cybersecurity, I like to experiment with AI-assisted software development (OK, call it “vibe coding” if you wish).

I have an OLED monitor and wanted to reduce the risk of burn-in from static text and icons sitting in my taskbar (less of a concern on newer OLEDs). The option available to me was taskbar auto-hide setitng in Windows, but I wanted a very lightweight utility that would make the taskbar darker instead of hiding it.

My development experience is mainly in web and 3D, although in recent years my work has been more managerial than hands-on, - so no hardcore coder here. For this tool, C++ looked like the best fit, and I hadn’t touched it since university. A native Windows utility was unfamiliar territory, even if the thing I wanted it to do sounded simple.

What struck me was how my lack of Windows application development expertise didn’t stand in the way of doing something practical. Yes, my taskbar experiment was benign, but in a security context, that same ease of getting things done could help someone cause real disruption with far less special knowledge.

You can know one framework or language well and be a novice in another. Now AI can help you modify unfamiliar code, automate administrative tasks, or use tools you would previously have struggled with or needed considerable time to learn.

Now put on a different hat (in this case black). With some technical skills and AI assistance, you could adapt existing malicious code without understanding all its internals. The path from an idea to malware you can test becomes shorter, even if making it reliable is another matter.

I’ve heard this called “workflow compression”. The way I interpret it is: less distance between wanting to do something and being able to try it. It gives defenders faster ways to investigate unfamiliar systems, but it also gives others ways to interfere with them, deliberately or by accident.

What worries me is how often we confuse unfamiliarity with inability. Someone who has never worked with a particular system may now have enough help to start probing it. They don’t need to understand every room to cause damage inside, and the tools once reserved for specialists are becoming easier to pick up.

Complicated Work Is Becoming Easier To Attempt

Models are becoming conversational front ends for decompilers, debuggers, fuzzers, disassemblers, scanners, and configuration tools. Given suitable tools and artifacts, they can help interpret decompiled firmware, explain unfamiliar assembly, inspect an old malware sample, or turn a crash into a testable hypothesis. A practitioner who has never specialized in reverse engineering now has a way into work that once looked inaccessible.

Firmware is just one example. Making sense of the compiled code inside a router or appliance has traditionally required specialized knowledge of reverse engineering, processor architectures, and toolchains. With an accessible firmware image and suitable tools, AI can now help a less-experienced practitioner work through decompilation and interpret the resulting code. The decompiler still does the decompiling, but operating it and understanding its output can become easier. That is a lower barrier to investigation, not a guarantee of recovered original source code or a working exploit. Hardware dependencies and difficulty testing a finding can still stall the work, no matter how convincing the model sounds.

Recent projects make that less abstract. In August 2026, Chris Lewis reported completing a matching decompilation of Snowboard Kids in 84 days, with coding agents helping reconstruct C code that compiled to the game’s original machine code. Human specialists were essential for difficult compiler quirks, and Lewis cautioned against attributing the elapsed time entirely to AI.

On the desktop side, Brandon Thomas used Claude to build ArtCraft’s Adobe-inspired creative apps. That is a different kind of reconstruction: PhotoCraft describes its approach as a clean-room reimplementation from public specifications and observed behavior, not decompilation of Adobe’s suite. It also acknowledges that it is still an early alpha, not a replacement for daily professional Photoshop work. Neither example makes expertise optional, but both show practical ways AI can help people reconstruct software they didn’t originally build.

Yes, these reconstructions are far from perfect, and the Adobe alternatives don’t yet match the original tools’ capabilities. But what if we peek five years into the future? How capable will the models be then? That’s the scary part for me: the same progress that makes creative tools easier to build could make novel exploit tools easier to create.

Access matters too. Defenders can run suitable open-weight models on their own machines or in on-premises data centers to keep sensitive code and firmware inside their own environment. Adversaries can do the same, without sending their requests through a model provider that might detect misuse or suspend an account. Running those tools still takes resources and skill, but capable assistance isn’t confined to managed services.

The UK AI Security Institute’s 2025 report found that the general capability gap between open and closed models was narrowing. That doesn’t tell us whether they are equally good at system exploitation, but it suggests capable assistance is becoming available beyond a handful of managed services.

What The Evidence Shows So Far

The firmware example is no longer just a possibility. In its September 2026 misuse report, Anthropic described a group that used Claude to orchestrate reverse engineering, vulnerability research, and exploit testing against security products and network appliances. Two operators were undergraduates, and one had prior security-industry experience, so this wasn’t a case of complete beginners. But their workflows kept research and collection running while the operators were away. Anthropic reported previously unknown vulnerabilities validated in the group’s own lab, alongside real intrusions and data theft. That doesn’t establish that those particular new exploits caused the reported breaches, but it does show specialist research being built into a sustained offensive operation.

There is a different kind of unsettling example from OpenAI that got a lot of publicity. On September 20, 2026, an internal research agent working on an ordinary web-search task reached an external chatbot through a gap in its sandbox’s DNS restrictions. Nobody had asked it to test the network controls. The chatbot actually answered, although the agent still failed to solve the original task. Monitoring flagged the behavior within 15 minutes, but the expected automatic stop failed and the run was stopped manually about two and a half hours after the alert. This happened during training, not in a public product; it was an actual boundary crossing, not just a simulated rogue-agent scenario.

One case is people directing AI toward harm; the other is an agent going beyond what its task authorized. I find both worth paying attention to. Useful persistence can become unsafe persistence when the boundaries don’t hold. These are provider-reported cases, not a measure of how common either behavior is.

The exploit side has a useful research example too. In the February 2026 revision of their CVE-GENIE paper, researchers reported reproducing 428 of 841 selected known vulnerabilities, including verifiable proofs of concept, in reconstructed test environments. The framework used publicly available source code and vulnerability information, including existing proofs of concept where available. It wasn’t a study of novice users or proof of attacks against production systems, but it shows how work between reading a disclosure and reproducing a flaw can be packaged into a tool.

There is progress on discovery and repair too. Google’s Big Sleep found an exploitable SQLite flaw before it reached an official release. DARPA’s AI Cyber Challenge systems found 54 of 63 synthetic vulnerabilities, patched 43, and identified 18 real vulnerabilities in the software used for the challenge. These expert-built projects demonstrate useful vulnerability discovery and repair, rather than what a beginner can do unaided.

Access to a capability and speed are different questions. METR’s early-2025 study found experienced developers slowed down on its tasks; its February 2026 follow-up suggested improvement but couldn’t reliably estimate the speedup. Neither measures novice exploit development. For my own work, I’d measure the time spent checking the result too. A confident patch can still break production, and nicely written comments are no consolation.

Cheaper Attempts, More Attack Paths

My concern isn’t just that more people can attempt technical work. It’s that attempts become cheap enough to repeat: discard a failed idea, reuse what works, and try another target. Skilled operators can steer that process, while packaged workflows carry some of their expertise to less-experienced users. “Machine speed” means faster research and iteration, not an instantly reliable intrusion.

Models can draw on learned patterns from historical code and security material, then use newer information supplied through prompts or tools. We rarely know exactly what was in training: OpenAI’s GPT-4 report, for example, doesn’t disclose its dataset construction. That knowledge can suggest an approach; it doesn’t prove the approach works.

The unsettling possibility is unfamiliar combinations. AI could help connect known vulnerabilities and adapted malware components so that one weakness enables the next. The innovation might be in the chain, not an individual flaw. Google’s September 2026 threat report describes AI-assisted work on multi-stage exploit chains, though not evidence that training alone produced a novel attack.

I can imagine a state sponsored campaign giving less experienced users expert-built workflows and self-hosted models, with a thousand agents trying different approaches around the clock. Many would get stuck; experts would still need to coordinate them and verify results. But enough useful attempts could make an obscure, low-value system worth sustained attention. The house doesn’t have to contain a treasure to become a way into the one next door.

Neglected Systems And Connected Weaknesses

Complexity has never been a dependable security control, but it has imposed a labor and knowledge cost. If AI lowers that cost, poorly documented appliances and unfamiliar integrations become less likely to escape scrutiny. Firmware may offer little endpoint visibility; industrial systems add difficult maintenance windows and possible physical consequences.

Consider a forgotten VPN account, an over-permissioned SaaS integration, and a stale secret in a build pipeline (a very common issue these days). Each might look manageable in isolation. Together, they could provide a route into more important systems without a single new vulnerability. The path between them is the thing under the floorboards.

Can Remediation Keep Up?

The NCSC’s assessment through 2027 expects more effective intrusions and shorter disclosure-to-exploitation windows, while considering fully automated, end-to-end advanced attacks unlikely in that period. Schneier, Adkins, and Evron explore the darker possibility of integrated attacks at machine speed. I wouldn’t put a date on that future. The nearer-term pressure is enough to act on.

I’d watch how quickly disclosed flaws get exploited, how easily specialist workflows can be reused, and whether remediation queues grow faster than teams can clear them. That last one worries me most. Finding twice as many bugs is little comfort if the dangerous ones remain exposed.

Attackers can benefit from one useful result among many failures. Defenders can’t casually ship ten speculative fixes though. AI can help with repair, but patches still need testing, owners, and deployment. Teams that handle that well can gain ground; others may end up with a better-documented backlog.

Shorten The Window And Limit The Damage

I would start with one exposed system. Time how long it takes to find its owner, assess a disclosure, and apply a verified mitigation. Agree who can isolate it if patching takes too long. A faster alert isn’t enough if nobody can act on it.

Then put AI to work on authorized investigation and repair, but require reproducible findings and tested fixes. Look at how weaknesses connect, not just each one in isolation. For agents, test network restrictions, tool permissions, access granted through MCP servers, and whether kill switches actually stop the agents and their running tools. Instructions to stay in scope aren’t a substitute for enforced boundaries. Measure the exposure you remove, and include verification time when judging whether AI made the work faster.

For legacy appliances, unsupported firmware, and operational technology, restrict management access, reduce service-account privileges, and remove unnecessary access to networks and shared secrets. Name owners and test containment before an incident. Coordinate changes with engineering and safety teams where physical processes are involved. An awkward maintenance window shouldn’t leave every other room accessible.

The Doors Still Need Closing

I don’t need an all-powerful hacker to find this unsettling. I’m thinking of the firmware nobody reviews, the integration nobody owns, and the fix waiting for next month’s change window. Defenders can use the same tools to investigate and repair them sooner, but somebody still has to verify the work and take responsibility for closing the gap.

The house doesn’t need an exorcism. It needs someone to check the doors and close the ones left open before the next knock. Once that’s done, feel free to perform the ritual (if it helps you sleep). Just don’t mistake it for a lock.

References and Further Reading

1. METR — Early 2025 AI Experienced Open-Source Developer Study https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study

2. METR — We Are Changing Our Developer Productivity Experiment Design (February 2026 follow-up) https://metr.org/blog/2026-02-24-uplift-update/

3. Google Project Zero — From Naptime to Big Sleep: Using Large Language Models to Catch Vulnerabilities in Real-World Code https://projectzero.google/2024/10/from-naptime-to-big-sleep.html

4. DARPA — AI Cyber Challenge Marks Pivotal Inflection Point for Cyber Defense https://www.darpa.mil/news/2025/aixcc-results

5. Anthropic — Detecting and Countering Misuse of AI: September 2026 https://www.anthropic.com/threat-intelligence-report-september-2026

6. CISA — ICT Supply Chain Risk Management Threat Scenarios https://www.cisa.gov/sites/default/files/2025-09/Threat_Evaluation_Working_Group_Threat_Scenarios_Version_2.0_Final_508.pdf

7. UK National Cyber Security Centre — Impact of AI on the Cyber Threat From Now to 2027 https://www.ncsc.gov.uk/report/impact-ai-cyber-threat-now-2027

8. UK AI Security Institute — Frontier AI Trends Report https://www.aisi.gov.uk/frontier-ai-trends-report

9. Bruce Schneier — Autonomous AI Hacking and the Future of Cybersecurity https://www.schneier.com/blog/archives/2025/10/autonomous-ai-hacking-and-the-future-of-cybersecurity.html

10. Ullah et al. — From CVE Entries to Verifiable Exploits: An Automated Multi-Agent Framework for Reproducing CVEs (February 2026 revision) https://arxiv.org/abs/2509.01835v2

11. Chris Lewis — Decompiling a Nintendo 64 Game in 84 Days (August 2026) https://blog.chrislewis.au/decompiling-a-nintendo-64-game-in-84-days/

12. TechSpot / Rob Thubron — “Software is over”: Developer uses AI to build free Adobe clones after frustration with cancellation fees (October 2026) https://www.techspot.com/news/114136-software-over-developer-uses-ai-build-free-adobe.html

13. ArtCraft — PhotoCraft: Project Documentation and Clean-Room Approach https://github.com/storytold/photocraft

14. OpenAI — GPT-4 Technical Report (2023; revised March 2024) https://arxiv.org/abs/2303.08774

15. Google Threat Intelligence Group — GTIG AI Threat Tracker: From Prompting to Autonomy – The Evolution of Adversarial AI (September 2026) https://cloud.google.com/blog/topics/threat-intelligence/from-prompting-to-autonomy-the-evolution-of-adversarial-ai

16. OpenAI — An Agent Used DNS to Reach an External Chatbot (September 20, 2026 incident; report updated September 25) https://alignment.openai.com/misalignment-reports/an-agent-used-dns-to-reach-an-external-chatbot/