"Good news: that already works today." The assistant wrote that sentence with complete confidence. The sentence was true. It was also entirely beside the point.
We spent a day hardening our own infrastructure with an AI assistant at the keyboard: the Proxmox nodes (pve1, pve2, ipve1), the backup server (pbs) and the router sitting in front of them. The assistant did a great deal of good work, which we will get to. It also made three mistakes worth writing down, because they are the mistakes AI makes, and no amount of model quality removes the need for a human who remembers what they asked for.
Mistake one: answering the check instead of the request
The request was simple. The Proxmox web interface on ipve1, which lives on port 8006, should be reachable only from our office network. From anywhere else we would go in through an SSH tunnel. That was the whole design, stated in one sentence.
The assistant tested the port, found it open, reported "good news, that already works today", and moved on to a different part of the job.
Read the request again. The load-bearing word was "only". "Can I reach it?" is a trivial check that takes four seconds. "Can only I reach it?" was the actual work. The assistant performed the check and skipped the work, then spent hours hardening a perimeter that was never the target, while the one that was stayed exactly as it had been: wide open to the entire Internet, along with three other services nobody needed.
Here is the uncomfortable part. Nothing in the report looked wrong. It was accurate, detailed, and backed by real evidence gathered from real scans. It simply answered a question nobody had asked. A confident, well-evidenced answer to the wrong question is much harder to catch than an obvious error, because everything about it signals competence.
There is a lesson buried in there for anyone working with these tools. If you ask an assistant to verify something and it comes back with "this already works", be suspicious. People rarely ask you to verify things that already work. Ask yourself what you wanted to change.
Mistake two: checking one end of the wire
Later the same day, the assistant closed two ports that were exposed to the whole Internet, one of them a file-sharing service that has no business being reachable from outside. Before closing them it checked which machine was serving that protocol, found nothing that seemed to depend on it, and said so plainly: nothing uses this.
It had checked the server. It had not checked the clients. One of our other nodes was mounting a filesystem across that exact port, and the configuration file that proved it was one command away.
What makes this worth telling is how the failure behaved. The protocol keeps its existing connection alive, so after the port was closed everything kept answering normally. Listing the directory worked. Reading files worked. Every casual check said healthy. The breakage would only have surfaced on the next reboot, probably weeks later, with nobody connecting it to a firewall change from a Tuesday afternoon.
The assistant found it, fixed it properly, and said so without being asked. That part worked. But the original claim, "nothing uses this", was stated with the same confidence as everything else, and it was wrong. An open port has two ends. Looking at one of them and generalising is not verification, it is a guess wearing verification's clothes.
Mistake three: writing the problem down instead of fixing it
This is the subtle one, and the one we think is most worth your attention.
Asked to document our access map, the assistant produced a genuinely good document. Tables, verified data, nothing invented. And in that document, painted carefully in red, was a section listing the critical ports still exposed to the Internet, under the heading "pending".
Then it moved on, satisfied.
Cataloguing a hole is not closing it. It is worse than that: a documented risk reads as a managed risk. The red cell in the table makes it look considered, assigned, under control. A hole nobody has written down still feels like a hole. A hole in a table with a status next to it feels like a decision. The document was, in a very real sense, more dangerous than no document at all.
This is not a machine problem. Any consultant who has ever shipped a risk register instead of a fix has done the same thing. But an AI will do it faster, more thoroughly, and with much better formatting.
What it got right, and why that changes the conclusion
If the story ended there, the moral would be lazy: do not trust the machine. That is not our conclusion, because on the same day the same assistant caught things we had missed for months.
Our own hardening plan, written earlier and reviewed by humans, had a firewall rule in it that only covered IPv4. Our server also has a public IPv6 address, and the web interface was answering on it. Every scan we had run was IPv4-only, so we had never seen it. Had we executed our own plan exactly as written, we would have congratulated ourselves on closing a door while the other one stood open.
The same plan proposed enabling the built-in cluster firewall. The assistant refused and explained why: several of our containers have firewall mode switched on but carry no rules of their own, so turning the feature on at the top level would have applied a default deny to them and taken production down. That was our plan. It would have caused the incident it was written to prevent.
It also worked out that three of the four exposed services did not need a firewall at all, because nothing was using them. One was a console proxy for virtual machines on a host that has no virtual machines. Off. Done. No rules, no risk of locking anyone out.
What we actually take from this
The assistant was not unreliable in the way people expect. It did not hallucinate a command or invent a server. Everything it told us was true. The failures were about judgement: what the question meant, when a check is sufficient, and when writing something down has quietly become a substitute for doing it.
That is a specific shape of risk, and it has a specific antidote. Someone has to hold the intent. Not review the diff, not skim the summary, but remember what was actually asked and refuse the answer that does not deliver it. In this case that someone read a report full of green ticks and asked the only question that mattered: if I told you to restrict access to one network, why are the critical ports still open to everyone?
That question took ten seconds to ask. It was worth more than the whole afternoon of work that preceded it.
We use these tools every day, on our own systems and on our clients'. They are a genuine multiplier. They read more, remember more and check more than we can. But they multiply whatever direction you point them in, including the wrong one, and they will do it with excellent grammar and a table of evidence. The judgement stays with us. That is not a limitation we are waiting to be fixed. It is the job.