A fast browse of LinkedIn, DevTok, and X would lead you to imagine that nearly each developer has jumped on board the vibe coding hype practice with full gusto. And whereas it’s not that far-fetched, with 84% of builders confirming they’re presently utilizing (or planning to make use of) AI coding instruments of their every day workflows, a full give up to vibe coding autonomous brokers remains to be uncommon. Stack Overflow’s 2025 AI Survey revealed that almost all respondents (72%) are usually not (but) vibe coding. Nonetheless, adoption is trending upwards, and AI is presently producing 41% of all code, for higher or worse.
Instruments like Cursor and Windsurf signify the newest technology of AI coding assistants, every with a robust autonomous mode that may make selections independently primarily based on preset parameters. The velocity and productiveness positive aspects are simple, however a worrying development is rising: many of those instruments are being deployed in enterprise environments, and these groups are usually not outfitted to deal with the inherent safety points related to their use. Human governance is paramount, and too few safety leaders are making an effort to modernize their safety applications to adequately defend themselves from the danger of AI-generated code.
If the tech stack lacks instruments that oversee not solely developer safety proficiency, but additionally the trustworthiness of authorized AI coding companions every developer makes use of, then it’s doubtless that efforts to uplift the general safety program and the builders working inside it will likely be wanting the suitable information insights to impact change.
AI and human governance ought to be a precedence
The drawing card of agentic fashions is their skill to work autonomously and make selections independently, and these being embedded into enterprise environments at scale with out applicable human governance is inevitably going to introduce safety points that aren’t significantly seen or straightforward to cease.
Lengthy-standing safety issues like delicate information publicity and inadequate logging and monitoring stay, and rising threats like reminiscence poisoning and gear poisoning are usually not points to take flippantly. CISOs should take steps to cut back developer danger, and supply steady studying and abilities verification inside their safety applications with a purpose to safely implement the assistance of agentic AI brokers.
Highly effective benchmarking lights your developer’s path
It’s very tough to make impactful, optimistic enhancements to a safety program primarily based solely on anecdotal accounts, restricted suggestions, and different information factors which are extra subjective in nature. Most of these information, whereas useful in correcting extra obvious faults (similar to a selected instrument repeatedly failing or personnel time being wasted on a low-value and irritating activity), will do little to uplift this system to a brand new degree. Sadly, the “individuals” a part of an enterprise safety (or, certainly, Safe by Design) initiative is notoriously tough to measure, and too typically uncared for as a chunk of the puzzle that should be a precedence to resolve.
That is the place governance instruments that ship information factors on particular person developer safety proficiency – categorized by language, framework and even trade – will be the distinction between executing one more flat coaching and observability train, versus correct developer danger administration, the place the instruments are working to gather the insights wanted to plug data gaps, filter security-proficient devs to essentially the most delicate initiatives, and importantly, monitor and approve the instruments they use of their day, similar to AI coding companions.
Evaluation of agentic AI coding instruments and LLMs
Three years on, we are able to confidently conclude that not all AI coding instruments are created equal. Extra research are rising that help in differentiating the strengths and weaknesses of every mannequin, for a wide range of purposes. Sonar’s latest research on the coding personalities of every mannequin was fairly eye-opening, revealing the totally different traits of fashions like Claude Sonnet 4, OpenCoder-8B, Llama 3.2 90B, GPT-4o, and Claude Sonnet 3.7, with perception into how their particular person approaches to coding have an effect on code high quality and, subsequently, related safety danger. Semgrep’s deep dive into the capabilities of AI coding brokers for detecting vulnerabilities additionally yielded blended outcomes, with findings that usually demonstrated {that a} security-focused immediate can already establish actual vulnerabilities in actual purposes. Nonetheless, relying on the vulnerability class, a excessive quantity of false positives created noisy, much less helpful outcomes.
Our personal distinctive benchmarking information helps a lot of Semgrep’s findings. We had been in a position to present that the most effective LLMs carry out comparably with proficient individuals at a variety of restricted safe coding duties. Nonetheless, there’s a important drop in consistency amongst LLMs throughout totally different phases of duties, languages, and vulnerability classes. Usually, high builders with safety proficiency outperform all LLMs, whereas common builders don’t.
With research like this in thoughts, we should not lose sight of what we as an trade are permitting into our codebases: AI coding brokers have growing autonomy, oversight and common use, and so they should be handled like every other human with their fingers on the instruments. This, in impact, requires cautious administration by way of assessing their safety proficiency, entry degree, commits and errors with the identical fervor because the human working them, with no exceptions. How reliable is the output of the instrument, and the way safety proficient is its operator?
If safety leaders can not reply these questions and plan accordingly, the assault floor will proceed to develop by the day. In the event you don’t know the place the code is coming from, ensure that it’s not getting in any repository, with no exceptions.
