AI

AI-Generated Code Security Risks for Web Apps

Eng. Donya Bino Published  ·  15 min read

If you've read a headline claiming that roughly half of AI-generated code fails security testing, the number itself is less interesting than what it describes, which is a specific and predictable operational problem rather than a verdict on AI, because teams are now shipping code faster than their review process can absorb, and the code they're shipping carries a defect rate that nobody has agreed how to handle.

That's the practical framing, and it's worth sitting with, because the danger here isn't that AI writes obviously broken code, since it doesn't, and the danger is that it writes code that compiles, runs, passes tests, and looks clean while being unsafe, which is a much harder thing to catch than a syntax error.

So here is why this happens, where the failures cluster, why your existing review habits miss them, and what actually reduces the risk without slowing your delivery down, including what a proper secure code review looks like when you bring someone in to do it.

Important Disclaimer

This article is intended for educational and defensive purposes only, and it describes general software security practices and reasoning rather than specific measurements or published findings. Any headline figure you see on this topic originates from third-party research, so verify it against the original publication before repeating it in your own materials. 

No single control eliminates risk, and a secure development process depends on layers rather than on any one tool. The author assumes no liability for missed vulnerabilities, business outcomes, or damages resulting from the use or misuse of this information.

Why AI Writes Insecure Code That Looks Correct

The core reason is that these models are trained to produce code that is plausible, and plausible means it resembles the code they learned from, which includes an enormous amount of insecure code written by humans over decades.

A model that has seen ten thousand examples of a database query assembled with string concatenation has learned that this is a normal way to build a query, because statistically it is, and it has no separate instinct telling it that the pattern is dangerous. It learns form, not judgement, and the judgement is what a human developer builds slowly through code review, incidents, and being told "don't do that" by someone more experienced.

The second reason is that the model has no model of your threat surface. It doesn't know which of your endpoints is public, which data is regulated, which service account can reach production, or where your trust boundaries sit, so it produces code that works in the happy path and has no opinion about who could abuse it.

The third reason is the objective function, which rewards code that runs and satisfies the prompt, and nothing in that objective rewards defence in depth, input validation, or authorisation checks, so those appear when the developer asks for them and disappear when they don't.

Why the Problem Is Structural Rather Than Careless

It's tempting to explain insecure AI output by saying the model made a mistake, and that framing leads to the wrong fix, because the model is doing exactly what it was built to do.

A human developer writing an authentication check thinks about who might call this endpoint, which is a habit formed by having been burned. A model writing the same handler is completing a pattern, and the pattern it completes is the common one, which is often the one without the check.

That's why "use a better model" doesn't solve it, because the difference between models is smaller than the difference between having a review process and not having one, and a marginally better model with no review will still ship injection flaws.

Where the Failures Cluster

The failures aren't evenly distributed, and knowing the hot spots tells you where to spend review effort.

  • Injection and cross-site scripting are the dominant categories, because they arise from ordinary patterns that models reproduce constantly, which means a query built by concatenation, a template that renders unescaped output, or a redirect that accepts a destination from the request.
  • Authorisation is the second cluster, because the model will faithfully implement the endpoint you described and won't spontaneously add the ownership check that stops one user reading another user's record, since nothing in the prompt implied that was part of the job.
  • Insecure defaults come third, which means debug flags left on, permissive CORS headers copied from an example, error responses that leak stack traces, and cryptographic choices made from whatever appeared most often in training data.
  • Secrets handling is the fourth, because models will happily hardcode a connection string or an API key into a configuration file to make the example run, and that habit propagates into real repositories.
  • Dependencies are the fifth and newest, because a model asked for a library will sometimes name one that doesn't exist, and an attacker who notices that pattern can register the name and wait for someone to install it.

Why Your Current Review Doesn't Catch It

Most teams already review code, and the review was designed for a different problem, which is why it underperforms here.

  • Fluency creates false confidence, because the generated code is well-formatted, consistently named, and idiomatically structured, so a reviewer skimming a diff sees competence and reads for logic rather than for trust boundaries.
  • Volume compresses attention, because the same developer who would have written a small change now produces a much larger one, and a large diff gets a shallower read per line, which is exactly the condition under which a subtle authorisation gap survives.
  • Tests don't cover it, because a test asserts that the feature works for valid input, and an injection flaw is a feature that works perfectly for valid input while doing something unexpected for malicious input.
  • Nobody asks the specific question, which is whether data reaching a sink has been validated, and that question is not part of most checklists because most reviewers assume the developer already thought about it.

What Actually Reduces the Risk

The answer isn't to slow down and it isn't to trust harder, and it's to move detection earlier so the review has less to catch.

  • Run static analysis on every commit rather than at release. The failure modes described above are detectable by tooling that looks for tainted data reaching a sink, and running it at the commit means the finding arrives while the developer still has the code in their head.
  • Add secret scanning before the push lands. A credential in a public repository is a clock that starts the moment the push completes, and the fix takes seconds at the commit and hours after the incident.
  • Verify dependencies before installing. If a package name looks unfamiliar, confirm it exists and has real maintainers, because the cost of a thirty-second check is trivial compared with a supply chain compromise.
  • Ask one security question in every review. For each new endpoint or data path, ask where the input comes from and what happens to it before it reaches the database, the shell, the file system, or the template, and make that the question rather than style.
  • Treat the model's output as a draft from a fast junior developer who has never seen your system. That framing changes what you look for, because you stop checking whether the code works and start checking whether it is safe.
  • Write the security requirement into the prompt. If you need an authorisation check, say so, and if you need parameterised queries, say so, because the model will produce what you ask for and won't infer what you left out.

Real Tools and Checks You Can Add This Week

Control

Tool or Practice

What It Catches

Static analysis

Semgrep, CodeQL, SonarQube

Injection, unsafe crypto, tainted data flow

Dependency scanning

Dependabot, Snyk, OWASP Dependency-Check

Known-vulnerable packages

Secret scanning

Gitleaks, TruffleHog, GitGuardian

Leaked keys and tokens in commits

Pre-commit hooks

pre-commit with SAST and secret rules

Blocks the commit rather than the release

Review checklist

One question about trust boundaries

Missing authorisation and validation

Package verification

Manual check before install

Hallucinated and malicious packages

Runtime scanning

ZAP baseline in CI

Injection and header issues in a running app

A minimal pre-commit configuration covers the two cheapest wins, which are leaked secrets and OWASP-pattern code, and it takes minutes to add.

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.18.0
    hooks:
      - id: gitleaks
  - repo: https://github.com/semgrep/semgrep
    rev: v1.55.0
    hooks:
      - id: semgrep
        args: ["--config", "p/owasp-top-ten", "--error"]

What a Secure Code Review Actually Covers

Everything above is what you can do yourself, and there's a point where doing it yourself stops being efficient, which is usually when the codebase is large, when the team is small, or when the application handles payments, personal data, or anything regulated.

A proper secure web development review is not a scanner report, and it's not a penetration test either, because it sits between the two and looks at the things automation can't see. In practice it covers four areas.

  • A review of your code for the failure patterns above, which means tracing input from each entry point to each sink, checking authorisation on every state-changing route, reviewing session and cookie handling, and verifying that secrets aren't committed or logged.
  • A review of your AI-assisted development workflow, which means looking at what your prompts actually ask for, whether static analysis and secret scanning are wired into CI, whether dependencies are verified, and whether the review process has a security question in it or just a style question.
  • A configuration and infrastructure review, because most web application compromises involve something outside the code, and that means transport configuration, security headers, file upload handling, cloud permissions, and the exposure of admin interfaces.
  • A prioritised remediation plan, because a list of two hundred findings is not actionable and a list of the seven things that actually matter this month is.

That's the shape of the engagement at Red Secure Tech, and it's designed to be the thing you do before you decide whether you need a full penetration test, because testing an application that still has an authorisation gap on every route is an expensive way to confirm what a code review would have told you in a week.

Real Two Scenarios

Scenario 1: The Feature That Passed Everything

The Setup

A small team uses an AI assistant to build a customer portal feature in a week rather than a month, and the code compiles cleanly, passes unit tests, and looks well-structured in review.

What Happened

The generated handler builds a database query by assembling a string from a request parameter, and because it returns correct results for normal input, every test passes and the reviewer approves it, since nothing about the code looks obviously wrong.

The Result

The injection flaw sits in production until a scanner or a researcher finds it, and the remediation takes an afternoon, but the exposure window was measured in weeks and nobody can say with certainty that it wasn't used.

The Lesson

The code was syntactically perfect and functionally correct, and it was insecure in a way that no correctness test would ever flag, which is the exact failure mode this whole discussion is about.

Scenario 2: The Review That Caught It

The Setup

Another team uses the same assistant for a similar feature but brings in an external review before launch, alongside adding a static analysis step to their pull requests.

What Happened

The review traces a request parameter from a public endpoint through three functions and into a query builder that concatenates it, confirms that no validation exists on the path, and also flags a second route where the handler fetches a record by ID without verifying that the caller owns it.

The Result

Both findings are fixed in a day, the launch happens on schedule, and the team adds the authorisation question to their review checklist so the same pattern gets caught internally next time.

The Lesson

The external review paid for itself in a single afternoon, and it found the class of flaw that automated scanning reports inconsistently and that internal review had stopped seeing because the code looked competent.

How to Spot AI-Generated Code That Needs a Second Look

  • The diff is large and well-formatted. Fluency is exactly what suppresses scrutiny, so slow down on the big clean-looking changes.
  • The query is assembled from a variable. Concatenation is the single most common injection path and it looks normal.
  • The endpoint has no ownership check. If the handler fetches by ID without verifying the caller owns the record, that's an authorisation gap.
  • CORS or debug settings look copied. Permissive defaults travel with examples and rarely get revisited.
  • A credential appears in a config file. Hardcoded values are a habit the model learned from sample code.
  • A dependency name is unfamiliar. Verify it exists before installing, because hallucinated packages are a real supply chain risk.
  • Error responses include internal details. Stack traces and database messages in responses are an information leak.

Quick Reference: Where AI Code Fails and What Catches It

Failure Category

What It Looks Like

Control

Injection

Query built from a variable

Static analysis, parameterization review

Cross-site scripting

Unescaped output in a template

Static analysis, framework auto-escaping

Authorization

Fetch by ID with no ownership check

Review question on trust boundaries

Insecure defaults

Permissive CORS, debug on

Configuration review, baseline hardening

Secrets

Credential in a config file

Secret scanning at commit

Dependencies

Unfamiliar package name

Manual verification before install

Information leak

Stack traces in responses

Error handling review

The Bottom Line

AI-generated code fails security tests not because it's broken but because it's plausible, and plausible code is drawn from a training set that is full of insecure patterns, which is why the output compiles, looks professional, and still ships injection and authorisation flaws.

The practical response is to move detection upstream, which means static analysis and secret scanning at the commit, one review question about where input comes from and where it goes, dependency verification before installation, and a written security requirement in the prompt so the model produces the check you actually need.

And when the codebase is large, the team is small, or the application matters enough that a missed authorisation gap would be a real incident, the sensible next step is a proper secure code review rather than a scanner subscription, because the flaws that hurt you are the ones automation reports inconsistently and reviewers stop seeing.

That's what we do at Red Secure Tech, and it's built to be the review you run before you spend money on a penetration test, not instead of one.

👉 Secure web development review: https://www.redsecuretech.co.uk/service/secure-web-development

FAQ

Why is AI-generated code often insecure?

Because models are trained to produce code that looks plausible, and the code they learned from includes a great deal of insecure human-written code, so they reproduce common patterns without the judgement that tells an experienced developer a pattern is dangerous.

Does AI-generated code fail security tests more than human code?

The reported figures suggest it carries a higher defect rate, and the more useful takeaway is that its failures are logic-level and invisible to correctness tests, which means they survive review unless you specifically look for them.

What vulnerabilities does AI-generated code introduce most often?

The recurring categories are injection and cross-site scripting, missing authorisation checks, insecure defaults such as permissive CORS or debug mode, hardcoded secrets, and occasionally dependencies that don't exist.

Why doesn't my code review catch these issues?

Because the code compiles and looks clean, the diffs are larger than before, and the flaws are authorisation and data-flow problems rather than style problems, which is why a specific question and an automated check outperform a general read.

Is it safe to use AI-generated code in production?

Yes, with the same controls you'd apply to any code, which means static analysis, secret scanning, dependency verification, and a reviewer who is accountable for the merge, and the risk is not in using AI but in treating its output as finished.

When should I bring in an external review rather than doing it myself?

When the codebase is large, the team is small, or the application handles payments, personal data, or anything regulated, because at that point the cost of a missed authorisation gap exceeds the cost of the review by a wide margin.

How do I get a secure code review for my web app?

Red Secure Tech runs secure web development reviews for applications built with or without AI assistance, covering code, configuration, and workflow, with a prioritised plan you can act on immediately. You can book or enquire here: https://www.redsecuretech.co.uk/service/secure-web-development

How do I handle the headline statistic responsibly?

Treat it as a third-party research claim rather than a given, keep the attribution intact if you cite it, and don't present it as a measurement of your own codebase, because it describes raw model output rather than reviewed output.

Professional Services

Explore Our Cybersecurity Services

Our insights are backed by hands-on service delivery. If your business needs professional cybersecurity support, our UK-based specialists are ready to help.

© 2016 – 2026 Red Secure Tech Ltd. Registered in England and Wales — Company No: 15581067