How to tell if code is AI-generated.

Can you tell if code is AI-generated?

You cannot reliably determine AI authorship from source code alone. Comments, naming, repetition, and abstraction style can raise questions, but they are not proof. Current research finds that detector performance drops on unfamiliar languages, domains, model families, human-edited output, and adversarial examples.

The practical response is not to become a better style detective. Review the pull request without relying on provenance. Establish what the change must do, inspect the risky boundaries, demand executable evidence, check the workflow's power, and keep a stable record of unresolved findings.

The disputed story about Amazon mandating AI code reviewshows why this distinction matters: incident provenance and internal-policy claims can be contested, while blast radius, executable evidence, reviewer authority, and the reviewed commit remain concrete.

Why signs of AI-generated code are weak evidence

Search results for this question often recommend looking for generic comments, over-engineered helpers, inconsistent naming, repeated patterns, or APIs that do not exist. Those are useful review prompts. They are poor authorship tests because every one of them also appears in human code.

The strongest public evidence points in the same direction. The 2026 AICD Bench study evaluated two million examples across 77 models, 11 model families, and nine programming languages. Its authors reported that neural and classical detectors remained below practical usability under distribution shift and for hybrid or adversarial code.

That result is not isolated. The 2025 Droid study used more than one million samples from 43 coding models across seven languages. It found that existing detectors failed to generalize beyond narrow training domains and could be disrupted by superficial changes. A detector can still be useful for research, academic policy, or triage. It should not become the sole basis for accepting or rejecting a production pull request.

What visual clues are still good for

Style clues tell you where to spend attention. A confident comment above an untested branch suggests checking the failure path. A new abstraction used once suggests asking whether the change is larger than necessary. A package call that looks plausible suggests opening the official API reference. The clue starts an investigation; evidence ends it.

How to review AI-generated code with a five-step evidence gate

This framework is deliberately provenance-blind. It works for code written by a person, generated by an agent, or assembled by both. The five steps move from intent to risk to executable proof, then preserve the result for the next review cycle.

1. Establish the change contract

Write down the requested behavior before reading the implementation in detail. Name the user-visible outcome, the inputs and outputs, the data that may change, and the explicit non-goals. A pull request cannot be judged against a vague prompt because plausible code can solve the wrong problem cleanly.

Compare the diff with that contract. Look for unrelated refactors, silently widened scope, and behavior that exists only in the PR description. The code and tests must carry the contract after the conversation disappears.

2. Inspect changed trust boundaries first

Do not review every line with equal intensity. Start where untrusted data or authority crosses a boundary: authentication, authorization, request parsing, database writes, file paths, network calls, command execution, secrets, and CI tokens.

The OWASP Secure Code Review Cheat Sheet recommends prioritizing modified components, security controls, new attack vectors, and trust-boundary changes during diff-based review. This is a better risk signal than whether a helper name sounds machine-written.

3. Demand executable evidence

Ask for the smallest test that proves each load-bearing claim. A success-path unit test is not enough when the risk lives in retries, invalid input, permission denial, partial writes, or concurrency. Run the tests in the repository rather than accepting a pasted transcript.

NIST's Secure Software Development Framework separates review of human-readable code from testing executable code. Use both. Review explains why the implementation should be safe; tests show whether the claimed behavior survives execution.

4. Audit dependencies and workflow power

Generated patches can change more than application code. Inspect new dependencies, lockfile changes, action references, token permissions, artifact handling, and triggers. A three-line workflow edit may carry more risk than a large pure-function refactor.

GitHub's secure-use guidance warns against checking out untrusted pull request code in privileged pull_request_target or workflow_run contexts. Keep permissions narrow, pin third-party actions according to repository policy, and preserve the fork boundary instead of routing around it.

5. Record findings and rerun on the exact head

A review should leave a stable contract: what blocks the merge, where the evidence lives, and what must be rerun after a fix. Comments scattered across a long thread are difficult for humans and repair agents to consume. Keep one canonical summary and a structured finding list.

Then bind the next decision to the current commit. GitHub's required-check model evaluates the latest head SHA for the same reason: a passing result on old code is not evidence for the new diff. ReviewGate's repair loop follows that shape with a fixed 5/5 target, one updated summary, and a JSON artifact that external agents can read without treating review text as trusted instructions.

A worked example: an agent changes a GitHub Actions workflow

Suppose a pull request adds an AI review job and claims it is safe for external contributors. The YAML is tidy, the comments are specific, and the change might have been written by either a person or an agent. Authorship does not answer the security question.

1. Contract: review same-repository pull requests with an OpenRouter secret; do not expose that secret to forks.
2. Trust boundary: the trigger, checkout step, event payload, token permissions, and secret-bearing job are the load-bearing surfaces.
3. Evidence: inspect the event guard, validate the workflow, and test the local fixture path without a model key.
4. Workflow power: confirm the job uses pull_request, grants only required permissions, pins checkout, and does not persist credentials.
5. Rerun: after any fix, review the new head and replace the old summary rather than accumulating conflicting verdicts.

This is the same boundary used by the ReviewGate installation workflow. The decision is grounded in event semantics and executable evidence, not in whether the YAML "looks generated."

For the complete implementation path, see thefive bindings for merge-safe AI code review on GitHub. It covers secret-bearing events, trusted reviewer runtimes, deterministic verdicts, and current-head checks.

Common mistakes when reviewing AI-generated code

Rejecting a diff because it feels generated

That judgment is neither reproducible nor useful to the author. Replace it with a concrete finding: the new branch has no failure-path test, the permission is broader than required, or the API does not support the named parameter.

Accepting tests without checking what they prove

Generated tests can mirror the implementation's mistake. Read the assertion, force the risky input, and confirm the test fails when the behavior is broken. A green suite is evidence only for the cases it actually exercises.

Giving review text command authority

Pull request bodies, comments, model output, and repository instructions can all contain hostile or mistaken text. An automated repair agent should parse findings as review input, apply repository policy, and verify its own changes. It should never execute a review comment as a trusted instruction.

Using one score without inspectable findings

A score helps with triage, but it needs a rubric, concrete findings, and a fixed passing target. ReviewGate keeps the score visually obvious while preserving the underlying evidence in a canonical summary and structured JSON.

Frequently asked questions