Which agent pull request gates should block a merge?

An agent can produce a clean diff, a persuasive description, and code that passes a quick review while still leaving an application exposed or impossible to recover. The merge decision should therefore depend on evidence the repository can measure, not on how confident the agent sounds or how small the patch looks.

I use seven blocking gates: tests, CodeQL, dependency review, secret scanning, authorization checks, migration rehearsal, and rollback verification. Each answers a different failure question. A green test suite cannot prove a new package is safe, and a clean static analysis result says nothing about whether a database migration locks the busiest table.

These gates apply to human and agent changes alike. Agent authorship changes the volume, speed, and shape of mistakes, but it does not justify a separate, weaker lane. If a proposed change cannot supply the same evidence as any other pull request, it is not ready to merge.

A gate must produce evidence, not advice

A merge gate should return a reproducible pass or fail result for the exact commit that will enter the protected branch. A comment that says "please review this dependency" is advice. A required check that identifies the package, version, advisory, and severity threshold is evidence.

That distinction matters because many security features default to reporting after the useful decision point. A scanner may create an alert, send email, or open an issue while the merge button remains available. Teams then say scanning is "enabled" even though it cannot stop the change. For every gate, verify four properties:

• It runs on the pull request's current head commit.
• Branch protection requires its named result.
• A skipped, timed out, or crashed job does not count as a pass.
• The result records enough detail to reproduce the decision.

Keep the policy in the repository. A small manifest makes review easier than a collection of settings known only to administrators:

merge_gates:
  tests: required
  codeql: required
  dependency_review: required
  secret_scan: required
  authorization: required
  migration_rehearsal: required_when_changed
  rollback_verification: required

The required_when_changed value is not a loophole. It means the gate first detects relevant files and then either performs the rehearsal or records a clean "not applicable" result. Do not let path filters leave a required check permanently pending, and do not let an agent decide that its own risky change is exempt.

Pin workflow permissions to the minimum each job needs. Pull request code is untrusted input, even when the branch belongs to your organization. A gate that exposes a write token or production secret to the code it examines can create a worse problem than the one it was meant to catch.

Protect the identity of the check as carefully as its logic. Branch rules usually require a status name, so two workflows that can report the same name may let a weaker job satisfy the rule. Give the policy job a unique name, restrict who can change its workflow, and require review from the owners of that file. When a merge queue creates a new merge commit, run the gates again on that commit or use a platform feature that binds results to the queued revision. Evidence for yesterday's head is not evidence for today's merge.

Treat the gate configuration itself as sensitive code. A pull request that changes a threshold, removes a path, downgrades a query pack, or adds an exception changes the meaning of every later green result. Show policy diffs prominently and require a maintainer who understands the affected control. An agent can propose such a change, but it should not gain easier passage merely because it edits the mechanism judging it.

Tests block observable regressions

The test gate should block any change that breaks specified behavior on the supported runtime and database versions. It must run the same build inputs that the merge commit will use, including locked dependencies, generated files, feature flags, and schema state.

Agents are particularly good at satisfying the nearest assertion. They may add a fallback that makes one test green while breaking error handling, pagination, concurrency, or an adjacent API contract. Require the pull request to add or change tests when it changes behavior, but do not measure quality by the number of new test lines. Review whether the test would fail if the implementation were removed or reverted.

A useful test gate has layers with separate names:

• Unit tests for local logic and boundary cases.
• Integration tests for database, queue, cache, and external service contracts.
• Contract tests for public request and response shapes.
• A small smoke test against the built artifact.

Run flaky tests to ground instead of granting automatic retries until green. One retry can collect diagnostic evidence, but the final status should expose the initial failure. Otherwise an agent can merge code whose only proven property is that it sometimes works.

The smoke test should start the artifact, exercise a health endpoint and one meaningful write path, then stop it cleanly. Testing source code without starting the packaged application misses missing files, bad environment defaults, broken migrations, and startup panics. For a web service, the result can be recorded in a compact shape:

{"commit":"abc123","build":"pass","startup_ms":842,"health":200,"write_read_cycle":"pass"}

Do not set a universal coverage percentage as the primary gate. Coverage can reveal an untested change, but a repository can reach a high percentage with weak assertions. Gate on required suites and changed behavior, then use coverage movement as review evidence.

Protect the tests from the implementation they judge. If a pull request both changes a rule and rewrites the assertions to accept the new result, the suite may pass while the contract quietly moves. Require a reviewer to compare changed tests with the public API, issue, or acceptance rule. For parsers, validators, billing logic, and access checks, add mutation testing or a small set of deliberately bad inputs. The useful question is whether the suite rejects a plausible wrong implementation, not whether the agent can make its own implementation satisfy assertions it also wrote.

Keep test artifacts when the gate fails. Store the failing seed for randomized tests, the exact database image, service logs with secrets removed, and the command that reproduces the run. A status without reproduction data sends the next agent or engineer back into guesswork. Cap artifact retention according to the repository's data rules, and never upload a production snapshot just because it makes a failure easy to reproduce.

CodeQL blocks known code paths to vulnerabilities

The CodeQL gate should block new high confidence findings in the languages and generated artifacts that CodeQL actually analyzes. It should not imply that a clean result proves the whole application secure.

GitHub describes CodeQL as compiling code into a queryable database and running queries over it. That model is useful because it follows data through code rather than matching only suspicious text. It is also bounded by language support, build success, query selection, and the code present during analysis. If the database build silently excludes a service, the green result covers less than reviewers think.

Use a workflow with explicit languages and a fixed query policy:

name: codeql
on: [pull_request]
permissions:
  contents: read
  security-events: write
jobs:
  analyze:
    strategy:
      matrix:
        language: [javascript-typescript, go]
    steps:
      - uses: actions/checkout@v4
      - uses: github/codeql-action/init@v3
        with:
          languages: ${{ matrix.language }}
          queries: security-extended
      - uses: github/codeql-action/autobuild@v3
      - uses: github/codeql-action/analyze@v3

Pin third party actions to reviewed commit digests in a real repository. Tags make the example readable, but a mutable tag expands the trust boundary of a required security job.

Decide what blocks before the first alert appears. I normally block new findings at the repository's agreed severity and precision threshold, while existing debt remains visible in a baseline. Blocking every historical result on day one encourages mass dismissal. Ignoring all existing results forever creates a permanent blind spot. Assign owners and due dates to the baseline instead.

Inspect the analyzed file set when services, languages, or build commands change. A pull request that introduces a new mobile client, generated resolver, or separate backend may need another analyzer or build step. "CodeQL passed" is meaningful only when reviewers can state what CodeQL examined.

Keep analysis failures distinct from clean analyses. If autobuild cannot compile a package, the job must report an infrastructure or configuration failure, not zero findings. Save the database creation log and the count of analyzed source files by language. Compare those counts with the base branch and flag a large unexplained drop. This catches an ordinary failure: a build edit excludes the vulnerable module, analysis becomes faster, and the security check turns green because it saw less code.

Review dismissals as policy changes, not cleanup. A false positive needs a concrete explanation tied to the code path and query. Suppression comments should be narrow, owned, and visible in the pull request diff. A repository wide exclusion for generated files may be appropriate, but first establish that no hand maintained templates or generator inputs sit in that directory.

Dependency review stops risk before installation

Dependency review should block a pull request when its dependency diff introduces a package or version that violates an explicit policy. That policy can include known advisory severity, denied licenses, unexpected package sources, and direct dependencies added without an owner.

This gate is different from a repository vulnerability alert. An alert tells you a vulnerable dependency exists in the branch. Dependency review asks whether this pull request makes the dependency graph worse. GitHub's dependency review compares manifests and lockfiles across the pull request, which makes the decision timely and attributable.

Require lockfile consistency. If an agent edits package.json but not the lockfile, or edits the lockfile without a matching manifest change, the job should fail. Install commands that resolve fresh versions during CI make the result nondeterministic and can test a graph different from the graph reviewers saw.

A compact policy might read:

dependency_policy:
  fail_on_severity: high
  deny_licenses:
    - AGPL-3.0
  allow_sources:
    - registry.npmjs.org
    - proxy.golang.org
  require_owner_for_direct_additions: true

The exact license list is a legal and product decision, not a value to copy blindly. The useful part is that the repository declares it and the check prints the package that triggered it.

Do not auto approve a package because its name resembles a suggested library. Agents can hallucinate package names, select abandoned forks, or add a large client for one trivial helper. The review output should show the new direct and transitive packages, source registry, resolved version, license, and advisory status. A reviewer can then ask whether existing code or a smaller dependency already solves the problem.

Fail closed if the dependency service cannot produce a diff. An unavailable advisory feed may justify holding the merge, not converting uncertainty into a green check. Emergency procedures can allow a documented override by a named maintainer, with the reason attached to the commit.

Check installation behavior in an isolated job with network access limited to approved registries. Lifecycle scripts and build plugins execute code during installation, so a package can be dangerous even when the application never imports it. Record whether a new dependency adds install scripts, native binaries, or an unfamiliar registry. Do not run that job with publishing credentials, cloud tokens, or a writable package cache shared with trusted builds.

Vendored code and container images belong in the same decision even though ordinary manifest review may miss them. Compare image digests, base image names, Git submodules, and checked in archives. Require immutable digests for release inputs. A friendly tag such as latest makes a pull request impossible to reproduce later because the bytes can change without another diff.

Secret scanning must inspect the diff and its history

Secret scanning should block when a pull request introduces a credential pattern or a verified live secret, even if the string appears in a test fixture, deleted file, generated bundle, or earlier commit in the pull request.

Push protection and pull request scanning solve related but different problems. Push protection can stop a recognized secret before it reaches the remote. A pull request gate examines what has already arrived and can cover contributors or token types that push protection missed. An alert without branch protection does not block merging.

Scan the full commit range against the base branch, not only the final filesystem. An agent can add a token in one commit and remove it in the next; the token remains in Git history and may already have reached logs or caches. Treat the result as exposed. Revoke or rotate the credential, remove it from the proposed history, and rerun the check.

Use synthetic fixtures that cannot authenticate. A fixture should clearly identify itself and match a locally defined test pattern instead of copying the shape of a real cloud key. Broad allowlists are dangerous because attackers and accidents both find the ignored directory eventually. Keep exceptions exact, reviewed, and close to the detector configuration.

Combine pattern detectors with entropy checks and, where the provider safely supports it, credential verification. Pattern matching has fewer network and disclosure risks but misses custom tokens. Verification can reduce uncertainty, yet it sends part of a candidate secret to another service and must never run against an untrusted endpoint supplied by the pull request. Document which detectors verify, which stay local, and what data leaves CI.

Scan common encoded forms and generated outputs without pretending every random string is a credential. Base64, URL encoding, and minified bundles can hide a secret that appeared plainly in a source step. Set the blocking threshold from tested fixtures, then review detector updates like other policy changes. A noisy scanner trains maintainers to dismiss results, while a silent scanner creates false confidence.

The gate output needs location without disclosure:

{"result":"fail","detector":"generic-api-token","commit":"abc123","path":"config/dev.env","line":7,"fingerprint":"sha256:8f2c..."}

Never print the full match into CI logs or a pull request comment. Masking after the scanner emits output may be too late because log systems, notifications, and job artifacts can copy it.

Secret scanning does not replace repository permission review. A workflow can read a production credential without placing it in the diff, and an agent can alter a deployment job to exfiltrate it. Keep secrets away from pull request jobs, restrict workflow permissions, and require human review for changes to CI definitions.

Authorization checks prove forbidden actions stay forbidden

The authorization gate should prove that each protected operation rejects the wrong actor and allows the intended actor at the service boundary. Login tests alone do not test authorization.

Teams routinely blur authentication, authorization, and user interface visibility. Authentication establishes who sent the request. Authorization decides whether that identity may perform this action on this object. Hiding an admin button in React changes neither decision on a Go API. If the backend accepts the request, the application remains exposed.

Build a permission matrix for changed endpoints and business actions. Keep it small enough to review, but include ownership and tenant boundaries:

read_private_project:
  anonymous: deny
  member: deny
  other_tenant: deny
  owner: allow
  admin: allow
update_project:
  anonymous: deny
  other_tenant: deny
  owner: allow

Generate tests from this matrix or encode equivalent table driven cases in the service language. Every deny case should call the real handler with realistic identifiers. Mocks that replace the authorization middleware can prove the route works while skipping the control under test.

Test object level access, not only roles. Two users may both have the member role while belonging to different organizations. Change the resource identifier and tenant identifier independently to catch insecure direct object references. Also test bulk endpoints, exports, background jobs, and GraphQL resolvers, which often bypass checks written for ordinary REST handlers.

Fail the gate when a new protected route has no policy mapping. That turns missing authorization from a reviewer hunch into a measurable defect. Keep the default on the server as deny. An explicit allow rule is easier to audit than scattered code that denies a few known bad cases.

Agents often reuse a nearby handler and preserve its happy path while dropping its ownership check. The permission matrix makes that omission visible. It also gives reviewers a stable contract when roles or tenant rules change later.

Exercise authorization after input normalization. Case folding, alternate identifier formats, duplicate query parameters, and nested object references can send equivalent requests through different code paths. Test the direct endpoint and any batch or import route that reaches the same operation. If a background worker performs the final write, propagate the actor and tenant context into the job rather than treating the worker as an all powerful trusted user.

Record the expected decision and the policy rule that produced it, but avoid exposing private object data in logs. A useful failure names the actor class, action, object class, and expected status. It does not dump the access token or full record. That evidence lets a reviewer distinguish a broken test fixture from a genuine privilege change.

Migration rehearsal measures locks and reversibility

The migration gate should apply every proposed schema change to a production shaped copy, run compatibility probes, and record duration, locks, and rollback behavior before merge. A migration that succeeds on an empty test database proves very little.

Use a sanitized snapshot or generated dataset with similar table sizes, indexes, constraints, and skew. Exact production data does not belong in pull request infrastructure. The goal is to recreate operational pressure without copying personal or confidential records.

Rehearse the actual deployment sequence. If old application instances remain active while the migration runs, test old code against the new schema and new code against the transitional schema. Adding a nullable column is usually compatible. Renaming a column in one step can break every old instance still serving traffic.

Record evidence in a machine readable result:

{"migration":"20260727_add_project_state","apply_ms":18420,"max_lock_ms":310,"old_app_probe":"pass","new_app_probe":"pass","down":"pass"}

Set thresholds per database and table class. A 300 millisecond lock may be harmless for one table and disruptive for another. Reviewers should see the chosen limit beside the measured result, plus the database version used during rehearsal.

For PostgreSQL, inspect operations that rewrite a table, validate a constraint, or build an index while blocking writes. Prefer expand and contract changes: add the new shape, deploy code that can use both shapes, backfill in controlled batches, switch reads, then remove the old shape in a later change. The extra pull request is cheaper than improvising during an outage.

Not every migration can have a safe down operation. Dropping a column loses data, and reversing a transformation may be ambiguous. In those cases, the gate should require a forward recovery procedure and a tested backup restore point. Calling an irreversible migration "rollback safe" because a down file exists is dishonest.

Test retries and partial failure. A deploy can stop after creating an index but before recording the migration as complete, and the next attempt must not corrupt state or fail forever. Interrupt the rehearsal at a controlled boundary, rerun it, and verify the schema and migration ledger agree. For a long backfill, prove that batches resume from a recorded cursor and that running a batch twice does not duplicate or erase data.

Check disk and replication effects as well as elapsed time. A table rewrite can consume temporary space, expand write ahead logs, and delay replicas after the primary operation appears complete. The gate does not need a perfect production forecast, but it should record these quantities on the rehearsal dataset and compare them with limits chosen by the database owner. Without that evidence, a fast local migration can still exhaust a production volume.

Rollback verification must run the recovery path

The rollback gate should deploy the candidate to an isolated environment, create representative state, trigger the supported rollback method, and prove the previous version still serves correct reads and writes. A written rollback plan is not verification.

Separate application rollback from data rollback. Repointing traffic to the previous binary may take seconds, while undoing a destructive schema or data transformation may be impossible. The gate should report both. If the old application cannot run against the new schema, mark the candidate as nonreversible and require a staged deployment plan.

A practical sequence is:

1. Deploy the current main commit and seed representative records.
2. Upgrade to the pull request artifact and exercise changed paths.
3. Create new records in the upgraded state.
4. Restore the previous artifact or snapshot using the documented control.
5. Run read, write, queue, and background job probes.

The check should retain artifact identifiers, snapshot identifiers, timestamps, and probe results. It should not retain credentials or copied customer data. Measure recovery time as evidence for your own operating target, not as a universal promise.

Snapshots help only if somebody proves they contain everything the application needs. Files, object storage, queue state, schema changes, and external side effects may live outside a server snapshot. List those boundaries in the result. A payment, email, or webhook already sent cannot be pulled back by restoring a database.

Make the rollback probe stronger than a health check. Read a record created before the upgrade, read one created after it, update both where compatibility permits, and process a queued job created by each application version. Compare user visible output, not only status codes. A server that returns 200 while dropping a new field or misreading an enum has not recovered.

Test the control used by the person on call. If production rollback requires selecting an artifact, restoring a snapshot, or changing traffic, the isolated rehearsal should use that same interface and permission path. A private script on one engineer's laptop is not an operational control. The evidence should show that the designated operator role can complete recovery without acquiring permanent administrator access.

Koder.ai supports source export, deployment and hosting, snapshots, and rollback, so applications built there can use those concrete artifacts in this gate rather than treating recovery as a paragraph in the pull request. The same rule applies on any platform: run the recovery control and probe the restored application.

One required check should summarize all seven

The final merge decision should require one stable policy check that verifies the seven underlying results for the exact head commit. Individual job names change, matrix jobs multiply, and optional paths skip work. A small aggregator prevents branch protection from drifting away from the policy.

Have each gate emit a signed or platform attested result containing the commit, policy version, outcome, and evidence location. The aggregator rejects missing, stale, neutral, or cancelled results. It should never infer success from a job that did not report.

{
  "commit": "abc123",
  "policy": "merge-gates-v3",
  "results": {
    "tests": "pass",
    "codeql": "pass",
    "dependency_review": "pass",
    "secret_scan": "pass",
    "authorization": "pass",
    "migration_rehearsal": "not_applicable",
    "rollback_verification": "pass"
  },
  "decision": "allow"
}

Define a narrow override path because emergencies happen. Require a named maintainer, a second approver, a written reason, an expiry, and a follow-up issue. Do not let the agent request or approve its own exception. Report overrides in the same place as ordinary gate results so they remain visible after the incident passes.

These checks will add minutes to some pull requests and much longer to migration changes. That is acceptable when the time buys specific evidence. Run cheap gates early, cancel superseded commits, cache trusted build inputs, and reserve full environment rehearsal for relevant paths. Do not weaken a gate merely to make the dashboard green faster.

Start by making current behavior observable. Put all seven names in one policy file, connect each to a required result, and force skipped work to explain itself. The first agent pull request that cannot produce that record has found a hole in the delivery system before it found one in production.