The enterprise vibe coding pilot in 30 days

An enterprise vibe coding pilot should prove that the team can operate, inspect, recover, and leave the platform under conditions that resemble production. Generating an attractive application quickly is useful, but it answers the least expensive question in the evaluation.

A contract should depend on recorded pass or fail results for source export, access control, data location, deployment, rollback, audit records, and developer handoff. If the vendor controls the test, explains away ambiguous results, or supplies missing steps during the final exercise, the pilot has measured vendor assistance rather than enterprise readiness.

The pilot measures exit cost as well as build speed

The pilot needs a frozen acceptance plan before anyone starts building. Otherwise every awkward result becomes a request for more time, a narrower interpretation, or a promise that the next release will fix it.

Choose one reference application that is small enough to finish but complicated enough to expose operational risk. It should have several user roles, tenant boundaries, persistent records, file handling, an external service, background work, secrets, and at least one database migration. A brochure site proves almost nothing about an enterprise application platform.

Record every test in an evidence file stored outside the platform. A simple structure keeps the result reviewable:

pilot:
  application: claims-intake-reference
  revision: 8f21c6a
  test_owner: enterprise-architecture
  vendor_observer: true
controls:
  source_export:
    result: pending
    evidence: []
    blocker_if_failed: true
  access_control:
    result: pending
    evidence: []
    blocker_if_failed: true
  data_location:
    result: pending
    evidence: []
    blocker_if_failed: true
exceptions:
  owner: procurement
  expires: 2026-09-30
  compensating_control: null

The revision identifies the exact application under test. Each evidence entry should point to material your team controls, such as an exported archive, terminal transcript, log file, identity configuration, recovery timing, or signed vendor response. Screenshots can support a result, but they rarely prove it alone because they omit requests, response codes, configuration history, and surrounding state.

Separate gates from preferences. Portability, tenant isolation, recoverability, data location, and audit integrity normally belong in the gate category. Editor convenience and generation speed may affect adoption, but a high score there cannot cancel a failed isolation test. Averaging all results into one cheerful score is a popular procurement mistake because it lets ten cosmetic passes conceal one dangerous failure.

Assign one enterprise owner to each control and one person who can declare failure. The vendor may observe and correct factual errors, but should not grade its own work. Record any assistance it gives. If vendor staff repair the export, modify a policy, or operate rollback, repeat the test without them before marking it passed.

A 30 day schedule works when the team tests evidence continuously. Freeze scope and build the reference application early, then reserve substantial time for destructive tests, clean environment rebuilds, identity failures, restore exercises, and handoff. Teams that develop until day 28 usually spend the closing meeting discussing features they never tested.

Source export must produce an independent build

Source export passes only when the enterprise can build, test, run, and modify the application in a clean environment without platform access. Possessing a directory full of code is not the same as having a portable application.

Export a pinned revision, record its checksum, and move it to a new repository controlled by the enterprise. Use a clean machine or disposable build worker with no vendor cookies, command credentials, package cache, generated files, or hidden environment variables. The receiving developer should have only the export and its documentation.

Run the repository's declared commands rather than commands supplied in a meeting. For a React and Go application, a transcript might have this shape:

$ npm ci
added 428 packages, and audited 429 packages
$ npm test
Test Suites: 18 passed, 18 total
$ go test ./...
ok   example/api/auth
ok   example/api/orders
$ go build ./cmd/server
$ ./server
configuration error: DATABASE_URL is required

That final error is a useful pass, not an embarrassment. It proves the program names a missing dependency instead of silently reaching a vendor service. After supplying documented configuration, the team should start the application, apply migrations, create a user, exercise an external integration through a test double, and run automated tests.

The Twelve-Factor App says an application should track one codebase in revision control and explicitly declare dependencies. Those rules remain useful, but they do not settle portability. Generated applications can declare public dependencies while still relying on proprietary identity brokers, deployment metadata, hosted functions, build plugins, or runtime endpoints. Your test must find those dependencies and classify which ones can be replaced.

Inspect the export for source maps, generated clients, migration files, test fixtures, build definitions, license notices, infrastructure configuration, and a dependency lock file. Search for hard coded service addresses, opaque binary components, copied secrets, and imports that resolve only inside the platform. The team must also know which artifacts it has the contractual right to use after termination. Technical possession cannot repair missing rights.

Database portability deserves its own check inside this gate. PostgreSQL documentation explains that pg_dump exports one database and produces a consistent snapshot, but it does not export cluster wide objects such as roles. A team that restores only the application database may discover that ownership and permission assumptions disappeared. Test schema creation, seed data, role recreation, extensions, and a restore into an enterprise controlled PostgreSQL instance.

Pass when an unfamiliar developer can reproduce the running system from the export using written instructions and replace every vendor runtime dependency or identify an accepted substitute. Fail when files are missing, builds call private services, schema history cannot recreate the database, secrets appear in the archive, or the vendor must intervene. A future export feature on a roadmap does not change the result.

Access control must survive direct requests

Access control passes when the server denies every unauthorized operation even if a user bypasses the generated interface. Hiding a button, route, or menu item tests presentation, not authorization.

Define roles and resources before generating the application. Use a small permission matrix that includes tenant boundaries and sensitive actions:

Run each denial through the browser and by calling the API directly. Change object identifiers, tenant identifiers, query filters, and request bodies. Try bulk endpoints separately because teams often protect the single record path and forget export, search, attachment, and batch update routes. Verify server enforcement after a client change removes all visual restrictions.

OWASP Application Security Verification Standard 4.0 places access control verification on trusted service layers and expects access to be denied by default. That advice matters more in generated systems because a polished interface can create false confidence. I have seen teams accept a role demo where the restricted user lacked an edit button, then discover that the same user could submit the edit request manually.

Authentication and authorization require separate verdicts. Authentication establishes who presented a credential. Authorization decides whether that identity may perform this action on this object now. Single sign on can pass while object authorization fails across every tenant.

Connect the enterprise identity provider and test joiner, mover, and leaver cases. Create a user, change the user's group, remove an elevated role, disable the account, and revoke active sessions. Measure how long each change takes to affect the application. Test emergency local accounts, service identities, API credentials, and platform administrators rather than limiting the exercise to ordinary application users.

OpenID Connect Core defines the sub claim as a locally unique identifier that is never reassigned within the issuer. Store and audit that stable identifier alongside a readable login name. Email addresses and display names change, so using them alone can corrupt ownership history or make two different people appear identical after account reuse.

Fail the gate if any lower privilege user can cross a tenant boundary, if administrative access bypasses recorded approval, if removed privileges survive beyond the agreed interval, or if the team cannot explain who can reach production data. Treat a vendor administrator as an access path, even when the access happens through support tooling rather than the application.

Data location needs a component level map

Data location passes only when the team can account for every material copy, processor, transfer, backup, and support path. Selecting a country for the application workload proves the location of that workload, not the location of all related data.

Start with categories rather than one vague question about residency. Include customer records, uploaded files, credentials, prompts, generated source, platform metadata, logs, traces, model requests and responses, backups, support attachments, and analytics. For each category, record where it enters, where it rests, which service processes it, how it moves, how long it remains, and who can access it.

This distinction catches a routine mistake: data residency, data processing location, and transfer control are related but different claims. A database may reside in one country while model inference, telemetry analysis, support access, or disaster recovery creates a transfer elsewhere. Procurement language that says data is "hosted" in a region often leaves those paths unanswered.

Use a seeded marker for each category, such as a unique project string or synthetic record identifier. Ask the vendor to show where that marker can appear in application storage, operational logs, backups, support systems, and model processing. Do not place real personal or regulated data in the pilot until legal and security reviewers accept the map.

Request documentary evidence for subprocessors, processing regions, support access, retention, deletion, encryption ownership, and disaster recovery. A verbal assurance from a sales call should remain an open item. If the platform uses several model providers, establish whether the enterprise can select or restrict them, where each processes requests, and whether prompts or outputs receive any provider retention.

Test deletion as an observable process. Delete a seeded record and then ask what remains in active storage, logs, snapshots, backups, and exported audit material. Immediate removal from every backup may be neither possible nor desirable, but the provider should state the retention and eventual expiry behavior precisely. Your legal team decides whether that behavior fits the obligation; the pilot team records what actually happens.

Pass when the data map is complete enough for security, privacy, and legal reviewers to approve each path and when configuration matches the documented placement. Fail when the provider answers only for the primary database, cannot identify model processing locations, permits unexplained support access, or treats backup geography as confidential. An unresolved location is not evidence of an acceptable location.

Deployment must be repeatable outside one browser session

Deployment passes when the team can release a pinned revision through a documented, repeatable process and prove exactly what reached each environment. A successful preview URL does not establish release control.

Create separate test and production like environments with distinct identities, secrets, databases, domains, and approval rules. The same source revision should move between them without copying hidden editor state. Configuration may differ, but the difference must be declared and reviewable.

Deploy the same revision twice from a clean state. Capture source revision, dependency lock checksums, build result, migration version, configuration references, approver, deployer, start and finish times, target environment, health check result, and resulting release identifier. Then compare the records. If the same input produces materially different software, the team needs an explanation before production use.

A release record can use this compact shape:

{
  "release_id": "rel-1042",
  "source_revision": "8f21c6a",
  "environment": "pilot-prod",
  "schema_version": "20260728_03",
  "requested_by": "oidc:00u81c",
  "approved_by": "oidc:00u19a",
  "result": "succeeded",
  "health_check": "passed"
}

Make the deployment fail on purpose. Remove a required secret, break a migration, deny access to an external service, and fail a health check. The system should stop safely, state which phase failed, preserve diagnostic evidence, and avoid presenting a partial release as healthy. A deployment UI that reports only "failed" leaves operators guessing during an incident.

Test separation of duties if policy requires it. The person who changes production code should not silently grant personal approval or alter the audit record. Also determine whether platform administrators, generated application administrators, and cloud operators have separate authority. These roles often collapse during a demo because one account creates everything.

Pass when another authorized operator can deploy a selected revision, see its configuration references, identify its approvals, and confirm its health without vendor help. Fail when deployment depends on the original chat session, an unnamed latest version, personal credentials, mutable generated artifacts, or undocumented manual work.

Rollback must cover code, schema, data, and side effects

Rollback passes when it restores a defined service state within the agreed time while keeping data loss within the agreed limit. Reverting application code alone can make an incident worse if the database or an external side effect has already moved forward.

Set a recovery time objective and recovery point objective before the exercise. Recovery time measures how long the service may remain disrupted. Recovery point measures how much committed data the business can lose. Teams regularly say "rollback took six minutes" without checking whether recent records vanished, which reports only half of the result.

Use an intentionally incompatible release. Version A stores a customer status as text. Version B migrates it into a new table, changes the API, emits a notification through a test service, and begins a background conversion. Add records before, during, and after the release, then interrupt the conversion and trigger rollback.

The first failure usually appears when Version A starts against the Version B schema. The old code expects a column that the migration removed. Restoring only the application therefore produces a second outage. Restoring a database snapshot may revive Version A, but it can discard records committed after the snapshot. Replaying those records can duplicate the external notification unless the integration uses an idempotency mechanism.

The team must choose a recovery design rather than assume one method fits every release. Compatible expand and contract migrations can let old and new code run against the same schema. A forward repair may be safer than reversal after an irreversible data transformation. Snapshot restoration can work when the business accepts its recovery point and the team has tested replay. Record which method applies to each migration class.

During the exercise, capture the detection time, decision time, operator, approval, application version, schema version, snapshot identity, restored records, lost records, replay result, queued jobs, and external calls. Validate business behavior after technical health checks pass. A green process monitor does not prove that permissions, balances, attachments, or workflow state remain correct.

Pass when operators execute the documented recovery path without vendor intervention, meet both recovery objectives, reconcile records, and explain every external side effect. Fail when rollback is an unlabeled button, schema compatibility is unknown, snapshots cannot be restored into an isolated environment, or the team cannot calculate data loss.

Audit records must reconstruct a disputed action

Audit capability passes when an investigator can determine who did what, to which object, when, from where, with what result, and under which authority. A chronological activity feed made for project collaboration is not necessarily an audit record.

NIST SP 800-53 Revision 5 separates account management in AC-2 from event logging and audit record generation in the AU controls. That separation is sensible. Identity administration determines which principal had access, while audit generation records how that principal used it. You need both histories to investigate a disputed deployment or data export.

NIST AU-3 calls for records that contain the event type, time, place, source, outcome, and associated identity. For this pilot, add tenant, target object, request correlation, previous and new security relevant values, authentication context, and approval reference where applicable. Do not record secret values, session tokens, full prompts containing restricted data, or sensitive record bodies merely to make the log look complete.

A useful event should resemble this:

{
  "event": "role.assignment.changed",
  "time": "2026-07-28T14:03:22Z",
  "actor_sub": "oidc:00u81c",
  "actor_role": "platform-admin",
  "tenant": "tenant-204",
  "target": "user-771",
  "change": {"from": "viewer", "to": "manager"},
  "outcome": "success",
  "request_id": "req-9918",
  "approval_id": "apr-118"
}

Generate events for authentication failures, role changes, session revocation, secret access, source export, data export, configuration change, deployment, rollback, snapshot use, domain change, support access, audit export, and changes to audit settings. Test failed attempts as well as successes. An investigator often needs the denial that preceded a successful privilege change.

Change a user's display name and email, then verify that earlier events remain tied to the stable identity. Compare platform events with application events and identity provider records through a shared request or session reference. Check clock consistency because a five minute skew can reverse the apparent order of approval and deployment.

Attempt to alter, delete, disable, and overflow the audit stream with the strongest pilot role. Verify retention, export format, pagination, time zone, filtering, and the delay before records become searchable. Export the records into enterprise controlled storage and confirm that the export contains stable field names suitable for investigation. A downloadable spreadsheet can help an analyst, but it should not be the only representation if cells truncate structured values.

Pass when a reviewer who did not attend the test can reconstruct a seeded incident from exported evidence and detect attempts to weaken logging. Fail when administrators can erase their own trail, identities cannot be correlated, failed actions disappear, support activity is invisible, or retention depends on an undocumented plan tier.

Developer handoff exposes hidden platform dependence

Developer handoff passes when a developer who did not build the pilot can maintain and release the exported application without the original builder or the platform. Code readability matters, but successful ownership transfer is the stronger test.

Give the receiving developer a clean environment, the source export, architecture notes, configuration reference, data model, migration history, test instructions, deployment procedure, recovery procedure, dependency inventory, and known limitations. Remove platform access during the exercise. The original builder may observe but should not answer implementation questions until time and blockers have been recorded.

Seed one ordinary defect, such as a tenant filter missing from a report query. Ask the developer to reproduce it, locate the authorization path, add a regression test, repair the query, make a small schema change, run the full test suite, deploy to the test environment, and explain the rollback path. This sequence exposes generated code that looks plausible but lacks consistent boundaries or test seams.

Judge the handoff through evidence, not style preference. Record setup time, undocumented dependencies, failed commands, unclear ownership, test coverage around the changed path, review findings, deployment result, and questions that required vendor knowledge. Require the developer to identify generated areas that can be edited safely and areas that the platform may overwrite after later chat changes.

Pay close attention to regeneration. Make a conventional code edit after export, import or reconnect the project if supported, and then request a platform generated change nearby. Determine whether the platform preserves, rewrites, duplicates, or silently conflicts with the manual edit. Teams need a declared operating model for mixed human and generated work; "developers can edit the code" does not explain what happens on the next generation.

Fail the handoff if the application lacks repeatable tests, the data model exists only in chat history, generated modules have no stable boundaries, manual changes vanish, or deployment still requires the first builder's account. Documentation generated by the same system can help, but the receiving developer must verify it against the code and runtime.

A clean handoff does not require every developer to admire the generated style. It requires a competent developer to predict change impact, test the behavior, review security sensitive paths, and operate the release without private knowledge.

The contract should preserve the evidence you proved

The contract should proceed only when every blocking control passes or the enterprise formally accepts a specific, time limited exception with a compensating control. Procurement should attach the evidence definitions to the commercial promise rather than rely on feature names.

For a Koder.ai evaluation, put its source export, deployment, hosting, custom domains, snapshots, rollback, planning mode, and country specific application placement through the same evidence rules; a feature name is an invitation to test, not proof.

Build the decision record around seven control verdicts. For each one, include the tested revision, environment, evidence owner, observed result, vendor assistance, defect reference, retest result, and contract consequence. Keep the raw artifacts in enterprise controlled storage so a later reviewer can distinguish what the team observed from what the parties discussed.

Do not turn an unresolved blocker into a vague contractual commitment to "support" portability, residency, or recovery. Define the artifact or behavior: a complete source export within a stated process, named processing locations, exportable audit fields, a tested restore path, or continued access to required build materials after termination. Set a remedy and an exit right for claims that matter to adoption.

Also protect the handoff conditions. Specify ownership and permitted use of generated source, access to exports, data return, deletion behavior, configuration retrieval, audit export, transition assistance, and the treatment of applications already deployed when the relationship ends. Commercial tiers can differ, but the team should know which tested controls depend on the selected tier before signing.

A conditional pass needs an owner and expiry date. Retest the actual correction in the same environment and update the original evidence record. A slide describing planned functionality does not close a failed test, and a demonstration on the vendor's prepared project does not prove the correction applies to yours.

The pilot has done its job when the decision remains clear after the excitement of the build has faded. If the team can export, restrict, locate, deploy, recover, investigate, and hand off the application under its own control, the contract rests on observed capability. If any of those gates still depends on an explanation, record the failure while it is cheap.