smart-org/docs/plan/reports/agent-3-phase-3.md

Agent 3 — Phase 3 report: live CI/CD + demo server

Date: 2026-10-14 · Branch: dev

What was done

3.1 Tables (models_delivery.py + migration a3b4c5d6e7f3)

pipeline_runs, pipeline_steps (bounded 64 KB log tail), releases, app_runtimes (one row per project — the primary key is the project id, because D5 gives each app exactly one demo container). alembic upgrade head → downgrade -1 → upgrade head → check all clean, no autogenerate drift.

3.2 delivery_service.py — DeliveryHooks

• on_app_created allocates the demo port, claims <org-slug>.<PREVIEW_ROOT_DOMAIN> through the existing DomainRouterPort, creates the runtime row, ensures the workspace and queues the first build. Idempotent: a retry reserves no second port and creates no second run.
• on_branch_pushed only acts on dev (a task branch is work in progress) and demotes older queued runs for the project. A run that has already started is left alone — cancelling it mid-flight would strand a container.

The demo port ledger is the database, not the adapter

PreviewDomainRepository (the preview_ports table) is the source of truth, not the router adapters' in-memory maps. The API and the delivery worker are separate processes; an in-process port map would hand the same port to two projects the moment a second process started. Two repository methods were added for the switch: set_port (re-point a service at a new port) and delete_port (release a transient reservation).

Blue/green, done properly

The replacement container is started on a staging port, never the live one:

1. deploy_demo reserves a transient staging port (demo-staging-<run>) and starts the new container on it, detached, --rm-less so it survives the call.
2. healthcheck polls http://127.0.0.1:<staging><healthcheck_path> — 20 tries, 3 s apart, any 2xx/3xx passes.
3. route reconfigures Caddy to the staging port, then swaps the ledger (delete_port staging → set_port demo), updates the runtime row, and only then stops the previous container.

Order in step 3 is load-bearing and was found by a test: preview_ports.port is unique platform-wide, so the staging row must be released before the demo row can move onto the port it held.

A failure at any step removes the failed replacement and releases its port, leaving the previous container serving. test_a_failed_build_keeps_the_previous_demo asserts exactly that, including that route never ran.

3.3 Worker (workers/delivery.py)

run_pass drains the queued runs oldest-first, then polls remotes (gated by DELIVERY_POLL_SECONDS). Runs are claimed atomically: PipelineRunRepository.claim is a conditional UPDATE … WHERE status='queued', so two worker processes can never build the same run. The worker survives a failing run — it records the error and continues.

3.4 Live logs

_LogBatcher coalesces each step's output into at most one event per 500 ms, at most 200 lines per event, and keeps a bounded tail. Redaction happens on the whole accumulated tail, not per chunk — a secret split across two pipe reads would slip past a per-chunk filter, and this was a real bug the redaction test caught.

3.5 Router (delivery.py)

GET /runs, GET /runs/{id} (with steps), GET /releases, GET /runtime need only org membership; POST /runs, POST /runs/{id}/cancel, POST /releases/rollback require projects.manage and are audited.

Files changed

File	Change
app/domain/delivery/models.py (+__init__.py)	new — run/step/release/runtime + status enums
app/infrastructure/db/models_delivery.py	the four tables
alembic/versions/a3b4c5d6e7f3_delivery_infra.py	new migration
app/infrastructure/repositories/delivery.py	new — 4 repositories incl. claim
app/infrastructure/repositories/projects.py	set_port, delete_port
app/application/services/delivery_service.py	new — hooks, pipeline, blue/green
app/application/container_delivery.py	real hooks + build_delivery_service
app/application/container.py	re-exports build_delivery_service
app/workers/delivery.py	real drain + poll loop
app/interfaces/http/routers/delivery.py	7 routes
app/interfaces/http/deps.py	get_delivery_service
app/infrastructure/adapters/docker_sandbox.py	run_step (fresh build/demo container)
tests/test_delivery.py	new — 21 tests + 1 docker-marked
tests/test_foundation.py	the delivery worker test now redirects the session factory

Test output

apps/api:  415 passed, 1 skipped (was 394; +21, +1 skipped docker test)
           ruff check .  -> All checks passed!
           mypy app      -> Success: no issues found in 132 source files
alembic:   upgrade head / downgrade -1 / upgrade head / check -> all clean
apps/web:  67 passed (unchanged)

Deviations from the TODO

1. package runs through the same container runner rather than a host tar. It keeps every step going through one hardened path, and a packaging failure is recorded but does not fail the run (the demo serves from the work tree).
2. A staging port per run, swapped into the demo ledger entry. The TODO says "start the new container on a temporary second port" without saying how the port is chosen; a fresh reservation per run is the only option that cannot collide with a concurrent deploy of another project.
3. healthcheck and route do not call the container runtime. They are a poll and a config write, so they appear in pipeline_steps but not in the container-call list. The step sequence in pipeline_steps is the full eight, as specified.
4. The worker's remote poll is gated in-process. DELIVERY_POLL_SECONDS is honoured per worker process. With several workers each would poll on its own schedule; the poll is idempotent (it compares against the last succeeded sha) so the cost is duplicated network, not duplicated builds.

Open questions / CHANGE REQUESTS

• CHANGE REQUEST (for integration): build_repo_workspace(session) returns a port bound to the caller's session. DeliveryService holds it for the lifetime of one request/pass, which is correct, but the worker must keep building the service per pass rather than caching it. Noted in the Phase 2 report as well.

Known limitations

• The route step relies on the Caddy adapter's own validate → revert → reload. Its routes_fingerprint optimisation lands in Phase 4.
• A demo rollback moves the route but not the container, so rolling back to a sha whose container has already been replaced serves the current container's code under the older release label. Making the container match requires keeping one container per release, which the single-app_runtimes-row design deliberately does not do. This is the honest limit of "rollback = a pointer move" and is documented rather than hidden.
• No production deploy path exists yet (Phase 5), by design.