smart-org/docs/plan/reports/agent-2-phase-1.md

Agent 2 — Phase 1 report: engine core (protocol, state machine, pipelines)

Date: 2026-10-14 · Branch: dev (this repo's integration branch) · Status: done, all checks green

Environment note: the TODO file assumes the ext4 worktree (/home/mamad/work/smart-org-wt/agent2, branch agent2/ai-workforce). This checkout is the dev repo itself and already contains the other agents' work (org wizard, invites, delivery infra), so the engine is being implemented here directly on dev at the user's instruction. No locked file was touched; no CHANGE REQUEST is needed so far.

What was done

1.1 app/domain/workforce/pipeline.py (new, pure)

• Stage (13 values: the eight pipeline stages plus the side states pending_user_decision, error_review, paused, cancelled, failed).
• PIPELINES: plan_code_test (default), plan_code_review_test, code_test, analysis_only (no worktree, no merge). Each is the ordered tuple of stages ending in completed.
• TRANSITIONS — every legal stage move, with the reasoning in comments: a paused or parked task resumes through triage (no paused_from column exists, and re-triage is the safe route); merge_request is platform-executed and nobody acts in it; terminal stages never move.
• next_stage(pipeline, current, outcome) with Outcome (ok, revise, reject, ask_user, error, resolve). ok walks the pipeline list one step; the others have fixed targets — work sent back always returns to the developer.
• STAGE_ROLE → (role, mode) or None for stages with no AI turn. Manager modes are triage / final / error (plus chat), every other role is work.
• resolve_pipeline(manager_choice, task_kind, app_default, has_active_reviewer): priority manager > kind > app default > plan_code_test; a review pipeline without an active reviewer falls back to plan_code_test.
• TASK_KIND_DEFAULTS: feature → plan_code_test/medium, bugfix → code_test/low, refactor → plan_code_review_test/high, docs → code_test/low, analysis → analysis_only/low.
• STAGE_STATUS: stage → coarse tasks.status (new/running/waiting_approval/done/failed/ cancelled) so existing APIs and the old worker keep working against one truthful column.
• TaskPipelineState dataclass: pipeline, kind, risk, stage, cycle/max_cycles, branch, last_diff_hash/same_diff_count, retry_count, waiver_reason, parent/depends, version (optimistic lock).

1.2 app/application/engine/protocol.py (new)

• Parser for [[TAG attr="v" ...]]body[[/TAG]] and [[TAG attr="v"/]], tolerant of whitespace, single- or double-quoted values.
• A body is literal: an outer tag wins and the scanner moves past its close, so brackets inside a body (a source file full of [[) are never re-parsed.
• TagName enum for all 28 tags; ALLOWED_TAGS is the per-role/mode matrix from the TODO (manager triage/final/error/chat, planner, developer, reviewer, tester) plus the six tool tags granted to working roles only — the manager never edits code.
• Unknown tags, disallowed tags, stray closings, unclosed tags and malformed attribute fragments are dropped and reported as one error line each, in order, for the model to read next turn.
• BODY_TAGS: only tags that carry a body must be closed; a forgotten / on a bodyless tag ([[RUN cmd="pytest"]]) is read as self-closing instead of failing the reply.
• Limits: max 3 CREATE_TASK, max 3 team changes across HIRE+UPDATE_EMPLOYEE+FIRE together, max 3 KNOWLEDGE per reply. Extras are dropped and reported.
• Typed readers: AssignTag, TaskDraft, TeamChange, KnowledgeItem, EditBlock (<<<OLD … === … NEW>>>), plus edit_blocks_of which refuses a malformed block rather than half-applying it. ASSIGN refuses an unknown pipeline name.

1.3 app/infrastructure/db/models_engine.py (stub filled) + migration a2b3c4d5e6f2

Four tables, all 1:1/side tables — no column was added to the locked tasks / employees:

• task_pipeline_state (PK task_id), with the denormalised organization_id so the scheduler's hot query stays single-table.
• employee_profiles (PK employee_id): ai_model_id, effort, roles CSV, hired_by, busy_task_id.
• engine_turns: id, task/run/employee, role, mode, stage, prompt_chars, reply_excerpt (Text, bounded on write), tags JSON, tool_calls, ok, error_kind, started/finished.
• gate_results: id, task, gate, ok, detail (bounded), created.

Migration alembic/versions/a2b3c4d5e6f2_ai_workforce.py: alembic upgrade head applies cleanly and alembic check reports no drift between the models and the migration chain.

Deviation (documented in the migration docstring): the TODO pins down_revision = "a0f0e1d2c3b4", which was right while each agent had its own branch. On the merged dev chain the heads already form a line (a3b4c5d6e7f4 → a1b2c3d4e5f1 → d5e6f7a8b9c0), so this migration chains after d5e6f7a8b9c0. Result: one linear head (a2b3c4d5e6f2), no merge revision needed.

1.4 app/application/engine/state.py (new)

• advance(states, task_id, expected_from, to): checks legality in the domain first, then the repository's compare-and-set (WHERE task_id AND stage AND version) performs the move. A lost race raises ConflictError ("این کار هم‌اکنون توسط موتور دیگری در حرکت است"), an unknown task raises NotFoundError.
• park_dependents(states, task_id): a failed or cancelled dependency moves every dependent to pending_user_decision (agybot bug M1). The function refuses to act on a healthy task — the guard is in the function, not in the caller — and leaves dependents that already wait on a human alone. A lost race on a dependent is skipped, not fatal.
• app/infrastructure/repositories/engine.py (new): PipelineStateRepository (advance/patch/ list_at_stages/list_dependents), EmployeeProfileRepository, EngineTurnRepository, GateResultRepository. advance/patch verify the version actually moved rather than trusting rowcount, so the check holds on SQLite and PostgreSQL alike.

1.5 Tests

• tests/test_engine_pipeline.py (24 tests): all four pipelines walk manager review → completed step by step, each step also legal; sent-back work returns to the developer; illegal transitions refused; the stage/role/mode map; stage → coarse status; pipeline resolution priority and the reviewer fallback; plus DB-backed tests for the version bump, the illegal move, exactly one winner in a concurrent advance, the 404, the dependency-parking rule (and the healthy-task guard), optimistic patch, cascade deletes and the employee profile round trip.
• tests/test_engine_protocol.py (34 tests): paired/self-closing parsing, whitespace tolerance, nested brackets staying literal, tag order, unknown/disallowed/stray/unclosed/malformed rejections, the full per-role matrix, toolkit-for-working-roles-only, typed readers (assign/task draft/team changes/knowledge/edit blocks) and all three limits.

Checks (PLAN-v2 §7.4)

cd apps/api && .venv/bin/python -m pytest -q        → 594 passed, 1 skipped (whole suite; 58 new)
cd apps/api && .venv/bin/ruff check .               → All checks passed!
cd apps/api && .venv/bin/mypy app                   → Success: no issues found in 144 source files
alembic upgrade head (scratch sqlite)               → applies through a2b3c4d5e6f2, single head
alembic check (scratch sqlite)                      → No new upgrade operations detected.

The web suite was not touched (no UI in Phase 1).

Deviations & notes for the reviewer

1. Implemented on dev instead of an agent2/ai-workforce worktree (user instruction; the worktree path in the TODO is a Linux path unavailable here).
2. Migration down_revision chained to the merged head (above) to keep one linear chain.
3. task_pipeline_state.organization_id added beyond the TODO's column list: it is denormalised from the task so the scheduler never joins back, and it keeps the tenant filter single-table.
4. resolve_pipeline applies the no-reviewer fallback to an explicit manager choice too. A review stage nobody can act on is a dead end; the manager who wants a reviewer should hire one.
5. Team-change limit counts HIRE + UPDATE_EMPLOYEE + FIRE together (one budget per turn), which is what "max 3 team changes per turn" means.
6. New repository module app/infrastructure/repositories/engine.py (the engine columns live beside models_engine.py); repositories/workforce.py was left untouched.

Open questions / CHANGE REQUESTs

• None for Phase 1. Expected for later phases: org automation policy behind ModelGrantPort (TODO 4.1 says to assume enabled until then) and the worker hook for memory expiry.

Next

Phase 2 — engine/prompt.py, engine/tools.py (sandbox execution, git refusal, path safety), engine/runner.py (run_turn with the scripted fake model, error classification, usage events) and tests/test_engine_runner.py + tests/engine_fakes.py.