smart-org/docs/plan/PLAN-v2.md

Smart Org v2 — Plan (shared by all agents)

Date: 2026-10-13 · Base branch: dev · Read this file fully before your own TODO file.

This file is the single source of truth for the v2 work. Three agents work in parallel, each from its own TODO file:

File	Agent	Area
docs/plan/TODO-0-foundation.md	Claude (DONE before the agents start)	shared contracts, DB foundation, wiring
docs/plan/TODO-agent-1-org-platform.md	Agent 1	invites, org wizard, app profile, sharing, owner model grants, memory, MCP
docs/plan/TODO-agent-2-ai-workforce.md	Agent 2	admin (manager) AI, employees, dynamic pipelines, agent loop, test/commit/push
docs/plan/TODO-agent-3-delivery-infra.md	Agent 3	docker sandbox, git workspaces, live CI/CD, demo server, Caddy domains, SSH/FTP deploy

---

1. Product vision (what the owner asked for)

1. One platform owner controls the whole application and its infrastructure: AI accounts, git, application rules, adding users, domains, sandboxes, models. Nobody else touches infra.
2. An invited user lands on an organization creation wizard. The wizard connects the org to our git (Gitea) or the user's own git and asks for the application type, language, framework, and so on. One organization = exactly one application. A user can create several orgs and can share an org with other people so they help manage it.
3. The wizard also asks (optionally) for SSH or FTP details to deploy the app to production. The platform can also run the app inside a sandbox as a live demo.
4. The user gives orders to the admin AI (manager). The manager brings in AI employees, built only from the models the owner grants to that org. They do the task, test it and push it.
5. A live CI/CD mechanism shows the deployment on our local demo sandbox server, live.
6. Caddy routes different domains to the applications.
7. This is the web version of /projects/agybot (Telegram bot). Reuse agybot's proven ideas, not its code.
8. Every org improves itself through memories: how its users like to manage the org, and how the app's logic and code work. It also exposes MCP so AI agents can control our app and read or push data.
9. Main difference from agybot: users never touch infrastructure or AI accounts. They only give orders and tasks. Tasks run through dynamic pipelines.

2. What already exists (audited 2026-10-13, the code is the truth, not old TODO.md ticks)

• Transport: the API is WebSocket-RPC only. app/main.py exposes only /health* and /api/ws. Every handler lives in an internal FastAPI app (app/interfaces/realtime/rpc_app.py) that is reached in-process. Realtime push uses channels org:<id>, task:<id>, run:<id>, thread:<id>, session:<id>, user:<id>, admin (app/application/ports/realtime.py, authz in app/interfaces/realtime/authz.py).
• Roles: platform roles owner (exactly one, enforced by a DB partial unique index), manager and user. Org roles are owner > admin > manager > member > viewer. There is no public signup. Accounts come from the owner (/admin/users) or from an org invite link.
• Admin console: agy (Google OAuth) accounts with pool rotation and 429 cooldown, plus model catalog, routing policies, rate cards, usage and error console.
• Projects: projects holds only name/slug/description/default_branch. There is no app profile and no "1 org = 1 app" rule. Gitea adapter (real) and mock. Preview domains through the Caddy adapter (real) and mock.
• Sandbox: full CRUD API, but only Daytona/E2B adapters (untested live) and a mock that runs nothing.
• Workforce: templates, employees, threads, tasks, runs. The run engine makes ONE LLM call per task and stores the text. Tool calls are recorded, never executed. No employee edits code, runs tests, commits or pushes.
• Missing entirely: org wizard, app profile, external git, SSH/FTP deploy, CI/CD, demo server, docker sandbox, memory system, MCP, per-org model grants.
• Tests: about 233 API tests (pytest, DB via Base.metadata.create_all in tests/conftest.py) and 64 web tests (vitest). Five-step alembic chain ending at d4e5f6a7b8c9.

3. What we take from agybot (concepts only, never import it)

agybot	v2 equivalent	Owner
Roles manager, planner, programmer, reviewer, tester (+ optional critic)	same role keys	Agent 2
[[TAG attr="..."]]body[[/TAG]] protocol, per-role allowed tags	same, in app/application/engine/protocol.py	Agent 2
Pipelines plan_code_test (default), plan_code_review_test, code_test, analysis_only	same names	Agent 2
Pipeline choice: manager's pipeline= > task-kind default > app default; review pipeline needs an active reviewer	same	Agent 2
Task statuses pending_manager_review → pending_analysis → pending_implementation → pending_review → pending_test → pending_final_review → merge_request → completed (+ pending_user_decision, error_review, paused, cancelled, failed)	same, with optimistic expected_from	Agent 2
Gates: real test run after WORK_REPORT, green gate (waiving needs a written reason), diff guard, no-progress stop (2 identical diffs), max cycles, transient-error retries, escalate model on red	same	Agent 2
Worktree per task task/<id>, merge into dev, push	same, through RepoWorkspacePort	Agent 3 (port), Agent 2 (use)
Manager HIRE/UPDATE_EMPLOYEE/FIRE (max 3 changes per turn, max 20 employees), only on org-shared models	same, only on owner-granted models	Agent 2
Org knowledge [[KNOWLEDGE]] (max 3 per turn, caps 1,500 chars/key and 6,000/org), lessons with 30-day TTL, project summaries per task	memory system	Agent 1
app_control_mcp / remote_dev_mcp	MCP gateway with scoped tokens	Agent 1
caddy_manager (single-label hosts under root, localhost targets, validate→revert→reload)	existing CaddyDomainRouterAdapter	Agent 3
ssh_manager.push_project, ftp_manager.deploy_project	deploy adapters	Agent 3
org_live_stream (ring buffer, polled)	realtime push events (no polling)	Agents 2 & 3

agybot bugs we must NOT repeat: deploy publishing the wrong code silently; git_publish stuck mid-merge; observe mode applying real state changes; a failed/cancelled dependency deadlocking its dependents; manager chat cost not counted in the budget; git operations without the repo lock; about 870 silent except Exception blocks; plaintext credentials; the AI running directly on the host; git status --porcelain hiding new dirs (always use -uall).

4. Architecture decisions (fixed, do not re-decide)

• D1. 1 org = 1 app. The app is the existing projects row (the table name stays). projects gets UNIQUE(organization_id). The UI calls it "Application" (اپلیکیشن). Route /projects becomes /app.
• D2. App profile. These are columns on projects: app_type, language, framework, runtime_version, package_manager, setup/build/test/start commands, app_port, healthcheck_path, git_source (platform|external), external_repo_url, wizard_completed_at. External git credentials and deploy targets live in their own tables, encrypted at rest with the existing crypto helper. They are never returned raw to the frontend (only has_secret: true and masked values).
• D3. Owner model grants. org_model_grants(org_id, ai_model_id, daily_credit_cap, is_active). The engine may only hire or run on granted models. Users never see model names; the UI shows the employee's role and a neutral "tier" label.
• D4. Execution engine = in-process agent loop. It uses the existing ModelProviderPort (owner's agy accounts, pool rotation) with the agybot tag protocol for actions and tools. Tools run inside the org's sandbox container through SandboxProviderPort. There are no agent CLIs in v1, so AI credentials never leave the server and users never see them.
• D5. Sandbox = local Docker (SANDBOX_PROVIDER=docker, the binary is configurable so podman works too). Per app there is one workspace container (employees work in it) and one demo container (runs the latest green build). Hardened: non-root, --cap-drop ALL, no-new-privileges, cpu/mem/pids limits, ports published on 127.0.0.1 only.
• D6. Git is done by the platform, never by the AI. RepoWorkspacePort runs git on the host with a gitdir that is never mounted into the container. Git is always called with explicit --git-dir/--work-tree, hooks disabled (core.hooksPath=/dev/null), -uall, core.quotePath=false, and a per-repo lock. Remote credentials go through GIT_ASKPASS or a temporary key file and never appear in a URL, log or exception.
• D7. CI/CD is platform-native. A push to dev (by the engine or seen by polling the remote) creates a pipeline_run. Steps: checkout → setup → build → test → package → deploy demo → healthcheck → route. Logs stream live as realtime events on org:<id> and a bounded tail is stored. A failed run keeps the previous demo running.
• D8. Production deploy is always human-triggered (org owner/admin, with confirmation and audit). SSH: release dir + current symlink + post-deploy command + rollback. FTP/FTPS: upload a release, keep the previous one for rollback. User-supplied hosts are SSRF-checked (no private, loopback or link-local IPs unless the owner allowlists them). SSH host key is pinned on first successful test (TOFU).
• D9. Domains. Each app gets an automatic demo host <org-slug>.<PREVIEW_ROOT_DOMAIN>. Only the owner can attach extra hosts. Targets are always 127.0.0.1:<allocated port>. The existing rules in app/domain/platform/rules.py stay authoritative.
• D10. Memory (org_memories) is the org's long-term self-knowledge: rules, preferences, lessons, facts and the code map, with caps, dedupe, TTL for lessons and pinning. Users can view, edit, pin and delete it. The engine reads it through MemoryPort.build_context and feeds it through MemoryPort.on_task_finished.
• D11. MCP gateway is a separate HTTP entry point (services/mcp-gateway) using the official mcp Python SDK (streamable HTTP). Bearer tokens are hashed, scoped, org-bound and expire. Every call is audited and org-enforced.
• D12. Platform invites. The owner can create an invite link for a new platform account, which becomes role manager and may create orgs. A logged-in manager with no org is sent to the wizard. Org invites (join an existing org) stay as they are.
• D13. Automation safety. Tasks run automatically up to merge into dev, followed by a demo deploy. Nothing goes to production without a human click. No code path may flip that default.

5. Contracts between agents (created by Claude in TODO-0, then FROZEN)

All contracts live in apps/api/app/application/ports/ and are typing.Protocols with a working default implementation, so each agent can test alone. Agents must not change a contract file. If you need a change, write a CHANGE REQUEST in your phase report, keep coding against the current signature, and add a local adapter or shim in your own files.

# ports/repo_workspace.py  — implemented by Agent 3, used by Agent 2 (and Agent 1 for test_remote)
class RepoWorkspacePort(Protocol):
    async def ensure_workspace(self, *, project_id: str) -> WorkspaceRef: ...            # clone/fetch app repo on host
    async def create_task_worktree(self, *, project_id: str, task_id: str, base: str = "dev") -> WorktreeRef: ...
    async def diff(self, *, worktree: WorktreeRef) -> DiffResult: ...                   # text + changed files (-uall)
    async def commit(self, *, worktree: WorktreeRef, message: str, author: str) -> str | None: ...  # sha or None if clean
    async def sync_with_base(self, *, worktree: WorktreeRef) -> MergeOutcome: ...       # merge base into task branch
    async def merge_into(self, *, project_id: str, branch: str, target: str = "dev") -> MergeOutcome: ...
    async def push(self, *, project_id: str, branch: str) -> PushOutcome: ...
    async def remove_worktree(self, *, worktree: WorktreeRef) -> None: ...
    async def head(self, *, project_id: str, branch: str) -> str | None: ...
    async def test_remote(self, *, url: str, credentials: RemoteCredentials) -> RemoteCheck: ...
# WorktreeRef has: project_id, task_id, branch, host_path, sandbox_path (path inside the workspace container)

# ports/delivery_hooks.py — implemented by Agent 3, called by Agent 1 (app created) and Agent 2 (pushed)
class DeliveryHooks(Protocol):
    async def on_app_created(self, *, organization_id: str, project_id: str) -> None: ...
    async def on_branch_pushed(self, *, organization_id: str, project_id: str, branch: str, sha: str, source: str) -> None: ...

# ports/deploy_target.py — implemented by Agent 3, called by Agent 1's wizard
class DeployTargetTester(Protocol):
    async def test_connection(self, *, target: DeployTargetSpec) -> DeployCheck: ...    # ok, message, fingerprint

# ports/memory.py — implemented by Agent 1, used by Agent 2
class MemoryPort(Protocol):
    async def build_context(self, *, organization_id: str, role: str, query: str, budget_chars: int = 2500) -> str: ...
    async def record(self, *, organization_id: str, kind: str, content: str, source: str,
                     evidence_task_id: str | None = None, key: str | None = None) -> str | None: ...
    async def on_task_finished(self, *, organization_id: str, task: TaskOutcome) -> None: ...

# ports/model_grants.py — implemented by Agent 1, used by Agent 2
class ModelGrantPort(Protocol):
    async def granted_models(self, *, organization_id: str) -> list[GrantedModel]: ...   # id, capabilities, tier, caps
    async def is_granted(self, *, organization_id: str, ai_model_id: str) -> bool: ...

# ports/workforce_bootstrap.py — implemented by Agent 2, called by Agent 1's wizard
class WorkforceBootstrapPort(Protocol):
    async def bootstrap_team(self, *, organization_id: str, project_id: str) -> None: ...  # creates the manager + default team

Existing contracts that stay as they are: SandboxProviderPort (Agent 3 adds the docker adapter), GitProviderPort, DomainRouterPort, ModelProviderPort, EventPublisher/RealtimeEvent.

Realtime event names (use the existing RealtimeEvent + get_publisher()):

Channel	Event types	Producer
task:<id>	task.status, task.step, task.gate	Agent 2
run:<id>	run.log, run.tool, run.status	Agent 2
org:<id>	task.created, team.changed	Agent 2
org:<id>	pipeline.queued, pipeline.step, pipeline.log, pipeline.status, release.live, deploy.status	Agent 3
org:<id>	memory.changed, org.updated	Agent 1

Log events are batched (at most one event per 500 ms per stream, max 200 lines per event).

6. File ownership (no two agents edit the same file)

Area	Agent 1	Agent 2	Agent 3
API routers	app_setup.py, memory.py, mcp_tokens.py, admin_grants.py, organizations.py, invites.py, admin_users.py, auth.py, knowledge.py	workforce.py	projects.py, sandbox.py, delivery.py, admin_domains.py
Services	org_setup_service.py, memory_service.py, model_grant_service.py, mcp_service.py, platform_users.py, knowledge_service.py	workforce_service.py, app/application/engine/***	project_service.py, sandbox_service.py, delivery_service.py, deploy_service.py
Adapters	—	—	everything in app/infrastructure/adapters/ (incl. registry.py, mock.py)
Repositories	identity.py, knowledge.py, memory.py*	workforce.py	projects.py, sandbox.py, delivery.py*
Domain	identity/, knowledge/, memory/, projects/catalog.py	workforce/	platform/, sandbox/, projects/models.py
DB tables	db/models_org.py*	db/models_engine.py*	db/models_delivery.py*
Wiring	container_org.py*	container_engine.py*	container_delivery.py*
Workers	—	workers/ai_runtime.py	workers/sandbox_gateway.py, workers/delivery.py*
Other API	app/interfaces/mcp/***, services/mcp-gateway/**	—	—
Web pages apps/web/src/app/	onboarding/, accept-invite/, (app)/settings/, (app)/memory/, (app)/knowledge/, (app)/admin/users/, (app)/admin/model-grants/, (app)/admin/mcp/*, (app)/admin/organizations/	(app)/manager/, (app)/tasks/, (app)/workforce/, (app)/traces/, (app)/dashboard/	(app)/app/, (app)/projects/ (delete after moving), (app)/admin/domains/, (app)/admin/sandbox/*, components/sandbox-panel.tsx
Web API client	lib/api-org.ts*	lib/api-engine.ts*	lib/api-delivery.ts*
Tests	tests/test_org_*.py, test_memory*.py, test_mcp*.py, test_model_grants*.py, test_invites.py, test_platform_hierarchy.py, web src/test/org-*.test.tsx	tests/test_engine_*.py, test_workforce*.py, test_workers.py, web src/test/engine-*.test.tsx	tests/test_docker_*.py, test_git_workspace*.py, test_delivery*.py, test_deploy*.py, test_project_repo.py, test_domain_isolation.py, test_platform_rules.py, web src/test/delivery-*.test.tsx
Alembic (one file per agent, more only if chained off your own)	a1b2c3d4e5f1_org_platform.py	a2b3c4d5e6f2_ai_workforce.py	a3b4c5d6e7f3_delivery_infra.py

* = new file (Claude creates an empty stub in TODO-0, the agent fills it).

Foundation-owned shared helpers (read/call only, never edit): app/infrastructure/db/columns.py (pk(), created() for your models_*.py files; never import from db/models.py there, it causes an import cycle), app/infrastructure/repositories/app_profile.py (AppProfileRepository: read the app profile, or write it with update_profile; this is how Agent 1 writes and Agents 2 and 3 read build/test commands, port, healthcheck), app/domain/projects/profile.py (AppProfile), app/application/ports/model_grants.py::tier_for. Wiring functions are in container_org.py / container_engine.py / container_delivery.py: you replace the body of your own builders (names and return types are frozen); container.py re-exports them.

Locked files (Claude only): db/models.py, container.py, rpc_app.py, workers/__main__.py, ports/*.py contracts, tests/conftest.py, pyproject.toml, package.json, pnpm-lock.yaml, apps/web/src/lib/api.ts, apps/web/src/lib/nav.ts, components/AppShell.tsx, alembic/env.py, root README.md/PLAN.md/TODO.md, docs/plan/PLAN-v2.md, every other agent's TODO file.

Append-only blocks. app/core/config.py and .env.example each have three marked blocks (# >>> agent-1 … # <<< agent-1, and so on). You may add lines only inside your own block.

Routers for every new area are already registered in rpc_app.py (empty stubs: app_setup, memory, mcp_tokens, admin_grants, delivery, admin_domains); the delivery worker is registered as delivery in workers/__main__.py. The admin console tab list now comes from ADMIN_ITEMS in lib/nav.ts (the layout no longer has its own copy). Each new migration uses down_revision = "a0f0e1d2c3b4" (the foundation). Claude adds the merge revision when the branches are merged.

7. Working rules for every agent

0. Environment: the main repo is on an exFAT disk (/projects, no symlinks, so no venv/node_modules there). Your worktree lives on ext4 under /home/mamad/work/smart-org-wt/<agent> with .venv and node_modules already installed. Use pnpm (in ~/.local/bin) and apps/api/.venv/bin/python. Python is 3.13 via uv.
1. Work only in your own git worktree and branch (given in your TODO file). Never cd outside it and never write to /tmp. Put scratch scripts in ./_scratch/ inside your worktree and delete them at the end.
2. Do NOT commit, push, merge or rebase. Claude reviews and commits each phase.
3. At the end of each phase, stop and write docs/plan/reports/agent-<N>-phase-<K>.md containing: what you did, files changed, test output summary, deviations, open questions, and CHANGE REQUESTs. Then wait.
4. Before declaring a phase done, run all checks (not only your tests) and paste the summary lines:

   cd apps/api && .venv/bin/python -m pytest -q && .venv/bin/ruff check . && .venv/bin/mypy app
   cd apps/web && pnpm test && pnpm exec tsc --noEmit && pnpm lint
   

   Every existing test must stay green. Never delete or weaken an existing test to make it pass. If an existing test encodes behaviour that this plan changes on purpose, update it and explain why in the report.
5. Every multi-tenant query is filtered by organization_id. Every destructive or production action has a confirmation and an audit event. No secret ever goes to the frontend, a log, an exception message, a realtime event or an LLM prompt.
6. No silent except Exception: pass. Catch narrow errors, or record them through app.infrastructure.error_log.
7. Users (non-owner) never see model names, providers, accounts, containers, ports or raw infra errors. Translate them to plain Persian messages.
8. UI language is Persian and RTL. Reuse packages/ui components and the existing motion classes.
9. Match the existing code style: hexagonal layers (domain has no FastAPI/SQLAlchemy imports), type hints everywhere, docstrings in the same tone as nearby code.
10. Do not run agy, do not call real external services in tests (use fakes). Integration tests that need docker, network or real git remotes must be skipped unless an env flag is set.

8. Integration (Claude, after the agents)

1. Review each phase report and diff, run the full checks, and commit on the agent's branch.
2. Merge the branches into dev one by one: Agent 3 → Agent 1 → Agent 2. Run the full tests after each merge.
3. Add the alembic merge revision for the three heads and run alembic upgrade head + alembic check.
4. Replace the Null/default contract implementations in container.py with the real ones and run the end-to-end smoke test: wizard → app created → demo domain → manager order → task runs → tests → push → pipeline → demo live → production deploy (to a local SSH/FTP test server).
5. Update README.md, TODO.md and the architecture doc, then commit.

9. Status (2026-10-14)

Written on 2026-10-14 from the code, an alembic run on a scratch SQLite database, and the statuses in todo-finish-plan.md. A box is ticked only when its task is status: done or its check passed.

• §8.1–8.3 (partly done)
  • Alembic: single head b9c0d1e2f3a4. upgrade head, a downgrade to d4e5f6a7b8c9 and upgrade head run cleanly, and alembic check reports no new operations.
  • The agent2/ai-workforce and agent3/delivery-infra branches are contained in dev.
  • agent1/org-platform (6 commits, 0016bb4..051bf1b) is not an ancestor of dev, and its committed tree differs from dev in 138 files. Its phases reached dev as separate commits (5095a29, 5fb85d6, dee2f72). Open: confirm that dev has everything from Agent 1, then merge or retire the branch.
• §8.4 done through T1–T26; T27 is this docs pass.
  • Double execution fixed (T2): the legacy run drain skips tasks that own a pipeline state.
  • Billing is prepaid (D-B): the owner credits each org, every usage event debits the wallet once, and a zero, negative or missing wallet pauses engine turns and the manager chat (T5–T13).
  • The engine honours the owner's org policy: automation switch, monthly credit budget and max_employees (T1, T4, T5).
  • A new organization gets a sandbox session that the engine starts on demand (T6).
  • End-to-end smoke test tests/test_v2_e2e_smoke.py (T7; closer to reality in T26): a thread made without a project, the real AiRuntimeWorker.drain(), and a sandbox session that the engine starts. The production-deploy leg is not in it.
  • Review fixes from the integration pass: the manager gives a task the org's project when the thread has none (T20); a review pipeline is chosen only with an active reviewer (T21); the scheduler tick reads a wide window, so skipped tasks cannot starve other organizations (T22); archived (fired) employees no longer count toward the hire cap (T23); a project id from another organization gets a 404 on threads and tasks, and the engine never uses another tenant's sandbox (T24); an explicit next (a pending invite) beats the onboarding redirect at login (T25).
  • Fixed: the HTTP message route POST /api/organizations/{org}/threads/{thread}/messages built its manager without pipeline states and employee profiles (workforce_service._manager_turns), so a manager order sent through it made tasks the engine never picked up. The manager now gets both, as container_engine._build_manager gives them. tests/test_integration_order_route.py posts the order over HTTP and checks its pipeline state.
• The production-deploy leg of the smoke test stays manual and has not been run. See docs/plan/reports/agent-3-final.md §6, steps 7–8, which need DEPLOY_IT=1.
• Contract extension ModelGrantPort.get_org_policy (T1), with DefaultOrgPolicy as the default.
• Engine UI, contracts and owner console (phase 5 and the follow-ups), tracked in todo-v2-final.md; its status: lines are the source of truth:
  • Contracts: the engine event types in @agy/contracts (task.step, task.gate, run.log, run.tool, team.changed) (T20).
  • apps/web/src/lib/api-engine.ts typed client (T21) and the phase-5 pages: /manager (T29–T30), /tasks (T31–T32), /tasks/[id] (T33–T36), /workforce (T37–T39), /dashboard and /traces (T40–T41).
  • Direct tasks (HTTP POST /tasks, MCP tasks_create) enter the engine at pending_manager_review (T14); the legacy start_run refuses them with 409.
  • Pause, resume, cancel and answer, in the engine API and in the UI, with confirm (T16, T18, T23).
  • Employee cap 7 per organization by default (T25).
  • Owner credit console /admin/credits and its client (T10–T11, T26–T27).
  • Docs: docs/plan/TODO-agent-2-ai-workforce.md ticked and docs/plan/reports/agent-2-final.md written (T43). Three boxes stay open there, each with its reason.