app: update service orchestration, environment configs, and documentation

2026-08-28 19:35:02 +03:30
parent 55c9c2a4cf
commit ad4c111318
6 changed files with 178 additions and 9 deletions
@@ -0,0 +1,102 @@
+# Copykar Improvements Implementation Plan
+
+> **For agentic workers:** Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Publish through the bot account, make history fetching controllable, allow
+automatic source→target routing, rebuild the dashboard from the real metric inventory,
+and track error resolution — with a Telegram note to admins after every task.
+
+**Architecture:** Same service layout (`collector` → `admin_bot` review → Redis per-target
+queues → `publisher`). Changes are additive: a new `targets.auto_source_ids` column drives
+auto-routing, `error_logs` gains resolution columns, and the admin bot's channel lists
+become button menus instead of one card per channel.
+
+**Tech Stack:** Python 3.12, Telethon 1.44, asyncpg/PostgreSQL 16, Redis 7,
+prometheus-client, Grafana 11.2, Docker Compose.
+
+**Spec:** User request of 2026-08-28 (8 items), recorded in the task list below.
+
+## Global Constraints
+
+- All admin-facing bot text is Persian; keep the existing emoji/HTML card style.
+- Telegram limits: media caption ≤ 1024 chars, text message ≤ 4096 chars, callback data ≤ 64 bytes.
+- No AI/model attribution in code comments or commit messages (user's GEMINI.md).
+- Minimal diffs — no drive-by refactors of untouched code.
+- Never commit `.env`. Do not commit at all without asking.
+- Every task ends with: tests pass, app rebuilt+restarted, Telegram note sent to admins.
+
+---
+
+### Task 1: Publish through the bot account, not the userbot
+
+**Files:** Modify `main.py`, `services/publisher.py`; Test `tests/test_publisher_failures.py`
+
+**Interfaces:** Produces — `PublisherService(..., client=<bot client>)`.
+
+- [ ] Pass `admin_bot.client` as the publisher client in `main.py`.
+- [ ] Publisher must not `client.start()` a client the admin bot already started.
+- [ ] Keep the permanent/transient failure split; a bot lacking rights raises
+      `ChatAdminRequiredError`, already classified permanent.
+- [ ] Update the broken-target message to say the **bot** needs admin rights.
+- [ ] Run suite, rebuild, restart, notify admins.
+
+### Task 2: Controllable history fetch
+
+**Files:** Modify `services/collector.py`, `services/admin_bot.py`; Test `tests/test_scrape_history.py`
+
+**Interfaces:** Produces — `scrape_channel_history(...) -> ScrapeResult` with fields
+`collected`, `already_stored`, `duplicates`, `scanned`.
+
+- [ ] Return a structured result instead of a bare int so the admin sees *why* nothing was added.
+- [ ] Report all four counters in the Persian progress message.
+- [ ] Add a `🔢 تعداد دلخواه` button per source → state `wait_fetch_count` → user sends a number (1–500).
+- [ ] Keep the 20/50 quick buttons.
+- [ ] Run suite, rebuild, restart, notify admins.
+
+### Task 3: Automatic source→target routing
+
+**Files:** Modify `db/database.py`, `db/repository.py`, `db/models.py`, `services/admin_bot.py`; Test `tests/test_auto_routing.py`
+
+**Interfaces:** Produces — `targets.auto_source_ids BIGINT[]`,
+`Repository.set_target_auto_sources(target_id, source_channel_ids)`,
+`Repository.get_targets_auto_routed_from(source_channel_id)`.
+
+- [ ] Add `auto_source_ids BIGINT[] DEFAULT '{}'` column + migration.
+- [ ] Target config card gains `🤖 ارسال خودکار از مبدا` button showing every source as a toggle.
+- [ ] After the review card is sent, auto-route: rewrite for each subscribed target and enqueue.
+- [ ] The admin review card is still posted — auto-routing never replaces it.
+- [ ] Run suite, rebuild, restart, notify admins.
+
+### Task 4: Channel lists as button menus
+
+**Files:** Modify `services/admin_bot.py`
+
+- [ ] `/sources` and `/targets` send ONE message with one button per channel.
+- [ ] `src_view:{id}` / `trg_view:{id}` open that channel's card with its options.
+- [ ] Every card gets a back button to its list.
+- [ ] Run suite, rebuild, restart, notify admins.
+
+### Task 5: Error resolution tracking + caption fix
+
+**Files:** Modify `db/database.py`, `db/repository.py`, `core/metrics.py`, `core/error_logger.py`, `services/admin_bot.py`; Test `tests/test_error_tracking.py`
+
+**Interfaces:** Produces — `error_logs.resolved/resolved_at/resolved_note`,
+`Repository.get_open_error_summary()`, `Repository.resolve_errors(error_type, service_name, note)`,
+gauges `copykar_errors_open`, counter `copykar_errors_resolved_total`.
+
+- [ ] Add resolution columns + migration.
+- [ ] Fix `MediaCaptionTooLongError`: clamp preview/review captions to Telegram limits.
+- [ ] `/errors` command lists open error groups with a `✅ رفع شد` button per group.
+- [ ] Publish `copykar_errors_open` and `copykar_errors_resolved_total`; refresh the gauge periodically.
+- [ ] Mark the already-fixed historical errors resolved.
+- [ ] Run suite, rebuild, restart, notify admins.
+
+### Task 6: Rebuild the Grafana dashboard
+
+**Files:** Rewrite `monitoring/grafana/dashboards/copykar.json`
+
+- [ ] Rebuild from the live metric inventory; every metric appears exactly once.
+- [ ] Remove the duplicated queue reading (`copykar_redis_queue_size` total vs per-target gauge).
+- [ ] Max 3 panels per row, Persian titles + descriptions.
+- [ ] Include the new error-health panels.
+- [ ] Restart Grafana, verify panels return data, notify admins.