diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..d74807b --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,59 @@ +# Copykar Project Memory & Agent Guidelines + +## 1. Project Overview & Architecture +Copykar is an automated content ingestion, AI rewriting, admin curation, and paced publishing fleet for Telegram channels and websites. + +### Key Components: +- **Collector Service (`services/collector.py`)**: Telethon Userbot client monitoring source Telegram channels, downloading media to `MEDIA_DIR`, extracting subjects/tags, performing semantic deduplication, and sending raw review cards to the admin review channel. +- **Website Collector Service (`services/website_collector.py` & `services/website_analyzer.py`)**: Automated HTTP crawler for web sources. Uses AI to analyze site DOM/API structures with user-specified custom extraction needs (`custom_instructions`), discovering data endpoints and extracting new articles. +- **AI Processing Pipeline (`core/llm.py` & `services/ai_processor.py`)**: + - Multi-provider resilient chain (OpenAI, Gemini, AGY CLI / local bridge on `host.docker.internal:8088`). + - Automatic fallback upon rate limits, errors, or timeouts. + - Multimodal Vision: Handles image analysis along with text. + - Target Channel Context Ingestion: Injects the last N published messages from the target channel into the AI prompt to preserve tone, style, and narrative continuity. + - Semantic Deduplication: Evaluates topic tags and content similarity with previous posts. +- **Admin Bot Panel (`services/admin_bot.py`)**: Telethon Bot account providing interactive management and instant review cards with 5 primary category hubs. +- **Publisher Service (`services/publisher.py`)**: Paced target delivery worker consuming queues with configurable intervals, sleep schedules, and FIFO/Random dispatch order. +- **Database & Persistence (`db/database.py`, `db/models.py`, `db/repository.py`)**: PostgreSQL with asyncpg, automatic migrations, and JSONB post tracking. Redis for queue management. +- **Observability (`core/metrics.py`)**: Prometheus metrics server on port 8008 + Grafana dashboards. + +--- + +## 2. Interaction & Workflow Rules +- **English Correction Rule**: Every user prompt must be corrected at the very top of each reply in the exact ASCII box format (`┌ ─ ┐ │ └ ┘`). +- **No AI Attribution**: Never mention AI, LLM, Gemini, Claude, Antigravity, or model names in commit messages, comments, PRs, or user-facing code. +- **Minimal Changes**: Smallest possible diff to achieve the goal. Do not refactor untouched code. +- **Git Workflow**: + - One logical change per commit. + - Stage files strictly by name (never `git add .`). + - Never commit `.env`, credentials, secrets, or temporary dumps. + - Never create or switch branches unless explicitly requested. + +--- + +## 3. Project Invariants & Domain Standards + +### 3.1 Categorization & Navigation (5 Primary Hubs) +The bot interface is strictly organized into 5 category hubs: +1. **`Copy` (مدیریت محتوا)**: Unreviewed posts queue (`/pending`), unified add source/target hub (`/add`), and channel topic categories (`/categories`). +2. **`AI` (تنظیمات هوش مصنوعی)**: Provider profiles, model selection, vision toggle, reasoning effort, fallback chain configuration, connectivity tests, and processing logs. +3. **`Bots` (کانال‌ها و ربات‌ها)**: Source Telegram channels (`/sources`), Target Telegram channels (`/targets`), Source websites (`/websites`), and collector userbot authentication (`/request_code`). +4. **`System` (مدیریت سیستم)**: Emergency fleet pause/resume (`/pause`, `/resume`), dynamic release notes (`/changes`), and comprehensive documentation (`/help`). +5. **`Monitor` (مانیتورینگ و آمار)**: Live queue & channel throughput statistics (`/stats`), unresolved error log review & bulk-fix (`/errors`), and Grafana time-range graphs. + +### 3.2 Release Notes Delivery +- **Never save release notes to a static file**: Do not create or keep `RELEASE_NOTES.md` on disk. +- **Dynamic Generation**: Always generate release notes dynamically strictly describing the changes of the current commit. +- **Direct Admin DMs**: Always send release notes via the Telegram bot directly to admin private DMs (`ADMIN_USER_IDS`), never to the review channel. + +### 3.3 Target Channel Context History +- Target channels support injecting recent published history (`context_message_count`) into AI rewriting prompts. +- History is queried via PostgreSQL JSONB array query against `published_to` matching `{"target_id": target.id}`, sorted chronologically (oldest to newest). + +### 3.4 Multimodal Vision & AGY Bridge +- When images are present, payload is structured with `{"type": "text", "text": ...}` and `{"type": "image_url", ...}`. +- The host-side AGY bridge (`agy_bridge.py` on port 8088) decodes image base64 into temporary files and executes the CLI with multimodal parameters. + +### 3.5 Button Permissions & Channel Callbacks +- Inline callbacks on review cards (`sel_trg:`, `rej:`, `del_msg:`) check `is_admin(event.sender_id)` against `ADMIN_USER_IDS`. +- The bot must have Administrator privileges in the review channel with permissions to edit and delete messages. diff --git a/GEMINI.md b/GEMINI.md new file mode 100644 index 0000000..d74807b --- /dev/null +++ b/GEMINI.md @@ -0,0 +1,59 @@ +# Copykar Project Memory & Agent Guidelines + +## 1. Project Overview & Architecture +Copykar is an automated content ingestion, AI rewriting, admin curation, and paced publishing fleet for Telegram channels and websites. + +### Key Components: +- **Collector Service (`services/collector.py`)**: Telethon Userbot client monitoring source Telegram channels, downloading media to `MEDIA_DIR`, extracting subjects/tags, performing semantic deduplication, and sending raw review cards to the admin review channel. +- **Website Collector Service (`services/website_collector.py` & `services/website_analyzer.py`)**: Automated HTTP crawler for web sources. Uses AI to analyze site DOM/API structures with user-specified custom extraction needs (`custom_instructions`), discovering data endpoints and extracting new articles. +- **AI Processing Pipeline (`core/llm.py` & `services/ai_processor.py`)**: + - Multi-provider resilient chain (OpenAI, Gemini, AGY CLI / local bridge on `host.docker.internal:8088`). + - Automatic fallback upon rate limits, errors, or timeouts. + - Multimodal Vision: Handles image analysis along with text. + - Target Channel Context Ingestion: Injects the last N published messages from the target channel into the AI prompt to preserve tone, style, and narrative continuity. + - Semantic Deduplication: Evaluates topic tags and content similarity with previous posts. +- **Admin Bot Panel (`services/admin_bot.py`)**: Telethon Bot account providing interactive management and instant review cards with 5 primary category hubs. +- **Publisher Service (`services/publisher.py`)**: Paced target delivery worker consuming queues with configurable intervals, sleep schedules, and FIFO/Random dispatch order. +- **Database & Persistence (`db/database.py`, `db/models.py`, `db/repository.py`)**: PostgreSQL with asyncpg, automatic migrations, and JSONB post tracking. Redis for queue management. +- **Observability (`core/metrics.py`)**: Prometheus metrics server on port 8008 + Grafana dashboards. + +--- + +## 2. Interaction & Workflow Rules +- **English Correction Rule**: Every user prompt must be corrected at the very top of each reply in the exact ASCII box format (`┌ ─ ┐ │ └ ┘`). +- **No AI Attribution**: Never mention AI, LLM, Gemini, Claude, Antigravity, or model names in commit messages, comments, PRs, or user-facing code. +- **Minimal Changes**: Smallest possible diff to achieve the goal. Do not refactor untouched code. +- **Git Workflow**: + - One logical change per commit. + - Stage files strictly by name (never `git add .`). + - Never commit `.env`, credentials, secrets, or temporary dumps. + - Never create or switch branches unless explicitly requested. + +--- + +## 3. Project Invariants & Domain Standards + +### 3.1 Categorization & Navigation (5 Primary Hubs) +The bot interface is strictly organized into 5 category hubs: +1. **`Copy` (مدیریت محتوا)**: Unreviewed posts queue (`/pending`), unified add source/target hub (`/add`), and channel topic categories (`/categories`). +2. **`AI` (تنظیمات هوش مصنوعی)**: Provider profiles, model selection, vision toggle, reasoning effort, fallback chain configuration, connectivity tests, and processing logs. +3. **`Bots` (کانال‌ها و ربات‌ها)**: Source Telegram channels (`/sources`), Target Telegram channels (`/targets`), Source websites (`/websites`), and collector userbot authentication (`/request_code`). +4. **`System` (مدیریت سیستم)**: Emergency fleet pause/resume (`/pause`, `/resume`), dynamic release notes (`/changes`), and comprehensive documentation (`/help`). +5. **`Monitor` (مانیتورینگ و آمار)**: Live queue & channel throughput statistics (`/stats`), unresolved error log review & bulk-fix (`/errors`), and Grafana time-range graphs. + +### 3.2 Release Notes Delivery +- **Never save release notes to a static file**: Do not create or keep `RELEASE_NOTES.md` on disk. +- **Dynamic Generation**: Always generate release notes dynamically strictly describing the changes of the current commit. +- **Direct Admin DMs**: Always send release notes via the Telegram bot directly to admin private DMs (`ADMIN_USER_IDS`), never to the review channel. + +### 3.3 Target Channel Context History +- Target channels support injecting recent published history (`context_message_count`) into AI rewriting prompts. +- History is queried via PostgreSQL JSONB array query against `published_to` matching `{"target_id": target.id}`, sorted chronologically (oldest to newest). + +### 3.4 Multimodal Vision & AGY Bridge +- When images are present, payload is structured with `{"type": "text", "text": ...}` and `{"type": "image_url", ...}`. +- The host-side AGY bridge (`agy_bridge.py` on port 8088) decodes image base64 into temporary files and executes the CLI with multimodal parameters. + +### 3.5 Button Permissions & Channel Callbacks +- Inline callbacks on review cards (`sel_trg:`, `rej:`, `del_msg:`) check `is_admin(event.sender_id)` against `ADMIN_USER_IDS`. +- The bot must have Administrator privileges in the review channel with permissions to edit and delete messages.