75 lines
6.3 KiB
Markdown
75 lines
6.3 KiB
Markdown
# Copykar Project Memory & Agent Guidelines
|
|
|
|
## 1. Project Overview & Architecture
|
|
Copykar is an automated content ingestion, AI rewriting, admin curation, and paced publishing fleet for Telegram channels and websites.
|
|
|
|
### Key Components:
|
|
- **Collector Service (`services/collector.py`)**: Telethon Userbot client monitoring source Telegram channels, downloading media to `MEDIA_DIR`, extracting subjects/tags, performing semantic deduplication, and sending raw review cards to the admin review channel.
|
|
- **Website Collector Service (`services/website_collector.py` & `services/website_analyzer.py`)**: Automated HTTP crawler for web sources. Uses AI to analyze site DOM/API structures with user-specified custom extraction needs (`custom_instructions`), discovering data endpoints and extracting new articles.
|
|
- **AI Processing Pipeline (`core/llm.py` & `services/ai_processor.py`)**:
|
|
- Multi-provider resilient chain (OpenAI, Gemini, AGY CLI / local bridge on `host.docker.internal:8088`).
|
|
- Automatic fallback upon rate limits, errors, or timeouts.
|
|
- Multimodal Vision: Handles image analysis along with text.
|
|
- Target Channel Context Ingestion: Injects the last N published messages from the target channel into the AI prompt to preserve tone, style, and narrative continuity.
|
|
- Semantic Deduplication: Evaluates topic tags and content similarity with previous posts.
|
|
- **Admin Bot Panel (`services/admin_bot.py`)**: Telethon Bot account providing interactive management and instant review cards with 5 primary category hubs.
|
|
- **Publisher Service (`services/publisher.py`)**: Paced target delivery worker consuming queues with configurable intervals, sleep schedules, and FIFO/Random dispatch order.
|
|
- **Database & Persistence (`db/database.py`, `db/models.py`, `db/repository.py`)**: PostgreSQL with asyncpg, automatic migrations, and JSONB post tracking. Redis for queue management.
|
|
- **Observability (`core/metrics.py`)**: Prometheus metrics server on port 8008 + Grafana dashboards.
|
|
|
|
---
|
|
|
|
## 2. Interaction & Workflow Rules
|
|
- **English Correction Rule**: Every user prompt must be corrected at the very top of each reply in the exact ASCII box format (`┌ ─ ┐ │ └ ┘`).
|
|
- **No AI Attribution**: Never mention AI, LLM, Gemini, Claude, Antigravity, or model names in commit messages, comments, PRs, or user-facing code.
|
|
- **Minimal Changes**: Smallest possible diff to achieve the goal. Do not refactor untouched code.
|
|
- **Git Workflow**:
|
|
- One logical change per commit.
|
|
- Stage files strictly by name (never `git add .`).
|
|
- Never commit `.env`, credentials, secrets, or temporary dumps.
|
|
- Never create or switch branches unless explicitly requested.
|
|
|
|
---
|
|
|
|
## 3. Project Invariants & Domain Standards
|
|
|
|
### 3.1 Categorization & Navigation (5 Primary Hubs)
|
|
The bot interface is strictly organized into 5 category hubs:
|
|
1. **`Copy` (مدیریت محتوا)**: Unreviewed posts queue (`/pending`), unified add source/target hub (`/add`), and channel topic categories (`/categories`).
|
|
2. **`AI` (تنظیمات هوش مصنوعی)**: Provider profiles, model selection, vision toggle, reasoning effort, fallback chain configuration, connectivity tests, and processing logs.
|
|
3. **`Bots` (کانالها و رباتها)**: Source Telegram channels (`/sources`), Target Telegram channels (`/targets`), Source websites (`/websites`), and collector userbot authentication (`/request_code`).
|
|
4. **`System` (مدیریت سیستم)**: Emergency fleet pause/resume (`/pause`, `/resume`), dynamic release notes (`/changes`), and comprehensive documentation (`/help`).
|
|
5. **`Monitor` (مانیتورینگ و آمار)**: Live queue & channel throughput statistics (`/stats`), unresolved error log review & bulk-fix (`/errors`), and Grafana time-range graphs.
|
|
|
|
### 3.2 Release Notes Delivery & Environment Configuration
|
|
- **Always use app `.env` file**: Always read bot credentials (`BOT_TOKEN`), admin IDs (`ADMIN_USER_IDS`), and configurations directly from `/projects/telegram-bots/copykar/.env`.
|
|
- **Never save release notes to a static file**: Do not create or keep `RELEASE_NOTES.md` on disk.
|
|
- **Dynamic Generation**: Always generate release notes dynamically strictly describing the changes of the current commit.
|
|
- **Direct Admin DMs Only**: Always dispatch release notes via the Telegram bot directly to admin private DMs (`ADMIN_USER_IDS`) using the `.env` configuration (and proxy `http://127.0.0.1:10809` if needed), never to any public or review channel.
|
|
|
|
### 3.3 Target Channel Context History
|
|
- Target channels support injecting recent published history (`context_message_count`) into AI rewriting prompts.
|
|
- History is queried via PostgreSQL JSONB array query against `published_to` matching `{"target_id": target.id}`, sorted chronologically (oldest to newest).
|
|
|
|
### 3.4 Multimodal Vision & AGY Bridge
|
|
- When images are present, payload is structured with `{"type": "text", "text": ...}` and `{"type": "image_url", ...}`.
|
|
- The host-side AGY bridge (`agy_bridge.py` on port 8088) decodes image base64 into temporary files and executes the CLI with multimodal parameters.
|
|
|
|
### 3.5 Button Permissions & Channel Callbacks
|
|
- Inline callbacks on review cards (`sel_trg:`, `rej:`, `del_msg:`) check `is_admin(event.sender_id)` against `ADMIN_USER_IDS`.
|
|
- The bot must have Administrator privileges in the review channel with permissions to edit and delete messages.
|
|
|
|
### 3.6 Self-Contained Project Memory
|
|
- Always record, update, and maintain all rules, architecture notes, domain standards, and operational memories about this project inside its own folder (`AGENTS.md` and `GEMINI.md`).
|
|
- This ensures any developer or collaborator receiving this repository has immediate, self-contained access to the project's memory and guidelines.
|
|
|
|
### 3.7 Persistent Data Storage Layout (`./data/`)
|
|
All service persistent data volumes are centralized within the project's `./data/` directory (git-ignored):
|
|
- `data/postgres/`: PostgreSQL database files. Uses `monitoring/postgres/libfakestat_musl.so` for Docker Desktop virtiofs permission compatibility.
|
|
- `data/redis/`: Redis RDB persistence and cache data.
|
|
- `data/app/`: Downloaded Telegram media and cached files.
|
|
- `data/sessions/`: Telethon SQLite session files for userbot and bot accounts.
|
|
- `data/prometheus/`: Prometheus metric time-series database.
|
|
- `data/grafana/`: Grafana dashboards and SQLite database.
|
|
- `data/pgadmin/`: pgAdmin configuration and query history.
|