Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
46251c635f | ||
|
|
a547d09fcd | ||
|
|
e94d247096 | ||
|
|
4f0c2d23de | ||
|
|
4e9895fbf4 | ||
|
|
b0edae98c6 | ||
|
|
d438c40542 | ||
|
|
722c88d436 | ||
|
|
5221442aeb | ||
|
|
85d81aa996 | ||
|
|
53e7b0e4ae | ||
|
|
d63eedf41b | ||
|
|
28fffb3acb | ||
|
|
4303b145b0 | ||
|
|
137aa806c0 | ||
|
|
e012b84969 | ||
|
|
727a87b73b | ||
|
|
d3ee8ebbd7 | ||
|
|
c216229c61 | ||
|
|
efd5f7b172 | ||
|
|
c14ae0250e | ||
|
|
2e34fb20b5 | ||
|
|
74ff49062f | ||
|
|
4c7051bc97 | ||
|
|
d9f58fb90e | ||
|
|
eb4d6ccdb0 | ||
|
|
9bcb019d3a | ||
|
|
4c75db1f74 | ||
|
|
75820125aa | ||
|
|
6c67e5042f | ||
|
|
a624066aab | ||
|
|
f6ff400344 | ||
|
|
5be5964098 | ||
|
|
399afe5ea5 | ||
|
|
a8c0da837a | ||
|
|
7d009c932f | ||
|
|
b47a6cc366 | ||
|
|
6146ea9eb0 | ||
|
|
61c7ac7d51 | ||
|
|
0eca44c11a | ||
|
|
79daea4ae1 | ||
|
|
7d94fbadb5 | ||
|
|
22485efe10 | ||
|
|
6c9bbd7fd7 | ||
|
|
3682f3ad61 | ||
|
|
2dd5f97143 | ||
|
|
064e731d25 | ||
|
|
e08bfaffba | ||
|
|
1d27fbde30 | ||
|
|
0152593815 | ||
|
|
8207d3ae02 | ||
|
|
787a5065bc | ||
|
|
f243807fc8 | ||
|
|
73feb75767 | ||
|
|
abe51b38d0 | ||
|
|
86ed3a966a | ||
|
|
1305cccfa5 | ||
|
|
bc3ceb5dfe | ||
|
|
bb8040d6fb | ||
|
|
aed541c70c | ||
|
|
54a1e6c042 | ||
|
|
d8b0c08aaa | ||
|
|
99c1498810 | ||
|
|
7567ac9191 | ||
|
|
4f5fb6d7c2 | ||
|
|
703a0ba32f | ||
|
|
35b00d7fd6 | ||
|
|
8ffbd7f9ec | ||
|
|
b7b49a3a63 | ||
|
|
fe9367b5e9 | ||
|
|
14958be302 | ||
|
|
cd319a32c2 | ||
|
|
3886c4f167 | ||
|
|
8cf453f86b | ||
|
|
48cd217479 | ||
|
|
04ac25c818 | ||
|
|
976b386e6b | ||
|
|
bb6fc6cd53 | ||
|
|
1adeff5e94 | ||
|
|
9c44cfc08f | ||
|
|
b76da2edd0 | ||
|
|
30bfccd0e7 | ||
|
|
c93c6adc7a | ||
|
|
aae5e0926a | ||
|
|
016d459861 | ||
|
|
0abe1a2bb4 | ||
|
|
cbd854d933 | ||
|
|
6aaee107a9 | ||
|
|
c6ffded8d7 | ||
|
|
392e1e534a | ||
|
|
3547598d37 | ||
|
|
0b7df7f721 | ||
|
|
7e464ef598 | ||
|
|
674bc5627f | ||
|
|
4042c80f00 | ||
|
|
fcc26a5b34 | ||
|
|
9681bdbd36 | ||
|
|
def5c39aa1 | ||
|
|
699a7c3857 | ||
|
|
0f078ab482 | ||
|
|
ba4c42f8f6 | ||
|
|
8e419c1832 | ||
|
|
6e18e8e565 | ||
|
|
7c28a0c36a | ||
|
|
ff8ea40d40 | ||
|
|
ca22042d18 | ||
|
|
9158ae3d8a | ||
|
|
1776283a95 | ||
|
|
31bbc15535 | ||
|
|
e48b9ef876 | ||
|
|
6974ae690d | ||
|
|
b84f1fe815 | ||
|
|
8d7fab97b9 | ||
|
|
d4ded77355 | ||
|
|
f6e6211871 | ||
|
|
049f1ef30a | ||
|
|
868278fdba | ||
|
|
926182a6cf | ||
|
|
883ad2c73b | ||
|
|
215af73388 | ||
|
|
26724c7675 | ||
|
|
bda0a0ef5e | ||
|
|
52ff29668e | ||
|
|
45c7e65bc4 | ||
|
|
cadd5607b8 | ||
|
|
2ce1e20af2 | ||
|
|
80d534d549 | ||
|
|
c3e847c249 | ||
|
|
b71e4a22a2 | ||
|
|
966000b0b0 | ||
|
|
a8b1775169 | ||
|
|
0c25cf020f | ||
|
|
2477543029 | ||
|
|
c944db54af | ||
|
|
9647667ae2 | ||
|
|
fc22795b63 | ||
|
|
1fabf21935 | ||
|
|
b0dd703f72 | ||
|
|
987d9ea109 | ||
|
|
e7f3de1ddc | ||
|
|
4fef0e8b74 | ||
|
|
45b73ec54a | ||
|
|
907ad537e2 | ||
|
|
51728bf3f2 | ||
|
|
8f76f70409 | ||
|
|
8c94d40766 | ||
|
|
1ffaf7478d | ||
|
|
2ad2bdcdff | ||
|
|
6664f24f20 | ||
|
|
0b7bbc78a6 | ||
|
|
fc963101b3 | ||
|
|
d28ad31aca | ||
|
|
67d81f81ff | ||
|
|
d6bc5ce85c | ||
|
|
5b2db1cf48 | ||
|
|
42bdec6c2f | ||
|
|
531f1aa82f | ||
|
|
486e4c4673 | ||
|
|
bea3b9ccbb |
@@ -0,0 +1,51 @@
|
||||
# План работы RF4 Spotter — Регрессионный аудит
|
||||
|
||||
На основе: [REGRESSION_AUDIT_2026-09-09.md](../docs/REGRESSION_AUDIT_2026-09-09.md)
|
||||
Дата: 2026-09-09
|
||||
|
||||
## Выполнено
|
||||
|
||||
### R01 ✅ — Caddy body_limit → request_body max_size
|
||||
- **Файл**: `deploy/Caddyfile`
|
||||
- **Проблема**: `body_limit 10M` не поддерживается в Caddy 2.10.2
|
||||
- **Решение**: `request_body { max_size 10M }`
|
||||
- **Верификация**: `caddy adapt` проходит без ошибок
|
||||
|
||||
### R02 ✅ — Activity API contract regression
|
||||
- **API**: `main.py` — `/api/v1/activity` возвращает `PaginatedActivityOut`
|
||||
- **Frontend**: все 4 потребителя обновлены:
|
||||
- `index.astro` ✅ (из предыдущего коммита)
|
||||
- `fish/[slug].astro` ✅
|
||||
- `waterbodies/[slug].astro` ✅
|
||||
- `waterbodies/[slug]/[fish].astro` ✅
|
||||
- `spots/[id].astro` ✅
|
||||
- **Тесты**: 4 теста обновлены под новый контракт
|
||||
- **Результат**: 76 passed, 1 skipped, 0 failures
|
||||
|
||||
### R14 ✅ — Registry источников больше не зависит от БД при парсинге CLI
|
||||
- **Файл**: `community_scheduler.py`
|
||||
- **Решение**: `_static_registry()` — без БД; `configured_sources(enabled_keys)` — с опциональной БД
|
||||
- **Тесты**: scheduler unit-тесты проходят без PostgreSQL
|
||||
|
||||
## Итоги тестов
|
||||
- **Python**: 76 passed, 1 skipped ✅
|
||||
- **Astro check**: 0 errors, 0 warnings, 0 hints ✅
|
||||
- **Caddy adapt**: passes ✅
|
||||
|
||||
## Коммиты
|
||||
1. `a37f9c4` — R01 Caddy body_limit → request_body
|
||||
2. `d962ba2` — R02 activity API contract + R14 static registry
|
||||
|
||||
## Осталось из регрессий (R03-R15)
|
||||
- R03: Research CLI cooldown broken (fcntl `r`/`r+` + encoding)
|
||||
- R04: Disabled-фильтрация обходит site cooldown
|
||||
- R05: Bootstrap небезопасен для источников
|
||||
- R06: Фильтры сигналов сравнивают slug с названием
|
||||
- R07: Ошибка главной остаётся HTTP 200
|
||||
- R08: Фильтры UI не доведены до responsive-состояния
|
||||
- R09: Пагинация (частично исправлено в U03)
|
||||
- R10: Ошибки формы теряют черновик
|
||||
- R11: Проверка URL после сетевого обращения
|
||||
- R12: Мониторинг не сигнализирует о зависшем импорте
|
||||
- R13: Доверие к IP клиента не ограничено proxy
|
||||
- R15: D04/D06/D07/D08 выполнены не полностью
|
||||
@@ -41,6 +41,8 @@ jobs:
|
||||
alembic current
|
||||
- name: Run Python tests
|
||||
run: pytest -q
|
||||
- name: Verify generated OpenAPI contract
|
||||
run: python apps/api/export_openapi.py --check
|
||||
|
||||
astro-build:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
name: Production bootstrap drill
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
schedule:
|
||||
- cron: "17 3 * * 6"
|
||||
|
||||
jobs:
|
||||
isolated-production-bootstrap:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "22"
|
||||
cache: npm
|
||||
cache-dependency-path: apps/web/package-lock.json
|
||||
- name: Install web dependencies
|
||||
run: npm --prefix apps/web ci
|
||||
- name: Run isolated production bootstrap
|
||||
shell: bash
|
||||
run: |
|
||||
set -o pipefail
|
||||
mkdir -p artifacts
|
||||
./deploy/test-production-bootstrap.sh 2>&1 | tee artifacts/production-bootstrap.log
|
||||
- name: Collect failure diagnostics
|
||||
if: failure()
|
||||
run: |
|
||||
docker compose --env-file .env.production.example -f compose.production.yaml ps -a > artifacts/compose-ps.txt 2>&1 || true
|
||||
cp -R apps/web/test-results artifacts/test-results 2>/dev/null || true
|
||||
cp -R apps/web/playwright-report artifacts/playwright-report 2>/dev/null || true
|
||||
- name: Upload drill diagnostics
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: production-bootstrap-${{ github.run_id }}
|
||||
path: artifacts
|
||||
if-no-files-found: error
|
||||
retention-days: 14
|
||||
@@ -0,0 +1,5 @@
|
||||
# Default ignored files
|
||||
/shelf/
|
||||
/workspace.xml
|
||||
# Environment-dependent path to Maven home directory
|
||||
/mavenHomeManager.xml
|
||||
@@ -0,0 +1,266 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project version="4">
|
||||
<component name="GigaCodeAgentSettings">
|
||||
<option name="autoApproveEdits" value="true" />
|
||||
<option name="autoApprovedCommands">
|
||||
<list>
|
||||
<Execute>
|
||||
<option name="pattern" value="mkdir -p /home/ik/git/rf4-help/.gigacode/plans" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="cd *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="ls *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="pip install *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="tail *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python pytest *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python -c " import ast import sys files = [ 'apps/api/app/main.py', 'apps/api/app/readiness.py', 'apps/api/app/schemas.py', 'apps/api/app/activity.py', 'apps/api/app/community_scheduler.py', 'apps/api/app/community_importer.py', 'rf4_research/community_cli.py', ] for f in files: try: with open(f) as fh: ast.parse(fh.read()) print(f'OK: {f}') except SyntaxError as e: print(f'ERROR: {f}: {e}') sys.exit(1) print('All Python files syntax OK') "" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="npm run *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="head *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="npm install *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="git add *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="git reset *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="git diff *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="git log *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="git commit *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="git status" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="echo *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="printf 'localhost {\n body_limit 10M\n}\n' > /tmp/test-caddyfile" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="cp /home/ik/git/rf4-help/deploy/Caddyfile /tmp/Caddyfile" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="printf 'example.com {\n body_limit 10M\n}\n' > /tmp/Caddyfile" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="printf 'example.com {\n request_body {\n max_size 10M\n }\n}\n' > /tmp/Caddyfile" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="docker run *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="sed 's/email {$ACME_EMAIL}/email test@test.com/' /home/ik/git/rf4-help/deploy/Caddyfile > /tmp/Caddyfile" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="grep *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="printf 'example.com {\n request_body {\n max_size 10M\n }\n handle {\n respond \"ok\"\n }\n}\n' > /tmp/Caddyfile" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python -c "import ast; ast.parse(open('rf4_research/community_cli.py').read()); print('OK')"" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python -c "import ast; ast.parse(open('apps/api/app/community_scheduler.py').read()); print('OK')"" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python -c "import ast; ast.parse(open('apps/api/app/main.py').read()); print('OK')"" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="node -e "try { fetch('http://localhost:1', { signal: AbortSignal.timeout(100) }); } catch(e) { console.log(e.constructor.name, e.message); }" 2>&1" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="node -e "fetch('http://httpbin.org/delay/10', { signal: AbortSignal.timeout(200) }).catch(e => console.log(e.constructor.name, e.message));" 2>&1" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="pip-compile requirements.txt --output-file requirements-lock.txt -q 2>&1" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="pip-compile requirements-dev.txt --output-file requirements-dev-lock.txt -q 2>&1" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="wc *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="alembic revision *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python -m app.cli --help 2>&1" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python -m rf4_research.community_cli --help 2>&1" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="alembic heads *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 -c " from urllib.request import Request, urlopen from urllib.error import HTTPError # Test with a URL that redirects try: req = Request('https://httpbin.org/redirect/1', headers={'User-Agent': 'test'}) resp = urlopen(req, timeout=5) print(f'Final URL: {resp.url}') print(f'Response URL: {resp.url}') except Exception as e: print(f'Error: {type(e).__name__}: {e}') " 2>&1" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="pwd" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 -c "from rf4_research import community_cli; print('Syntax OK')"" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/append_tests.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 pytest *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_redirect_test.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a04.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a04_v2.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="git show *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="git branch *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/test_double_reservation_bug.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a02_double_reservation.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/test_debug.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/test_a02_final.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/test_a02_v2.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/test_a02_code.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a04_pagination.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a01_monitoring.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a01_tests.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="*" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a01_tests_v2.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a04_filters.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a05_sessionstorage.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a06_tests.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a07_review_note.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a08_errorpage.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a08_index.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a11_pip_audit.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a04_pagination_final.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a08_index_unavailable.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a08_spots.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a08_records.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a01_vars.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a01_test.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a07_auto_publish.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a07_review_note_auto.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 /tmp/fix_a09_cli.py" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python3 -c "from app.cli import main; import sys; sys.argv = ['cli', '--help']; main()" 2>&1" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="find *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="git push *" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python -c " from sqlalchemy import create_engine, select from sqlalchemy.orm import Session from sqlalchemy.pool import StaticPool from app.database import Base from app.models import SubmissionAttempt import hashlib, hmac engine = create_engine('sqlite://', connect_args={'check_same_thread': False}, poolclass=StaticPool) Base.metadata.create_all(engine) with Session(engine) as db: # Simulate first request key = 'idem-test-key-001' key_hash = hmac.new('change-rate-limit-secret'.encode(), key.encode(), hashlib.sha256).hexdigest() print(f'Key hash: {key_hash}') # Check before storing existing = db.scalar(select(SubmissionAttempt).where(SubmissionAttempt.idempotency_key == key_hash)) print(f'Found before: {existing}') # Store from datetime import datetime, timezone db.add(SubmissionAttempt(client_hash='test', idempotency_key=key_hash, created_at=datetime.now(timezone.utc))) db.commit() # Check after storing existing = db.scalar(select(SubmissionAttempt).where(SubmissionAttempt.idempotency_key == key_hash)) print(f'Found after: {existing}') if existing: print(f' idempotency_key: {existing.idempotency_key}') "" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python -c " from sqlalchemy import create_engine, select from sqlalchemy.orm import Session from sqlalchemy.pool import StaticPool from app.database import Base from app.models import SubmissionAttempt import hashlib, hmac engine = create_engine('sqlite://', connect_args={'check_same_thread': False}, poolclass=StaticPool) Base.metadata.create_all(engine) with Session(engine) as db: key = 'idem-test-key-001' key_hash = hmac.new('change-rate-limit-secret'.encode(), key.encode(), hashlib.sha256).hexdigest() print(f'Key hash: {key_hash}') existing = db.scalar(select(SubmissionAttempt).where(SubmissionAttempt.idempotency_key == key_hash)) print(f'Found before: {existing}') from datetime import datetime, timezone db.add(SubmissionAttempt(client_hash='test', idempotency_key=key_hash, created_at=datetime.now(timezone.utc))) db.commit() existing = db.scalar(select(SubmissionAttempt).where(SubmissionAttempt.idempotency_key == key_hash)) print(f'Found after: {existing}') if existing: print(f' idempotency_key: {existing.idempotency_key}') "" />
|
||||
</Execute>
|
||||
<Execute>
|
||||
<option name="pattern" value="python -c " from sqlalchemy import create_engine, inspect, text from sqlalchemy.pool import StaticPool from app.database import Base from app.models import SubmissionAttempt engine = create_engine('sqlite://', connect_args={'check_same_thread': False}, poolclass=StaticPool) Base.metadata.create_all(engine) inspector = inspect(engine) columns = inspector.get_columns('submission_attempt') print('Columns:', [c['name'] for c in columns]) "" />
|
||||
</Execute>
|
||||
</list>
|
||||
</option>
|
||||
<option name="autoApprovedReads">
|
||||
<list>
|
||||
<Read>
|
||||
<option name="pattern" value="/path/to/apps/web/src/pages/index.astro" />
|
||||
</Read>
|
||||
<Read>
|
||||
<option name="pattern" value="/path/to/apps/web/src/lib/api.ts" />
|
||||
</Read>
|
||||
<Read>
|
||||
<option name="pattern" value="/path/to/rf4_spotter/rf4_research/community_cli.py" />
|
||||
</Read>
|
||||
</list>
|
||||
</option>
|
||||
</component>
|
||||
</project>
|
||||
@@ -0,0 +1,6 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project version="4">
|
||||
<component name="ProjectRootManager">
|
||||
<output url="file://$PROJECT_DIR$/out" />
|
||||
</component>
|
||||
</project>
|
||||
@@ -0,0 +1,8 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project version="4">
|
||||
<component name="ProjectModuleManager">
|
||||
<modules>
|
||||
<module fileurl="file://$PROJECT_DIR$/.idea/rf4-help.iml" filepath="$PROJECT_DIR$/.idea/rf4-help.iml" />
|
||||
</modules>
|
||||
</component>
|
||||
</project>
|
||||
@@ -0,0 +1,9 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<module type="JAVA_MODULE" version="4">
|
||||
<component name="NewModuleRootManager" inherit-compiler-output="true">
|
||||
<exclude-output />
|
||||
<content url="file://$MODULE_DIR$" />
|
||||
<orderEntry type="inheritedJdk" />
|
||||
<orderEntry type="sourceFolder" forTests="false" />
|
||||
</component>
|
||||
</module>
|
||||
@@ -0,0 +1,6 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project version="4">
|
||||
<component name="VcsDirectoryMappings">
|
||||
<mapping directory="" vcs="Git" />
|
||||
</component>
|
||||
</project>
|
||||
@@ -0,0 +1,65 @@
|
||||
# Repository Guidelines
|
||||
|
||||
## Project Structure & Module Organization
|
||||
|
||||
RF4 Spotter is an Astro/FastAPI/PostgreSQL application with an offline
|
||||
research and media-ingestion toolkit. The main areas are:
|
||||
|
||||
- `apps/api/` — FastAPI application, routers, models, migrations, and API tests.
|
||||
- `apps/web/` — Astro pages, components, styles, unit tests, and Playwright tests.
|
||||
- `rf4_research/` — source parsers, media manifest tooling, and CLI commands.
|
||||
- `tests/` — Python research/tooling tests and fixtures.
|
||||
- `data/media/` — versioned manifest and content-addressed local media files.
|
||||
- `docs/` — specification, roadmap, ADRs, runbooks, and acceptance procedures.
|
||||
- `compose.yaml` — local PostgreSQL, API, web, and supporting services.
|
||||
|
||||
Keep generated reports and temporary downloads outside committed paths unless a
|
||||
task explicitly requires versioning them.
|
||||
|
||||
## Build, Test, and Development Commands
|
||||
|
||||
From the repository root:
|
||||
|
||||
```bash
|
||||
.venv/bin/pytest -q # Python suites
|
||||
npm --prefix apps/web run check # Astro type/template checks
|
||||
npm --prefix apps/web run build # Check and production build
|
||||
npm --prefix apps/web run test:unit # Web unit tests
|
||||
WEB_URL=http://127.0.0.1:4321 npm --prefix apps/web run test:e2e
|
||||
docker compose up --build # Full local stack
|
||||
```
|
||||
|
||||
For media work, use `.venv/bin/python -m rf4_research.media_cli --audit` and
|
||||
respect the manifest’s cooldown and approval states. Do not hotlink or replace
|
||||
approved media without explicit review.
|
||||
|
||||
## Coding Style & Naming Conventions
|
||||
|
||||
Use four-space indentation for Python and two spaces for Astro/TypeScript.
|
||||
Prefer typed Python functions, `snake_case` for Python identifiers, and
|
||||
`camelCase` for TypeScript variables/functions. Astro components use
|
||||
`PascalCase.astro`; tests use descriptive `test_*.py` or `*.test.ts` names.
|
||||
Keep UI text and data-source labels explicit and accessible; run `astro check`
|
||||
before committing web changes.
|
||||
|
||||
## Testing Guidelines
|
||||
|
||||
Add focused regression tests beside the affected suite. Python tests use
|
||||
pytest; web behavior uses Node unit tests and Playwright. Run the smallest
|
||||
relevant test first, then the full suite before handoff. Never use live external
|
||||
sources in tests; use fixtures or isolated Docker services.
|
||||
|
||||
## Commit & Pull Request Guidelines
|
||||
|
||||
Use imperative, concise commit subjects with the repository’s existing scope
|
||||
style, such as `feat:`, `fix:`, `data:`, or `chore:`. Keep commits focused and
|
||||
exclude unrelated user files. Pull requests should describe behavior changes,
|
||||
verification commands, migration or configuration impact, and screenshots for
|
||||
visual/UI work. Call out any media provenance, approval, or rollback decision.
|
||||
|
||||
## Security & Configuration Tips
|
||||
|
||||
Do not commit secrets, production `.env` files, tokens, or private payloads.
|
||||
Use local fixtures and documented environment variables. Preserve the strict
|
||||
CSP, admin authentication barriers, source attribution, and media provenance
|
||||
when changing application code.
|
||||
@@ -6,7 +6,11 @@ RF4 Spotter — неофициальный сервис свежих точек
|
||||
|
||||
## Статус разработки
|
||||
|
||||
**Проверка 10 сентября 2026 (`4f68d6b`): к деплою пока не готов.** Python: **107 passed, 1 skipped**; Astro check/build, web unit и Caddy adapt проходят. Исправлены синтаксис Caddy и потребители activity envelope, но остаются цикл readiness/scheduler, неатомарный cooldown, ошибки пагинации/фильтров и другие недоработки. [Актуальный план A01–A13](docs/RECOVERY_PLAN_2026-09-10.md), [промпт исполнителю](docs/RECOVERY_PROMPT.md). Начинать с A01, затем A02. Bootstrap больше не запускает реальные парсеры, но требует обновления проверки миграции и изолированной proxy/scheduler-приёмки. Ниже описаны реализованные возможности, а не гарантия приёмки текущей ревизии.
|
||||
**Проверка 20 сентября 2026: локальный кодовый контур проходит сборочные и тестовые gates, внешний запуск ждёт сервер и его настройки.** Пакет восстановления A01–A13 закрыт. Python: **191 passed, 1 skipped**; Astro check/build и web unit проходят. Граф миграций имеет единственную голову `0020`; OpenAPI artifact синхронизирован с FastAPI (`36 paths`). Изолированный production bootstrap с чистыми томами повторно пройден; живая БД, реальные секреты и импорт внешних источников по-прежнему требуют отдельной инфраструктурной/разрешённой приёмки.
|
||||
|
||||
Актуальные следующие задачи находятся только в [ROADMAP](docs/ROADMAP.md). Старые планы и аудиты сохранены как история и больше не задают порядок работ. До внешнего запуска нужны сервер, DNS/TLS, production-секреты, публичные контакты, внешний backup и канал уведомлений.
|
||||
|
||||
Разрешённые интеграции и недостающие первичные подтверждения сведены в [реестр разрешений](docs/source-permissions.md). Перед открытой публикацией пустые поля реестра являются блокером конкретного источника, особенно для изображений.
|
||||
|
||||
Web Docker-образ устанавливает зависимости через `npm ci` по lock-файлу и удаляет devDependencies после сборки. Локальные `.env` исключены из web build context.
|
||||
|
||||
@@ -18,23 +22,55 @@ Web Docker-образ устанавливает зависимости чере
|
||||
|
||||
Повторный импорт изменённой опубликованной записи переводит её на ручную проверку и снимает прежний улов с активности (с учётом TTL кэша). После сопоставления и подтверждения обновляется тот же улов; дубликат не создаётся. Автоматическое обнаружение удалённых оригиналов пока не реализовано.
|
||||
|
||||
Идёт исправление аудита: актуальные изменения и ограничения перечислены в [AUDIT_FIXES.md](docs/AUDIT_FIXES.md). Production Compose включает community scheduler; страницы rules/privacy реализованы. Для запуска остаются сервер, DNS/TLS, секреты, внешний backup и контакты. Шкала 72 часов использует полную выборку по времени поступления; одинаковые поля разных источников больше не считаются доказательством одного события. Фоновая публикация обновляет кэш API в пределах TTL, не мгновенно.
|
||||
История исправлений аудита сохранена в [AUDIT_FIXES.md](docs/AUDIT_FIXES.md) и [RECOVERY_FIXES_REPORT.md](docs/RECOVERY_FIXES_REPORT.md). Production Compose включает community scheduler; страницы rules/privacy реализованы. Для запуска остаются сервер, DNS/TLS, секреты, внешний backup и контакты. Шкала 72 часов использует полную выборку по времени поступления; одинаковые поля разных источников больше не считаются доказательством одного события. Фоновая публикация обновляет кэш API в пределах TTL, не мгновенно.
|
||||
|
||||
Предыдущий полный аудит 8 сентября: [отчёт](docs/PROJECT_AUDIT_2026-09-08.md). T01/T02 исправили маршруты формы и конфликт admin-аутентификации; `sh deploy/test-proxy-routing.sh` проверял Astro redirects, API, Basic/Bearer и отказы на прежней ревизии. Синтаксис Caddy теперь исправен. Подключение scheduler к исходящей сети добавлено; актуальная интеграционная приёмка входит в [план A01–A13](docs/RECOVERY_PLAN_2026-09-10.md).
|
||||
Предыдущие аудиты и план восстановления доступны в `docs/` как исторические материалы. Их незакрытые на момент составления чекбоксы не являются текущим backlog; статусы сведены в [итоговый отчёт](docs/RECOVERY_FIXES_REPORT.md).
|
||||
|
||||
На ширинах 320, 390, 768 и 1280 px ранее проверено отсутствие горизонтального переполнения основных страниц. Это не полная визуальная приёмка: аудит обнаружил неверную desktop-компоновку фильтров; наполненные карточки, длинные названия, клавиатура и zoom остаются отдельной задачей.
|
||||
На ширинах 320, 390, 768 и 1280 px проверено отсутствие горизонтального переполнения основных страниц; исправлена desktop-компоновка фильтров. Наполненные карточки, длинные названия, клавиатура и zoom остаются постоянной частью визуальной приёмки каждого крупного UI-пакета.
|
||||
|
||||
Функциональный MVP и локальный production-контур готовятся к открытой альфе: официальный импорт, пользовательские заявки, модерация, объяснимый индекс, staging внешних источников, адаптивный Astro UI, миграции, резервное копирование, retention, мониторинг и security/accessibility-проверки реализованы. На всех страницах подключён компактный баннер открытой альфы со ссылками на статус, правила и отправку улова. В production Compose включён community scheduler; локально он запускается отдельным профилем. Публичный запуск блокируют покупка и настройка сервера, DNS/TLS, реальные секреты, внешний backup, канал уведомлений; публичный адрес обратной связи ещё не задан.
|
||||
|
||||
Главная деградирует по секциям: activity, community signals и справочники загружаются независимо. Частичный отказ сохраняет доступные данные и возвращает HTTP 200 с `X-RF4-Partial` и запретом кэширования; общий 503 возникает только при отказе всех частей.
|
||||
|
||||
RF4DB/RF4-STAT/RF4MAP/RF4 Posts сначала принимаются в изолированный staging. Полные записи с ранее подтверждёнными алиасами источника публикуются автоматически; новые соответствия и неполные записи остаются на ручной проверке. Admin API предлагает точные ранее подтверждённые алиасы отдельно от mapping-действия и запрещает молча переназначать alias другой сущности. Для разрешённых community-источников действует интервал не менее 30 минут на сайт, общий для всех его endpoint. Открытая альфа не использует продуктовый allowlist: интерфейс показывает весь корректно загруженный разрешённый каталог, сохраняя требования полноты и модерации.
|
||||
|
||||
Жизненный цикл первоисточника учитывается консервативно: изменённая опубликованная запись снимается с активности до повторной ручной проверки, а исчезнувшая — только после подтверждённого ответа `missing` во время разрешённого планового обращения. Временные ошибки и блокировка доступа не удаляют данные. Admin provenance показывает результат и время последней проверки; повторно появившаяся запись также требует подтверждения модератором.
|
||||
|
||||
На сайте у каждой записи отображается источник, а у агрегированной активности — все вошедшие в расчёт источники. Неполные community-наблюдения публикуются сразу в отдельной ленте «Полевые сигналы» с предупреждением и перечнем отсутствующих полей; до подтверждения полноты они не влияют на индекс клёва. Лента раскрывается серверной кнопкой «Показать ещё», сохраняет выбранные фильтры и ограничена 48 сигналами на страницу. Визуально объединяются только повторы одного ID источника; похожие записи разных площадок остаются самостоятельными наблюдениями. Sidebar лидера скрывается при единственном результате, чтобы не повторять ту же карточку.
|
||||
|
||||
Базовый SEO-контур готов для `rf4spotter.ru`: страницы имеют уникальные метаданные, canonical, Open Graph/Twitter Card, фирменное изображение 1200×630 и JSON-LD; доступны динамические `/robots.txt` и `/sitemap.xml`, административные и ошибочные страницы закрыты от индексации, добавлена собственная страница 404. Индексируемые каталоги рыб и водоёмов, detail-страницы и сочетания водоём + рыба строятся из актуального разрешённого справочника и включаются в sitemap. Подключены фирменные favicon/app icons, web manifest и production-кэширование статических ресурсов. Карточки активности показывают единый паспорт данных: источники, свежесть, полноту и уровень доверия. На `/status` опубликована легенда цветов всех источников и статусов качества.
|
||||
Базовый SEO-контур готов для `rf4spotter.ru`: страницы имеют уникальные метаданные, canonical, Open Graph/Twitter Card, фирменное изображение 1200×630 и JSON-LD; доступны динамические `/robots.txt` и `/sitemap.xml`, административные и ошибочные страницы закрыты от индексации, добавлена собственная страница 404. Индексируемые каталоги рыб и водоёмов, detail-страницы и сочетания водоём + рыба строятся из актуального разрешённого справочника и включаются в sitemap. Подключены резкие favicon/app icons из SVG-мастера, отдельные полнофоновые maskable-иконки, web manifest и production-кэширование статических ресурсов. Карточки активности показывают единый паспорт данных: источники, свежесть, полноту и уровень доверия. На `/status` опубликована легенда цветов всех источников и статусов качества.
|
||||
|
||||
Публичные точки используют постоянные читаемые адреса вида `/spots/kuori-85x92`; старые UUID-адреса остаются совместимыми и перенаправляются на канонический URL. На странице точки координаты дополнительно показаны фирменным радаром, который не имитирует отсутствующую географию водоёма, а уловы за 72 часа — шкалой-леской с 12-часовым шагом. Каждый улов показывает источник, относительную свежесть и точное время UTC; время получения явно отделено от времени улова. Карточки активности и каталог дополнены лёгкими SVG-силуэтами рыб без внешних графических зависимостей. Пустые и аварийные состояния используют собственную CSS-иллюстрацию поплавка; анимация учитывает системное ограничение движения.
|
||||
Публичные точки используют постоянные читаемые адреса вида `/spots/kuori-85x92`; старые UUID-адреса остаются совместимыми и перенаправляются на канонический URL. На странице точки координаты дополнительно показаны фирменным радаром, который не имитирует отсутствующую географию водоёма, а уловы за 72 часа — шкалой-леской с 12-часовым шагом. Каждый улов показывает источник, относительную свежесть и точное время UTC; время получения явно отделено от времени улова. Каталоги оформлены как полевой атлас: тёмный seal показывает объём справочника, карточки рыб используют смысловые SVG-силуэты, а каждый водоём — собственный детерминированный абстрактный отпечаток берега, волн, точки и индекса. На странице сочетания оба знака собираются в единую атласную эмблему, detail-иерархию связывает breadcrumb-леска с текстовыми узлами, а боковые переходы повторяют знаки связанных сущностей. Это не карта и не игровая география. Пустые состояния используют статичную CSS-иллюстрацию поплавка; смысловые анимации полностью учитывают системное ограничение движения.
|
||||
|
||||
Все пять community-парсеров подключены к отдельному scheduler-процессу. Попытка резервируется в PostgreSQL до HTTP-запроса, поэтому ошибки тоже расходуют cooldown. Блокировка и минимальный интервал 1800 секунд действуют на весь домен; endpoint одного сайта выбираются по самому давнему запуску и не голодают. Ручной production-запуск использует тот же журнал: `docker compose exec api python -m app.cli fetch-community rf4stat-fishing`. Локально scheduler включается профилем `docker compose --profile scheduler up -d`; detail-URL RF4MAP/RF4 Posts задаются переменными окружения.
|
||||
Все пять community-парсеров подключены к отдельному scheduler-процессу. Попытка резервируется в PostgreSQL до HTTP-запроса, поэтому ошибки тоже расходуют cooldown. Блокировка и минимальный интервал 1800 секунд действуют на весь домен; endpoint одного сайта выбираются по самому давнему запуску и не голодают. Тот же запрос служит проверкой точной исходной ссылки: `404/410` означает `missing`, `401/403/429` — `blocked`, остальные сбои — `temporary_error`; отдельного link-checker и дополнительных обращений нет. Пропажа элемента из агрегатного списка сама по себе удалением не считается. Ручной production-запуск использует тот же журнал: `docker compose exec api python -m app.cli fetch-community rf4stat-fishing`. Локально scheduler включается профилем `docker compose --profile scheduler up -d`; detail-URL RF4MAP/RF4 Posts задаются переменными окружения.
|
||||
|
||||
Медиасборщик индексирует разрешённые изображения отдельно от публичного каталога: manifest хранит исходную страницу, URL, предполагаемый тип сущности и время обнаружения, а оригиналы сохраняются по SHA-256 без hotlink. После явного разрешения владельца от 14 сентября все скачанные и целостные материалы опубликованы в `/media`; публичный API отдаёт Git-копии по content-addressed URL, а каждая карточка показывает плашку и прямую ссылку на источник. Будущие загрузки по-прежнему не одобряются автоматически. Локальный `media_cli --audit` без сетевых запросов проверяет хэши, файлы, MIME, размеры, approved-сопоставления и отсутствие бесхозных оригиналов.
|
||||
|
||||
Актуальный offline-срез от 20.09.2026: 704 записи manifest, 466 `approved`, 226 `superseded`, 1 `duplicate` и 11 `invalid`; audit проходит без ошибок и orphan-файлов. У 252 из 253 рыбных изображений есть 1024×1024 WebP, один 48×48 fallback сохранён из-за отсутствия проверенной альтернативы. Производные WebP/AVIF и provenance хранятся в Git; визуальная browser-приёмка остаётся отдельным пунктом B25.
|
||||
|
||||
`python -m rf4_research.media_cli --coverage` сравнивает manifest с датированным `data/media/catalog-baseline.json`: отдельно считает уникальные нормализованные подписи и кандидатов без подписи, поэтому альтернативные URL не завышают покрытие. Актуальный manifest содержит 704 записи: 466 approved, 226 superseded, 1 duplicate и 11 invalid; queue-представления больше нет. Опубликованы 253 fish-файла, все 149 найденных изображений снастей/приманок и 64 справочных материала; общий target снастей остаётся `null` до проверяемого полного счётчика. Водоёмы имеют отдельный canonical index из 19 карточек; detail-наполнение остаётся W02–W08.
|
||||
|
||||
`python -m rf4_research.media_cli --quality-report` выполняет offline-проверку разрешения опубликованных рыб. Контур quality-upgrade обработал 226 прямых RF4DB-альтернатив: 252 из 253 рыбных изображений теперь имеют 1024×1024 WebP, один 48×48 fallback сохранён из-за отсутствия проверенной альтернативы. `--queue-quality-upgrades` по-прежнему переводит только прямые альтернативы низкоразрешённых published-файлов в безопасную очередь, не снимая текущую версию с публикации; B21/B22 завершены, а browser-приёмка остаётся отдельным пунктом B25.
|
||||
|
||||
`python -m rf4_research.media_cli --queue-plan` без сетевых запросов объединяет manifest с общим cooldown-state: показывает queued-состав каждого домена, оставшееся время и наиболее полезный следующий asset с приоритетом водоёмов и рыб. Разрешённое media-окно загружается командой `--download-batch --batch-limit 40`: до 40 assets на домен под одной резервацией, затем 30 минут до нового batch. Блокировка/rate limit/сетевая ошибка останавливает домен сразу, три последовательных невалидных ответа — досрочно. Все файлы остаются в карантине до ручного review. Точный поимённый список отсутствующих сущностей появится только после получения канонического перечня; разница между двумя несогласованными каталогами не выдаётся за доказанный gap.
|
||||
|
||||
Актуальный внешний ориентир — 19 водоёмов и 252 вида рыб; локальная альфа пока содержит 2+2 справочные сущности. Media-manifest включает 704 записи из RF4MAP, RF4DB и официального руководства: 456 опубликованы, 227 являются альтернативными дубликатами, 10 ожидают загрузки и 11 URL невалидны. Все 456 оригиналов находятся в Git; подтверждённых entity-карт водоёмов пока нет. Полное число «снастей» не заявляется: приманки — лишь одна часть каталога наряду с удилищами, катушками, лесками, крючками и оснастками.
|
||||
|
||||
Исследовательские RF4-ассеты в `data/media/files/` версионируются обычным Git вместе с `data/media/manifest.json`, чтобы клон репозитория был самодостаточным и не зависел от локального кэша. Это не относится к пользовательским скриншотам: они по-прежнему хранятся в MinIO/S3 и не попадают в Git.
|
||||
|
||||
Для измерений на собственном сервере подготовлен read-only `deploy/load-smoke.py`: он считает p50/p95/max и HTTP-коды для activity/records, а при наличии `ADMIN_TOKEN` — staging/moderation. Методика и безопасные ступени нагрузки описаны в [docs/load-testing.md](docs/load-testing.md); локальные цифры не выдаются за production baseline.
|
||||
|
||||
До сервера запросы проверяются командой `./deploy/test-query-plans.sh`: session-local TEMP-fixture на 100 000 уловов не меняет рабочую БД и требует индексные планы для activity, records и spot detail, а также выполнение пяти публичных планов быстрее 250 мс. Методика и последний локальный результат находятся в [docs/query-performance.md](docs/query-performance.md); новые индексы по текущему измерению не требуются.
|
||||
|
||||
В production MinIO root credentials доступны только одноразовому init-контейнеру. API использует отдельного пользователя с доступом исключительно к `S3_BUCKET`: просмотр bucket, чтение, запись и удаление его объектов без глобального списка bucket и без права создавать новые.
|
||||
|
||||
Production release отделяет Alembic от runtime: одноразовый `migrate` должен успешно завершиться до запуска новой версии API. Перед изменением схемы создаётся backup; совместимый rollback возвращает предыдущие images, несовместимый — восстанавливает предрелизную копию данных вместо непроверенного `alembic downgrade`.
|
||||
|
||||
Путь обновления схемы проверяется изолированным `deploy/test-release-upgrade.sh`: предыдущая ревизия получает контрольную запись, обновляется до head, после чего проверяются версия, сохранность записи и новые колонки.
|
||||
|
||||
Тяжёлый production bootstrap вынесен в отдельный ручной/еженедельный CI workflow с 30-минутным timeout и сохраняемыми diagnostics; обычный push по-прежнему использует быстрый Compose E2E.
|
||||
|
||||
Публичный API зафиксирован генерируемым [OpenAPI-контрактом](docs/api-contract.md): CI сравнивает `apps/api/openapi.json` с фактической схемой FastAPI, поэтому рефакторинг routers не может незаметно изменить URL, параметры или response models. Catalog, activity/spots, public data, submissions и admin API принадлежат отдельным `APIRouter`; `main.py` служит компактной точкой сборки приложения, а проверка доверенных proxy и persistent rate limit изолированы в `submission_security`.
|
||||
|
||||
После повторных ошибок scheduler увеличивает паузу экспоненциально до 24 часов и возвращается к 30 минутам после успеха. Публичная страница `/status` показывает свежесть и состояние источников без URL запросов, внутренних ошибок и другой диагностической информации.
|
||||
|
||||
@@ -45,6 +81,10 @@ Production-контур для домена `rf4spotter.ru`, TLS, секреты
|
||||
Политика минимизации данных и ежедневная dry-run-first очистка описаны в [`docs/data-retention.md`](docs/data-retention.md).
|
||||
|
||||
Host-side мониторинг контейнеров, readiness, диска, объёма PostgreSQL/MinIO, резервных копий и TLS описан в [`docs/production-monitoring.md`](docs/production-monitoring.md).
|
||||
|
||||
Минимальные SLI открытой альфы, стартовые пороги и правила безопасной телеметрии собраны в [observability plan](docs/observability.md). До выбора сервера используются существующие JSON-логи, readiness, diagnostics и host monitor; отдельный metrics-стек заранее не добавляется.
|
||||
|
||||
Принятые границы стека, memory-cache, scheduler и хранилищ зафиксированы в [архитектурных решениях](docs/architecture-decisions.md). Порядок действий при заполнении диска, отказах PostgreSQL/MinIO, зависшем импорте, ошибке миграции и утечке секрета находится в [incident runbook](docs/incident-runbook.md).
|
||||
Ежедневный systemd timer создаёт проверяемую копию до retention-очистки, а production Compose ограничивает рост JSON-логов контейнеров.
|
||||
|
||||
Фактическое состояние DNS/TLS домена и серверный чек-лист ведутся в [`docs/deployment-status.md`](docs/deployment-status.md).
|
||||
@@ -63,9 +103,13 @@ FastAPI ─ PostgreSQL 17
|
||||
|
||||
Наружу production-профиль публикует только Caddy. PostgreSQL, API, Astro и MinIO находятся в Docker-сетях. Caddy завершает TLS и защищает административные страницы Basic Auth; административный API отдельно проверяет Bearer-токен в FastAPI. Basic не накладывается на API-запросы.
|
||||
|
||||
Production CSP ограничивает browser-запросы текущим доменом и отдельным files-доменом для изображений, запрещает plugins, frames, inline handlers и attributes, `unsafe-inline`, eval, wildcard и HTTP. Page scripts и scoped styles выпускаются отдельными same-origin `_astro`-ассетами; динамический JSON-LD получает новый криптографический nonce на каждый SSR-ответ. Caddy сохраняет эту policy и задаёт строгий fallback для служебных ответов. Локальный Compose отдельно разрешает только loopback API/MinIO; детали и проверка описаны в [CSP inventory](docs/csp-inventory.md).
|
||||
|
||||
Тема по умолчанию следует системному `prefers-color-scheme`, а переключатель в header позволяет выбрать системную, светлую или тёмную палитру. Выбор сохраняется в cookie и применяется Astro при SSR без localStorage-only flash и ослабления CSP; browser chrome синхронизируется парными `theme-color`. Публичные и административные поверхности, формы, таблицы, provenance/status-плашки и фирменная SVG/CSS-графика используют семантические light/dark-токены и базовый forced-colors layer; ограничения и оставшаяся визуальная приёмка описаны в [dark-theme.md](docs/dark-theme.md).
|
||||
|
||||
Gitea Actions workflow `.gitea/workflows/ci.yml` на каждый push и pull request проверяет Python, миграции на чистой PostgreSQL, Astro build и полный Compose/Playwright-сценарий. При падении E2E сохраняются логи контейнеров и Playwright-артефакты.
|
||||
|
||||
Актуальная инвентаризация источников и правила подключения адаптеров находятся в [`docs/data-source-audit.md`](docs/data-source-audit.md). Разрешённый технический пилот RF4DB/RF4-STAT описан в [`docs/community-source-pilot.md`](docs/community-source-pilot.md), а статус разрешений и лимитов — в [`docs/data-permissions.md`](docs/data-permissions.md). Данные сохраняются только в промежуточный staging и не влияют на индекс без явной проверки и публикации администратором.
|
||||
Актуальная инвентаризация источников и правила подключения адаптеров находятся в [`docs/data-source-audit.md`](docs/data-source-audit.md). Разрешённый технический пилот RF4DB/RF4-STAT описан в [`docs/community-source-pilot.md`](docs/community-source-pilot.md), а статус разрешений и лимитов — в [`docs/data-permissions.md`](docs/data-permissions.md). Все наблюдения сначала попадают в staging. Полные записи с уже подтверждёнными алиасами могут публиковаться автоматически; новые соответствия требуют модерации, а неполные записи показываются отдельно и не влияют на индекс.
|
||||
|
||||
Один ограниченный снимок публичных карточек можно получить исследовательским CLI:
|
||||
|
||||
@@ -86,7 +130,7 @@ python -m rf4_research.community_cli rf4db --limit 25 \
|
||||
| docker compose exec -T api python -m app.cli stage-community-json --input -
|
||||
```
|
||||
|
||||
Staging проверяет происхождение URL и диапазоны значений. Источники по умолчанию выключены; автоматического преобразования в одобренные уловы нет. Ручная очередь доступна по адресу <http://localhost:4321/admin/external-sources>. Публикация разрешена только после сопоставления канонических рыбы и водоёма и при наличии координат и веса.
|
||||
Staging проверяет происхождение URL и диапазоны значений. Источники локально включаются явно; production-профиль запускает разрешённый scheduler. Ручная очередь доступна по адресу <http://localhost:4321/admin/external-sources>. Автопубликация разрешена только для полных наблюдений с ранее подтверждёнными каноническими соответствиями рыбы и водоёма; остальные записи не обходят модерацию.
|
||||
|
||||
## Запуск через Docker
|
||||
|
||||
@@ -104,7 +148,7 @@ docker compose up --build
|
||||
- readiness PostgreSQL, MinIO и импорта с версией/revision сборки: <http://localhost:8000/ready>;
|
||||
- консоль MinIO: <http://localhost:9001>.
|
||||
|
||||
Контейнер API сам выполняет `alembic upgrade head`, затем идемпотентный seed. PostgreSQL хранит данные в именованном volume `postgres_data`, а MinIO — в `minio_data`. Compose ожидает readiness PostgreSQL и MinIO перед API, а API-контейнер проверяет `/ready`. Версия и commit SHA задаются через `APP_VERSION`/`APP_REVISION`; те же значения доступны администратору в `/api/v1/admin/diagnostics`. Официальный импорт по умолчанию необязателен; при включённом scheduler установите `OFFICIAL_IMPORT_REQUIRED=true`, тогда отсутствующий, неуспешный или просроченный запуск сделает readiness отрицательным.
|
||||
Одноразовые контейнеры `migrate` и `minio-init` перед запуском API соответственно применяют `alembic upgrade head` и идемпотентно создают локальный `S3_BUCKET`; приложение само не создаёт схему или bucket. PostgreSQL хранит данные в именованном volume `postgres_data`, а MinIO — в `minio_data`. Compose ожидает readiness PostgreSQL, MinIO и успешное завершение обоих init-контейнеров перед API, а API-контейнер проверяет `/ready`. Версия и commit SHA задаются через `APP_VERSION`/`APP_REVISION`; те же значения доступны администратору в `/api/v1/admin/diagnostics`. Здоровье импортов диагностическое и не мешает API или scheduler восстановиться после сбоя.
|
||||
|
||||
API и scheduler пишут по одной JSON-записи на событие. HTTP-лог содержит только сгенерированный `request_id`, метод, путь без query string, статус и длительность; IP, заголовок авторизации и пользовательский payload не журналируются. `X-Request-ID` возвращается клиенту. Стандартный access-log Uvicorn отключён. Уровень управляется `LOG_LEVEL`. Публичный агрегат активности кэшируется в памяти процесса на 20 секунд (до 128 ключей) и очищается после публикации, модерации или удаления через этот процесс API; изменения scheduler видны после TTL; `X-Cache` показывает `HIT`/`MISS`. Защищённый `/api/v1/admin/diagnostics` скачивает JSON только с идентификатором сборки и агрегированными счётчиками, без имён игроков, исходных URL, payload и ошибок парсеров.
|
||||
|
||||
@@ -140,7 +184,7 @@ docker compose up --build
|
||||
## Что реализовано
|
||||
|
||||
- FastAPI и SQLAlchemy 2;
|
||||
- PostgreSQL 17 и миграции Alembic до `0013`;
|
||||
- PostgreSQL 17 и линейные миграции Alembic до `0020`;
|
||||
- идемпотентный seed с двумя точками и свежими демо-уловами;
|
||||
- `GET /api/v1/activity` с фильтрами периода, водоёма, рыбы, способа и сортировки;
|
||||
- `GET /api/v1/spots/{id}` и `/catches`;
|
||||
@@ -226,9 +270,21 @@ curl -H "Authorization: Bearer change-me-in-production" \
|
||||
|
||||
Если создание записи прошло успешно, а загрузка скриншота завершилась ошибкой, форма сохраняет на один час ID заявки и одноразовый секрет в защищённой `HttpOnly` cookie и предлагает повторить только загрузку изображения. Повторно отправлять сам улов не требуется; один UUID заявки не даёт права изменить чужую запись.
|
||||
|
||||
Очередь модерации доступна по адресу <http://localhost:4321/admin/moderation>. Администратор вводит `ADMIN_TOKEN`; интерфейс держит его только в памяти открытой страницы и не сохраняет в URL или браузерном хранилище.
|
||||
Единый dashboard доступен по адресу <http://localhost:4321/admin>, очереди — `/admin/moderation` и `/admin/external-sources`. Dashboard показывает объём очередей, состояние источников и последние импорты без внутренних URL и текстов ошибок. Администратор вводит `ADMIN_TOKEN`; интерфейс держит его только в памяти открытой страницы и не сохраняет в URL или браузерном хранилище. После 15 минут бездействия или ответа API `401` сессия очищается; также доступен явный выход.
|
||||
|
||||
Администратор может одобрить, отклонить или удалить сообщение. Удаление очищает ник, комментарий, исходную ссылку и объект скриншота, исключает запись из статистики, но сохраняет обезличенный факт действия в журнале аудита.
|
||||
В production HTML административных страниц дополнительно закрыт Caddy Basic Auth, а API независимо проверяет Bearer-токен. Неуспешные попытки API-входа считаются в БД по HMAC-идентификатору адреса и временно блокируются после десяти ошибок за десять минут; успешная авторизация очищает ошибки клиента. Интерфейс открывает только ссылки со схемой `http` или `https`; данные источника не могут подставить исполняемую URL-схему в ссылку или превью.
|
||||
|
||||
Администратор может одобрить, отклонить или удалить сообщение. Очередь внешних наблюдений фильтруется на сервере по источнику и полноте, ищет рыбу/водоём и сортируется по свежести либо риску до применения пагинации; риск поднимает неполные и несопоставленные записи. Во время решения вся карточка блокируется; при ошибке введённая причина остаётся на месте, а после успеха интерфейс сообщает результат и переводит фокус к следующей записи. Удаление очищает ник, комментарий, исходную ссылку и объект скриншота, исключает запись из статистики, но сохраняет обезличенный факт действия в журнале аудита.
|
||||
|
||||
Для карточки с клавиатурным фокусом доступны подсказанные в интерфейсе быстрые клавиши одобрения, сопоставления и публикации. Они не срабатывают в полях ввода; отклонение и удаление требуют явного нажатия кнопки.
|
||||
|
||||
Административный API внешней очереди возвращает безопасный provenance: время первого и последнего обнаружения, время проверки, отсутствующие поля и только разрешённые скалярные поля исходной записи. Неизвестные ключи и вложенные служебные структуры в ответ не попадают.
|
||||
|
||||
В карточке внешнего наблюдения provenance доступен в отдельном раскрываемом блоке: временная линия, отсутствующие и исходные разрешённые поля видны до сопоставления и публикации.
|
||||
|
||||
`GET /api/v1/admin/moderation-history` объединяет историю решений по пользовательским уловам и внешним наблюдениям; последние события видны на dashboard. Ответ содержит только тип и UUID сущности, время, действие, оператора и причину — без ников, исходных URL и parser payload. Dashboard выгружает отдельный `moderation-history-export`: в нём дополнительно исключены UUID, оператор и свободный текст причины, остаются только время, тип, действие и признак необходимости подтверждения.
|
||||
|
||||
Решения в обеих очередях используют optimistic locking: API возвращает `moderation_version`, а изменяющий запрос обязан прислать увиденное значение. Проверка выполняется под блокировкой строки; если другая вкладка уже решила запись, сервер отвечает `409`, UI обновляет очередь и не перезаписывает более новое решение.
|
||||
|
||||
## Эксплуатация production
|
||||
|
||||
@@ -245,7 +301,7 @@ Production-логи структурированы в JSON и не содерж
|
||||
- нет пользовательских аккаунтов, OCR, Telegram-бота и уведомлений о клёве;
|
||||
- community-источники автоматически включают в активность только полные наблюдения с подтверждёнными external-ID алиасами; fallback alias по имени ещё не используется автопубликацией. Неполные наблюдения видны в «Полевых сигналах», но не влияют на индекс;
|
||||
- offset pagination рассчитана на пилотные объёмы, не на бесконечную ленту;
|
||||
- прежний локальный Lighthouse показывал LCP 9,3 с; после него hero уменьшен с 1,6 МБ до 71 КБ и получил высокий приоритет загрузки, повторный production-замер выполняется после размещения;
|
||||
- локальный Lighthouse production-сборки 12 сентября показал performance 100, LCP 1,66 с, CLS 0,023 и TBT 9 мс; это лабораторный baseline, полевой INP и серверные метрики появятся после размещения;
|
||||
- один сервер остаётся точкой отказа, поэтому обязательны внешний backup и мониторинг;
|
||||
- текущий публичный DNS/TLS не подтверждён, см. [статус развёртывания](docs/deployment-status.md).
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@ RUN pip install --no-cache-dir -r requirements-lock.txt
|
||||
RUN useradd --create-home --uid 10001 rf4
|
||||
COPY --chown=rf4:rf4 apps/api .
|
||||
COPY --chown=rf4:rf4 rf4_research ./rf4_research
|
||||
COPY --chown=rf4:rf4 data/media ./data/media
|
||||
USER rf4
|
||||
EXPOSE 8000
|
||||
CMD ["sh", "-c", "alembic upgrade head && python -m app.seed && uvicorn app.main:app --host 0.0.0.0 --port 8000 --no-access-log"]
|
||||
CMD ["sh", "-c", "python -m app.seed && uvicorn app.main:app --host 0.0.0.0 --port 8000 --no-access-log"]
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
"""Add persistent administrative authentication rate limit."""
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
revision = "0014"
|
||||
down_revision = "20260910_recovery"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"admin_auth_attempt",
|
||||
sa.Column("id", sa.Uuid(), nullable=False),
|
||||
sa.Column("client_hash", sa.String(length=64), nullable=False),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
)
|
||||
op.create_index("ix_admin_auth_attempt_client_hash", "admin_auth_attempt", ["client_hash"])
|
||||
op.create_index("ix_admin_auth_attempt_created_at", "admin_auth_attempt", ["created_at"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("ix_admin_auth_attempt_created_at", table_name="admin_auth_attempt")
|
||||
op.drop_index("ix_admin_auth_attempt_client_hash", table_name="admin_auth_attempt")
|
||||
op.drop_table("admin_auth_attempt")
|
||||
@@ -0,0 +1,22 @@
|
||||
"""add optimistic moderation versions
|
||||
|
||||
Revision ID: 0015
|
||||
Revises: 0014
|
||||
"""
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
revision = "0015"
|
||||
down_revision = "0014"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column("catch_report", sa.Column("moderation_version", sa.Integer(), nullable=False, server_default="0"))
|
||||
op.add_column("external_observation", sa.Column("moderation_version", sa.Integer(), nullable=False, server_default="0"))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("external_observation", "moderation_version")
|
||||
op.drop_column("catch_report", "moderation_version")
|
||||
@@ -0,0 +1,24 @@
|
||||
"""track source record lifecycle checks
|
||||
|
||||
Revision ID: 0016
|
||||
Revises: 0015
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "0016"
|
||||
down_revision = "0015"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column("external_observation", sa.Column("source_check_status", sa.String(length=30)))
|
||||
op.add_column("external_observation", sa.Column("source_checked_at", sa.DateTime(timezone=True)))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("external_observation", "source_checked_at")
|
||||
op.drop_column("external_observation", "source_check_status")
|
||||
@@ -0,0 +1,37 @@
|
||||
"""add canonical waterbody provenance fields
|
||||
|
||||
Revision ID: 0017
|
||||
Revises: 0016
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "0017"
|
||||
down_revision = "0016"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column("waterbody", sa.Column("source_system", sa.String(length=50), nullable=True))
|
||||
op.add_column("waterbody", sa.Column("source_external_id", sa.String(length=200), nullable=True))
|
||||
op.add_column("waterbody", sa.Column("source_url", sa.Text(), nullable=True))
|
||||
op.add_column("waterbody", sa.Column("description", sa.Text(), nullable=True))
|
||||
op.add_column("waterbody", sa.Column("source_checked_at", sa.DateTime(timezone=True), nullable=True))
|
||||
op.create_index(
|
||||
"uq_waterbody_source_identity",
|
||||
"waterbody",
|
||||
["source_system", "source_external_id"],
|
||||
unique=True,
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("uq_waterbody_source_identity", table_name="waterbody")
|
||||
op.drop_column("waterbody", "source_checked_at")
|
||||
op.drop_column("waterbody", "description")
|
||||
op.drop_column("waterbody", "source_url")
|
||||
op.drop_column("waterbody", "source_external_id")
|
||||
op.drop_column("waterbody", "source_system")
|
||||
@@ -0,0 +1,26 @@
|
||||
"""store source coordinate text and precision
|
||||
|
||||
Revision ID: 0018
|
||||
Revises: 0017
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "0018"
|
||||
down_revision = "0017"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column("external_observation", sa.Column("coordinate_raw", sa.String(length=200), nullable=True))
|
||||
op.add_column("external_observation", sa.Column("coordinate_precision", sa.String(length=20), nullable=True))
|
||||
op.execute("UPDATE external_observation SET coordinate_precision = CASE WHEN x IS NOT NULL AND y IS NOT NULL THEN 'exact' ELSE 'missing' END")
|
||||
op.alter_column("external_observation", "coordinate_precision", nullable=False, server_default="missing")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("external_observation", "coordinate_precision")
|
||||
op.drop_column("external_observation", "coordinate_raw")
|
||||
@@ -0,0 +1,22 @@
|
||||
"""store canonical waterbody fish count
|
||||
|
||||
Revision ID: 0019
|
||||
Revises: 0018
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "0019"
|
||||
down_revision = "0018"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column("waterbody", sa.Column("fish_species_count", sa.Integer(), nullable=True))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("waterbody", "fish_species_count")
|
||||
@@ -0,0 +1,24 @@
|
||||
"""store canonical waterbody detail facts
|
||||
|
||||
Revision ID: 0020
|
||||
Revises: 0019
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "0020"
|
||||
down_revision = "0019"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
for name in ("source_aliases", "source_fish_species", "source_image_urls", "source_point_urls"):
|
||||
op.add_column("waterbody", sa.Column(name, sa.JSON(), nullable=True))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
for name in ("source_point_urls", "source_image_urls", "source_fish_species", "source_aliases"):
|
||||
op.drop_column("waterbody", name)
|
||||
@@ -0,0 +1,65 @@
|
||||
"""add canonical tackle items and rig components
|
||||
|
||||
Revision ID: 0021
|
||||
Revises: 0020
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "0021"
|
||||
down_revision = "0020"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"tackle_item",
|
||||
sa.Column("id", sa.Uuid(), primary_key=True),
|
||||
sa.Column("name", sa.String(200), nullable=False),
|
||||
sa.Column("normalized_name", sa.String(200), nullable=False, unique=True),
|
||||
sa.Column("category", sa.String(20), nullable=False),
|
||||
sa.Column("subcategory", sa.String(100)),
|
||||
sa.Column("brand", sa.String(100)),
|
||||
sa.Column("family", sa.String(100)),
|
||||
sa.Column("unlock_level", sa.Integer()),
|
||||
sa.Column("source_system", sa.String(50)),
|
||||
sa.Column("source_external_id", sa.String(200)),
|
||||
sa.Column("source_url", sa.Text()),
|
||||
sa.Column("source_checked_at", sa.DateTime(timezone=True)),
|
||||
sa.Column("raw_payload", sa.JSON()),
|
||||
sa.UniqueConstraint("source_system", "source_external_id"),
|
||||
sa.CheckConstraint(
|
||||
"category IN ('bait', 'lure', 'rod', 'reel', 'line', 'hook', 'rig', 'float', 'sinker', 'other')",
|
||||
name="ck_tackle_item_category",
|
||||
),
|
||||
)
|
||||
op.create_table(
|
||||
"rig",
|
||||
sa.Column("id", sa.Uuid(), primary_key=True),
|
||||
sa.Column("name", sa.String(200), nullable=False),
|
||||
sa.Column("normalized_name", sa.String(200), nullable=False, unique=True),
|
||||
sa.Column("source_system", sa.String(50)),
|
||||
sa.Column("source_external_id", sa.String(200)),
|
||||
sa.Column("source_url", sa.Text()),
|
||||
sa.Column("source_checked_at", sa.DateTime(timezone=True)),
|
||||
sa.Column("raw_payload", sa.JSON()),
|
||||
)
|
||||
op.create_table(
|
||||
"rig_component",
|
||||
sa.Column("id", sa.Uuid(), primary_key=True),
|
||||
sa.Column("rig_id", sa.Uuid(), sa.ForeignKey("rig.id"), nullable=False),
|
||||
sa.Column("tackle_item_id", sa.Uuid(), sa.ForeignKey("tackle_item.id")),
|
||||
sa.Column("role", sa.String(50), nullable=False),
|
||||
sa.Column("position", sa.Integer(), nullable=False),
|
||||
sa.Column("raw_value", sa.String(200)),
|
||||
sa.UniqueConstraint("rig_id", "position"),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_table("rig_component")
|
||||
op.drop_table("rig")
|
||||
op.drop_table("tackle_item")
|
||||
@@ -0,0 +1,40 @@
|
||||
"""preserve ordered gear evidence on catches
|
||||
|
||||
Revision ID: 0022
|
||||
Revises: 0021
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "0022"
|
||||
down_revision = "0021"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"catch_tackle_component",
|
||||
sa.Column("id", sa.Uuid(), primary_key=True),
|
||||
sa.Column("catch_report_id", sa.Uuid(), sa.ForeignKey("catch_report.id"), nullable=False),
|
||||
sa.Column("tackle_item_id", sa.Uuid(), sa.ForeignKey("tackle_item.id")),
|
||||
sa.Column("rig_id", sa.Uuid(), sa.ForeignKey("rig.id")),
|
||||
sa.Column("role", sa.String(50), nullable=False),
|
||||
sa.Column("position", sa.Integer(), nullable=False),
|
||||
sa.Column("raw_value", sa.String(200), nullable=False),
|
||||
sa.Column("source_system", sa.String(50)),
|
||||
sa.Column("source_external_id", sa.String(200)),
|
||||
sa.Column("source_url", sa.Text()),
|
||||
sa.Column("raw_payload", sa.JSON()),
|
||||
sa.UniqueConstraint("catch_report_id", "position"),
|
||||
sa.CheckConstraint(
|
||||
"NOT (tackle_item_id IS NOT NULL AND rig_id IS NOT NULL)",
|
||||
name="ck_catch_tackle_one_canonical_target",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_table("catch_tackle_component")
|
||||
@@ -0,0 +1,30 @@
|
||||
"""add recovery idempotency and import history columns
|
||||
|
||||
Revision ID: 20260910_recovery
|
||||
Revises: 48094a7d1b92
|
||||
"""
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
revision = "20260910_recovery"
|
||||
down_revision = "48094a7d1b92"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column("submission_attempt", sa.Column("idempotency_key", sa.String(128), nullable=True))
|
||||
op.add_column("submission_attempt", sa.Column("catch_report_id", sa.Uuid(), nullable=True))
|
||||
op.add_column("submission_attempt", sa.Column("payload_hash", sa.String(64), nullable=True))
|
||||
op.create_foreign_key("fk_submission_attempt_report", "submission_attempt", "catch_report", ["catch_report_id"], ["id"])
|
||||
op.create_index("ix_submission_attempt_idempotency_key", "submission_attempt", ["idempotency_key"], unique=True)
|
||||
op.add_column("import_record_event", sa.Column("changes", sa.JSON(), nullable=True))
|
||||
op.add_column("import_record_event", sa.Column("provenance", sa.JSON(), nullable=True))
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("import_record_event", "provenance")
|
||||
op.drop_column("import_record_event", "changes")
|
||||
op.drop_index("ix_submission_attempt_idempotency_key", table_name="submission_attempt")
|
||||
op.drop_constraint("fk_submission_attempt_report", "submission_attempt", type_="foreignkey")
|
||||
op.drop_column("submission_attempt", "catch_report_id")
|
||||
op.drop_column("submission_attempt", "payload_hash")
|
||||
op.drop_column("submission_attempt", "idempotency_key")
|
||||
@@ -61,6 +61,9 @@ def activity_rows(
|
||||
confidence = min(confidence, 50)
|
||||
elif len(players) == 2:
|
||||
confidence = min(confidence, 65)
|
||||
coordinate_precisions = {_coordinate_precision(item) for item in items}
|
||||
coordinate_precision = max(coordinate_precisions, key=_precision_rank)
|
||||
coordinate_sources = sorted({_source_system(item) for item in items})
|
||||
latest = max(_aware(r.reported_at) for r in items)
|
||||
baits = Counter(r.bait.name for r in items if r.bait)
|
||||
freshness_text = _freshness_text(now - latest)
|
||||
@@ -74,7 +77,9 @@ def activity_rows(
|
||||
max_weight_g=max(r.weight_g for r in items), last_confirmed_at=latest,
|
||||
activity_score=activity, confidence_score=confidence,
|
||||
explanation=_explanation(len(items), len(players), freshness_text, activity, confidence),
|
||||
sources=sorted({_source_system(item) for item in items}),
|
||||
sources=coordinate_sources,
|
||||
coordinate_precision=coordinate_precision,
|
||||
coordinate_sources=coordinate_sources,
|
||||
))
|
||||
return sorted(result, key=lambda row: (row.activity_score, row.last_confirmed_at), reverse=True)
|
||||
|
||||
@@ -90,6 +95,16 @@ def _source_system(report: CatchReport) -> str:
|
||||
return "manual-import"
|
||||
|
||||
|
||||
def _coordinate_precision(report: CatchReport) -> str:
|
||||
provenance = (report.raw_payload or {}).get("provenance", {})
|
||||
value = provenance.get("coordinate_precision") if isinstance(provenance, dict) else None
|
||||
return value if value in {"exact", "approximate", "area", "missing"} else "exact"
|
||||
|
||||
|
||||
def _precision_rank(value: str) -> int:
|
||||
return {"exact": 0, "approximate": 1, "area": 2, "missing": 3}[value]
|
||||
|
||||
|
||||
def _aware(value: datetime) -> datetime:
|
||||
return value if value.tzinfo else value.replace(tzinfo=timezone.utc)
|
||||
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
from datetime import datetime, timedelta, timezone
|
||||
import hashlib
|
||||
import hmac
|
||||
|
||||
from fastapi import HTTPException, Request
|
||||
from sqlalchemy import delete, func, select, text
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .models import AdminAuthAttempt
|
||||
from .submission_security import RateLimitConfig, client_address
|
||||
|
||||
|
||||
class AdminAuthConfig(RateLimitConfig):
|
||||
admin_token: str
|
||||
admin_auth_attempt_limit: int
|
||||
admin_auth_window_seconds: int
|
||||
|
||||
|
||||
def verify_admin(
|
||||
request: Request,
|
||||
db: Session,
|
||||
authorization: str | None,
|
||||
config: AdminAuthConfig,
|
||||
) -> str:
|
||||
now = datetime.now(timezone.utc)
|
||||
cutoff = now - timedelta(seconds=config.admin_auth_window_seconds)
|
||||
client = client_address(request, config.trusted_proxy_cidrs)
|
||||
client_hash = hmac.new(
|
||||
config.rate_limit_secret.encode(), f"admin:{client}".encode(), hashlib.sha256
|
||||
).hexdigest()
|
||||
if db.get_bind().dialect.name == "postgresql":
|
||||
lock_key = int(client_hash[:16], 16) & 0x7FFF_FFFF_FFFF_FFFF
|
||||
db.execute(text("SELECT pg_advisory_xact_lock(:lock_key)"), {"lock_key": lock_key})
|
||||
db.execute(delete(AdminAuthAttempt).where(AdminAuthAttempt.created_at < now - timedelta(days=1)))
|
||||
failures = db.scalar(
|
||||
select(func.count()).select_from(AdminAuthAttempt).where(
|
||||
AdminAuthAttempt.client_hash == client_hash,
|
||||
AdminAuthAttempt.created_at >= cutoff,
|
||||
)
|
||||
) or 0
|
||||
if failures >= config.admin_auth_attempt_limit:
|
||||
db.commit()
|
||||
raise HTTPException(
|
||||
status_code=429,
|
||||
detail="too many admin authentication attempts",
|
||||
headers={"Retry-After": str(config.admin_auth_window_seconds)},
|
||||
)
|
||||
expected = f"Bearer {config.admin_token}"
|
||||
if not authorization or not hmac.compare_digest(authorization, expected):
|
||||
db.add(AdminAuthAttempt(client_hash=client_hash, created_at=now))
|
||||
db.commit()
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="invalid admin token",
|
||||
headers={"WWW-Authenticate": "Bearer"},
|
||||
)
|
||||
db.execute(delete(AdminAuthAttempt).where(AdminAuthAttempt.client_hash == client_hash))
|
||||
db.commit()
|
||||
return "admin"
|
||||
@@ -15,3 +15,28 @@ def audit_catalog(db: Session) -> dict[str, int]:
|
||||
"incomplete_published_staging": db.scalar(select(func.count()).select_from(ExternalObservation).where(ExternalObservation.status == "published", or_(ExternalObservation.fish_id.is_(None), ExternalObservation.waterbody_id.is_(None), ExternalObservation.x.is_(None), ExternalObservation.y.is_(None), ExternalObservation.weight_g.is_(None), ExternalObservation.catch_report_id.is_(None)))) or 0,
|
||||
}
|
||||
return {"fishes": count(Fish), "waterbodies": count(Waterbody), "reports": count(CatchReport), "staging": count(ExternalObservation), **failures, "failures": sum(failures.values())}
|
||||
|
||||
|
||||
def audit_waterbody_catalog(db: Session, expected_ids: set[str]) -> dict:
|
||||
"""Check a verified RF4DB snapshot without withdrawing legacy rows."""
|
||||
rows = list(db.scalars(select(Waterbody).where(Waterbody.source_system == "rf4db")))
|
||||
observed_ids = [str(row.source_external_id) for row in rows if row.source_external_id]
|
||||
observed = set(observed_ids)
|
||||
duplicate_ids = sorted({item for item in observed_ids if observed_ids.count(item) > 1})
|
||||
missing = sorted(expected_ids - observed)
|
||||
unexpected = sorted(observed - expected_ids)
|
||||
provenance_issues = sorted(
|
||||
str(row.source_external_id)
|
||||
for row in rows
|
||||
if not row.source_external_id or not row.source_url or not row.source_checked_at
|
||||
)
|
||||
failures = len(missing) + len(duplicate_ids) + len(provenance_issues)
|
||||
return {
|
||||
"expected": len(expected_ids),
|
||||
"observed": len(observed),
|
||||
"missing_source_external_ids": missing,
|
||||
"unexpected_source_external_ids": unexpected,
|
||||
"duplicate_source_external_ids": duplicate_ids,
|
||||
"provenance_issues": provenance_issues,
|
||||
"failures": failures,
|
||||
}
|
||||
|
||||
@@ -8,10 +8,10 @@ from dataclasses import asdict
|
||||
from .config import settings
|
||||
from .database import SessionLocal
|
||||
from .importer import import_records
|
||||
from .community_importer import stage_observations
|
||||
from .community_importer import stage_observations, update_waterbody_detail, update_waterbody_details, upsert_waterbody_catalog
|
||||
from .retention import RetentionPolicy, apply_retention
|
||||
from .storage import delete_screenshot
|
||||
from .catalog_audit import audit_catalog
|
||||
from .catalog_audit import audit_catalog, audit_waterbody_catalog
|
||||
from .community_scheduler import run_source, configured_sources
|
||||
|
||||
# Static registry for argparse choices — no DB required for --help
|
||||
@@ -34,11 +34,20 @@ def main() -> int:
|
||||
community = sub.add_parser("stage-community-json")
|
||||
community.add_argument("--input", default="-", help="JSON array path or - for stdin")
|
||||
community.add_argument("--limit", type=int, default=500)
|
||||
waterbodies = sub.add_parser("import-waterbody-catalog")
|
||||
waterbodies.add_argument("--input", required=True, help="JSON snapshot path or - for stdin")
|
||||
waterbodies.add_argument("--limit", type=int, default=100)
|
||||
detail = sub.add_parser("import-waterbody-detail")
|
||||
detail.add_argument("--input", required=True, help="JSON detail snapshot path")
|
||||
details = sub.add_parser("import-waterbody-details")
|
||||
details.add_argument("--input", required=True, help="JSON array of detail snapshots")
|
||||
fetch_community = sub.add_parser("fetch-community")
|
||||
fetch_community.add_argument("source", choices=STATIC_SOURCE_CHOICES)
|
||||
cleanup = sub.add_parser("cleanup-retention")
|
||||
cleanup.add_argument("--apply", action="store_true", help="apply changes; default is dry-run")
|
||||
sub.add_parser("audit-catalog")
|
||||
waterbody_audit = sub.add_parser("audit-waterbody-catalog")
|
||||
waterbody_audit.add_argument("--input", required=True, help="JSON snapshot path")
|
||||
args = parser.parse_args()
|
||||
with SessionLocal() as session:
|
||||
if args.command == "import-records":
|
||||
@@ -57,6 +66,43 @@ def main() -> int:
|
||||
parser.error("input must be a JSON array")
|
||||
created, updated = stage_observations(session, payload[:args.limit])
|
||||
print(f"staged: created={created} updated={updated}")
|
||||
elif args.command == "import-waterbody-catalog":
|
||||
if not 1 <= args.limit <= 500:
|
||||
parser.error("--limit must be between 1 and 500")
|
||||
stream = sys.stdin if args.input == "-" else open(args.input, encoding="utf-8")
|
||||
try:
|
||||
snapshot = json.load(stream)
|
||||
finally:
|
||||
if stream is not sys.stdin:
|
||||
stream.close()
|
||||
if isinstance(snapshot, dict):
|
||||
payload = snapshot.get("items")
|
||||
source_system = snapshot.get("source_system")
|
||||
if isinstance(payload, list) and isinstance(source_system, str):
|
||||
payload = [
|
||||
{"source_system": source_system, **item}
|
||||
for item in payload if isinstance(item, dict)
|
||||
]
|
||||
else:
|
||||
payload = snapshot
|
||||
if not isinstance(payload, list):
|
||||
parser.error("input must be a JSON array or an object with an items array")
|
||||
created, updated = upsert_waterbody_catalog(session, payload[:args.limit])
|
||||
print(f"waterbodies: created={created} updated={updated}")
|
||||
elif args.command == "import-waterbody-detail":
|
||||
with open(args.input, encoding="utf-8") as stream:
|
||||
payload = json.load(stream)
|
||||
if not isinstance(payload, dict):
|
||||
parser.error("input must be a JSON object")
|
||||
update_waterbody_detail(session, payload)
|
||||
print(f"waterbody detail: updated={payload.get('source_external_id', 'unknown')}")
|
||||
elif args.command == "import-waterbody-details":
|
||||
with open(args.input, encoding="utf-8") as stream:
|
||||
payload = json.load(stream)
|
||||
if not isinstance(payload, list):
|
||||
parser.error("input must be a JSON array")
|
||||
created, updated = update_waterbody_details(session, payload)
|
||||
print(f"waterbody details: created={created} updated={updated}")
|
||||
elif args.command == "fetch-community":
|
||||
# A09: Verify source is enabled at runtime (not just in static choices)
|
||||
enabled = configured_sources()
|
||||
@@ -76,6 +122,24 @@ def main() -> int:
|
||||
)
|
||||
counts = apply_retention(session, policy=policy, dry_run=not args.apply, delete_object=delete_screenshot)
|
||||
print(json.dumps({"mode": "apply" if args.apply else "dry-run", "policy": asdict(policy), "counts": counts}, ensure_ascii=False))
|
||||
elif args.command == "audit-waterbody-catalog":
|
||||
stream = sys.stdin if args.input == "-" else open(args.input, encoding="utf-8")
|
||||
try:
|
||||
snapshot = json.load(stream)
|
||||
finally:
|
||||
if stream is not sys.stdin:
|
||||
stream.close()
|
||||
items = snapshot.get("items") if isinstance(snapshot, dict) else snapshot
|
||||
if not isinstance(items, list):
|
||||
parser.error("input must be a JSON array or an object with an items array")
|
||||
expected_ids = {
|
||||
str(item["source_external_id"])
|
||||
for item in items
|
||||
if isinstance(item, dict) and item.get("source_external_id")
|
||||
}
|
||||
result = audit_waterbody_catalog(session, expected_ids)
|
||||
print(json.dumps(result, ensure_ascii=False))
|
||||
return 1 if result["failures"] else 0
|
||||
else:
|
||||
result = audit_catalog(session)
|
||||
print(json.dumps(result, ensure_ascii=False))
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import hashlib
|
||||
import re
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any, Iterable
|
||||
from urllib.parse import urlparse
|
||||
@@ -26,12 +28,153 @@ SOURCE_HOSTS = {
|
||||
"rf4map": {"rf4map.ru"},
|
||||
"rf4posts-spot": {"rf4-posts.com"},
|
||||
}
|
||||
COORDINATE_PRECISIONS = frozenset({"exact", "approximate", "area", "missing"})
|
||||
|
||||
|
||||
class CommunityImportError(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
def upsert_waterbody_catalog(
|
||||
session: Session, rows: Iterable[dict[str, Any]], *, fetched_at: datetime | None = None,
|
||||
) -> tuple[int, int]:
|
||||
"""Apply a validated canonical waterbody snapshot without destructive sync.
|
||||
|
||||
Rows are matched by the RF4DB source identity first and by an exact existing
|
||||
name second. Missing rows are deliberately left untouched: an incomplete
|
||||
response must never withdraw a previously known waterbody.
|
||||
"""
|
||||
fetched_at = fetched_at or datetime.now(timezone.utc)
|
||||
created = updated = 0
|
||||
for raw in rows:
|
||||
payload = _json_payload(raw)
|
||||
if payload.get("source_system") != "rf4db":
|
||||
raise CommunityImportError("waterbody catalog requires source_system=rf4db")
|
||||
external_id = _required(payload, "source_external_id", 200)
|
||||
name = _required(payload, "name", 200)
|
||||
source_url = _required(payload, "source_url", 2000)
|
||||
parsed_url = urlparse(source_url)
|
||||
if parsed_url.scheme != "https" or parsed_url.hostname not in {"rf4db.com", "www.rf4db.com"}:
|
||||
raise CommunityImportError("waterbody source_url does not match rf4db")
|
||||
unlock_level = _integer(payload.get("unlock_level"), minimum=0, maximum=1_000)
|
||||
unlock_label = _required(payload, "unlock_label", 50)
|
||||
fish_species_count = _integer(payload.get("fish_species_count"), minimum=0, maximum=10_000)
|
||||
if fish_species_count is None:
|
||||
raise CommunityImportError("invalid fish_species_count")
|
||||
|
||||
item = session.scalar(select(Waterbody).where(
|
||||
Waterbody.source_system == "rf4db",
|
||||
Waterbody.source_external_id == external_id,
|
||||
))
|
||||
if item is None:
|
||||
item = session.scalar(select(Waterbody).where(Waterbody.name_ru == name))
|
||||
if item is None:
|
||||
item = Waterbody(
|
||||
slug=_catalog_slug(session, name, external_id),
|
||||
name_ru=name,
|
||||
unlock_level=unlock_level,
|
||||
)
|
||||
session.add(item)
|
||||
created += 1
|
||||
else:
|
||||
updated += 1
|
||||
item.name_ru = name
|
||||
item.unlock_level = unlock_level
|
||||
item.fish_species_count = fish_species_count
|
||||
item.source_system = "rf4db"
|
||||
item.source_external_id = external_id
|
||||
item.source_url = source_url
|
||||
item.source_checked_at = fetched_at
|
||||
session.commit()
|
||||
return created, updated
|
||||
|
||||
|
||||
def update_waterbody_detail(
|
||||
session: Session, detail: dict[str, Any], *, fetched_at: datetime | None = None,
|
||||
) -> bool:
|
||||
"""Persist one complete RF4DB detail snapshot without assigning media roles."""
|
||||
fetched_at = fetched_at or datetime.now(timezone.utc)
|
||||
payload = _validate_waterbody_detail(detail)
|
||||
_apply_waterbody_detail(session, payload, fetched_at=fetched_at)
|
||||
session.commit()
|
||||
return True
|
||||
|
||||
|
||||
def update_waterbody_details(
|
||||
session: Session, details: Iterable[dict[str, Any]], *, fetched_at: datetime | None = None,
|
||||
) -> tuple[int, int]:
|
||||
"""Validate and apply a detail batch in one transaction."""
|
||||
fetched_at = fetched_at or datetime.now(timezone.utc)
|
||||
payloads = [_validate_waterbody_detail(detail) for detail in details]
|
||||
external_ids = [str(payload["source_external_id"]) for payload in payloads]
|
||||
if len(external_ids) != len(set(external_ids)):
|
||||
raise CommunityImportError("waterbody detail batch contains duplicate source identities")
|
||||
updated = 0
|
||||
for payload in payloads:
|
||||
_apply_waterbody_detail(session, payload, fetched_at=fetched_at)
|
||||
updated += 1
|
||||
session.commit()
|
||||
return 0, updated
|
||||
|
||||
|
||||
def _validate_waterbody_detail(detail: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = _json_payload(detail)
|
||||
if payload.get("source_system") != "rf4db":
|
||||
raise CommunityImportError("waterbody detail requires source_system=rf4db")
|
||||
external_id = _required(payload, "source_external_id", 200)
|
||||
source_url = _required(payload, "source_url", 2000)
|
||||
parsed_url = urlparse(source_url)
|
||||
if parsed_url.scheme != "https" or parsed_url.hostname not in {"rf4db.com", "www.rf4db.com", "download.rf4db.com"}:
|
||||
raise CommunityImportError("waterbody detail source_url does not match rf4db")
|
||||
_required(payload, "name", 200)
|
||||
_optional(payload, "description", 20_000)
|
||||
_string_list(payload, "aliases", 100, 200)
|
||||
_string_list(payload, "fish_species", 10_000, 200)
|
||||
_string_list(payload, "image_urls", 100, 2_000)
|
||||
_string_list(payload, "point_urls", 10_000, 2_000)
|
||||
return payload
|
||||
|
||||
|
||||
def _apply_waterbody_detail(session: Session, payload: dict[str, Any], *, fetched_at: datetime) -> None:
|
||||
external_id = str(payload["source_external_id"])
|
||||
source_url = str(payload["source_url"])
|
||||
item = session.scalar(select(Waterbody).where(
|
||||
Waterbody.source_system == "rf4db", Waterbody.source_external_id == external_id,
|
||||
))
|
||||
if item is None:
|
||||
raise CommunityImportError("waterbody detail has no imported catalog identity")
|
||||
item.description = _optional(payload, "description", 20_000)
|
||||
item.source_aliases = _string_list(payload, "aliases", 100, 200)
|
||||
item.source_fish_species = _string_list(payload, "fish_species", 10_000, 200)
|
||||
item.source_image_urls = _string_list(payload, "image_urls", 100, 2_000)
|
||||
item.source_point_urls = _string_list(payload, "point_urls", 10_000, 2_000)
|
||||
item.source_url = source_url
|
||||
item.source_checked_at = fetched_at
|
||||
|
||||
|
||||
def _catalog_slug(session: Session, name: str, external_id: str) -> str:
|
||||
base = re.sub(r"[^a-z0-9а-яё]+", "-", name.casefold(), flags=re.IGNORECASE).strip("-")
|
||||
base = base or "waterbody"
|
||||
candidate = base[:100]
|
||||
if session.scalar(select(Waterbody.id).where(Waterbody.slug == candidate)) is None:
|
||||
return candidate
|
||||
suffix = hashlib.sha256(external_id.encode()).hexdigest()[:10]
|
||||
return f"{base[:89]}-{suffix}"
|
||||
|
||||
|
||||
def _string_list(payload: dict[str, Any], key: str, max_items: int, max_length: int) -> list[str]:
|
||||
value = payload.get(key)
|
||||
if not isinstance(value, list) or len(value) > max_items:
|
||||
raise CommunityImportError(f"invalid {key}")
|
||||
result = []
|
||||
for item in value:
|
||||
text = str(item).strip()
|
||||
if not text or len(text) > max_length:
|
||||
raise CommunityImportError(f"invalid {key}")
|
||||
result.append(text)
|
||||
return list(dict.fromkeys(result))
|
||||
|
||||
|
||||
def stage_observations(
|
||||
session: Session, records: Iterable[dict[str, Any]], *, fetched_at: datetime | None = None,
|
||||
) -> tuple[int, int]:
|
||||
@@ -65,9 +208,12 @@ def stage_observations(
|
||||
"waterbody_external_id": _optional(payload, "waterbody_external_id", 200),
|
||||
"x": _integer(payload.get("x"), maximum=10_000),
|
||||
"y": _integer(payload.get("y"), maximum=10_000),
|
||||
"coordinate_raw": _coordinate_raw(payload),
|
||||
"coordinate_precision": _coordinate_precision(payload),
|
||||
"weight_g": _integer(payload.get("weight_g"), minimum=1, maximum=3_000_000),
|
||||
"published_at": _datetime(payload.get("published_at")),
|
||||
"last_seen_at": fetched_at, "payload": payload,
|
||||
"source_check_status": "available", "source_checked_at": fetched_at,
|
||||
}
|
||||
if observation is None:
|
||||
observation = ExternalObservation(
|
||||
@@ -82,6 +228,7 @@ def stage_observations(
|
||||
changed = any(getattr(observation, key) != values[key] for key in (
|
||||
"source_url", "fish_name", "fish_external_id", "waterbody_name",
|
||||
"waterbody_external_id", "x", "y", "weight_g",
|
||||
"coordinate_raw", "coordinate_precision",
|
||||
)) or observation.payload != payload
|
||||
if observation.status != "rejected" and changed and observation.catch_report is not None:
|
||||
observation.catch_report.moderation_status = ModerationStatus.pending
|
||||
@@ -89,8 +236,17 @@ def stage_observations(
|
||||
observation.fish = None
|
||||
observation.waterbody = None
|
||||
observation.review_note = "Source record changed; manual mapping and publication required"
|
||||
observation.reviewed_at = fetched_at
|
||||
observation.moderation_version += 1
|
||||
for key, value in values.items():
|
||||
setattr(observation, key, value)
|
||||
if observation.status == "withdrawn":
|
||||
observation.status = "staged"
|
||||
observation.fish = None
|
||||
observation.waterbody = None
|
||||
observation.review_note = "Source record reappeared; manual confirmation required"
|
||||
observation.reviewed_at = fetched_at
|
||||
observation.moderation_version += 1
|
||||
updated += 1
|
||||
touched.append(observation)
|
||||
session.commit()
|
||||
@@ -176,6 +332,27 @@ def _optional(payload: dict[str, Any], key: str, limit: int) -> str | None:
|
||||
return value or None
|
||||
|
||||
|
||||
def _coordinate_raw(payload: dict[str, Any]) -> str | None:
|
||||
value = str(payload.get("coordinate_raw") or "").strip()
|
||||
if len(value) > 200:
|
||||
raise CommunityImportError("invalid coordinate_raw")
|
||||
if value:
|
||||
return value
|
||||
x, y = payload.get("x"), payload.get("y")
|
||||
return f"{x}:{y}" if isinstance(x, int) and isinstance(y, int) else None
|
||||
|
||||
|
||||
def _coordinate_precision(payload: dict[str, Any]) -> str:
|
||||
value = str(payload.get("coordinate_precision") or "").strip().casefold()
|
||||
if not value:
|
||||
return "exact" if isinstance(payload.get("x"), int) and isinstance(payload.get("y"), int) else "missing"
|
||||
if value not in COORDINATE_PRECISIONS:
|
||||
raise CommunityImportError("invalid coordinate_precision")
|
||||
if value == "exact" and (not isinstance(payload.get("x"), int) or not isinstance(payload.get("y"), int)):
|
||||
raise CommunityImportError("exact coordinates require x and y")
|
||||
return value
|
||||
|
||||
|
||||
def _integer(value: Any, *, minimum: int = -10_000, maximum: int) -> int | None:
|
||||
if value is None:
|
||||
return None
|
||||
|
||||
@@ -11,6 +11,8 @@ from .models import (
|
||||
Bait, BaitKind, CatchReport, ExternalEntityAlias, ExternalObservation,
|
||||
Fish, ModerationStatus, SourceType, Spot, Waterbody,
|
||||
)
|
||||
from .tackle_components import replace_tackle_components
|
||||
from rf4_research.gear_components import from_catch_fields
|
||||
|
||||
|
||||
class ExternalReviewError(ValueError):
|
||||
@@ -104,6 +106,8 @@ def publish_observation(session: Session, observation: ExternalObservation) -> C
|
||||
"external_observation_id": str(observation.id),
|
||||
"source_system": observation.source_system,
|
||||
"source_external_id": observation.source_external_id,
|
||||
"coordinate_raw": observation.coordinate_raw,
|
||||
"coordinate_precision": observation.coordinate_precision,
|
||||
},
|
||||
"original": observation.payload,
|
||||
},
|
||||
@@ -116,6 +120,17 @@ def publish_observation(session: Session, observation: ExternalObservation) -> C
|
||||
setattr(report, key, value)
|
||||
session.add(report)
|
||||
session.flush()
|
||||
replace_tackle_components(
|
||||
session,
|
||||
report,
|
||||
from_catch_fields(
|
||||
bait=observation.payload.get("bait"),
|
||||
rig_type=observation.payload.get("rig_type"),
|
||||
),
|
||||
source_system=observation.source_system,
|
||||
source_url=observation.source_url,
|
||||
raw_payload={"origin": "community_observation", "observation_id": str(observation.id)},
|
||||
)
|
||||
observation.catch_report = report
|
||||
observation.status = "published"
|
||||
observation.reviewed_at = now
|
||||
|
||||
@@ -14,6 +14,7 @@ from .config import settings
|
||||
from .database import SessionLocal
|
||||
from .logging_config import configure_logging
|
||||
from .models import CommunityImportRun, DataSource
|
||||
from .source_lifecycle import classify_source_failure, record_scheduled_source_check
|
||||
|
||||
logger = logging.getLogger("rf4.community_scheduler")
|
||||
MAX_BACKOFF_SECONDS = 24 * 60 * 60
|
||||
@@ -102,11 +103,27 @@ def run_source(source_system: str, *, now: datetime | None = None) -> bool:
|
||||
html = fetch_html(url)
|
||||
records = parser(html, source_url=url) if source_system in {"rf4map", "rf4posts-spot"} else parser(html)
|
||||
created, updated = stage_observations(session, [asdict(item) for item in records])
|
||||
record_scheduled_source_check(
|
||||
session, source_system=source_system, source_url=url,
|
||||
status="available", checked_at=current,
|
||||
)
|
||||
run.status, run.rows_seen, run.rows_created, run.rows_updated = "success", len(records), created, updated
|
||||
except Exception as exc:
|
||||
session.rollback()
|
||||
source_status = classify_source_failure(exc)
|
||||
checked = datetime.now(timezone.utc)
|
||||
affected = record_scheduled_source_check(
|
||||
session,
|
||||
source_system=source_system,
|
||||
source_url=url,
|
||||
status=source_status,
|
||||
checked_at=checked,
|
||||
)
|
||||
run.status, run.error_summary = "failed", f"{type(exc).__name__}: {str(exc)[:500]}"
|
||||
logger.exception("community import failed", extra={"event":"community_import_failed", "source_system":source_system})
|
||||
logger.exception("community import failed", extra={
|
||||
"event":"community_import_failed", "source_system":source_system,
|
||||
"source_check_status": source_status, "affected_observations": affected,
|
||||
})
|
||||
run.finished_at = datetime.now(timezone.utc); session.commit()
|
||||
return True
|
||||
|
||||
|
||||
@@ -31,6 +31,8 @@ class Settings(BaseSettings):
|
||||
rf4map_point_url: str = "https://rf4map.ru/points/275"
|
||||
rf4posts_spot_url: str = "https://rf4-posts.com/ru/spots/d0c6d9c6-4ebf-49a7-98a8-9a562553a8ee"
|
||||
rate_limit_secret: str = "change-rate-limit-secret"
|
||||
admin_auth_attempt_limit: int = Field(default=10, ge=3, le=100)
|
||||
admin_auth_window_seconds: int = Field(default=600, ge=60, le=3600)
|
||||
log_level: str = "INFO"
|
||||
cors_origins: list[str] = Field(default_factory=lambda: ["http://localhost:4321", "http://127.0.0.1:4321"])
|
||||
trusted_proxy_cidrs: list[str] = Field(default_factory=lambda: ["127.0.0.1/32", "::1/128"])
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import Depends
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .database import get_session
|
||||
|
||||
Db = Annotated[Session, Depends(get_session)]
|
||||
@@ -16,6 +16,8 @@ from .models import (
|
||||
Bait, BaitKind, CatchReport, Fish, ImportRecordEvent, ImportStatus, ModerationStatus,
|
||||
OfficialRecordImport, SourceType, Waterbody,
|
||||
)
|
||||
from .tackle_components import replace_tackle_components
|
||||
from rf4_research.gear_components import from_catch_fields
|
||||
|
||||
|
||||
USER_AGENT = "RF4-Spotter/0.1 (public records importer)"
|
||||
@@ -206,6 +208,14 @@ def _import_records_locked(session: Session, *, url: str, region: str, category:
|
||||
if old_val != new_val:
|
||||
changed_fields[field] = {"old": old_val, "new": new_val}
|
||||
if changed_fields:
|
||||
# Keep the queryable normalized record in sync with its
|
||||
# versioned source payload.
|
||||
report.fish = fish
|
||||
report.waterbody = waterbody
|
||||
report.bait = bait
|
||||
report.weight_g = raw.weight_g
|
||||
report.player_name = raw.player
|
||||
report.caught_at = caught
|
||||
report.raw_payload = payload
|
||||
report.source_url = url
|
||||
session.add(ImportRecordEvent(
|
||||
@@ -214,6 +224,14 @@ def _import_records_locked(session: Session, *, url: str, region: str, category:
|
||||
provenance={"source_system": "rf4-official", "source_url": url, "source_external_id": key},
|
||||
))
|
||||
run.rows_updated += 1
|
||||
replace_tackle_components(
|
||||
session,
|
||||
report,
|
||||
from_catch_fields(bait=raw.bait, rig_type=None),
|
||||
source_system="rf4-official",
|
||||
source_url=url,
|
||||
raw_payload={"origin": "official_record", "source_external_id": key},
|
||||
)
|
||||
run.status = ImportStatus.success
|
||||
run.finished_at = datetime.now(timezone.utc)
|
||||
session.commit()
|
||||
|
||||
@@ -1,35 +1,28 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections import Counter
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from ipaddress import IPv4Address, IPv6Address, IPv4Network, IPv6Network
|
||||
import hashlib
|
||||
import hmac
|
||||
import logging
|
||||
import secrets
|
||||
import time as time_module
|
||||
from typing import Annotated, Literal
|
||||
from uuid import UUID
|
||||
import uuid
|
||||
|
||||
import httpx
|
||||
from fastapi import Depends, FastAPI, File, Header, HTTPException, Query, Request, Response, UploadFile
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import JSONResponse
|
||||
from sqlalchemy import delete, func, select, text
|
||||
from sqlalchemy.orm import Session, joinedload
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .activity import activity_rows
|
||||
from .database import get_session
|
||||
from .config import settings
|
||||
from .community_review import ExternalReviewError, map_observation, publish_observation, reject_observation, suggest_aliases
|
||||
from .importer import ImportAlreadyRunning, ImportSourceError, import_records, normalize
|
||||
from .dependencies import Db
|
||||
from .logging_config import configure_logging
|
||||
from .models import Bait, BaitKind, CatchReport, CommunityImportRun, DataSource, ExternalObservation, Fish, ModerationEvent, ModerationStatus, OfficialRecordImport, SourceType, Spot, SubmissionAttempt, Waterbody
|
||||
from .readiness import readiness_report
|
||||
from .public_cache import public_cache
|
||||
from .schemas import ActivityOut, AdminCatchReportOut, BaitOut, CatchOut, CatchReportAccepted, CatchReportCreate, CatchReportCreated, ExternalAliasSuggestionOut, ExternalObservationDecision, ExternalObservationMapping, ExternalObservationOut, ExternalObservationPublished, FishOut, ImportRunOut, ImportRunPublicOut, ModerationUpdate, OfficialRecordOut, PaginatedActivityOut, PaginatedOfficialRecordOut, PublicObservationOut, SourceStatusOut, SpotOut, WaterbodyOut
|
||||
from .storage import ScreenshotError, client as storage_client, delete_screenshot, signed_screenshot_url, upload_screenshot
|
||||
from .routers.activity import router as activity_router
|
||||
from .routers.analytics import router as analytics_router
|
||||
from .routers.admin import router as admin_router
|
||||
from .routers.catalog import router as catalog_router
|
||||
from .routers.media import router as media_router
|
||||
from .routers.public_data import router as public_data_router
|
||||
from .routers.submissions import router as submissions_router
|
||||
from .storage import client as storage_client
|
||||
from .submission_security import check_rate_limit
|
||||
from .submission_security import is_trusted_proxy as _is_trusted_proxy
|
||||
|
||||
|
||||
configure_logging(settings.log_level)
|
||||
@@ -41,7 +34,6 @@ app.add_middleware(
|
||||
allow_methods=["GET", "POST", "PATCH", "DELETE"],
|
||||
allow_headers=["Authorization", "Content-Type"],
|
||||
)
|
||||
Db = Annotated[Session, Depends(get_session)]
|
||||
|
||||
|
||||
@app.middleware("http")
|
||||
@@ -96,526 +88,14 @@ def ready(db: Db) -> JSONResponse:
|
||||
)
|
||||
|
||||
|
||||
@app.get("/api/v1/fishes", response_model=list[FishOut])
|
||||
def fishes(db: Db, limit: int = Query(200, ge=1, le=500), offset: int = Query(0, ge=0)) -> list[Fish]:
|
||||
return list(db.scalars(select(Fish).order_by(Fish.name_ru, Fish.id).offset(offset).limit(limit)))
|
||||
|
||||
|
||||
@app.get("/api/v1/waterbodies", response_model=list[WaterbodyOut])
|
||||
def waterbodies(db: Db, limit: int = Query(200, ge=1, le=500), offset: int = Query(0, ge=0)) -> list[Waterbody]:
|
||||
return list(db.scalars(select(Waterbody).order_by(Waterbody.name_ru, Waterbody.id).offset(offset).limit(limit)))
|
||||
|
||||
|
||||
@app.get("/api/v1/baits", response_model=list[BaitOut])
|
||||
def baits(db: Db, limit: int = Query(200, ge=1, le=500), offset: int = Query(0, ge=0)) -> list[Bait]:
|
||||
return list(db.scalars(select(Bait).order_by(Bait.name, Bait.id).offset(offset).limit(limit)))
|
||||
|
||||
|
||||
@app.get("/api/v1/public-spot-pages")
|
||||
def public_spot_pages(db: Db, limit: int = Query(500, ge=1, le=500), offset: int = Query(0, ge=0)) -> list[str]:
|
||||
rows = db.execute(select(Waterbody.slug, Spot.x, Spot.y, Fish.slug)
|
||||
.select_from(CatchReport).join(Spot, CatchReport.spot_id == Spot.id)
|
||||
.join(Waterbody, Spot.waterbody_id == Waterbody.id).join(Fish, CatchReport.fish_id == Fish.id)
|
||||
.where(CatchReport.moderation_status == ModerationStatus.approved, CatchReport.deleted_at.is_(None))
|
||||
.distinct().order_by(Waterbody.slug, Spot.x, Spot.y, Fish.slug).offset(offset).limit(limit))
|
||||
return [path for water, x, y, fish in rows for path in
|
||||
(f"/spots/{water}-{x}x{y}", f"/waterbodies/{water}/{fish}")]
|
||||
|
||||
|
||||
@app.get("/api/v1/activity", response_model=PaginatedActivityOut)
|
||||
def activity(
|
||||
db: Db, response: Response, hours: int = Query(24),
|
||||
waterbody: str | None = None, fish: str | None = None,
|
||||
method: str | None = None,
|
||||
sort: Literal["activity", "confidence", "freshness"] = "activity",
|
||||
limit: int = Query(20, ge=1, le=100), offset: int = Query(0, ge=0),
|
||||
) -> PaginatedActivityOut:
|
||||
if hours not in {6, 12, 24, 72}:
|
||||
raise HTTPException(status_code=422, detail="hours must be one of: 6, 12, 24, 72")
|
||||
response.headers["Cache-Control"] = "no-store"
|
||||
generation = public_cache.generation()
|
||||
cache_key = ("activity", hours, waterbody, fish, method, sort, limit, offset)
|
||||
cached = public_cache.get(cache_key, settings.public_cache_seconds)
|
||||
if cached is not None:
|
||||
response.headers["X-Cache"] = "HIT"
|
||||
return cached
|
||||
rows = activity_rows(db, hours=hours, waterbody=waterbody, fish=fish, method=method)
|
||||
total = len(rows)
|
||||
keys = {
|
||||
"activity": lambda r: (r.activity_score, r.confidence_score, r.last_confirmed_at, str(r.spot_id)),
|
||||
"confidence": lambda r: (r.confidence_score, r.activity_score, r.last_confirmed_at, str(r.spot_id)),
|
||||
"freshness": lambda r: (r.last_confirmed_at, r.activity_score, r.confidence_score, str(r.spot_id)),
|
||||
}
|
||||
rows.sort(key=keys[sort], reverse=True)
|
||||
page = rows[offset:offset + limit]
|
||||
response.headers["X-Cache"] = "MISS"
|
||||
return public_cache.set(cache_key, PaginatedActivityOut(items=page, total=total, limit=limit, offset=offset), generation=generation)
|
||||
|
||||
|
||||
def _spot_or_404(db: Session, spot_id: UUID) -> Spot:
|
||||
spot = db.scalar(select(Spot).options(joinedload(Spot.waterbody)).where(Spot.id == spot_id))
|
||||
if spot is None:
|
||||
raise HTTPException(status_code=404, detail="spot not found")
|
||||
return spot
|
||||
|
||||
|
||||
@app.get("/api/v1/spots/resolve", response_model=SpotOut)
|
||||
def resolve_spot(
|
||||
db: Db, waterbody: str, x: int = Query(ge=-10_000, le=10_000),
|
||||
y: int = Query(ge=-10_000, le=10_000),
|
||||
) -> SpotOut:
|
||||
spot = db.scalar(select(Spot).options(joinedload(Spot.waterbody)).join(Spot.waterbody).where(
|
||||
Waterbody.slug == waterbody, Spot.x == x, Spot.y == y,
|
||||
))
|
||||
if spot is None:
|
||||
raise HTTPException(status_code=404, detail="spot not found")
|
||||
return spot_detail(spot.id, db)
|
||||
|
||||
|
||||
@app.get("/api/v1/spots/{spot_id}", response_model=SpotOut)
|
||||
def spot_detail(spot_id: UUID, db: Db) -> SpotOut:
|
||||
spot = _spot_or_404(db, spot_id)
|
||||
reports = list(db.scalars(select(CatchReport).options(joinedload(CatchReport.bait)).where(CatchReport.spot_id == spot.id, CatchReport.moderation_status == ModerationStatus.approved, CatchReport.deleted_at.is_(None))))
|
||||
now = datetime.now(timezone.utc)
|
||||
def count_since(delta: timedelta) -> int:
|
||||
return sum(_aware(r.reported_at) >= now - delta for r in reports)
|
||||
bait_counts = Counter(r.bait.name for r in reports if r.bait)
|
||||
return SpotOut(id=spot.id, waterbody_slug=spot.waterbody.slug, waterbody=spot.waterbody.name_ru, x=spot.x, y=spot.y, description=spot.description, catches_24h=count_since(timedelta(hours=24)), catches_3d=count_since(timedelta(days=3)), catches_7d=count_since(timedelta(days=7)), top_baits=[name for name, _ in bait_counts.most_common(5)])
|
||||
|
||||
|
||||
@app.get("/api/v1/spots/{spot_id}/catches", response_model=list[CatchOut])
|
||||
def spot_catches(spot_id: UUID, db: Db, limit: int = Query(50, ge=1, le=100), offset: int = Query(0, ge=0)) -> list[CatchOut]:
|
||||
_spot_or_404(db, spot_id)
|
||||
reports = list(db.scalars(select(CatchReport).options(joinedload(CatchReport.fish), joinedload(CatchReport.bait)).where(CatchReport.spot_id == spot_id, CatchReport.moderation_status == ModerationStatus.approved, CatchReport.deleted_at.is_(None)).order_by(CatchReport.reported_at.desc(), CatchReport.id.desc()).offset(offset).limit(limit)))
|
||||
return [CatchOut(id=r.id, fish=r.fish.name_ru, weight_g=r.weight_g, bait=r.bait.name if r.bait else None, player_name=r.player_name, caught_at=r.caught_at, reported_at=r.reported_at, retrieve_method=r.retrieve_method, retrieve_speed=r.retrieve_speed, source_system=_report_source(r), source_url=r.source_url) for r in reports]
|
||||
|
||||
|
||||
@app.get("/api/v1/spots/{spot_id}/timeline")
|
||||
def spot_timeline(spot_id: UUID, db: Db) -> list[dict]:
|
||||
_spot_or_404(db, spot_id)
|
||||
now = datetime.now(timezone.utc)
|
||||
buckets = []
|
||||
for index in range(6):
|
||||
start = now - timedelta(hours=(6 - index) * 12)
|
||||
end = start + timedelta(hours=12)
|
||||
count = db.scalar(select(func.count()).select_from(CatchReport).where(
|
||||
CatchReport.spot_id == spot_id,
|
||||
CatchReport.moderation_status == ModerationStatus.approved,
|
||||
CatchReport.deleted_at.is_(None),
|
||||
CatchReport.reported_at >= start, CatchReport.reported_at < end,
|
||||
)) or 0
|
||||
buckets.append({"start": start.isoformat(), "end": end.isoformat(), "count": count})
|
||||
return buckets
|
||||
|
||||
|
||||
def _report_source(report: CatchReport) -> str:
|
||||
provenance = (report.raw_payload or {}).get("provenance", {})
|
||||
if isinstance(provenance, dict) and provenance.get("source_system"):
|
||||
return str(provenance["source_system"])
|
||||
if report.source_type == SourceType.official_record:
|
||||
return "rf4-official"
|
||||
if report.source_type == SourceType.user:
|
||||
return "players"
|
||||
return "manual-import"
|
||||
|
||||
|
||||
@app.get("/api/v1/community-observations", response_model=list[PublicObservationOut])
|
||||
def community_observations(
|
||||
db: Db, limit: int = Query(12, ge=1, le=50), offset: int = Query(0, ge=0),
|
||||
waterbody: str | None = None, fish: str | None = None,
|
||||
) -> list[PublicObservationOut]:
|
||||
query = select(ExternalObservation).join(ExternalObservation.source).options(joinedload(ExternalObservation.source)).where(
|
||||
ExternalObservation.catch_report_id.is_(None),
|
||||
ExternalObservation.status != "rejected",
|
||||
DataSource.enabled.is_(True),
|
||||
)
|
||||
if waterbody:
|
||||
query = query.join(ExternalObservation.waterbody).where(Waterbody.slug == waterbody)
|
||||
if fish:
|
||||
query = query.join(ExternalObservation.fish).where(Fish.slug == fish)
|
||||
items = list(db.scalars(query.order_by(ExternalObservation.last_seen_at.desc(), ExternalObservation.id.desc()).offset(offset).limit(limit)))
|
||||
result: list[PublicObservationOut] = []
|
||||
for item in items:
|
||||
missing = []
|
||||
if item.x is None or item.y is None:
|
||||
missing.append("координаты")
|
||||
if item.weight_g is None:
|
||||
missing.append("вес")
|
||||
result.append(PublicObservationOut(
|
||||
id=item.id, source_system=item.source_system, source_name=item.source.name,
|
||||
source_url=item.source_url, fish_name=item.fish_name, waterbody_name=item.waterbody_name,
|
||||
x=item.x, y=item.y, weight_g=item.weight_g, last_seen_at=item.last_seen_at,
|
||||
missing_fields=missing, quality="incomplete" if missing else "unverified",
|
||||
))
|
||||
return result
|
||||
|
||||
|
||||
@app.get("/api/v1/source-status", response_model=list[SourceStatusOut])
|
||||
def source_status(db: Db) -> list[SourceStatusOut]:
|
||||
now = datetime.now(timezone.utc)
|
||||
result = []
|
||||
for source in db.scalars(select(DataSource).order_by(DataSource.name)):
|
||||
runs = list(db.scalars(select(CommunityImportRun).where(CommunityImportRun.source_system == source.key).order_by(CommunityImportRun.started_at.desc()).limit(20)))
|
||||
latest = runs[0] if runs else None
|
||||
success = next((run for run in runs if run.status == "success"), None)
|
||||
if not source.enabled:
|
||||
state = "disabled"
|
||||
elif latest is None:
|
||||
state = "waiting"
|
||||
elif latest.status == "failed":
|
||||
state = "source_changed" if "CommunityParseError" in (latest.error_summary or "") else "temporarily_limited"
|
||||
elif _aware(latest.started_at) < now - timedelta(seconds=settings.community_import_interval_seconds * 2):
|
||||
state = "stale"
|
||||
else:
|
||||
state = "healthy"
|
||||
result.append(SourceStatusOut(source_system=source.key, name=source.name, status=state,
|
||||
last_started_at=latest.started_at if latest else None, last_success_at=success.started_at if success else None,
|
||||
observations=db.scalar(select(func.count()).select_from(ExternalObservation).where(ExternalObservation.source_system == source.key)) or 0))
|
||||
return result
|
||||
|
||||
|
||||
@app.get("/api/v1/records", response_model=PaginatedOfficialRecordOut)
|
||||
def records(
|
||||
db: Db, fish: str | None = None, waterbody: str | None = None,
|
||||
category: str | None = None, limit: int = Query(50, ge=1, le=100),
|
||||
offset: int = Query(0, ge=0),
|
||||
) -> PaginatedOfficialRecordOut:
|
||||
query = select(CatchReport).options(joinedload(CatchReport.fish), joinedload(CatchReport.waterbody), joinedload(CatchReport.bait)).where(CatchReport.source_type == SourceType.official_record)
|
||||
if fish:
|
||||
query = query.join(CatchReport.fish).where(Fish.slug == fish)
|
||||
if waterbody:
|
||||
query = query.join(CatchReport.waterbody).where(Waterbody.slug == waterbody)
|
||||
if category:
|
||||
query = query.where(CatchReport.raw_payload["category"].as_string() == category)
|
||||
# Count total before pagination
|
||||
total = db.scalar(select(func.count()).select_from(CatchReport).where(CatchReport.source_type == SourceType.official_record)) or 0
|
||||
if fish:
|
||||
total = db.scalar(select(func.count()).select_from(CatchReport).join(CatchReport.fish).where(Fish.slug == fish, CatchReport.source_type == SourceType.official_record)) or 0
|
||||
if waterbody:
|
||||
total = db.scalar(select(func.count()).select_from(CatchReport).join(CatchReport.waterbody).where(Waterbody.slug == waterbody, CatchReport.source_type == SourceType.official_record)) or 0
|
||||
if category:
|
||||
total = db.scalar(select(func.count()).select_from(CatchReport).where(CatchReport.source_type == SourceType.official_record, CatchReport.raw_payload["category"].as_string() == category)) or 0
|
||||
items = list(db.scalars(query.order_by(CatchReport.caught_at.desc(), CatchReport.weight_g.desc(), CatchReport.id.desc()).offset(offset).limit(limit)))
|
||||
return PaginatedOfficialRecordOut(
|
||||
items=[OfficialRecordOut(id=r.id, fish=r.fish.name_ru, weight_g=r.weight_g, waterbody=r.waterbody.name_ru, bait=r.bait.name if r.bait else None, player_name=r.player_name, record_date=r.caught_at, category=(r.raw_payload or {}).get("category"), region=(r.raw_payload or {}).get("region"), source_url=r.source_url) for r in items],
|
||||
total=total, limit=limit, offset=offset,
|
||||
)
|
||||
|
||||
|
||||
def _admin(authorization: Annotated[str | None, Header()] = None) -> str:
|
||||
expected = f"Bearer {settings.admin_token}"
|
||||
if not authorization or not hmac.compare_digest(authorization, expected):
|
||||
raise HTTPException(status_code=401, detail="invalid admin token", headers={"WWW-Authenticate": "Bearer"})
|
||||
return "admin"
|
||||
|
||||
|
||||
@app.get("/api/v1/admin/diagnostics")
|
||||
def admin_diagnostics(db: Db, _: Annotated[str, Depends(_admin)]) -> JSONResponse:
|
||||
report_counts = {status.value: count for status, count in db.execute(
|
||||
select(CatchReport.moderation_status, func.count()).group_by(CatchReport.moderation_status)
|
||||
)}
|
||||
observation_counts = {status: count for status, count in db.execute(
|
||||
select(ExternalObservation.status, func.count()).group_by(ExternalObservation.status)
|
||||
)}
|
||||
payload = {
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
"build": {"version": settings.app_version, "revision": settings.app_revision, "environment": settings.deployment_environment},
|
||||
"counts": {
|
||||
"catch_reports": report_counts,
|
||||
"external_observations": observation_counts,
|
||||
"data_sources": db.scalar(select(func.count()).select_from(DataSource)) or 0,
|
||||
"enabled_data_sources": db.scalar(select(func.count()).select_from(DataSource).where(DataSource.enabled.is_(True))) or 0,
|
||||
"official_import_runs": db.scalar(select(func.count()).select_from(OfficialRecordImport)) or 0,
|
||||
"community_import_runs": db.scalar(select(func.count()).select_from(CommunityImportRun)) or 0,
|
||||
},
|
||||
}
|
||||
return JSONResponse(payload, headers={"Content-Disposition": "attachment; filename=rf4spotter-diagnostics.json"})
|
||||
|
||||
|
||||
@app.get("/api/v1/imports", response_model=list[ImportRunPublicOut])
|
||||
def imports(db: Db, limit: int = Query(20, ge=1, le=100), offset: int = Query(0, ge=0)) -> list[OfficialRecordImport]:
|
||||
return list(db.scalars(select(OfficialRecordImport).order_by(OfficialRecordImport.started_at.desc(), OfficialRecordImport.id.desc()).offset(offset).limit(limit)))
|
||||
|
||||
|
||||
@app.get("/api/v1/admin/imports", response_model=list[ImportRunOut])
|
||||
def admin_imports(
|
||||
db: Db,
|
||||
_: Annotated[str, Depends(_admin)],
|
||||
limit: int = Query(20, ge=1, le=100),
|
||||
offset: int = Query(0, ge=0),
|
||||
) -> list[OfficialRecordImport]:
|
||||
query = select(OfficialRecordImport).order_by(
|
||||
OfficialRecordImport.started_at.desc(), OfficialRecordImport.id.desc()
|
||||
).offset(offset).limit(limit)
|
||||
return list(db.scalars(query))
|
||||
|
||||
|
||||
@app.post("/api/v1/admin/imports/official-records", response_model=ImportRunOut, status_code=201)
|
||||
def admin_start_official_import(db: Db, _: Annotated[str, Depends(_admin)]) -> OfficialRecordImport:
|
||||
try:
|
||||
return import_records(
|
||||
db,
|
||||
url=settings.official_records_url,
|
||||
region=settings.official_records_region,
|
||||
category=settings.official_records_category,
|
||||
)
|
||||
except ImportAlreadyRunning as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
except (ImportSourceError, httpx.HTTPError) as exc:
|
||||
raise HTTPException(status_code=502, detail=f"official records import failed: {exc}") from exc
|
||||
|
||||
|
||||
def _external_out(item: ExternalObservation) -> ExternalObservationOut:
|
||||
return ExternalObservationOut(
|
||||
id=item.id, source_system=item.source_system, source_external_id=item.source_external_id,
|
||||
source_url=item.source_url, fish_name=item.fish_name, fish_external_id=item.fish_external_id,
|
||||
waterbody_name=item.waterbody_name, waterbody_external_id=item.waterbody_external_id,
|
||||
x=item.x, y=item.y, weight_g=item.weight_g, published_at=item.published_at,
|
||||
last_seen_at=item.last_seen_at, status=item.status,
|
||||
fish_slug=item.fish.slug if item.fish else None,
|
||||
waterbody_slug=item.waterbody.slug if item.waterbody else None,
|
||||
catch_report_id=item.catch_report_id, review_note=item.review_note,
|
||||
)
|
||||
|
||||
|
||||
@app.get("/api/v1/admin/external-observations", response_model=list[ExternalObservationOut])
|
||||
def admin_external_observations(
|
||||
db: Db, _: Annotated[str, Depends(_admin)],
|
||||
status: Literal["staged", "mapped", "ready", "published", "rejected", "review"] | None = None,
|
||||
source_system: str | None = None, limit: int = Query(50, ge=1, le=200), offset: int = Query(0, ge=0),
|
||||
) -> list[ExternalObservationOut]:
|
||||
query = select(ExternalObservation).options(
|
||||
joinedload(ExternalObservation.fish), joinedload(ExternalObservation.waterbody),
|
||||
)
|
||||
if status == "review":
|
||||
query = query.where(ExternalObservation.status.in_(["staged", "mapped", "ready"]))
|
||||
elif status:
|
||||
query = query.where(ExternalObservation.status == status)
|
||||
if source_system:
|
||||
query = query.where(ExternalObservation.source_system == source_system)
|
||||
items = db.scalars(query.order_by(ExternalObservation.last_seen_at.desc(), ExternalObservation.id.desc()).offset(offset).limit(limit))
|
||||
return [_external_out(item) for item in items]
|
||||
|
||||
|
||||
@app.get("/api/v1/admin/external-observations/{observation_id}/alias-suggestions", response_model=ExternalAliasSuggestionOut)
|
||||
def admin_external_alias_suggestions(
|
||||
observation_id: UUID, db: Db, _: Annotated[str, Depends(_admin)],
|
||||
) -> ExternalAliasSuggestionOut:
|
||||
observation = db.get(ExternalObservation, observation_id)
|
||||
if observation is None:
|
||||
raise HTTPException(status_code=404, detail="external observation not found")
|
||||
fish, waterbody = suggest_aliases(db, observation)
|
||||
return ExternalAliasSuggestionOut(
|
||||
fish_slug=fish.slug if fish else None,
|
||||
waterbody_slug=waterbody.slug if waterbody else None,
|
||||
)
|
||||
|
||||
|
||||
@app.patch("/api/v1/admin/external-observations/{observation_id}/mapping", response_model=ExternalObservationOut)
|
||||
def admin_map_external_observation(
|
||||
observation_id: UUID, payload: ExternalObservationMapping, db: Db,
|
||||
_: Annotated[str, Depends(_admin)],
|
||||
) -> ExternalObservationOut:
|
||||
observation = db.get(ExternalObservation, observation_id)
|
||||
fish = db.scalar(select(Fish).where(Fish.slug == payload.fish_slug))
|
||||
waterbody = db.scalar(select(Waterbody).where(Waterbody.slug == payload.waterbody_slug))
|
||||
if observation is None:
|
||||
raise HTTPException(status_code=404, detail="external observation not found")
|
||||
if fish is None or waterbody is None:
|
||||
raise HTTPException(status_code=422, detail="unknown fish or waterbody")
|
||||
try:
|
||||
return _external_out(map_observation(db, observation, fish, waterbody, note=payload.note))
|
||||
except ExternalReviewError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.post("/api/v1/admin/external-observations/{observation_id}/publish", response_model=ExternalObservationPublished)
|
||||
def admin_publish_external_observation(
|
||||
observation_id: UUID, db: Db, _: Annotated[str, Depends(_admin)],
|
||||
) -> ExternalObservationPublished:
|
||||
observation = db.get(ExternalObservation, observation_id)
|
||||
if observation is None:
|
||||
raise HTTPException(status_code=404, detail="external observation not found")
|
||||
try:
|
||||
report = publish_observation(db, observation)
|
||||
except ExternalReviewError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
public_cache.invalidate()
|
||||
return ExternalObservationPublished(observation_id=observation.id, catch_report_id=report.id, status=observation.status)
|
||||
|
||||
|
||||
@app.patch("/api/v1/admin/external-observations/{observation_id}/reject", response_model=ExternalObservationOut)
|
||||
def admin_reject_external_observation(
|
||||
observation_id: UUID, payload: ExternalObservationDecision, db: Db,
|
||||
_: Annotated[str, Depends(_admin)],
|
||||
) -> ExternalObservationOut:
|
||||
observation = db.get(ExternalObservation, observation_id)
|
||||
if observation is None:
|
||||
raise HTTPException(status_code=404, detail="external observation not found")
|
||||
try:
|
||||
return _external_out(reject_observation(db, observation, reason=payload.reason))
|
||||
except ExternalReviewError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.post("/api/v1/catch-reports", response_model=CatchReportAccepted, status_code=201)
|
||||
def create_catch_report(
|
||||
payload: CatchReportCreate, request: Request, db: Db,
|
||||
idempotency_key: Annotated[str | None, Header()] = None,
|
||||
) -> CatchReportAccepted:
|
||||
if payload.website:
|
||||
raise HTTPException(status_code=400, detail="invalid submission")
|
||||
# A05: Server-side idempotency — check BEFORE rate limit to avoid polluting table
|
||||
if idempotency_key:
|
||||
key_hash = hmac.new(settings.rate_limit_secret.encode(), idempotency_key.encode(), hashlib.sha256).hexdigest()
|
||||
cutoff = datetime.now(timezone.utc) - timedelta(minutes=5)
|
||||
# Force refresh from database to see committed data from previous requests
|
||||
db.expire_all()
|
||||
existing = db.scalar(
|
||||
select(SubmissionAttempt).where(
|
||||
SubmissionAttempt.idempotency_key == key_hash,
|
||||
SubmissionAttempt.created_at >= cutoff,
|
||||
)
|
||||
)
|
||||
if existing is not None:
|
||||
# Return 200 with idempotent flag — client can retry safely
|
||||
logger.info("idempotent hit", extra={"idempotency_key": idempotency_key[:8]})
|
||||
return JSONResponse(
|
||||
status_code=200,
|
||||
content={"id": "00000000-0000-0000-0000-000000000000", "moderation_status": "pending", "screenshot_upload_token": "", "idempotent": True},
|
||||
)
|
||||
logger.info("idempotency check miss", extra={"idempotency_key": idempotency_key[:8]})
|
||||
_check_rate_limit(request, db)
|
||||
fish = db.scalar(select(Fish).where(Fish.slug == payload.fish_slug))
|
||||
waterbody = db.scalar(select(Waterbody).where(Waterbody.slug == payload.waterbody_slug))
|
||||
if fish is None or waterbody is None:
|
||||
raise HTTPException(status_code=422, detail="unknown fish or waterbody")
|
||||
spot = db.scalar(select(Spot).where(Spot.waterbody_id == waterbody.id, Spot.x == payload.x, Spot.y == payload.y))
|
||||
if spot is None:
|
||||
spot = Spot(waterbody=waterbody, x=payload.x, y=payload.y)
|
||||
db.add(spot)
|
||||
bait = None
|
||||
if payload.bait_name and payload.bait_name.strip():
|
||||
key = normalize(payload.bait_name)
|
||||
bait = db.scalar(select(Bait).where(Bait.normalized_name == key))
|
||||
if bait is None:
|
||||
bait = Bait(name=payload.bait_name.strip(), normalized_name=key, kind=BaitKind.unknown)
|
||||
db.add(bait)
|
||||
upload_token = secrets.token_urlsafe(32)
|
||||
report = CatchReport(fish=fish, spot=spot, waterbody=waterbody, bait=bait, weight_g=payload.weight_g, fishing_method=payload.fishing_method, rig_type=payload.rig_type, retrieve_method=payload.retrieve_method, retrieve_speed=payload.retrieve_speed, caught_at=payload.caught_at, reported_at=datetime.now(timezone.utc), player_name=payload.player_name, source_type=SourceType.user, source_url=payload.source_url, source_confidence=60, moderation_status=ModerationStatus.pending, raw_payload={"comment": payload.comment} if payload.comment else None, screenshot_upload_token_hash=hashlib.sha256(upload_token.encode()).hexdigest())
|
||||
db.add(report)
|
||||
db.commit()
|
||||
# Store idempotency key if provided
|
||||
if idempotency_key:
|
||||
key_hash = hmac.new(settings.rate_limit_secret.encode(), idempotency_key.encode(), hashlib.sha256).hexdigest()
|
||||
db.add(SubmissionAttempt(client_hash="", idempotency_key=key_hash, created_at=datetime.now(timezone.utc)))
|
||||
db.commit()
|
||||
logger.info("idempotency key stored", extra={"idempotency_key": idempotency_key[:8]})
|
||||
return CatchReportAccepted(id=report.id, moderation_status=report.moderation_status.value, screenshot_upload_token=upload_token, idempotent=False)
|
||||
|
||||
|
||||
@app.post("/api/v1/catch-reports/{report_id}/screenshot", status_code=204, response_class=Response)
|
||||
def add_screenshot(
|
||||
report_id: UUID, db: Db, screenshot: UploadFile = File(),
|
||||
upload_token: Annotated[str | None, Header(alias="X-Upload-Token")] = None,
|
||||
) -> Response:
|
||||
report = db.get(CatchReport, report_id)
|
||||
if report is None or report.source_type != SourceType.user or report.moderation_status != ModerationStatus.pending:
|
||||
raise HTTPException(status_code=404, detail="pending catch report not found")
|
||||
supplied_hash = hashlib.sha256((upload_token or "").encode()).hexdigest()
|
||||
if not report.screenshot_upload_token_hash or not hmac.compare_digest(report.screenshot_upload_token_hash, supplied_hash):
|
||||
raise HTTPException(status_code=401, detail="invalid screenshot upload token")
|
||||
if report.screenshot_key:
|
||||
raise HTTPException(status_code=409, detail="screenshot already uploaded")
|
||||
raw = screenshot.file.read(settings.screenshot_max_bytes + 1)
|
||||
try:
|
||||
report.screenshot_key = upload_screenshot(raw, filename=screenshot.filename, content_type=screenshot.content_type)
|
||||
except ScreenshotError as exc:
|
||||
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
||||
report.screenshot_upload_token_hash = None
|
||||
db.commit()
|
||||
return Response(status_code=204)
|
||||
|
||||
|
||||
@app.get("/api/v1/admin/catch-reports", response_model=list[AdminCatchReportOut])
|
||||
def admin_reports(db: Db, _: Annotated[str, Depends(_admin)], status: ModerationStatus = ModerationStatus.pending, limit: int = Query(50, ge=1, le=100), offset: int = Query(0, ge=0)) -> list[AdminCatchReportOut]:
|
||||
reports = list(db.scalars(select(CatchReport).options(joinedload(CatchReport.fish), joinedload(CatchReport.waterbody), joinedload(CatchReport.spot), joinedload(CatchReport.bait)).where(CatchReport.source_type == SourceType.user, CatchReport.moderation_status == status, CatchReport.deleted_at.is_(None)).order_by(CatchReport.reported_at, CatchReport.id).offset(offset).limit(limit)))
|
||||
return [AdminCatchReportOut(id=r.id, fish=r.fish.name_ru, waterbody=r.waterbody.name_ru, coordinates=f"{r.spot.x}:{r.spot.y}" if r.spot else "—", weight_g=r.weight_g, bait=r.bait.name if r.bait else None, player_name=r.player_name, reported_at=r.reported_at, moderation_status=r.moderation_status.value, comment=(r.raw_payload or {}).get("comment"), screenshot_url=signed_screenshot_url(r.screenshot_key) if r.screenshot_key else None) for r in reports]
|
||||
|
||||
|
||||
@app.patch("/api/v1/admin/catch-reports/{report_id}", response_model=CatchReportCreated)
|
||||
def moderate_report(report_id: UUID, payload: ModerationUpdate, db: Db, moderator: Annotated[str, Depends(_admin)]) -> CatchReportCreated:
|
||||
report = db.get(CatchReport, report_id)
|
||||
if report is None or report.source_type != SourceType.user or report.deleted_at is not None:
|
||||
raise HTTPException(status_code=404, detail="catch report not found")
|
||||
previous = report.moderation_status
|
||||
report.moderation_status = ModerationStatus(payload.status)
|
||||
db.add(ModerationEvent(catch_report=report, created_at=datetime.now(timezone.utc), previous_status=previous, new_status=report.moderation_status, moderator=moderator, reason=payload.reason))
|
||||
db.commit()
|
||||
public_cache.invalidate()
|
||||
return CatchReportCreated(id=report.id, moderation_status=report.moderation_status.value)
|
||||
|
||||
|
||||
@app.delete("/api/v1/admin/catch-reports/{report_id}", status_code=204, response_class=Response)
|
||||
def delete_report(report_id: UUID, db: Db, moderator: Annotated[str, Depends(_admin)]) -> Response:
|
||||
report = db.get(CatchReport, report_id)
|
||||
if report is None or report.source_type != SourceType.user or report.deleted_at is not None:
|
||||
raise HTTPException(status_code=404, detail="catch report not found")
|
||||
previous = report.moderation_status
|
||||
if report.screenshot_key:
|
||||
try:
|
||||
delete_screenshot(report.screenshot_key)
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=502, detail="screenshot deletion failed") from exc
|
||||
report.moderation_status = ModerationStatus.rejected
|
||||
report.deleted_at = datetime.now(timezone.utc)
|
||||
report.player_name = None
|
||||
report.source_url = None
|
||||
report.screenshot_key = None
|
||||
report.raw_payload = None
|
||||
db.add(ModerationEvent(catch_report=report, created_at=report.deleted_at, previous_status=previous, new_status=ModerationStatus.rejected, moderator=moderator, reason="user report deleted and anonymized"))
|
||||
db.commit()
|
||||
public_cache.invalidate()
|
||||
return Response(status_code=204)
|
||||
|
||||
|
||||
def _is_trusted_proxy(address: str, trusted_cidrs: list[str]) -> bool:
|
||||
"""Check if address is in trusted proxy CIDRs."""
|
||||
try:
|
||||
addr = IPv4Address(address) if ":" not in address else IPv6Address(address)
|
||||
except ValueError:
|
||||
return False
|
||||
for cidr in trusted_cidrs:
|
||||
try:
|
||||
network = IPv4Network(cidr) if ":" not in cidr else IPv6Network(cidr)
|
||||
if addr in network:
|
||||
return True
|
||||
except ValueError:
|
||||
continue
|
||||
return False
|
||||
app.include_router(catalog_router)
|
||||
app.include_router(media_router)
|
||||
app.include_router(activity_router)
|
||||
app.include_router(analytics_router)
|
||||
app.include_router(public_data_router)
|
||||
app.include_router(admin_router)
|
||||
app.include_router(submissions_router)
|
||||
|
||||
|
||||
def _check_rate_limit(request: Request, db: Session) -> None:
|
||||
now = datetime.now(timezone.utc)
|
||||
cutoff = now - timedelta(minutes=10)
|
||||
# Extract real client IP from forwarded headers
|
||||
client = request.client.host if request.client else "unknown"
|
||||
forwarded = request.headers.get("x-forwarded-for")
|
||||
# Only trust X-Forwarded-For if connection came from a trusted proxy
|
||||
if forwarded and request.client and _is_trusted_proxy(request.client.host, settings.trusted_proxy_cidrs):
|
||||
client = forwarded.split(",")[0].strip()
|
||||
client_hash = hmac.new(settings.rate_limit_secret.encode(), client.encode(), hashlib.sha256).hexdigest()
|
||||
if db.get_bind().dialect.name == "postgresql":
|
||||
lock_key = int(client_hash[:16], 16) & 0x7FFF_FFFF_FFFF_FFFF
|
||||
db.execute(text("SELECT pg_advisory_xact_lock(:lock_key)"), {"lock_key": lock_key})
|
||||
db.execute(delete(SubmissionAttempt).where(SubmissionAttempt.created_at < now - timedelta(days=1)))
|
||||
recent = db.scalar(select(func.count()).select_from(SubmissionAttempt).where(SubmissionAttempt.client_hash == client_hash, SubmissionAttempt.created_at >= cutoff)) or 0
|
||||
if recent >= 5:
|
||||
db.commit()
|
||||
raise HTTPException(status_code=429, detail="too many submissions")
|
||||
db.add(SubmissionAttempt(client_hash=client_hash, created_at=now))
|
||||
db.commit()
|
||||
|
||||
|
||||
def _aware(value: datetime) -> datetime:
|
||||
return value if value.tzinfo else value.replace(tzinfo=timezone.utc)
|
||||
check_rate_limit(request, db, settings)
|
||||
|
||||
@@ -0,0 +1,121 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
MEDIA_ROOT = Path(os.environ.get("MEDIA_ROOT", "data/media")).resolve()
|
||||
|
||||
|
||||
def published_assets(entity_type: str | None = None) -> list[dict]:
|
||||
manifest = json.loads((MEDIA_ROOT / "manifest.json").read_text(encoding="utf-8"))
|
||||
result = []
|
||||
for item in manifest.get("assets", []):
|
||||
if item.get("status") != "approved" or not item.get("sha256") or not item.get("local_path"):
|
||||
continue
|
||||
if entity_type and item.get("entity_type") != entity_type:
|
||||
continue
|
||||
source_page = str(item.get("source_page") or "")
|
||||
source = "rf4db" if "rf4db.com" in source_page else "rf4map" if "rf4map.ru" in source_page else "rf4-official"
|
||||
result.append({
|
||||
"id": item["sha256"],
|
||||
"entity_type": item.get("entity_type"),
|
||||
"entity_key": item.get("entity_key"),
|
||||
"label": item.get("label"),
|
||||
"width": item.get("width"),
|
||||
"height": item.get("height"),
|
||||
"content_type": item.get("content_type"),
|
||||
"image_url": f"/api/v1/media/assets/{item['sha256']}",
|
||||
"source_system": source,
|
||||
"source_url": source_page,
|
||||
"variants": [
|
||||
{
|
||||
"role": variant.get("role"),
|
||||
"format": variant.get("format"),
|
||||
"width": variant.get("width"),
|
||||
"height": variant.get("height"),
|
||||
"url": f"/api/v1/media/assets/{variant['sha256']}",
|
||||
}
|
||||
for variant in item.get("derivatives", [])
|
||||
if variant.get("sha256") and variant.get("local_path")
|
||||
],
|
||||
})
|
||||
return sorted(result, key=lambda item: (str(item["entity_type"]), str(item["label"] or "").casefold(), item["id"]))
|
||||
|
||||
|
||||
def published_file(digest: str) -> tuple[Path, str] | None:
|
||||
if len(digest) != 64 or any(char not in "0123456789abcdef" for char in digest):
|
||||
return None
|
||||
manifest = json.loads((MEDIA_ROOT / "manifest.json").read_text(encoding="utf-8"))
|
||||
item = next((row for row in manifest.get("assets", []) if row.get("status") == "approved" and row.get("sha256") == digest), None)
|
||||
media_type = None
|
||||
local_path = None
|
||||
if item:
|
||||
media_type = item.get("content_type")
|
||||
local_path = item.get("local_path")
|
||||
else:
|
||||
for row in manifest.get("assets", []):
|
||||
if row.get("status") != "approved":
|
||||
continue
|
||||
variant = next((candidate for candidate in row.get("derivatives", []) if candidate.get("sha256") == digest), None)
|
||||
if variant:
|
||||
media_type = variant.get("content_type")
|
||||
local_path = variant.get("local_path")
|
||||
break
|
||||
if not local_path:
|
||||
return None
|
||||
target = (MEDIA_ROOT / local_path).resolve()
|
||||
if not target.is_relative_to(MEDIA_ROOT.resolve()) or not target.is_file():
|
||||
return None
|
||||
return target, str(media_type or "application/octet-stream")
|
||||
|
||||
|
||||
def review_assets(entity_type: str | None = None, status: str | None = None) -> list[dict]:
|
||||
manifest = json.loads((MEDIA_ROOT / "manifest.json").read_text(encoding="utf-8"))
|
||||
result = []
|
||||
for item in manifest.get("assets", []):
|
||||
item_status = str(item.get("status") or "")
|
||||
if item_status not in {"approved", "upgrade_queued", "upgrade_stored"} or (status and item_status != status):
|
||||
continue
|
||||
if entity_type and item.get("entity_type") != entity_type:
|
||||
continue
|
||||
digest = str(item.get("sha256") or "")
|
||||
if len(digest) != 64 or not item.get("local_path"):
|
||||
continue
|
||||
source_page = str(item.get("source_page") or "")
|
||||
source = "rf4db" if "rf4db.com" in source_page else "rf4map" if "rf4map.ru" in source_page else "rf4-official"
|
||||
result.append({
|
||||
"id": digest,
|
||||
"status": item_status,
|
||||
"entity_type": item.get("entity_type"),
|
||||
"entity_key": item.get("entity_key"),
|
||||
"label": item.get("label"),
|
||||
"width": item.get("width"),
|
||||
"height": item.get("height"),
|
||||
"content_type": item.get("content_type"),
|
||||
"image_url": f"/api/v1/admin/media/assets/{digest}",
|
||||
"asset_url": item.get("asset_url", ""),
|
||||
"source_system": source,
|
||||
"source_url": source_page,
|
||||
"duplicate_of": item.get("duplicate_of"),
|
||||
"supersedes": item.get("supersedes"),
|
||||
"derivatives": [{
|
||||
"role": variant.get("role"), "format": variant.get("format"),
|
||||
"width": variant.get("width"), "height": variant.get("height"),
|
||||
} for variant in item.get("derivatives", [])],
|
||||
})
|
||||
return sorted(result, key=lambda item: (str(item["status"]), str(item["entity_type"]), str(item["label"] or "").casefold(), item["id"]))
|
||||
|
||||
|
||||
def review_file(digest: str) -> tuple[Path, str] | None:
|
||||
if len(digest) != 64 or any(char not in "0123456789abcdef" for char in digest):
|
||||
return None
|
||||
manifest = json.loads((MEDIA_ROOT / "manifest.json").read_text(encoding="utf-8"))
|
||||
item = next((row for row in manifest.get("assets", []) if row.get("sha256") == digest and row.get("status") in {"approved", "upgrade_queued", "upgrade_stored"}), None)
|
||||
if not item or not item.get("local_path"):
|
||||
return None
|
||||
target = (MEDIA_ROOT / item["local_path"]).resolve()
|
||||
if not target.is_relative_to(MEDIA_ROOT.resolve()) or not target.is_file():
|
||||
return None
|
||||
return target, str(item.get("content_type") or "application/octet-stream")
|
||||
@@ -49,6 +49,16 @@ class Waterbody(Base):
|
||||
slug: Mapped[str] = mapped_column(String(100), unique=True)
|
||||
name_ru: Mapped[str] = mapped_column(String(200), unique=True)
|
||||
unlock_level: Mapped[int | None]
|
||||
fish_species_count: Mapped[int | None]
|
||||
source_system: Mapped[str | None] = mapped_column(String(50))
|
||||
source_external_id: Mapped[str | None] = mapped_column(String(200))
|
||||
source_url: Mapped[str | None] = mapped_column(Text)
|
||||
description: Mapped[str | None] = mapped_column(Text)
|
||||
source_aliases: Mapped[list[str] | None] = mapped_column(JSON)
|
||||
source_fish_species: Mapped[list[str] | None] = mapped_column(JSON)
|
||||
source_image_urls: Mapped[list[str] | None] = mapped_column(JSON)
|
||||
source_point_urls: Mapped[list[str] | None] = mapped_column(JSON)
|
||||
source_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class Bait(Base):
|
||||
@@ -59,6 +69,61 @@ class Bait(Base):
|
||||
kind: Mapped[BaitKind] = mapped_column(Enum(BaitKind))
|
||||
|
||||
|
||||
class TackleItem(Base):
|
||||
"""Canonical gear item; legacy Bait rows remain source-compatible."""
|
||||
|
||||
__tablename__ = "tackle_item"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("source_system", "source_external_id"),
|
||||
CheckConstraint(
|
||||
"category IN ('bait', 'lure', 'rod', 'reel', 'line', 'hook', 'rig', 'float', 'sinker', 'other')",
|
||||
name="ck_tackle_item_category",
|
||||
),
|
||||
)
|
||||
id: Mapped[uuid.UUID] = mapped_column(primary_key=True, default=uuid.uuid4)
|
||||
name: Mapped[str] = mapped_column(String(200))
|
||||
normalized_name: Mapped[str] = mapped_column(String(200), unique=True)
|
||||
category: Mapped[str] = mapped_column(String(20))
|
||||
subcategory: Mapped[str | None] = mapped_column(String(100))
|
||||
brand: Mapped[str | None] = mapped_column(String(100))
|
||||
family: Mapped[str | None] = mapped_column(String(100))
|
||||
unlock_level: Mapped[int | None]
|
||||
source_system: Mapped[str | None] = mapped_column(String(50))
|
||||
source_external_id: Mapped[str | None] = mapped_column(String(200))
|
||||
source_url: Mapped[str | None] = mapped_column(Text)
|
||||
source_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
raw_payload: Mapped[dict | None] = mapped_column(JSON)
|
||||
rig_components: Mapped[list["RigComponent"]] = relationship(back_populates="tackle_item")
|
||||
|
||||
|
||||
class Rig(Base):
|
||||
"""A named rig/setup kept separate from individual tackle items."""
|
||||
|
||||
__tablename__ = "rig"
|
||||
id: Mapped[uuid.UUID] = mapped_column(primary_key=True, default=uuid.uuid4)
|
||||
name: Mapped[str] = mapped_column(String(200))
|
||||
normalized_name: Mapped[str] = mapped_column(String(200), unique=True)
|
||||
source_system: Mapped[str | None] = mapped_column(String(50))
|
||||
source_external_id: Mapped[str | None] = mapped_column(String(200))
|
||||
source_url: Mapped[str | None] = mapped_column(Text)
|
||||
source_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
raw_payload: Mapped[dict | None] = mapped_column(JSON)
|
||||
components: Mapped[list["RigComponent"]] = relationship(back_populates="rig")
|
||||
|
||||
|
||||
class RigComponent(Base):
|
||||
__tablename__ = "rig_component"
|
||||
__table_args__ = (UniqueConstraint("rig_id", "position"),)
|
||||
id: Mapped[uuid.UUID] = mapped_column(primary_key=True, default=uuid.uuid4)
|
||||
rig_id: Mapped[uuid.UUID] = mapped_column(ForeignKey("rig.id"))
|
||||
tackle_item_id: Mapped[uuid.UUID | None] = mapped_column(ForeignKey("tackle_item.id"))
|
||||
role: Mapped[str] = mapped_column(String(50))
|
||||
position: Mapped[int] = mapped_column(Integer)
|
||||
raw_value: Mapped[str | None] = mapped_column(String(200))
|
||||
rig: Mapped[Rig] = relationship(back_populates="components")
|
||||
tackle_item: Mapped[TackleItem | None] = relationship(back_populates="rig_components")
|
||||
|
||||
|
||||
class Spot(Base):
|
||||
__tablename__ = "spot"
|
||||
__table_args__ = (UniqueConstraint("waterbody_id", "x", "y"),)
|
||||
@@ -95,10 +160,39 @@ class CatchReport(Base):
|
||||
screenshot_upload_token_hash: Mapped[str | None] = mapped_column(String(64))
|
||||
deleted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
raw_payload: Mapped[dict | None] = mapped_column(JSON)
|
||||
moderation_version: Mapped[int] = mapped_column(default=0)
|
||||
fish: Mapped[Fish] = relationship()
|
||||
spot: Mapped[Spot | None] = relationship()
|
||||
waterbody: Mapped[Waterbody] = relationship()
|
||||
bait: Mapped[Bait | None] = relationship()
|
||||
tackle_components: Mapped[list["CatchTackleComponent"]] = relationship(back_populates="catch_report")
|
||||
|
||||
|
||||
class CatchTackleComponent(Base):
|
||||
"""Ordered gear evidence; unresolved raw values are valid and preserved."""
|
||||
|
||||
__tablename__ = "catch_tackle_component"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("catch_report_id", "position"),
|
||||
CheckConstraint(
|
||||
"NOT (tackle_item_id IS NOT NULL AND rig_id IS NOT NULL)",
|
||||
name="ck_catch_tackle_one_canonical_target",
|
||||
),
|
||||
)
|
||||
id: Mapped[uuid.UUID] = mapped_column(primary_key=True, default=uuid.uuid4)
|
||||
catch_report_id: Mapped[uuid.UUID] = mapped_column(ForeignKey("catch_report.id"))
|
||||
tackle_item_id: Mapped[uuid.UUID | None] = mapped_column(ForeignKey("tackle_item.id"))
|
||||
rig_id: Mapped[uuid.UUID | None] = mapped_column(ForeignKey("rig.id"))
|
||||
role: Mapped[str] = mapped_column(String(50))
|
||||
position: Mapped[int] = mapped_column(Integer)
|
||||
raw_value: Mapped[str] = mapped_column(String(200))
|
||||
source_system: Mapped[str | None] = mapped_column(String(50))
|
||||
source_external_id: Mapped[str | None] = mapped_column(String(200))
|
||||
source_url: Mapped[str | None] = mapped_column(Text)
|
||||
raw_payload: Mapped[dict | None] = mapped_column(JSON)
|
||||
catch_report: Mapped[CatchReport] = relationship(back_populates="tackle_components")
|
||||
tackle_item: Mapped[TackleItem | None] = relationship()
|
||||
rig: Mapped[Rig | None] = relationship()
|
||||
|
||||
|
||||
class OfficialRecordImport(Base):
|
||||
@@ -136,7 +230,17 @@ class SubmissionAttempt(Base):
|
||||
__tablename__ = "submission_attempt"
|
||||
id: Mapped[uuid.UUID] = mapped_column(primary_key=True, default=uuid.uuid4)
|
||||
client_hash: Mapped[str] = mapped_column(String(64), index=True)
|
||||
idempotency_key: Mapped[str | None] = mapped_column(String(128), index=True)
|
||||
idempotency_key: Mapped[str | None] = mapped_column(String(128), unique=True, index=True)
|
||||
catch_report_id: Mapped[uuid.UUID | None] = mapped_column(ForeignKey("catch_report.id"), nullable=True)
|
||||
payload_hash: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), index=True)
|
||||
catch_report: Mapped[CatchReport | None] = relationship()
|
||||
|
||||
|
||||
class AdminAuthAttempt(Base):
|
||||
__tablename__ = "admin_auth_attempt"
|
||||
id: Mapped[uuid.UUID] = mapped_column(primary_key=True, default=uuid.uuid4)
|
||||
client_hash: Mapped[str] = mapped_column(String(64), index=True)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), index=True)
|
||||
|
||||
|
||||
@@ -176,6 +280,8 @@ class ExternalObservation(Base):
|
||||
waterbody_external_id: Mapped[str | None] = mapped_column(String(200))
|
||||
x: Mapped[int | None]
|
||||
y: Mapped[int | None]
|
||||
coordinate_raw: Mapped[str | None] = mapped_column(String(200))
|
||||
coordinate_precision: Mapped[str] = mapped_column(String(20), default="missing")
|
||||
weight_g: Mapped[int | None]
|
||||
published_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
first_seen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
|
||||
@@ -187,6 +293,9 @@ class ExternalObservation(Base):
|
||||
catch_report_id: Mapped[uuid.UUID | None] = mapped_column(ForeignKey("catch_report.id"), unique=True)
|
||||
review_note: Mapped[str | None] = mapped_column(Text)
|
||||
reviewed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
moderation_version: Mapped[int] = mapped_column(default=0)
|
||||
source_check_status: Mapped[str | None] = mapped_column(String(30))
|
||||
source_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
source: Mapped[DataSource] = relationship()
|
||||
fish: Mapped[Fish | None] = relationship()
|
||||
waterbody: Mapped[Waterbody | None] = relationship()
|
||||
|
||||
@@ -6,6 +6,7 @@ from typing import Any
|
||||
from sqlalchemy import func, select, text
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .config import settings
|
||||
from .models import CommunityImportRun, DataSource, ImportStatus, OfficialRecordImport
|
||||
|
||||
|
||||
@@ -32,7 +33,7 @@ def readiness_report(
|
||||
ready = False
|
||||
|
||||
try:
|
||||
s3.list_buckets()
|
||||
s3.head_bucket(Bucket=settings.s3_bucket)
|
||||
components["minio"] = {"status": "ready"}
|
||||
except Exception:
|
||||
components["minio"] = {"status": "unavailable"}
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
"""HTTP route groups for the RF4 Spotter API."""
|
||||
@@ -0,0 +1,113 @@
|
||||
from collections import Counter
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Query, Response
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session, joinedload, selectinload
|
||||
|
||||
from ..activity import activity_rows
|
||||
from ..config import settings
|
||||
from ..dependencies import Db
|
||||
from ..models import CatchReport, ModerationStatus, SourceType, Spot, Waterbody
|
||||
from ..public_cache import public_cache
|
||||
from ..schemas import CatchOut, PaginatedActivityOut, SpotOut
|
||||
from ..time_utils import aware
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/api/v1/activity", response_model=PaginatedActivityOut)
|
||||
def activity(db: Db, response: Response, hours: int = Query(24), waterbody: str | None = None,
|
||||
fish: str | None = None, method: str | None = None,
|
||||
sort: Literal["activity", "confidence", "freshness"] = "activity",
|
||||
limit: int = Query(20, ge=1, le=100), offset: int = Query(0, ge=0)) -> PaginatedActivityOut:
|
||||
if hours not in {6, 12, 24, 72}:
|
||||
raise HTTPException(status_code=422, detail="hours must be one of: 6, 12, 24, 72")
|
||||
response.headers["Cache-Control"] = "no-store"
|
||||
generation = public_cache.generation()
|
||||
cache_key = ("activity", hours, waterbody, fish, method, sort, limit, offset)
|
||||
cached = public_cache.get(cache_key, settings.public_cache_seconds)
|
||||
if cached is not None:
|
||||
response.headers["X-Cache"] = "HIT"
|
||||
return cached
|
||||
rows = activity_rows(db, hours=hours, waterbody=waterbody, fish=fish, method=method)
|
||||
total = len(rows)
|
||||
keys = {
|
||||
"activity": lambda row: (row.activity_score, row.confidence_score, row.last_confirmed_at, str(row.spot_id)),
|
||||
"confidence": lambda row: (row.confidence_score, row.activity_score, row.last_confirmed_at, str(row.spot_id)),
|
||||
"freshness": lambda row: (row.last_confirmed_at, row.activity_score, row.confidence_score, str(row.spot_id)),
|
||||
}
|
||||
rows.sort(key=keys[sort], reverse=True)
|
||||
response.headers["X-Cache"] = "MISS"
|
||||
return public_cache.set(cache_key, PaginatedActivityOut(items=rows[offset:offset + limit], total=total, limit=limit, offset=offset), generation=generation)
|
||||
|
||||
|
||||
def _spot_or_404(db: Session, spot_id: UUID) -> Spot:
|
||||
spot = db.scalar(select(Spot).options(joinedload(Spot.waterbody)).where(Spot.id == spot_id))
|
||||
if spot is None:
|
||||
raise HTTPException(status_code=404, detail="spot not found")
|
||||
return spot
|
||||
|
||||
|
||||
@router.get("/api/v1/spots/resolve", response_model=SpotOut)
|
||||
def resolve_spot(db: Db, waterbody: str, x: int = Query(ge=-10_000, le=10_000), y: int = Query(ge=-10_000, le=10_000)) -> SpotOut:
|
||||
spot = db.scalar(select(Spot).options(joinedload(Spot.waterbody)).join(Spot.waterbody).where(Waterbody.slug == waterbody, Spot.x == x, Spot.y == y))
|
||||
if spot is None:
|
||||
raise HTTPException(status_code=404, detail="spot not found")
|
||||
return spot_detail(spot.id, db)
|
||||
|
||||
|
||||
@router.get("/api/v1/spots/{spot_id}", response_model=SpotOut)
|
||||
def spot_detail(spot_id: UUID, db: Db) -> SpotOut:
|
||||
spot = _spot_or_404(db, spot_id)
|
||||
reports = list(db.scalars(select(CatchReport).options(joinedload(CatchReport.bait)).where(CatchReport.spot_id == spot.id, CatchReport.moderation_status == ModerationStatus.approved, CatchReport.deleted_at.is_(None))))
|
||||
now = datetime.now(timezone.utc)
|
||||
bait_counts = Counter(report.bait.name for report in reports if report.bait)
|
||||
def count_since(delta: timedelta) -> int:
|
||||
return sum(aware(report.reported_at) >= now - delta for report in reports)
|
||||
|
||||
provenance = [
|
||||
(report.raw_payload or {}).get("provenance", {})
|
||||
for report in reports
|
||||
if isinstance((report.raw_payload or {}).get("provenance", {}), dict)
|
||||
]
|
||||
precisions = [item.get("coordinate_precision") for item in provenance]
|
||||
precision = max((value for value in precisions if value in {"exact", "approximate", "area", "missing"}), key={"exact": 0, "approximate": 1, "area": 2, "missing": 3}.get, default="exact")
|
||||
sources = sorted({str(item.get("source_system")) for item in provenance if item.get("source_system")}) or ["players"]
|
||||
return SpotOut(id=spot.id, waterbody_slug=spot.waterbody.slug, waterbody=spot.waterbody.name_ru, x=spot.x, y=spot.y, description=spot.description, catches_24h=count_since(timedelta(hours=24)), catches_3d=count_since(timedelta(days=3)), catches_7d=count_since(timedelta(days=7)), top_baits=[name for name, _ in bait_counts.most_common(5)], coordinate_precision=precision, coordinate_sources=sources)
|
||||
|
||||
|
||||
def _report_source(report: CatchReport) -> str:
|
||||
provenance = (report.raw_payload or {}).get("provenance", {})
|
||||
if isinstance(provenance, dict) and provenance.get("source_system"):
|
||||
return str(provenance["source_system"])
|
||||
if report.source_type == SourceType.official_record:
|
||||
return "rf4-official"
|
||||
return "players" if report.source_type == SourceType.user else "manual-import"
|
||||
|
||||
|
||||
@router.get("/api/v1/spots/{spot_id}/catches", response_model=list[CatchOut])
|
||||
def spot_catches(spot_id: UUID, db: Db, limit: int = Query(50, ge=1, le=100), offset: int = Query(0, ge=0)) -> list[CatchOut]:
|
||||
_spot_or_404(db, spot_id)
|
||||
reports = list(db.scalars(select(CatchReport).options(joinedload(CatchReport.fish), joinedload(CatchReport.bait), selectinload(CatchReport.tackle_components)).where(CatchReport.spot_id == spot_id, CatchReport.moderation_status == ModerationStatus.approved, CatchReport.deleted_at.is_(None)).order_by(CatchReport.reported_at.desc(), CatchReport.id.desc()).offset(offset).limit(limit)))
|
||||
return [CatchOut(id=report.id, fish=report.fish.name_ru, weight_g=report.weight_g, bait=report.bait.name if report.bait else None, player_name=report.player_name, caught_at=report.caught_at, reported_at=report.reported_at, retrieve_method=report.retrieve_method, retrieve_speed=report.retrieve_speed, source_system=_report_source(report), source_url=report.source_url, tackle_components=[{
|
||||
"id": component.id, "role": component.role, "position": component.position,
|
||||
"raw_value": component.raw_value, "tackle_item_id": component.tackle_item_id,
|
||||
"rig_id": component.rig_id, "source_system": component.source_system,
|
||||
"source_url": component.source_url,
|
||||
} for component in sorted(report.tackle_components, key=lambda value: value.position)]) for report in reports]
|
||||
|
||||
|
||||
@router.get("/api/v1/spots/{spot_id}/timeline")
|
||||
def spot_timeline(spot_id: UUID, db: Db) -> list[dict]:
|
||||
_spot_or_404(db, spot_id)
|
||||
now = datetime.now(timezone.utc)
|
||||
buckets = []
|
||||
for index in range(6):
|
||||
start = now - timedelta(hours=(6 - index) * 12)
|
||||
end = start + timedelta(hours=12)
|
||||
count = db.scalar(select(func.count()).select_from(CatchReport).where(CatchReport.spot_id == spot_id, CatchReport.moderation_status == ModerationStatus.approved, CatchReport.deleted_at.is_(None), CatchReport.reported_at >= start, CatchReport.reported_at < end)) or 0
|
||||
buckets.append({"start": start.isoformat(), "end": end.isoformat(), "count": count})
|
||||
return buckets
|
||||
@@ -0,0 +1,426 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Annotated, Literal
|
||||
from uuid import UUID
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, Header, HTTPException, Query, Request, Response
|
||||
from fastapi.responses import FileResponse, JSONResponse
|
||||
from sqlalchemy import case, func, or_, select
|
||||
from sqlalchemy.orm import joinedload
|
||||
|
||||
from ..admin_security import verify_admin
|
||||
from ..community_review import ExternalReviewError, map_observation, publish_observation, reject_observation, suggest_aliases
|
||||
from ..config import settings
|
||||
from ..dependencies import Db
|
||||
from ..importer import ImportAlreadyRunning, ImportSourceError, import_records
|
||||
from rf4_research.media_assets import publish_quality_upgrades, rollback_quality_upgrade
|
||||
|
||||
from ..media_catalog import MEDIA_ROOT, review_assets, review_file
|
||||
from ..models import CatchReport, CommunityImportRun, DataSource, ExternalObservation, Fish, ModerationEvent, ModerationStatus, OfficialRecordImport, SourceType, Waterbody
|
||||
from ..public_cache import public_cache
|
||||
from ..schemas import AdminCatchReportOut, AdminMediaDecision, AdminMediaReviewOut, AdminMediaRollback, AdminModerationHistoryOut, AdminSourceStatusOut, CatchReportCreated, ExternalAliasSuggestionOut, ExternalObservationAction, ExternalObservationDecision, ExternalObservationMapping, ExternalObservationOut, ExternalObservationPublished, ImportRunOut, ModerationUpdate
|
||||
from ..storage import delete_screenshot, signed_screenshot_url
|
||||
from ..time_utils import aware
|
||||
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def _admin(request: Request, db: Db, authorization: Annotated[str | None, Header()] = None) -> str:
|
||||
return verify_admin(request, db, authorization, settings)
|
||||
|
||||
|
||||
@router.get("/api/v1/admin/media/catalog", response_model=list[AdminMediaReviewOut])
|
||||
def admin_media_catalog(
|
||||
_: Annotated[str, Depends(_admin)],
|
||||
entity_type: str | None = Query(None, pattern="^(fish|waterbody|tackle|reference)$"),
|
||||
status: str | None = Query(None, pattern="^(approved|upgrade_queued|upgrade_stored)$"),
|
||||
limit: int = Query(50, ge=1, le=100),
|
||||
offset: int = Query(0, ge=0),
|
||||
) -> list[AdminMediaReviewOut]:
|
||||
return review_assets(entity_type, status)[offset:offset + limit]
|
||||
|
||||
|
||||
@router.get("/api/v1/admin/media/assets/{digest}", response_class=FileResponse)
|
||||
def admin_media_asset(digest: str, _: Annotated[str, Depends(_admin)]) -> FileResponse:
|
||||
item = review_file(digest)
|
||||
if not item:
|
||||
raise HTTPException(status_code=404, detail="Media review asset not found")
|
||||
path, media_type = item
|
||||
return FileResponse(path, media_type=media_type, headers={"Cache-Control": "private, no-store"})
|
||||
|
||||
|
||||
@router.post("/api/v1/admin/media/upgrades/publish")
|
||||
def admin_publish_media_upgrades(
|
||||
payload: AdminMediaDecision,
|
||||
_: Annotated[str, Depends(_admin)],
|
||||
) -> dict[str, int]:
|
||||
"""Atomically publish all stored quality upgrades after an explicit decision."""
|
||||
try:
|
||||
return publish_quality_upgrades(MEDIA_ROOT / "manifest.json", note=payload.note)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.post("/api/v1/admin/media/upgrades/rollback")
|
||||
def admin_rollback_media_upgrade(
|
||||
payload: AdminMediaRollback,
|
||||
_: Annotated[str, Depends(_admin)],
|
||||
) -> dict[str, str]:
|
||||
"""Restore one superseded fallback while retaining the reviewed candidate."""
|
||||
try:
|
||||
return rollback_quality_upgrade(
|
||||
MEDIA_ROOT / "manifest.json", asset_url=payload.asset_url, note=payload.note,
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.get("/api/v1/admin/diagnostics")
|
||||
def admin_diagnostics(db: Db, _: Annotated[str, Depends(_admin)]) -> JSONResponse:
|
||||
report_counts = {status.value: count for status, count in db.execute(
|
||||
select(CatchReport.moderation_status, func.count()).group_by(CatchReport.moderation_status)
|
||||
)}
|
||||
observation_counts = {status: count for status, count in db.execute(
|
||||
select(ExternalObservation.status, func.count()).group_by(ExternalObservation.status)
|
||||
)}
|
||||
payload = {
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
"build": {"version": settings.app_version, "revision": settings.app_revision, "environment": settings.deployment_environment},
|
||||
"counts": {
|
||||
"catch_reports": report_counts,
|
||||
"external_observations": observation_counts,
|
||||
"data_sources": db.scalar(select(func.count()).select_from(DataSource)) or 0,
|
||||
"enabled_data_sources": db.scalar(select(func.count()).select_from(DataSource).where(DataSource.enabled.is_(True))) or 0,
|
||||
"official_import_runs": db.scalar(select(func.count()).select_from(OfficialRecordImport)) or 0,
|
||||
"community_import_runs": db.scalar(select(func.count()).select_from(CommunityImportRun)) or 0,
|
||||
},
|
||||
}
|
||||
return JSONResponse(payload, headers={"Content-Disposition": "attachment; filename=rf4spotter-diagnostics.json"})
|
||||
|
||||
|
||||
@router.get("/api/v1/admin/moderation-history", response_model=list[AdminModerationHistoryOut])
|
||||
def admin_moderation_history(
|
||||
db: Db,
|
||||
_: Annotated[str, Depends(_admin)],
|
||||
limit: int = Query(50, ge=1, le=200),
|
||||
offset: int = Query(0, ge=0),
|
||||
) -> list[AdminModerationHistoryOut]:
|
||||
report_events = list(db.scalars(
|
||||
select(ModerationEvent).order_by(ModerationEvent.created_at.desc()).limit(limit + offset)
|
||||
))
|
||||
external_events = list(db.scalars(
|
||||
select(ExternalObservation).where(ExternalObservation.reviewed_at.is_not(None))
|
||||
.order_by(ExternalObservation.reviewed_at.desc()).limit(limit + offset)
|
||||
))
|
||||
history = [AdminModerationHistoryOut(
|
||||
entity_type="catch_report", entity_id=event.catch_report_id,
|
||||
decided_at=event.created_at, action=event.new_status.value,
|
||||
moderator=event.moderator, reason=event.reason,
|
||||
) for event in report_events]
|
||||
history.extend(AdminModerationHistoryOut(
|
||||
entity_type="external_observation", entity_id=observation.id,
|
||||
decided_at=observation.reviewed_at, action=observation.status,
|
||||
moderator=None, reason=observation.review_note,
|
||||
) for observation in external_events if observation.reviewed_at is not None)
|
||||
history.sort(key=lambda event: aware(event.decided_at), reverse=True)
|
||||
return history[offset:offset + limit]
|
||||
|
||||
|
||||
@router.get("/api/v1/admin/moderation-history-export")
|
||||
def admin_moderation_history_export(
|
||||
db: Db,
|
||||
_: Annotated[str, Depends(_admin)],
|
||||
limit: int = Query(1000, ge=1, le=5000),
|
||||
) -> JSONResponse:
|
||||
"""Return an anonymized, analysis-safe decision export."""
|
||||
events = admin_moderation_history(db, _, limit=limit, offset=0)
|
||||
payload = {
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
"count": len(events),
|
||||
"events": [{
|
||||
"entity_type": event.entity_type,
|
||||
"decided_at": event.decided_at.isoformat(),
|
||||
"action": event.action,
|
||||
"requires_confirmation": event.requires_confirmation,
|
||||
} for event in events],
|
||||
}
|
||||
return JSONResponse(payload, headers={
|
||||
"Content-Disposition": "attachment; filename=rf4spotter-moderation-history.json",
|
||||
})
|
||||
|
||||
|
||||
@router.get("/api/v1/admin/imports", response_model=list[ImportRunOut])
|
||||
def admin_imports(
|
||||
db: Db,
|
||||
_: Annotated[str, Depends(_admin)],
|
||||
limit: int = Query(20, ge=1, le=100),
|
||||
offset: int = Query(0, ge=0),
|
||||
) -> list[OfficialRecordImport]:
|
||||
query = select(OfficialRecordImport).order_by(
|
||||
OfficialRecordImport.started_at.desc(), OfficialRecordImport.id.desc()
|
||||
).offset(offset).limit(limit)
|
||||
return list(db.scalars(query))
|
||||
|
||||
|
||||
@router.post("/api/v1/admin/imports/official-records", response_model=ImportRunOut, status_code=201)
|
||||
def admin_start_official_import(db: Db, _: Annotated[str, Depends(_admin)]) -> OfficialRecordImport:
|
||||
try:
|
||||
return import_records(
|
||||
db,
|
||||
url=settings.official_records_url,
|
||||
region=settings.official_records_region,
|
||||
category=settings.official_records_category,
|
||||
)
|
||||
except ImportAlreadyRunning as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
except (ImportSourceError, httpx.HTTPError) as exc:
|
||||
raise HTTPException(status_code=502, detail=f"official records import failed: {exc}") from exc
|
||||
|
||||
|
||||
@router.get("/api/v1/admin/source-status", response_model=list[AdminSourceStatusOut])
|
||||
def admin_source_status(db: Db, _: Annotated[str, Depends(_admin)]) -> list[AdminSourceStatusOut]:
|
||||
"""Return safe operational details needed by the owner dashboard."""
|
||||
now = datetime.now(timezone.utc)
|
||||
result: list[AdminSourceStatusOut] = []
|
||||
for source in db.scalars(select(DataSource).order_by(DataSource.name)):
|
||||
runs = list(db.scalars(
|
||||
select(CommunityImportRun)
|
||||
.where(CommunityImportRun.source_system == source.key)
|
||||
.order_by(CommunityImportRun.started_at.desc()).limit(20)
|
||||
))
|
||||
latest = runs[0] if runs else None
|
||||
success = next((run for run in runs if run.status == "success"), None)
|
||||
recent_failures = sum(
|
||||
1 for run in runs
|
||||
if run.status == "failed" and aware(run.started_at) >= now - timedelta(hours=24)
|
||||
)
|
||||
next_allowed = (
|
||||
aware(latest.started_at) + timedelta(seconds=settings.community_import_interval_seconds)
|
||||
if latest else None
|
||||
)
|
||||
cooldown_seconds = max(0, int((next_allowed - now).total_seconds())) if next_allowed else 0
|
||||
if not source.enabled:
|
||||
state = "disabled"
|
||||
elif latest is None:
|
||||
state = "waiting"
|
||||
elif latest.status == "failed":
|
||||
state = "source_changed" if "CommunityParseError" in (latest.error_summary or "") else "temporarily_limited"
|
||||
elif aware(latest.started_at) < now - timedelta(seconds=settings.community_import_interval_seconds * 2):
|
||||
state = "stale"
|
||||
else:
|
||||
state = "healthy"
|
||||
result.append(AdminSourceStatusOut(
|
||||
source_system=source.key,
|
||||
name=source.name,
|
||||
status=state,
|
||||
last_started_at=latest.started_at if latest else None,
|
||||
last_success_at=success.started_at if success else None,
|
||||
next_allowed_at=next_allowed,
|
||||
cooldown_seconds=cooldown_seconds,
|
||||
recent_failures_24h=recent_failures,
|
||||
backoff_recommended=recent_failures >= 5,
|
||||
))
|
||||
return result
|
||||
|
||||
|
||||
def _external_out(item: ExternalObservation) -> ExternalObservationOut:
|
||||
allowed_payload = {
|
||||
key: value for key, value in (item.payload or {}).items()
|
||||
if key in {
|
||||
"bait", "fishing_method", "rig_type", "retrieve_method", "retrieve_speed",
|
||||
"player_name", "published_at", "region", "category",
|
||||
} and (value is None or isinstance(value, (str, int, float, bool)))
|
||||
}
|
||||
missing_fields = []
|
||||
if item.x is None or item.y is None:
|
||||
missing_fields.append("coordinates")
|
||||
if item.weight_g is None:
|
||||
missing_fields.append("weight_g")
|
||||
return ExternalObservationOut(
|
||||
id=item.id, source_system=item.source_system, source_external_id=item.source_external_id,
|
||||
source_url=item.source_url, fish_name=item.fish_name, fish_external_id=item.fish_external_id,
|
||||
waterbody_name=item.waterbody_name, waterbody_external_id=item.waterbody_external_id,
|
||||
x=item.x, y=item.y, weight_g=item.weight_g, published_at=item.published_at,
|
||||
first_seen_at=item.first_seen_at, last_seen_at=item.last_seen_at, reviewed_at=item.reviewed_at,
|
||||
status=item.status,
|
||||
fish_slug=item.fish.slug if item.fish else None,
|
||||
waterbody_slug=item.waterbody.slug if item.waterbody else None,
|
||||
catch_report_id=item.catch_report_id, review_note=item.review_note,
|
||||
missing_fields=missing_fields, source_payload=allowed_payload,
|
||||
moderation_version=item.moderation_version,
|
||||
source_check_status=item.source_check_status, source_checked_at=item.source_checked_at,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/api/v1/admin/external-observations", response_model=list[ExternalObservationOut])
|
||||
def admin_external_observations(
|
||||
db: Db, _: Annotated[str, Depends(_admin)],
|
||||
status: Literal["staged", "mapped", "ready", "published", "rejected", "withdrawn", "review"] | None = None,
|
||||
source_system: str | None = None,
|
||||
completeness: Literal["all", "complete", "incomplete"] = "all",
|
||||
order: Literal["newest", "oldest", "risk"] = "newest",
|
||||
q: str | None = Query(None, max_length=100),
|
||||
limit: int = Query(50, ge=1, le=200), offset: int = Query(0, ge=0),
|
||||
) -> list[ExternalObservationOut]:
|
||||
query = select(ExternalObservation).options(
|
||||
joinedload(ExternalObservation.fish), joinedload(ExternalObservation.waterbody),
|
||||
)
|
||||
if status == "review":
|
||||
query = query.where(ExternalObservation.status.in_(["staged", "mapped", "ready"]))
|
||||
elif status:
|
||||
query = query.where(ExternalObservation.status == status)
|
||||
if source_system:
|
||||
query = query.where(ExternalObservation.source_system == source_system)
|
||||
if completeness == "complete":
|
||||
query = query.where(
|
||||
ExternalObservation.x.is_not(None), ExternalObservation.y.is_not(None),
|
||||
ExternalObservation.weight_g.is_not(None),
|
||||
)
|
||||
elif completeness == "incomplete":
|
||||
query = query.where(or_(
|
||||
ExternalObservation.x.is_(None), ExternalObservation.y.is_(None),
|
||||
ExternalObservation.weight_g.is_(None),
|
||||
))
|
||||
if q and q.strip():
|
||||
term = q.strip()
|
||||
query = query.where(or_(
|
||||
ExternalObservation.fish_name.icontains(term, autoescape=True),
|
||||
ExternalObservation.waterbody_name.icontains(term, autoescape=True),
|
||||
))
|
||||
if order == "risk":
|
||||
incomplete = case(
|
||||
(or_(ExternalObservation.x.is_(None), ExternalObservation.y.is_(None), ExternalObservation.weight_g.is_(None)), 0),
|
||||
else_=1,
|
||||
)
|
||||
workflow = case(
|
||||
(ExternalObservation.status == "staged", 0),
|
||||
(ExternalObservation.status == "mapped", 1),
|
||||
else_=2,
|
||||
)
|
||||
ordering = (incomplete, workflow, ExternalObservation.last_seen_at.asc(), ExternalObservation.id.desc())
|
||||
else:
|
||||
direction = ExternalObservation.last_seen_at.asc() if order == "oldest" else ExternalObservation.last_seen_at.desc()
|
||||
ordering = (direction, ExternalObservation.id.desc())
|
||||
items = db.scalars(query.order_by(*ordering).offset(offset).limit(limit))
|
||||
return [_external_out(item) for item in items]
|
||||
|
||||
|
||||
@router.get("/api/v1/admin/external-observations/{observation_id}/alias-suggestions", response_model=ExternalAliasSuggestionOut)
|
||||
def admin_external_alias_suggestions(
|
||||
observation_id: UUID, db: Db, _: Annotated[str, Depends(_admin)],
|
||||
) -> ExternalAliasSuggestionOut:
|
||||
observation = db.get(ExternalObservation, observation_id)
|
||||
if observation is None:
|
||||
raise HTTPException(status_code=404, detail="external observation not found")
|
||||
fish, waterbody = suggest_aliases(db, observation)
|
||||
return ExternalAliasSuggestionOut(
|
||||
fish_slug=fish.slug if fish else None,
|
||||
waterbody_slug=waterbody.slug if waterbody else None,
|
||||
)
|
||||
|
||||
|
||||
@router.patch("/api/v1/admin/external-observations/{observation_id}/mapping", response_model=ExternalObservationOut)
|
||||
def admin_map_external_observation(
|
||||
observation_id: UUID, payload: ExternalObservationMapping, db: Db,
|
||||
_: Annotated[str, Depends(_admin)],
|
||||
) -> ExternalObservationOut:
|
||||
observation = db.scalar(select(ExternalObservation).where(ExternalObservation.id == observation_id).with_for_update())
|
||||
fish = db.scalar(select(Fish).where(Fish.slug == payload.fish_slug))
|
||||
waterbody = db.scalar(select(Waterbody).where(Waterbody.slug == payload.waterbody_slug))
|
||||
if observation is None:
|
||||
raise HTTPException(status_code=404, detail="external observation not found")
|
||||
if fish is None or waterbody is None:
|
||||
raise HTTPException(status_code=422, detail="unknown fish or waterbody")
|
||||
if observation.moderation_version != payload.expected_version:
|
||||
raise HTTPException(status_code=409, detail="observation changed; reload the queue")
|
||||
observation.moderation_version += 1
|
||||
try:
|
||||
return _external_out(map_observation(db, observation, fish, waterbody, note=payload.note))
|
||||
except ExternalReviewError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.post("/api/v1/admin/external-observations/{observation_id}/publish", response_model=ExternalObservationPublished)
|
||||
def admin_publish_external_observation(
|
||||
observation_id: UUID, payload: ExternalObservationAction, db: Db, _: Annotated[str, Depends(_admin)],
|
||||
) -> ExternalObservationPublished:
|
||||
observation = db.scalar(select(ExternalObservation).where(ExternalObservation.id == observation_id).with_for_update())
|
||||
if observation is None:
|
||||
raise HTTPException(status_code=404, detail="external observation not found")
|
||||
if observation.moderation_version != payload.expected_version:
|
||||
raise HTTPException(status_code=409, detail="observation changed; reload the queue")
|
||||
observation.moderation_version += 1
|
||||
try:
|
||||
report = publish_observation(db, observation)
|
||||
except ExternalReviewError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
public_cache.invalidate()
|
||||
return ExternalObservationPublished(observation_id=observation.id, catch_report_id=report.id, status=observation.status)
|
||||
|
||||
|
||||
@router.patch("/api/v1/admin/external-observations/{observation_id}/reject", response_model=ExternalObservationOut)
|
||||
def admin_reject_external_observation(
|
||||
observation_id: UUID, payload: ExternalObservationDecision, db: Db,
|
||||
_: Annotated[str, Depends(_admin)],
|
||||
) -> ExternalObservationOut:
|
||||
observation = db.scalar(select(ExternalObservation).where(ExternalObservation.id == observation_id).with_for_update())
|
||||
if observation is None:
|
||||
raise HTTPException(status_code=404, detail="external observation not found")
|
||||
if observation.moderation_version != payload.expected_version:
|
||||
raise HTTPException(status_code=409, detail="observation changed; reload the queue")
|
||||
observation.moderation_version += 1
|
||||
try:
|
||||
return _external_out(reject_observation(db, observation, reason=payload.reason))
|
||||
except ExternalReviewError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.get("/api/v1/admin/catch-reports", response_model=list[AdminCatchReportOut])
|
||||
def admin_reports(db: Db, _: Annotated[str, Depends(_admin)], status: ModerationStatus = ModerationStatus.pending, limit: int = Query(50, ge=1, le=100), offset: int = Query(0, ge=0)) -> list[AdminCatchReportOut]:
|
||||
reports = list(db.scalars(select(CatchReport).options(joinedload(CatchReport.fish), joinedload(CatchReport.waterbody), joinedload(CatchReport.spot), joinedload(CatchReport.bait)).where(CatchReport.source_type == SourceType.user, CatchReport.moderation_status == status, CatchReport.deleted_at.is_(None)).order_by(CatchReport.reported_at, CatchReport.id).offset(offset).limit(limit)))
|
||||
return [AdminCatchReportOut(id=r.id, fish=r.fish.name_ru, waterbody=r.waterbody.name_ru, coordinates=f"{r.spot.x}:{r.spot.y}" if r.spot else "—", weight_g=r.weight_g, bait=r.bait.name if r.bait else None, player_name=r.player_name, reported_at=r.reported_at, moderation_status=r.moderation_status.value, comment=(r.raw_payload or {}).get("comment"), screenshot_url=signed_screenshot_url(r.screenshot_key) if r.screenshot_key else None, moderation_version=r.moderation_version) for r in reports]
|
||||
|
||||
|
||||
@router.patch("/api/v1/admin/catch-reports/{report_id}", response_model=CatchReportCreated)
|
||||
def moderate_report(report_id: UUID, payload: ModerationUpdate, db: Db, moderator: Annotated[str, Depends(_admin)]) -> CatchReportCreated:
|
||||
report = db.scalar(select(CatchReport).where(CatchReport.id == report_id).with_for_update())
|
||||
if report is None or report.source_type != SourceType.user or report.deleted_at is not None:
|
||||
raise HTTPException(status_code=404, detail="catch report not found")
|
||||
if report.moderation_version != payload.expected_version:
|
||||
raise HTTPException(status_code=409, detail="report changed; reload the queue")
|
||||
previous = report.moderation_status
|
||||
report.moderation_status = ModerationStatus(payload.status)
|
||||
report.moderation_version += 1
|
||||
db.add(ModerationEvent(catch_report=report, created_at=datetime.now(timezone.utc), previous_status=previous, new_status=report.moderation_status, moderator=moderator, reason=payload.reason))
|
||||
db.commit()
|
||||
public_cache.invalidate()
|
||||
return CatchReportCreated(id=report.id, moderation_status=report.moderation_status.value)
|
||||
|
||||
|
||||
@router.delete("/api/v1/admin/catch-reports/{report_id}", status_code=204, response_class=Response)
|
||||
def delete_report(report_id: UUID, db: Db, moderator: Annotated[str, Depends(_admin)], expected_version: int = Query(ge=0)) -> Response:
|
||||
report = db.scalar(select(CatchReport).where(CatchReport.id == report_id).with_for_update())
|
||||
if report is None or report.source_type != SourceType.user or report.deleted_at is not None:
|
||||
raise HTTPException(status_code=404, detail="catch report not found")
|
||||
if report.moderation_version != expected_version:
|
||||
raise HTTPException(status_code=409, detail="report changed; reload the queue")
|
||||
previous = report.moderation_status
|
||||
if report.screenshot_key:
|
||||
try:
|
||||
delete_screenshot(report.screenshot_key)
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=502, detail="screenshot deletion failed") from exc
|
||||
report.moderation_status = ModerationStatus.rejected
|
||||
report.moderation_version += 1
|
||||
report.deleted_at = datetime.now(timezone.utc)
|
||||
report.player_name = None
|
||||
report.source_url = None
|
||||
report.screenshot_key = None
|
||||
report.raw_payload = None
|
||||
db.add(ModerationEvent(catch_report=report, created_at=report.deleted_at, previous_status=previous, new_status=ModerationStatus.rejected, moderator=moderator, reason="user report deleted and anonymized"))
|
||||
db.commit()
|
||||
public_cache.invalidate()
|
||||
return Response(status_code=204)
|
||||
@@ -0,0 +1,66 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections import defaultdict
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from fastapi import APIRouter, Query
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session, joinedload, selectinload
|
||||
|
||||
from ..dependencies import Db
|
||||
from ..models import CatchReport, Fish, ModerationStatus, Spot, Waterbody
|
||||
from ..schemas import TackleCombinationOut
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/api/v1/analytics/tackle", response_model=list[TackleCombinationOut])
|
||||
def tackle_combinations(
|
||||
db: Db,
|
||||
waterbody: str | None = None,
|
||||
fish: str | None = None,
|
||||
method: str | None = None,
|
||||
hours: int = Query(72, ge=24, le=168),
|
||||
min_samples: int = Query(3, ge=1, le=100),
|
||||
min_players: int = Query(2, ge=1, le=100),
|
||||
) -> list[TackleCombinationOut]:
|
||||
now = datetime.now(timezone.utc)
|
||||
query = select(CatchReport).options(
|
||||
joinedload(CatchReport.fish), joinedload(CatchReport.waterbody),
|
||||
selectinload(CatchReport.tackle_components),
|
||||
).where(
|
||||
CatchReport.moderation_status == ModerationStatus.approved,
|
||||
CatchReport.deleted_at.is_(None),
|
||||
CatchReport.reported_at >= now - timedelta(hours=hours),
|
||||
)
|
||||
if waterbody:
|
||||
query = query.join(Waterbody, CatchReport.waterbody_id == Waterbody.id).where(Waterbody.slug == waterbody)
|
||||
if fish:
|
||||
query = query.join(Fish, CatchReport.fish_id == Fish.id).where(Fish.slug == fish)
|
||||
if method:
|
||||
query = query.where(CatchReport.fishing_method == method)
|
||||
|
||||
groups: dict[tuple[str, str], list[CatchReport]] = defaultdict(list)
|
||||
for report in db.scalars(query):
|
||||
for component in report.tackle_components:
|
||||
if component.raw_value.strip():
|
||||
groups[(component.role, component.raw_value.strip())].append(report)
|
||||
|
||||
result = []
|
||||
for (role, value), reports in groups.items():
|
||||
unique_reports = {report.id: report for report in reports}
|
||||
players = {report.player_name.strip().casefold() for report in unique_reports.values() if report.player_name and report.player_name.strip()}
|
||||
catches = len(unique_reports)
|
||||
unique_players = len(players)
|
||||
last_seen = max(report.reported_at for report in unique_reports.values())
|
||||
enough = catches >= min_samples and unique_players >= min_players
|
||||
result.append(TackleCombinationOut(
|
||||
role=role, value=value, catches=catches, unique_players=unique_players,
|
||||
last_seen_at=last_seen, status="recommendation" if enough else "insufficient_data",
|
||||
explanation=(
|
||||
"Достаточно независимых наблюдений для рекомендации."
|
||||
if enough else
|
||||
f"Данных мало: нужно минимум {min_samples} наблюдения и {min_players} независимых игрока."
|
||||
),
|
||||
))
|
||||
return sorted(result, key=lambda item: (item.status != "recommendation", -item.catches, -item.unique_players, item.role, item.value))
|
||||
@@ -0,0 +1,99 @@
|
||||
from uuid import UUID
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Query
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import selectinload
|
||||
|
||||
from ..dependencies import Db
|
||||
from ..models import Bait, CatchReport, Fish, ModerationStatus, Rig, Spot, TackleItem, Waterbody
|
||||
from ..schemas import BaitOut, FishOut, PaginatedTackleItemOut, RigOut, TackleItemOut, WaterbodyOut
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/api/v1/fishes", response_model=list[FishOut])
|
||||
def fishes(db: Db, limit: int = Query(200, ge=1, le=500), offset: int = Query(0, ge=0)) -> list[Fish]:
|
||||
return list(db.scalars(select(Fish).order_by(Fish.name_ru, Fish.id).offset(offset).limit(limit)))
|
||||
|
||||
|
||||
@router.get("/api/v1/waterbodies", response_model=list[WaterbodyOut])
|
||||
def waterbodies(db: Db, limit: int = Query(200, ge=1, le=500), offset: int = Query(0, ge=0)) -> list[Waterbody]:
|
||||
return list(db.scalars(select(Waterbody).order_by(Waterbody.name_ru, Waterbody.id).offset(offset).limit(limit)))
|
||||
|
||||
|
||||
@router.get("/api/v1/baits", response_model=list[BaitOut])
|
||||
def baits(db: Db, limit: int = Query(200, ge=1, le=500), offset: int = Query(0, ge=0)) -> list[Bait]:
|
||||
return list(db.scalars(select(Bait).order_by(Bait.name, Bait.id).offset(offset).limit(limit)))
|
||||
|
||||
|
||||
def _item_missing_fields(item: TackleItem) -> list[str]:
|
||||
return [field for field, value in (
|
||||
("subcategory", item.subcategory), ("brand", item.brand),
|
||||
("family", item.family), ("unlock_level", item.unlock_level),
|
||||
("source_url", item.source_url), ("source_checked_at", item.source_checked_at),
|
||||
) if value is None]
|
||||
|
||||
|
||||
@router.get("/api/v1/tackle/items", response_model=PaginatedTackleItemOut)
|
||||
def tackle_items(
|
||||
db: Db,
|
||||
category: str | None = Query(None, pattern="^(bait|lure|rod|reel|line|hook|rig|float|sinker|other)$"),
|
||||
brand: str | None = None,
|
||||
family: str | None = None,
|
||||
unlock_level: int | None = Query(None, ge=0),
|
||||
limit: int = Query(50, ge=1, le=100),
|
||||
offset: int = Query(0, ge=0),
|
||||
) -> PaginatedTackleItemOut:
|
||||
query = select(TackleItem)
|
||||
if category:
|
||||
query = query.where(TackleItem.category == category)
|
||||
if brand:
|
||||
query = query.where(TackleItem.brand == brand)
|
||||
if family:
|
||||
query = query.where(TackleItem.family == family)
|
||||
if unlock_level is not None:
|
||||
query = query.where(TackleItem.unlock_level == unlock_level)
|
||||
total = db.scalar(query.with_only_columns(func.count(TackleItem.id), maintain_column_froms=True).order_by(None)) or 0
|
||||
items = list(db.scalars(query.order_by(TackleItem.name, TackleItem.id).offset(offset).limit(limit)))
|
||||
return PaginatedTackleItemOut(
|
||||
items=[TackleItemOut.model_validate(item).model_copy(update={"missing_fields": _item_missing_fields(item)}) for item in items],
|
||||
total=total, limit=limit, offset=offset,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/api/v1/tackle/items/{item_id}", response_model=TackleItemOut)
|
||||
def tackle_item(item_id: UUID, db: Db) -> TackleItemOut:
|
||||
item = db.get(TackleItem, item_id)
|
||||
if item is None:
|
||||
raise HTTPException(status_code=404, detail="tackle item not found")
|
||||
return TackleItemOut.model_validate(item).model_copy(update={"missing_fields": _item_missing_fields(item)})
|
||||
|
||||
|
||||
@router.get("/api/v1/tackle/rigs/{rig_id}", response_model=RigOut)
|
||||
def rig_detail(rig_id: UUID, db: Db) -> RigOut:
|
||||
rig = db.scalar(select(Rig).options(selectinload(Rig.components)).where(Rig.id == rig_id))
|
||||
if rig is None:
|
||||
raise HTTPException(status_code=404, detail="rig not found")
|
||||
missing = [field for field, value in (
|
||||
("source_url", rig.source_url), ("source_checked_at", rig.source_checked_at),
|
||||
) if value is None]
|
||||
return RigOut(
|
||||
id=rig.id, name=rig.name, source_system=rig.source_system,
|
||||
source_external_id=rig.source_external_id, source_url=rig.source_url,
|
||||
source_checked_at=rig.source_checked_at, missing_fields=missing,
|
||||
components=[{
|
||||
"id": component.id, "role": component.role, "position": component.position,
|
||||
"raw_value": component.raw_value, "tackle_item_id": component.tackle_item_id,
|
||||
} for component in sorted(rig.components, key=lambda value: value.position)],
|
||||
)
|
||||
|
||||
|
||||
@router.get("/api/v1/public-spot-pages")
|
||||
def public_spot_pages(db: Db, limit: int = Query(500, ge=1, le=500), offset: int = Query(0, ge=0)) -> list[str]:
|
||||
rows = db.execute(select(Waterbody.slug, Spot.x, Spot.y, Fish.slug)
|
||||
.select_from(CatchReport).join(Spot, CatchReport.spot_id == Spot.id)
|
||||
.join(Waterbody, Spot.waterbody_id == Waterbody.id).join(Fish, CatchReport.fish_id == Fish.id)
|
||||
.where(CatchReport.moderation_status == ModerationStatus.approved, CatchReport.deleted_at.is_(None))
|
||||
.distinct().order_by(Waterbody.slug, Spot.x, Spot.y, Fish.slug).offset(offset).limit(limit))
|
||||
return [path for water, x, y, fish in rows for path in
|
||||
(f"/spots/{water}-{x}x{y}", f"/waterbodies/{water}/{fish}")]
|
||||
@@ -0,0 +1,24 @@
|
||||
from fastapi import APIRouter, HTTPException, Query
|
||||
from fastapi.responses import FileResponse
|
||||
|
||||
from ..media_catalog import published_assets, published_file
|
||||
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/api/v1/media/catalog")
|
||||
def media_catalog(entity_type: str | None = Query(None, pattern="^(fish|waterbody|tackle|reference)$")) -> list[dict]:
|
||||
return published_assets(entity_type)
|
||||
|
||||
|
||||
@router.get("/api/v1/media/assets/{digest}", response_class=FileResponse)
|
||||
def media_asset(digest: str) -> FileResponse:
|
||||
item = published_file(digest)
|
||||
if not item:
|
||||
raise HTTPException(status_code=404, detail="Media asset not found")
|
||||
path, media_type = item
|
||||
return FileResponse(path, media_type=media_type, headers={
|
||||
"Cache-Control": "public, max-age=31536000, immutable",
|
||||
"ETag": f'"{digest}"',
|
||||
})
|
||||
@@ -0,0 +1,170 @@
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from fastapi import APIRouter, Query
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import joinedload
|
||||
|
||||
from ..config import settings
|
||||
from ..dependencies import Db
|
||||
from ..models import (
|
||||
CatchReport,
|
||||
CommunityImportRun,
|
||||
DataSource,
|
||||
ExternalObservation,
|
||||
Fish,
|
||||
OfficialRecordImport,
|
||||
SourceType,
|
||||
Waterbody,
|
||||
)
|
||||
from ..schemas import (
|
||||
ImportRunPublicOut,
|
||||
OfficialRecordOut,
|
||||
PaginatedOfficialRecordOut,
|
||||
PublicObservationOut,
|
||||
SourceStatusOut,
|
||||
)
|
||||
from ..time_utils import aware
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/api/v1/community-observations", response_model=list[PublicObservationOut])
|
||||
def community_observations(
|
||||
db: Db,
|
||||
limit: int = Query(12, ge=1, le=50),
|
||||
offset: int = Query(0, ge=0),
|
||||
waterbody: str | None = None,
|
||||
fish: str | None = None,
|
||||
) -> list[PublicObservationOut]:
|
||||
query = select(ExternalObservation).join(ExternalObservation.source).options(
|
||||
joinedload(ExternalObservation.source)
|
||||
).where(
|
||||
ExternalObservation.catch_report_id.is_(None),
|
||||
ExternalObservation.status.not_in(["rejected", "withdrawn"]),
|
||||
DataSource.enabled.is_(True),
|
||||
)
|
||||
if waterbody:
|
||||
query = query.join(ExternalObservation.waterbody).where(Waterbody.slug == waterbody)
|
||||
if fish:
|
||||
query = query.join(ExternalObservation.fish).where(Fish.slug == fish)
|
||||
items = list(db.scalars(
|
||||
query.order_by(ExternalObservation.last_seen_at.desc(), ExternalObservation.id.desc())
|
||||
.offset(offset).limit(limit)
|
||||
))
|
||||
result: list[PublicObservationOut] = []
|
||||
for item in items:
|
||||
missing = []
|
||||
if item.x is None or item.y is None:
|
||||
missing.append("координаты")
|
||||
if item.weight_g is None:
|
||||
missing.append("вес")
|
||||
result.append(PublicObservationOut(
|
||||
id=item.id,
|
||||
source_system=item.source_system,
|
||||
source_name=item.source.name,
|
||||
source_url=item.source_url,
|
||||
fish_name=item.fish_name,
|
||||
waterbody_name=item.waterbody_name,
|
||||
x=item.x,
|
||||
y=item.y,
|
||||
weight_g=item.weight_g,
|
||||
last_seen_at=item.last_seen_at,
|
||||
missing_fields=missing,
|
||||
quality="incomplete" if missing else "unverified",
|
||||
))
|
||||
return result
|
||||
|
||||
|
||||
@router.get("/api/v1/source-status", response_model=list[SourceStatusOut])
|
||||
def source_status(db: Db) -> list[SourceStatusOut]:
|
||||
now = datetime.now(timezone.utc)
|
||||
result = []
|
||||
for source in db.scalars(select(DataSource).order_by(DataSource.name)):
|
||||
runs = list(db.scalars(
|
||||
select(CommunityImportRun)
|
||||
.where(CommunityImportRun.source_system == source.key)
|
||||
.order_by(CommunityImportRun.started_at.desc()).limit(20)
|
||||
))
|
||||
latest = runs[0] if runs else None
|
||||
success = next((run for run in runs if run.status == "success"), None)
|
||||
if not source.enabled:
|
||||
state = "disabled"
|
||||
elif latest is None:
|
||||
state = "waiting"
|
||||
elif latest.status == "failed":
|
||||
state = "source_changed" if "CommunityParseError" in (latest.error_summary or "") else "temporarily_limited"
|
||||
elif aware(latest.started_at) < now - timedelta(seconds=settings.community_import_interval_seconds * 2):
|
||||
state = "stale"
|
||||
else:
|
||||
state = "healthy"
|
||||
result.append(SourceStatusOut(
|
||||
source_system=source.key,
|
||||
name=source.name,
|
||||
status=state,
|
||||
last_started_at=latest.started_at if latest else None,
|
||||
last_success_at=success.started_at if success else None,
|
||||
observations=db.scalar(
|
||||
select(func.count()).select_from(ExternalObservation)
|
||||
.where(ExternalObservation.source_system == source.key)
|
||||
) or 0,
|
||||
))
|
||||
return result
|
||||
|
||||
|
||||
@router.get("/api/v1/records", response_model=PaginatedOfficialRecordOut)
|
||||
def records(
|
||||
db: Db,
|
||||
fish: str | None = None,
|
||||
waterbody: str | None = None,
|
||||
category: str | None = None,
|
||||
limit: int = Query(50, ge=1, le=100),
|
||||
offset: int = Query(0, ge=0),
|
||||
) -> PaginatedOfficialRecordOut:
|
||||
query = select(CatchReport).options(
|
||||
joinedload(CatchReport.fish),
|
||||
joinedload(CatchReport.waterbody),
|
||||
joinedload(CatchReport.bait),
|
||||
).where(CatchReport.source_type == SourceType.official_record)
|
||||
if fish:
|
||||
query = query.join(CatchReport.fish).where(Fish.slug == fish)
|
||||
if waterbody:
|
||||
query = query.join(CatchReport.waterbody).where(Waterbody.slug == waterbody)
|
||||
if category:
|
||||
query = query.where(CatchReport.raw_payload["category"].as_string() == category)
|
||||
total = db.scalar(
|
||||
query.with_only_columns(func.count(CatchReport.id), maintain_column_froms=True).order_by(None)
|
||||
) or 0
|
||||
items = list(db.scalars(
|
||||
query.order_by(CatchReport.caught_at.desc(), CatchReport.weight_g.desc(), CatchReport.id.desc())
|
||||
.offset(offset).limit(limit)
|
||||
))
|
||||
return PaginatedOfficialRecordOut(
|
||||
items=[OfficialRecordOut(
|
||||
id=item.id,
|
||||
fish=item.fish.name_ru,
|
||||
weight_g=item.weight_g,
|
||||
waterbody=item.waterbody.name_ru,
|
||||
bait=item.bait.name if item.bait else None,
|
||||
player_name=item.player_name,
|
||||
record_date=item.caught_at,
|
||||
category=(item.raw_payload or {}).get("category"),
|
||||
region=(item.raw_payload or {}).get("region"),
|
||||
source_url=item.source_url,
|
||||
) for item in items],
|
||||
total=total,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/api/v1/imports", response_model=list[ImportRunPublicOut])
|
||||
def imports(
|
||||
db: Db,
|
||||
limit: int = Query(20, ge=1, le=100),
|
||||
offset: int = Query(0, ge=0),
|
||||
) -> list[OfficialRecordImport]:
|
||||
return list(db.scalars(
|
||||
select(OfficialRecordImport)
|
||||
.order_by(OfficialRecordImport.started_at.desc(), OfficialRecordImport.id.desc())
|
||||
.offset(offset).limit(limit)
|
||||
))
|
||||
@@ -0,0 +1,125 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import logging
|
||||
import secrets
|
||||
from typing import Annotated
|
||||
from uuid import UUID
|
||||
|
||||
from fastapi import APIRouter, File, Header, HTTPException, Request, Response, UploadFile
|
||||
from fastapi.responses import JSONResponse
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
|
||||
from ..config import settings
|
||||
from ..dependencies import Db
|
||||
from ..importer import normalize
|
||||
from ..models import Bait, BaitKind, CatchReport, Fish, ModerationStatus, SourceType, Spot, SubmissionAttempt, Waterbody
|
||||
from ..schemas import CatchReportAccepted, CatchReportCreate
|
||||
from ..storage import ScreenshotError, upload_screenshot
|
||||
from ..submission_security import check_rate_limit
|
||||
from ..tackle_components import replace_tackle_components
|
||||
from rf4_research.gear_components import from_catch_fields
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger("rf4.api.submissions")
|
||||
|
||||
|
||||
@router.post("/api/v1/catch-reports", response_model=CatchReportAccepted, status_code=201)
|
||||
def create_catch_report(payload: CatchReportCreate, request: Request, db: Db, idempotency_key: Annotated[str | None, Header()] = None) -> CatchReportAccepted:
|
||||
if payload.website:
|
||||
raise HTTPException(status_code=400, detail="invalid submission")
|
||||
payload_hash = hashlib.sha256(json.dumps(payload.model_dump(mode="json"), sort_keys=True, separators=(",", ":")).encode()).hexdigest()
|
||||
key_hash = hmac.new(settings.rate_limit_secret.encode(), idempotency_key.encode(), hashlib.sha256).hexdigest() if idempotency_key else None
|
||||
if key_hash:
|
||||
cutoff = datetime.now(timezone.utc) - timedelta(minutes=5)
|
||||
db.expire_all()
|
||||
existing = db.scalar(select(SubmissionAttempt).where(SubmissionAttempt.idempotency_key == key_hash, SubmissionAttempt.created_at >= cutoff))
|
||||
if existing is not None:
|
||||
logger.info("idempotent hit", extra={"idempotency_key": idempotency_key[:8]})
|
||||
if existing.payload_hash and not hmac.compare_digest(existing.payload_hash, payload_hash):
|
||||
raise HTTPException(status_code=409, detail="Idempotency-Key was already used with different payload")
|
||||
if existing.catch_report is None:
|
||||
raise HTTPException(status_code=409, detail="idempotency record is incomplete; retry with a new key")
|
||||
replay_token = _replay_token(key_hash)
|
||||
if not hmac.compare_digest(hashlib.sha256(replay_token.encode()).hexdigest(), existing.catch_report.screenshot_upload_token_hash or ""):
|
||||
raise HTTPException(status_code=409, detail="idempotency record token mismatch; retry with a new key")
|
||||
return JSONResponse(status_code=200, content=_accepted(existing.catch_report, replay_token, True))
|
||||
logger.info("idempotency check miss", extra={"idempotency_key": idempotency_key[:8]})
|
||||
check_rate_limit(request, db, settings)
|
||||
fish = db.scalar(select(Fish).where(Fish.slug == payload.fish_slug))
|
||||
waterbody = db.scalar(select(Waterbody).where(Waterbody.slug == payload.waterbody_slug))
|
||||
if fish is None or waterbody is None:
|
||||
raise HTTPException(status_code=422, detail="unknown fish or waterbody")
|
||||
spot = db.scalar(select(Spot).where(Spot.waterbody_id == waterbody.id, Spot.x == payload.x, Spot.y == payload.y))
|
||||
if spot is None:
|
||||
spot = Spot(waterbody=waterbody, x=payload.x, y=payload.y)
|
||||
db.add(spot)
|
||||
bait = _bait(db, payload.bait_name)
|
||||
upload_token = _replay_token(key_hash) if key_hash else secrets.token_urlsafe(32)
|
||||
report = CatchReport(fish=fish, spot=spot, waterbody=waterbody, bait=bait, weight_g=payload.weight_g, fishing_method=payload.fishing_method, rig_type=payload.rig_type, retrieve_method=payload.retrieve_method, retrieve_speed=payload.retrieve_speed, caught_at=payload.caught_at, reported_at=datetime.now(timezone.utc), player_name=payload.player_name, source_type=SourceType.user, source_url=payload.source_url, source_confidence=60, moderation_status=ModerationStatus.pending, raw_payload={"comment": payload.comment} if payload.comment else None, screenshot_upload_token_hash=hashlib.sha256(upload_token.encode()).hexdigest())
|
||||
db.add(report)
|
||||
replace_tackle_components(
|
||||
db,
|
||||
report,
|
||||
from_catch_fields(bait=payload.bait_name, rig_type=payload.rig_type),
|
||||
source_system="user",
|
||||
source_url=payload.source_url,
|
||||
raw_payload={"origin": "user_submission"},
|
||||
)
|
||||
if key_hash:
|
||||
db.add(SubmissionAttempt(client_hash="", idempotency_key=key_hash, catch_report=report, payload_hash=payload_hash, created_at=datetime.now(timezone.utc)))
|
||||
try:
|
||||
db.commit()
|
||||
except IntegrityError:
|
||||
db.rollback()
|
||||
winner = db.scalar(select(SubmissionAttempt).where(SubmissionAttempt.idempotency_key == key_hash))
|
||||
if winner and winner.catch_report:
|
||||
return JSONResponse(status_code=200, content=_accepted(winner.catch_report, _replay_token(key_hash), True))
|
||||
raise
|
||||
logger.info("idempotency key stored", extra={"idempotency_key": idempotency_key[:8]})
|
||||
else:
|
||||
db.commit()
|
||||
return CatchReportAccepted(id=report.id, moderation_status=report.moderation_status.value, screenshot_upload_token=upload_token, idempotent=False)
|
||||
|
||||
|
||||
@router.post("/api/v1/catch-reports/{report_id}/screenshot", status_code=204, response_class=Response)
|
||||
def add_screenshot(report_id: UUID, db: Db, screenshot: UploadFile = File(), upload_token: Annotated[str | None, Header(alias="X-Upload-Token")] = None) -> Response:
|
||||
report = db.get(CatchReport, report_id)
|
||||
if report is None or report.source_type != SourceType.user or report.moderation_status != ModerationStatus.pending:
|
||||
raise HTTPException(status_code=404, detail="pending catch report not found")
|
||||
supplied_hash = hashlib.sha256((upload_token or "").encode()).hexdigest()
|
||||
if not report.screenshot_upload_token_hash or not hmac.compare_digest(report.screenshot_upload_token_hash, supplied_hash):
|
||||
raise HTTPException(status_code=401, detail="invalid screenshot upload token")
|
||||
if report.screenshot_key:
|
||||
raise HTTPException(status_code=409, detail="screenshot already uploaded")
|
||||
raw = screenshot.file.read(settings.screenshot_max_bytes + 1)
|
||||
try:
|
||||
report.screenshot_key = upload_screenshot(raw, filename=screenshot.filename, content_type=screenshot.content_type)
|
||||
except ScreenshotError as exc:
|
||||
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
||||
report.screenshot_upload_token_hash = None
|
||||
db.commit()
|
||||
return Response(status_code=204)
|
||||
|
||||
|
||||
def _replay_token(key_hash: str) -> str:
|
||||
return hmac.new(settings.rate_limit_secret.encode(), (key_hash + ":upload").encode(), hashlib.sha256).hexdigest()
|
||||
|
||||
|
||||
def _accepted(report: CatchReport, token: str, idempotent: bool) -> dict[str, object]:
|
||||
return {"id": str(report.id), "moderation_status": report.moderation_status.value, "screenshot_upload_token": token, "idempotent": idempotent}
|
||||
|
||||
|
||||
def _bait(db: Db, value: str | None) -> Bait | None:
|
||||
if not value or not value.strip():
|
||||
return None
|
||||
key = normalize(value)
|
||||
bait = db.scalar(select(Bait).where(Bait.normalized_name == key))
|
||||
if bait is None:
|
||||
bait = Bait(name=value.strip(), normalized_name=key, kind=BaitKind.unknown)
|
||||
db.add(bait)
|
||||
return bait
|
||||
@@ -20,6 +20,16 @@ class WaterbodyOut(BaseModel):
|
||||
slug: str
|
||||
name_ru: str
|
||||
unlock_level: int | None
|
||||
fish_species_count: int | None
|
||||
source_system: str | None
|
||||
source_external_id: str | None
|
||||
source_url: str | None
|
||||
description: str | None
|
||||
source_aliases: list[str] | None
|
||||
source_fish_species: list[str] | None
|
||||
source_image_urls: list[str] | None
|
||||
source_point_urls: list[str] | None
|
||||
source_checked_at: datetime | None
|
||||
|
||||
|
||||
class BaitOut(BaseModel):
|
||||
@@ -30,6 +40,48 @@ class BaitOut(BaseModel):
|
||||
kind: str
|
||||
|
||||
|
||||
class TackleItemOut(BaseModel):
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
id: UUID
|
||||
name: str
|
||||
category: str
|
||||
subcategory: str | None
|
||||
brand: str | None
|
||||
family: str | None
|
||||
unlock_level: int | None
|
||||
source_system: str | None
|
||||
source_external_id: str | None
|
||||
source_url: str | None
|
||||
source_checked_at: datetime | None
|
||||
missing_fields: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class PaginatedTackleItemOut(BaseModel):
|
||||
items: list[TackleItemOut]
|
||||
total: int
|
||||
limit: int
|
||||
offset: int
|
||||
|
||||
|
||||
class RigComponentOut(BaseModel):
|
||||
id: UUID
|
||||
role: str
|
||||
position: int
|
||||
raw_value: str | None
|
||||
tackle_item_id: UUID | None
|
||||
|
||||
|
||||
class RigOut(BaseModel):
|
||||
id: UUID
|
||||
name: str
|
||||
source_system: str | None
|
||||
source_external_id: str | None
|
||||
source_url: str | None
|
||||
source_checked_at: datetime | None
|
||||
missing_fields: list[str] = Field(default_factory=list)
|
||||
components: list[RigComponentOut]
|
||||
|
||||
|
||||
class ActivityOut(BaseModel):
|
||||
spot_id: UUID
|
||||
waterbody_slug: str
|
||||
@@ -48,6 +100,8 @@ class ActivityOut(BaseModel):
|
||||
confidence_score: int
|
||||
explanation: str
|
||||
sources: list[str]
|
||||
coordinate_precision: str
|
||||
coordinate_sources: list[str]
|
||||
|
||||
|
||||
class PaginatedActivityOut(BaseModel):
|
||||
@@ -57,6 +111,16 @@ class PaginatedActivityOut(BaseModel):
|
||||
offset: int
|
||||
|
||||
|
||||
class TackleCombinationOut(BaseModel):
|
||||
role: str
|
||||
value: str
|
||||
catches: int
|
||||
unique_players: int
|
||||
last_seen_at: datetime
|
||||
status: str
|
||||
explanation: str
|
||||
|
||||
|
||||
class CatchOut(BaseModel):
|
||||
id: UUID
|
||||
fish: str
|
||||
@@ -69,6 +133,18 @@ class CatchOut(BaseModel):
|
||||
retrieve_speed: int | None
|
||||
source_system: str
|
||||
source_url: str | None
|
||||
tackle_components: list["CatchTackleComponentOut"]
|
||||
|
||||
|
||||
class CatchTackleComponentOut(BaseModel):
|
||||
id: UUID
|
||||
role: str
|
||||
position: int
|
||||
raw_value: str
|
||||
tackle_item_id: UUID | None
|
||||
rig_id: UUID | None
|
||||
source_system: str | None
|
||||
source_url: str | None
|
||||
|
||||
|
||||
class SpotOut(BaseModel):
|
||||
@@ -82,6 +158,8 @@ class SpotOut(BaseModel):
|
||||
catches_3d: int
|
||||
catches_7d: int
|
||||
top_baits: list[str]
|
||||
coordinate_precision: str
|
||||
coordinate_sources: list[str]
|
||||
|
||||
|
||||
class OfficialRecordOut(BaseModel):
|
||||
@@ -199,11 +277,13 @@ class AdminCatchReportOut(BaseModel):
|
||||
moderation_status: str
|
||||
comment: str | None
|
||||
screenshot_url: str | None
|
||||
moderation_version: int
|
||||
|
||||
|
||||
class ModerationUpdate(BaseModel):
|
||||
status: str
|
||||
reason: str | None = Field(default=None, max_length=1000)
|
||||
expected_version: int = Field(ge=0)
|
||||
|
||||
@field_validator("status")
|
||||
@classmethod
|
||||
@@ -226,28 +306,50 @@ class ExternalObservationOut(BaseModel):
|
||||
y: int | None
|
||||
weight_g: int | None
|
||||
published_at: datetime | None
|
||||
first_seen_at: datetime
|
||||
last_seen_at: datetime
|
||||
reviewed_at: datetime | None
|
||||
status: str
|
||||
fish_slug: str | None
|
||||
waterbody_slug: str | None
|
||||
catch_report_id: UUID | None
|
||||
review_note: str | None
|
||||
missing_fields: list[str]
|
||||
source_payload: dict[str, str | int | float | bool | None]
|
||||
moderation_version: int
|
||||
source_check_status: str | None
|
||||
source_checked_at: datetime | None
|
||||
|
||||
|
||||
class ExternalObservationMapping(BaseModel):
|
||||
fish_slug: str
|
||||
waterbody_slug: str
|
||||
note: str | None = Field(default=None, max_length=1000)
|
||||
expected_version: int = Field(ge=0)
|
||||
|
||||
|
||||
class ExternalAliasSuggestionOut(BaseModel):
|
||||
fish_slug: str | None
|
||||
waterbody_slug: str | None
|
||||
|
||||
|
||||
class AdminModerationHistoryOut(BaseModel):
|
||||
entity_type: str
|
||||
entity_id: UUID
|
||||
decided_at: datetime
|
||||
action: str
|
||||
moderator: str | None
|
||||
reason: str | None
|
||||
requires_confirmation: bool = True
|
||||
|
||||
|
||||
class ExternalObservationDecision(BaseModel):
|
||||
reason: str = Field(min_length=1, max_length=1000)
|
||||
expected_version: int = Field(ge=0)
|
||||
|
||||
|
||||
class ExternalObservationAction(BaseModel):
|
||||
expected_version: int = Field(ge=0)
|
||||
|
||||
|
||||
class ExternalObservationPublished(BaseModel):
|
||||
@@ -263,3 +365,48 @@ class SourceStatusOut(BaseModel):
|
||||
last_started_at: datetime | None
|
||||
last_success_at: datetime | None
|
||||
observations: int
|
||||
|
||||
|
||||
class AdminSourceStatusOut(BaseModel):
|
||||
source_system: str
|
||||
name: str
|
||||
status: str
|
||||
last_started_at: datetime | None
|
||||
last_success_at: datetime | None
|
||||
next_allowed_at: datetime | None
|
||||
cooldown_seconds: int
|
||||
recent_failures_24h: int
|
||||
backoff_recommended: bool
|
||||
|
||||
|
||||
class AdminMediaDerivativeOut(BaseModel):
|
||||
role: str | None
|
||||
format: str | None
|
||||
width: int | None
|
||||
height: int | None
|
||||
|
||||
|
||||
class AdminMediaReviewOut(BaseModel):
|
||||
id: str
|
||||
status: str
|
||||
entity_type: str | None
|
||||
entity_key: str | None
|
||||
label: str | None
|
||||
width: int | None
|
||||
height: int | None
|
||||
content_type: str | None
|
||||
image_url: str
|
||||
asset_url: str
|
||||
source_system: str
|
||||
source_url: str
|
||||
duplicate_of: str | None
|
||||
supersedes: str | None
|
||||
derivatives: list[AdminMediaDerivativeOut]
|
||||
|
||||
|
||||
class AdminMediaDecision(BaseModel):
|
||||
note: str = Field(min_length=1, max_length=1000)
|
||||
|
||||
|
||||
class AdminMediaRollback(AdminMediaDecision):
|
||||
asset_url: str = Field(min_length=1, max_length=2000)
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from typing import Literal
|
||||
from urllib.error import HTTPError
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .models import ExternalObservation, ModerationStatus
|
||||
|
||||
|
||||
SourceCheckStatus = Literal["available", "missing", "temporary_error", "blocked"]
|
||||
|
||||
|
||||
def classify_source_failure(exc: Exception) -> SourceCheckStatus:
|
||||
"""Classify the result of the scheduled request without retrying it."""
|
||||
if isinstance(exc, HTTPError):
|
||||
if exc.code in {404, 410}:
|
||||
return "missing"
|
||||
if exc.code in {401, 403, 429}:
|
||||
return "blocked"
|
||||
return "temporary_error"
|
||||
|
||||
|
||||
def record_source_check(
|
||||
session: Session,
|
||||
observation: ExternalObservation,
|
||||
status: SourceCheckStatus,
|
||||
*,
|
||||
checked_at: datetime | None = None,
|
||||
) -> ExternalObservation:
|
||||
"""Persist a check performed during an already scheduled source request.
|
||||
|
||||
Only an authoritative 404/410-style ``missing`` result withdraws published
|
||||
data. Transient errors and access blocks remain diagnostic and never remove
|
||||
an observation from activity.
|
||||
"""
|
||||
_apply_source_check(observation, status, checked_at or datetime.now(timezone.utc))
|
||||
session.commit()
|
||||
return observation
|
||||
|
||||
|
||||
def _apply_source_check(
|
||||
observation: ExternalObservation,
|
||||
status: SourceCheckStatus,
|
||||
checked_at: datetime,
|
||||
) -> None:
|
||||
"""Mutate one observation; the caller owns the transaction boundary."""
|
||||
current = checked_at
|
||||
observation.source_check_status = status
|
||||
observation.source_checked_at = current
|
||||
if status == "missing" and observation.status != "withdrawn":
|
||||
if observation.catch_report is not None:
|
||||
observation.catch_report.moderation_status = ModerationStatus.pending
|
||||
observation.status = "withdrawn"
|
||||
observation.review_note = "Source record missing; withdrawn pending moderator review"
|
||||
observation.reviewed_at = current
|
||||
observation.moderation_version += 1
|
||||
|
||||
|
||||
def record_scheduled_source_check(
|
||||
session: Session,
|
||||
*,
|
||||
source_system: str,
|
||||
source_url: str,
|
||||
status: SourceCheckStatus,
|
||||
checked_at: datetime | None = None,
|
||||
) -> int:
|
||||
"""Apply one scheduled request result only to observations with that exact URL.
|
||||
|
||||
Aggregate pages cannot prove that an omitted record was deleted, so absence
|
||||
from a parsed listing is deliberately ignored.
|
||||
"""
|
||||
observations = list(session.scalars(select(ExternalObservation).where(
|
||||
ExternalObservation.source_system == source_system,
|
||||
ExternalObservation.source_url == source_url,
|
||||
)))
|
||||
current = checked_at or datetime.now(timezone.utc)
|
||||
for observation in observations:
|
||||
_apply_source_check(observation, status, current)
|
||||
session.commit()
|
||||
return len(observations)
|
||||
@@ -6,7 +6,6 @@ from functools import lru_cache
|
||||
|
||||
import boto3
|
||||
from botocore.client import BaseClient
|
||||
from botocore.exceptions import ClientError
|
||||
from PIL import Image, UnidentifiedImageError
|
||||
|
||||
from .config import settings
|
||||
@@ -63,10 +62,7 @@ def upload_screenshot(raw: bytes, *, filename: str | None = None, content_type:
|
||||
body, extension, mime = prepare_image(raw)
|
||||
key = f"reports/{uuid.uuid4()}.{extension}"
|
||||
s3 = client()
|
||||
try:
|
||||
s3.head_bucket(Bucket=settings.s3_bucket)
|
||||
except ClientError:
|
||||
s3.create_bucket(Bucket=settings.s3_bucket)
|
||||
s3.head_bucket(Bucket=settings.s3_bucket)
|
||||
s3.put_object(Bucket=settings.s3_bucket, Key=key, Body=body, ContentType=mime)
|
||||
return key
|
||||
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
from datetime import datetime, timedelta, timezone
|
||||
import hashlib
|
||||
import hmac
|
||||
from ipaddress import IPv4Address, IPv4Network, IPv6Address, IPv6Network, ip_address
|
||||
from typing import Protocol
|
||||
|
||||
from fastapi import HTTPException, Request
|
||||
from sqlalchemy import delete, func, select, text
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .models import SubmissionAttempt
|
||||
|
||||
|
||||
class RateLimitConfig(Protocol):
|
||||
rate_limit_secret: str
|
||||
trusted_proxy_cidrs: list[str]
|
||||
|
||||
|
||||
def is_trusted_proxy(address: str, trusted_cidrs: list[str]) -> bool:
|
||||
try:
|
||||
addr = IPv4Address(address) if ":" not in address else IPv6Address(address)
|
||||
except ValueError:
|
||||
return False
|
||||
for cidr in trusted_cidrs:
|
||||
try:
|
||||
network = IPv4Network(cidr) if ":" not in cidr else IPv6Network(cidr)
|
||||
if addr in network:
|
||||
return True
|
||||
except ValueError:
|
||||
continue
|
||||
return False
|
||||
|
||||
|
||||
def client_address(request: Request, trusted_cidrs: list[str]) -> str:
|
||||
client = request.client.host if request.client else "unknown"
|
||||
forwarded = request.headers.get("x-forwarded-for")
|
||||
if forwarded and request.client and is_trusted_proxy(request.client.host, trusted_cidrs):
|
||||
candidate = forwarded.split(",")[0].strip()
|
||||
try:
|
||||
return str(ip_address(candidate))
|
||||
except ValueError:
|
||||
return client
|
||||
return client
|
||||
|
||||
|
||||
def check_rate_limit(request: Request, db: Session, config: RateLimitConfig) -> None:
|
||||
now = datetime.now(timezone.utc)
|
||||
cutoff = now - timedelta(minutes=10)
|
||||
client = client_address(request, config.trusted_proxy_cidrs)
|
||||
client_hash = hmac.new(config.rate_limit_secret.encode(), client.encode(), hashlib.sha256).hexdigest()
|
||||
if db.get_bind().dialect.name == "postgresql":
|
||||
lock_key = int(client_hash[:16], 16) & 0x7FFF_FFFF_FFFF_FFFF
|
||||
db.execute(text("SELECT pg_advisory_xact_lock(:lock_key)"), {"lock_key": lock_key})
|
||||
db.execute(delete(SubmissionAttempt).where(SubmissionAttempt.created_at < now - timedelta(days=1)))
|
||||
recent = db.scalar(
|
||||
select(func.count()).select_from(SubmissionAttempt).where(
|
||||
SubmissionAttempt.client_hash == client_hash,
|
||||
SubmissionAttempt.created_at >= cutoff,
|
||||
)
|
||||
) or 0
|
||||
if recent >= 5:
|
||||
db.commit()
|
||||
raise HTTPException(status_code=429, detail="too many submissions")
|
||||
db.add(SubmissionAttempt(client_hash=client_hash, created_at=now))
|
||||
db.commit()
|
||||
@@ -0,0 +1,39 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterable
|
||||
|
||||
from sqlalchemy import delete
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from rf4_research.gear_components import GearComponentIdentity
|
||||
|
||||
from .models import CatchReport, CatchTackleComponent
|
||||
|
||||
|
||||
def replace_tackle_components(
|
||||
session: Session,
|
||||
report: CatchReport,
|
||||
components: Iterable[GearComponentIdentity],
|
||||
*,
|
||||
source_system: str | None,
|
||||
source_url: str | None = None,
|
||||
raw_payload: dict | None = None,
|
||||
) -> None:
|
||||
"""Replace the ordered evidence for a report while keeping imports idempotent."""
|
||||
session.flush()
|
||||
session.execute(
|
||||
delete(CatchTackleComponent).where(CatchTackleComponent.catch_report_id == report.id)
|
||||
)
|
||||
session.add_all(
|
||||
CatchTackleComponent(
|
||||
catch_report_id=report.id,
|
||||
role=component.role,
|
||||
position=component.position,
|
||||
raw_value=component.raw_value,
|
||||
source_system=source_system,
|
||||
source_external_id=component.source_external_id,
|
||||
source_url=source_url,
|
||||
raw_payload=raw_payload,
|
||||
)
|
||||
for component in components
|
||||
)
|
||||
@@ -0,0 +1,5 @@
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
def aware(value: datetime) -> datetime:
|
||||
return value if value.tzinfo else value.replace(tzinfo=timezone.utc)
|
||||
@@ -0,0 +1,34 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from app.main import app
|
||||
|
||||
|
||||
TARGET = Path(__file__).with_name("openapi.json")
|
||||
|
||||
|
||||
def rendered_contract() -> str:
|
||||
return json.dumps(app.openapi(), ensure_ascii=False, indent=2, sort_keys=True) + "\n"
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="Generate or verify the RF4 Spotter OpenAPI contract")
|
||||
parser.add_argument("--check", action="store_true")
|
||||
args = parser.parse_args()
|
||||
rendered = rendered_contract()
|
||||
if args.check:
|
||||
if not TARGET.exists() or TARGET.read_text(encoding="utf-8") != rendered:
|
||||
parser.exit(1, "OpenAPI contract is stale; run apps/api/export_openapi.py\n")
|
||||
print(f"OpenAPI contract is current: {len(app.openapi()['paths'])} paths")
|
||||
return 0
|
||||
TARGET.write_text(rendered, encoding="utf-8")
|
||||
print(f"Wrote {TARGET}: {len(app.openapi()['paths'])} paths")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,71 @@
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.database import Base
|
||||
from app.models import CatchReport, CatchTackleComponent, Fish, ModerationStatus, SourceType, Spot, Waterbody
|
||||
from app.routers.analytics import tackle_combinations
|
||||
|
||||
|
||||
def test_tackle_recommendation_requires_samples_and_independent_players() -> None:
|
||||
now = datetime.now(timezone.utc)
|
||||
engine = create_engine("sqlite://")
|
||||
Base.metadata.create_all(engine)
|
||||
with Session(engine) as db:
|
||||
waterbody = Waterbody(slug="lake", name_ru="Озеро", unlock_level=1)
|
||||
fish = Fish(slug="pike", name_ru="Щука", trophy_weight_g=10_000)
|
||||
spot = Spot(waterbody=waterbody, x=10, y=20)
|
||||
db.add_all([waterbody, fish, spot])
|
||||
db.flush()
|
||||
for index, player in enumerate(("One", "One", "Two")):
|
||||
report = CatchReport(
|
||||
fish=fish, waterbody=waterbody, spot=spot, weight_g=1000,
|
||||
caught_at=now - timedelta(hours=1), reported_at=now - timedelta(hours=1),
|
||||
player_name=player, source_type=SourceType.user, source_confidence=80,
|
||||
moderation_status=ModerationStatus.approved,
|
||||
)
|
||||
report.tackle_components.append(CatchTackleComponent(role="lure", position=0, raw_value="Spinner #1"))
|
||||
db.add(report)
|
||||
db.commit()
|
||||
|
||||
rows = tackle_combinations(db, waterbody="lake", fish="pike", method=None, hours=72, min_samples=3, min_players=2)
|
||||
assert len(rows) == 1
|
||||
assert (rows[0].status, rows[0].catches, rows[0].unique_players) == ("recommendation", 3, 2)
|
||||
|
||||
rows = tackle_combinations(db, waterbody="lake", fish="pike", method=None, hours=72, min_samples=3, min_players=3)
|
||||
assert rows[0].status == "insufficient_data"
|
||||
|
||||
engine.dispose()
|
||||
|
||||
|
||||
def test_tackle_analytics_handles_empty_and_multicomponent_observations() -> None:
|
||||
now = datetime.now(timezone.utc)
|
||||
engine = create_engine("sqlite://")
|
||||
Base.metadata.create_all(engine)
|
||||
with Session(engine) as db:
|
||||
waterbody = Waterbody(slug="empty-check", name_ru="Проверка", unlock_level=1)
|
||||
fish = Fish(slug="perch", name_ru="Окунь", trophy_weight_g=5_000)
|
||||
spot = Spot(waterbody=waterbody, x=1, y=2)
|
||||
report = CatchReport(
|
||||
fish=fish, waterbody=waterbody, spot=spot, weight_g=500,
|
||||
caught_at=now, reported_at=now, player_name="Player",
|
||||
source_type=SourceType.user, source_confidence=80,
|
||||
moderation_status=ModerationStatus.approved,
|
||||
)
|
||||
report.tackle_components.extend([
|
||||
CatchTackleComponent(role="lure", position=0, raw_value="Spinner #1"),
|
||||
CatchTackleComponent(role="rig", position=1, raw_value="Rig #1"),
|
||||
CatchTackleComponent(role="lure", position=2, raw_value="Spinner #1"),
|
||||
CatchTackleComponent(role="lure", position=3, raw_value=" "),
|
||||
])
|
||||
db.add(report)
|
||||
db.commit()
|
||||
|
||||
rows = tackle_combinations(db, waterbody="empty-check", fish="perch", method=None, hours=72, min_samples=1, min_players=1)
|
||||
assert {(row.role, row.value, row.catches) for row in rows} == {
|
||||
("lure", "Spinner #1", 1), ("rig", "Rig #1", 1),
|
||||
}
|
||||
assert tackle_combinations(db, waterbody="missing", fish=None, method=None, hours=72) == []
|
||||
|
||||
engine.dispose()
|
||||
@@ -12,7 +12,8 @@ from app.database import Base, get_session
|
||||
from app.community_importer import stage_observations
|
||||
from app.importer import ImportAlreadyRunning
|
||||
from app.main import app
|
||||
from app.models import Bait, BaitKind, CatchReport, DataSource, ExternalEntityAlias, ExternalObservation, Fish, ImportStatus, ModerationEvent, ModerationStatus, OfficialRecordImport, SourceType, Spot, SubmissionAttempt, Waterbody
|
||||
from app.models import Bait, BaitKind, CatchReport, CatchTackleComponent, DataSource, ExternalEntityAlias, ExternalObservation, Fish, ImportStatus, ModerationEvent, ModerationStatus, OfficialRecordImport, SourceType, Spot, SubmissionAttempt, Waterbody
|
||||
from app.routers import admin as admin_router
|
||||
|
||||
|
||||
engine = create_engine("sqlite://", connect_args={"check_same_thread": False}, poolclass=StaticPool)
|
||||
@@ -56,11 +57,37 @@ def test_activity_filters_and_explains_score() -> None:
|
||||
assert payload["items"][0]["sources"] == ["manual-import"]
|
||||
|
||||
|
||||
def test_waterbody_catalog_exposes_nullable_source_provenance() -> None:
|
||||
response = client.get("/api/v1/waterbodies")
|
||||
assert response.status_code == 200
|
||||
item = next(row for row in response.json() if row["slug"] == "test-lake")
|
||||
assert item["source_system"] is None
|
||||
assert item["source_external_id"] is None
|
||||
assert item["source_url"] is None
|
||||
assert item["description"] is None
|
||||
assert item["source_checked_at"] is None
|
||||
|
||||
|
||||
def test_invalid_period_is_rejected() -> None:
|
||||
assert client.get("/api/v1/activity?hours=13").status_code == 422
|
||||
assert client.get("/api/v1/activity?sort=unknown").status_code == 422
|
||||
|
||||
|
||||
def test_published_media_catalog_and_content_addressed_file() -> None:
|
||||
catalog = client.get("/api/v1/media/catalog?entity_type=fish")
|
||||
assert catalog.status_code == 200
|
||||
assert catalog.json()
|
||||
item = catalog.json()[0]
|
||||
image = client.get(item["image_url"])
|
||||
assert image.status_code == 200
|
||||
assert image.headers["content-type"].startswith("image/")
|
||||
assert image.headers["cache-control"] == "public, max-age=31536000, immutable"
|
||||
variant = client.get(item["variants"][0]["url"])
|
||||
assert variant.status_code == 200
|
||||
assert variant.headers["content-type"].startswith("image/")
|
||||
assert client.get("/api/v1/media/assets/not-a-hash").status_code == 404
|
||||
|
||||
|
||||
def test_review_queue_filters_before_pagination() -> None:
|
||||
with Session(engine) as db:
|
||||
stage_observations(db, [{
|
||||
@@ -139,6 +166,54 @@ def test_public_source_status_hides_internal_details() -> None:
|
||||
assert all("error_summary" not in item and "source_url" not in item for item in response.json())
|
||||
|
||||
|
||||
def test_admin_source_status_requires_auth_and_exposes_safe_cooldown_fields() -> None:
|
||||
assert client.get("/api/v1/admin/source-status").status_code == 401
|
||||
response = client.get("/api/v1/admin/source-status", headers={"Authorization": "Bearer change-me-in-production"})
|
||||
assert response.status_code == 200
|
||||
assert response.json()
|
||||
assert all({"status", "cooldown_seconds", "recent_failures_24h", "backoff_recommended"} <= set(item) for item in response.json())
|
||||
assert all({"source_system", "name", "last_started_at", "last_success_at", "next_allowed_at"} <= set(item) for item in response.json())
|
||||
assert all("error_summary" not in item and "base_url" not in item for item in response.json())
|
||||
|
||||
|
||||
def test_admin_media_review_requires_auth() -> None:
|
||||
assert client.get("/api/v1/admin/media/catalog").status_code == 401
|
||||
response = client.get("/api/v1/admin/media/catalog?status=approved&limit=2", headers={"Authorization": "Bearer change-me-in-production"})
|
||||
assert response.status_code == 200
|
||||
assert len(response.json()) <= 2
|
||||
if response.json():
|
||||
assert {"status", "width", "height", "source_system", "source_url", "derivatives"} <= set(response.json()[0])
|
||||
assert all({"role", "format", "width", "height"} <= set(derivative) for derivative in response.json()[0]["derivatives"])
|
||||
|
||||
|
||||
def test_admin_media_decisions_require_auth_and_note(monkeypatch) -> None:
|
||||
assert client.post("/api/v1/admin/media/upgrades/publish", json={"note": "publish"}).status_code == 401
|
||||
assert client.post("/api/v1/admin/media/upgrades/rollback", json={"asset_url": "https://example.test/a", "note": "rollback"}).status_code == 401
|
||||
|
||||
monkeypatch.setattr(admin_router, "publish_quality_upgrades", lambda path, note: {"published": 2, "retained_fallbacks": 2})
|
||||
publish = client.post(
|
||||
"/api/v1/admin/media/upgrades/publish",
|
||||
json={"note": "visual review complete"},
|
||||
headers={"Authorization": "Bearer change-me-in-production"},
|
||||
)
|
||||
assert publish.status_code == 200
|
||||
assert publish.json() == {"published": 2, "retained_fallbacks": 2}
|
||||
|
||||
monkeypatch.setattr(admin_router, "rollback_quality_upgrade", lambda path, asset_url, note: {"rolled_back": asset_url, "restored": "https://example.test/fallback"})
|
||||
rollback = client.post(
|
||||
"/api/v1/admin/media/upgrades/rollback",
|
||||
json={"asset_url": "https://example.test/a", "note": "fallback is preferred"},
|
||||
headers={"Authorization": "Bearer change-me-in-production"},
|
||||
)
|
||||
assert rollback.status_code == 200
|
||||
assert rollback.json()["rolled_back"] == "https://example.test/a"
|
||||
assert client.post(
|
||||
"/api/v1/admin/media/upgrades/publish",
|
||||
json={"note": ""},
|
||||
headers={"Authorization": "Bearer change-me-in-production"},
|
||||
).status_code == 422
|
||||
|
||||
|
||||
def test_liveness_does_not_probe_dependencies() -> None:
|
||||
response = client.get("/health?token=must-not-be-logged")
|
||||
assert response.json() == {"status": "ok"}
|
||||
@@ -158,6 +233,17 @@ def test_admin_diagnostics_exposes_build_identity_only_to_admin() -> None:
|
||||
serialized = response.text.lower()
|
||||
for forbidden in ("player_name", "source_url", "error_summary", "raw_payload", "admin_token", "s3_"):
|
||||
assert forbidden not in serialized
|
||||
assert client.get("/api/v1/admin/moderation-history").status_code == 401
|
||||
history = client.get("/api/v1/admin/moderation-history", headers={"Authorization": "Bearer change-me-in-production"})
|
||||
assert history.status_code == 200
|
||||
for forbidden in ("player_name", "source_url", "raw_payload", "screenshot"):
|
||||
assert forbidden not in history.text.lower()
|
||||
export = client.get("/api/v1/admin/moderation-history-export", headers={"Authorization": "Bearer change-me-in-production"})
|
||||
assert export.status_code == 200
|
||||
assert export.headers["Content-Disposition"] == "attachment; filename=rf4spotter-moderation-history.json"
|
||||
assert set(export.json()) == {"generated_at", "count", "events"}
|
||||
for forbidden in ("entity_id", "moderator", "reason", "player_name", "source_url", "raw_payload"):
|
||||
assert forbidden not in export.text.lower()
|
||||
|
||||
|
||||
def test_spot_detail_and_catches() -> None:
|
||||
@@ -265,18 +351,24 @@ def test_records_pagination_returns_correct_total_and_offset() -> None:
|
||||
|
||||
|
||||
def test_user_report_requires_moderation_before_activity() -> None:
|
||||
created = client.post("/api/v1/catch-reports", json={"fish_slug": "pike", "waterbody_slug": "test-lake", "x": 77, "y": 88, "weight_g": 5500, "bait_name": "Новая приманка", "player_name": "Reporter"})
|
||||
created = client.post("/api/v1/catch-reports", json={"fish_slug": "pike", "waterbody_slug": "test-lake", "x": 77, "y": 88, "weight_g": 5500, "bait_name": "Новая приманка", "rig_type": "Спиннинг", "player_name": "Reporter"})
|
||||
assert created.status_code == 201
|
||||
assert created.headers["Cache-Control"] == "no-store"
|
||||
assert created.json()["moderation_status"] == "pending"
|
||||
report_id = created.json()["id"]
|
||||
with Session(engine) as db:
|
||||
components = db.scalars(select(CatchTackleComponent).where(CatchTackleComponent.catch_report_id == UUID(report_id)).order_by(CatchTackleComponent.position)).all()
|
||||
assert [(component.role, component.raw_value) for component in components] == [("lure", "Новая приманка"), ("rig", "Спиннинг")]
|
||||
headers = {"Authorization": "Bearer change-me-in-production"}
|
||||
pending = client.get("/api/v1/admin/catch-reports", headers=headers)
|
||||
assert pending.status_code == 200
|
||||
assert pending.headers["Cache-Control"] == "no-store"
|
||||
assert any(item["id"] == report_id for item in pending.json())
|
||||
approved = client.patch(f"/api/v1/admin/catch-reports/{report_id}", headers=headers, json={"status": "approved", "reason": "fixture verified"})
|
||||
approved = client.patch(f"/api/v1/admin/catch-reports/{report_id}", headers=headers, json={"status": "approved", "reason": "fixture verified", "expected_version": 0})
|
||||
assert approved.status_code == 200
|
||||
stale = client.patch(f"/api/v1/admin/catch-reports/{report_id}", headers=headers, json={"status": "rejected", "reason": "stale tab", "expected_version": 0})
|
||||
assert stale.status_code == 409
|
||||
assert "reload" in stale.json()["detail"]
|
||||
activity = client.get("/api/v1/activity?waterbody=test-lake&fish=pike&hours=24").json()
|
||||
assert any(item["x"] == 77 and item["catches"] == 1 for item in activity["items"])
|
||||
|
||||
@@ -301,19 +393,19 @@ def test_external_observation_requires_mapping_and_complete_data_before_publicat
|
||||
ExternalObservation.source_external_id == "review-complete"
|
||||
))
|
||||
headers = {"Authorization": "Bearer change-me-in-production"}
|
||||
premature = client.post(f"/api/v1/admin/external-observations/{observation_id}/publish", headers=headers)
|
||||
premature = client.post(f"/api/v1/admin/external-observations/{observation_id}/publish", headers=headers, json={"expected_version": 0})
|
||||
assert premature.status_code == 409
|
||||
mapped = client.patch(
|
||||
f"/api/v1/admin/external-observations/{observation_id}/mapping", headers=headers,
|
||||
json={"fish_slug": "pike", "waterbody_slug": "test-lake", "note": "verified fixture"},
|
||||
json={"fish_slug": "pike", "waterbody_slug": "test-lake", "note": "verified fixture", "expected_version": 0},
|
||||
)
|
||||
assert mapped.status_code == 200
|
||||
assert mapped.json()["status"] == "ready"
|
||||
published = client.post(f"/api/v1/admin/external-observations/{observation_id}/publish", headers=headers)
|
||||
published = client.post(f"/api/v1/admin/external-observations/{observation_id}/publish", headers=headers, json={"expected_version": 1})
|
||||
assert published.status_code == 200
|
||||
assert published.json()["status"] == "published"
|
||||
repeated = client.post(f"/api/v1/admin/external-observations/{observation_id}/publish", headers=headers)
|
||||
assert repeated.json()["catch_report_id"] == published.json()["catch_report_id"]
|
||||
repeated = client.post(f"/api/v1/admin/external-observations/{observation_id}/publish", headers=headers, json={"expected_version": 1})
|
||||
assert repeated.status_code == 409
|
||||
with Session(engine) as db:
|
||||
observation = db.get(ExternalObservation, observation_id)
|
||||
report = db.get(CatchReport, observation.catch_report_id)
|
||||
@@ -339,17 +431,28 @@ def test_incomplete_external_observation_is_publicly_labelled_but_not_counted()
|
||||
assert signal["source_system"] == "rf4db"
|
||||
assert signal["quality"] == "incomplete"
|
||||
assert signal["missing_fields"] == ["вес"]
|
||||
assert all(item["x"] != 32 or item["y"] != 42 for item in client.get("/api/v1/activity").json()["items"])
|
||||
headers = {"Authorization": "Bearer change-me-in-production"}
|
||||
incomplete = client.get("/api/v1/admin/external-observations?status=review&completeness=incomplete&q=Pike", headers=headers)
|
||||
assert any(item["id"] == str(observation_id) for item in incomplete.json())
|
||||
provenance = next(item for item in incomplete.json() if item["id"] == str(observation_id))
|
||||
assert provenance["missing_fields"] == ["weight_g"]
|
||||
assert provenance["first_seen_at"] and provenance["last_seen_at"]
|
||||
assert set(provenance["source_payload"]) <= {"bait", "fishing_method", "rig_type", "retrieve_method", "retrieve_speed", "player_name", "published_at", "region", "category"}
|
||||
complete = client.get("/api/v1/admin/external-observations?status=review&completeness=complete&q=Pike", headers=headers)
|
||||
assert all(item["id"] != str(observation_id) for item in complete.json())
|
||||
prioritized = client.get("/api/v1/admin/external-observations?status=review&order=risk", headers=headers)
|
||||
assert prioritized.status_code == 200
|
||||
assert prioritized.json()[0]["weight_g"] is None
|
||||
assert all(item["x"] != 32 or item["y"] != 42 for item in client.get("/api/v1/activity").json()["items"])
|
||||
mapped = client.patch(
|
||||
f"/api/v1/admin/external-observations/{observation_id}/mapping", headers=headers,
|
||||
json={"fish_slug": "pike", "waterbody_slug": "test-lake"},
|
||||
json={"fish_slug": "pike", "waterbody_slug": "test-lake", "expected_version": 0},
|
||||
)
|
||||
assert mapped.json()["status"] == "mapped"
|
||||
assert client.post(f"/api/v1/admin/external-observations/{observation_id}/publish", headers=headers).status_code == 409
|
||||
assert client.post(f"/api/v1/admin/external-observations/{observation_id}/publish", headers=headers, json={"expected_version": 1}).status_code == 409
|
||||
rejected = client.patch(
|
||||
f"/api/v1/admin/external-observations/{observation_id}/reject", headers=headers,
|
||||
json={"reason": "weight is absent"},
|
||||
json={"reason": "weight is absent", "expected_version": 1},
|
||||
)
|
||||
assert rejected.json()["status"] == "rejected"
|
||||
assert all(item["id"] != str(observation_id) for item in client.get("/api/v1/community-observations").json())
|
||||
@@ -371,7 +474,7 @@ def test_admin_can_start_and_list_official_import(monkeypatch) -> None:
|
||||
db.refresh(run)
|
||||
return run
|
||||
|
||||
monkeypatch.setattr("app.main.import_records", fake_import)
|
||||
monkeypatch.setattr("app.routers.admin.import_records", fake_import)
|
||||
headers = {"Authorization": "Bearer change-me-in-production"}
|
||||
started = client.post("/api/v1/admin/imports/official-records", headers=headers)
|
||||
assert started.status_code == 201
|
||||
@@ -382,14 +485,14 @@ def test_admin_can_start_and_list_official_import(monkeypatch) -> None:
|
||||
def busy_import(*args, **kwargs):
|
||||
raise ImportAlreadyRunning("official import is already running")
|
||||
|
||||
monkeypatch.setattr("app.main.import_records", busy_import)
|
||||
monkeypatch.setattr("app.routers.admin.import_records", busy_import)
|
||||
conflict = client.post("/api/v1/admin/imports/official-records", headers=headers)
|
||||
assert conflict.status_code == 409
|
||||
|
||||
|
||||
def test_pending_report_accepts_one_validated_screenshot(monkeypatch) -> None:
|
||||
created = client.post("/api/v1/catch-reports", json={"fish_slug": "pike", "waterbody_slug": "test-lake", "x": 91, "y": 92, "weight_g": 4200}).json()
|
||||
monkeypatch.setattr("app.main.upload_screenshot", lambda raw, **metadata: "reports/test.jpg" if raw == b"image-bytes" and metadata == {"filename": "catch.jpg", "content_type": "image/jpeg"} else "unexpected")
|
||||
monkeypatch.setattr("app.routers.submissions.upload_screenshot", lambda raw, **metadata: "reports/test.jpg" if raw == b"image-bytes" and metadata == {"filename": "catch.jpg", "content_type": "image/jpeg"} else "unexpected")
|
||||
upload_url = f"/api/v1/catch-reports/{created['id']}/screenshot"
|
||||
assert client.post(upload_url, files={"screenshot": ("catch.jpg", b"image-bytes", "image/jpeg")}).status_code == 401
|
||||
assert client.post(upload_url, headers={"X-Upload-Token": "wrong"}, files={"screenshot": ("catch.jpg", b"image-bytes", "image/jpeg")}).status_code == 401
|
||||
@@ -406,9 +509,9 @@ def test_admin_delete_anonymizes_report_removes_screenshot_and_keeps_audit(monke
|
||||
report.screenshot_key = "reports/private.jpg"
|
||||
db.commit()
|
||||
deleted_keys: list[str] = []
|
||||
monkeypatch.setattr("app.main.delete_screenshot", deleted_keys.append)
|
||||
monkeypatch.setattr("app.routers.admin.delete_screenshot", deleted_keys.append)
|
||||
headers = {"Authorization": "Bearer change-me-in-production"}
|
||||
response = client.delete(f"/api/v1/admin/catch-reports/{created['id']}", headers=headers)
|
||||
response = client.delete(f"/api/v1/admin/catch-reports/{created['id']}?expected_version=0", headers=headers)
|
||||
assert response.status_code == 204
|
||||
assert deleted_keys == ["reports/private.jpg"]
|
||||
with Session(engine) as db:
|
||||
@@ -420,7 +523,7 @@ def test_admin_delete_anonymizes_report_removes_screenshot_and_keeps_audit(monke
|
||||
event = db.query(ModerationEvent).filter_by(catch_report_id=report.id).order_by(ModerationEvent.created_at.desc()).first()
|
||||
assert event is not None
|
||||
assert event.reason == "user report deleted and anonymized"
|
||||
assert client.delete(f"/api/v1/admin/catch-reports/{created['id']}", headers=headers).status_code == 404
|
||||
assert client.delete(f"/api/v1/admin/catch-reports/{created['id']}?expected_version=0", headers=headers).status_code == 404
|
||||
|
||||
|
||||
def test_catch_report_idempotency_key_prevents_duplicates(monkeypatch) -> None:
|
||||
@@ -439,3 +542,8 @@ def test_catch_report_idempotency_key_prevents_duplicates(monkeypatch) -> None:
|
||||
second = client.post("/api/v1/catch-reports", json=payload, headers=headers)
|
||||
assert second.status_code == 200, f"Expected 200, got {second.status_code}. Response: {second.json()}"
|
||||
assert second.json()["idempotent"] is True
|
||||
assert second.json()["id"] == report_id
|
||||
assert second.json()["screenshot_upload_token"] == first.json()["screenshot_upload_token"]
|
||||
changed = dict(payload, weight_g=7800)
|
||||
conflict = client.post("/api/v1/catch-reports", json=changed, headers=headers)
|
||||
assert conflict.status_code == 409
|
||||
|
||||
@@ -3,7 +3,7 @@ from datetime import datetime, timezone
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.catalog_audit import audit_catalog
|
||||
from app.catalog_audit import audit_catalog, audit_waterbody_catalog
|
||||
from app.database import Base
|
||||
from app.models import CatchReport, Fish, ModerationStatus, SourceType, Spot, Waterbody
|
||||
|
||||
@@ -24,3 +24,24 @@ def test_catalog_audit_checks_the_whole_catalog() -> None:
|
||||
assert result["reports"] == 1
|
||||
assert result["invalid_coordinates"] == 1
|
||||
assert result["failures"] == 1
|
||||
|
||||
|
||||
def test_waterbody_catalog_audit_reports_snapshot_gaps_without_legacy_rows() -> None:
|
||||
engine = create_engine("sqlite://")
|
||||
Base.metadata.create_all(engine)
|
||||
with Session(engine) as db:
|
||||
now = datetime.now(timezone.utc)
|
||||
db.add_all([
|
||||
Waterbody(
|
||||
slug="lake", name_ru="Озеро", source_system="rf4db",
|
||||
source_external_id="level_001_lake", source_url="https://rf4db.com/ru/maps/level_001_lake",
|
||||
source_checked_at=now,
|
||||
),
|
||||
Waterbody(slug="legacy", name_ru="Старое озеро"),
|
||||
])
|
||||
db.commit()
|
||||
result = audit_waterbody_catalog(db, {"level_001_lake", "level_002_river"})
|
||||
assert result["expected"] == 2
|
||||
assert result["observed"] == 1
|
||||
assert result["missing_source_external_ids"] == ["level_002_river"]
|
||||
assert result["failures"] == 1
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.database import Base
|
||||
from app.models import CatchReport, CatchTackleComponent, Fish, ModerationStatus, SourceType, Waterbody
|
||||
|
||||
|
||||
def test_catch_keeps_ordered_unresolved_gear_evidence() -> None:
|
||||
engine = create_engine("sqlite:///:memory:")
|
||||
Base.metadata.create_all(engine)
|
||||
with Session(engine) as session:
|
||||
fish = Fish(slug="pike", name_ru="Щука")
|
||||
waterbody = Waterbody(slug="lake", name_ru="Озеро")
|
||||
report = CatchReport(
|
||||
fish=fish, waterbody=waterbody, weight_g=1000,
|
||||
reported_at=datetime(2026, 9, 20, tzinfo=timezone.utc), source_type=SourceType.manual_import,
|
||||
source_confidence=50, moderation_status=ModerationStatus.pending,
|
||||
)
|
||||
report.tackle_components.extend([
|
||||
CatchTackleComponent(role="lure", position=0, raw_value="Spiker #2"),
|
||||
CatchTackleComponent(role="rig", position=1, raw_value="Method Popup"),
|
||||
])
|
||||
session.add(report)
|
||||
session.commit()
|
||||
saved = session.get(CatchReport, report.id)
|
||||
assert saved is not None
|
||||
assert [(row.position, row.role, row.raw_value) for row in saved.tackle_components] == [
|
||||
(0, "lure", "Spiker #2"), (1, "rig", "Method Popup"),
|
||||
]
|
||||
@@ -7,10 +7,11 @@ import pytest
|
||||
from sqlalchemy import create_engine, func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.community_importer import CommunityImportError, stage_observations
|
||||
from app.community_importer import CommunityImportError, stage_observations, update_waterbody_detail, update_waterbody_details, upsert_waterbody_catalog
|
||||
from app.community_review import ExternalReviewError, map_observation, publish_observation, suggest_aliases
|
||||
from app.source_lifecycle import record_scheduled_source_check, record_source_check
|
||||
from app.database import Base
|
||||
from app.models import CatchReport, DataSource, ExternalEntityAlias, ExternalObservation, Fish, Waterbody
|
||||
from app.models import CatchReport, CatchTackleComponent, DataSource, ExternalEntityAlias, ExternalObservation, Fish, Waterbody
|
||||
from rf4_research.community_sources import parse_rf4db_catches, parse_rf4map_point, parse_rf4posts_spot
|
||||
|
||||
|
||||
@@ -41,6 +42,20 @@ def record(source: str = "rf4db", external_id: str = "catch-1") -> dict[str, obj
|
||||
}
|
||||
|
||||
|
||||
def waterbody_row(**overrides: object) -> dict[str, object]:
|
||||
row: dict[str, object] = {
|
||||
"source_system": "rf4db",
|
||||
"source_external_id": "level_001_mosquito",
|
||||
"source_url": "https://rf4db.com/ru/maps/level_001_mosquito",
|
||||
"name": "оз. Комариное",
|
||||
"unlock_level": 1,
|
||||
"unlock_label": "1",
|
||||
"fish_species_count": 20,
|
||||
}
|
||||
row.update(overrides)
|
||||
return row
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def db() -> Session:
|
||||
engine = create_engine("sqlite://")
|
||||
@@ -68,6 +83,77 @@ def test_staging_is_idempotent_and_preserves_first_seen(db: Session) -> None:
|
||||
assert source is not None and source.enabled is True
|
||||
|
||||
|
||||
def test_waterbody_catalog_upsert_is_idempotent_and_non_destructive(db: Session) -> None:
|
||||
first = datetime(2026, 9, 16, 10, tzinfo=timezone.utc)
|
||||
assert upsert_waterbody_catalog(db, [waterbody_row()], fetched_at=first) == (1, 0)
|
||||
item = db.scalar(select(Waterbody).where(Waterbody.source_external_id == "level_001_mosquito"))
|
||||
assert item is not None
|
||||
assert item.slug == "оз-комариное"
|
||||
assert item.source_checked_at.replace(tzinfo=timezone.utc) == first
|
||||
assert item.fish_species_count == 20
|
||||
|
||||
assert upsert_waterbody_catalog(db, [waterbody_row(name="Озеро Комариное", unlock_level=2)], fetched_at=first) == (0, 1)
|
||||
item = db.scalar(select(Waterbody).where(Waterbody.source_external_id == "level_001_mosquito"))
|
||||
assert item is not None
|
||||
assert (item.name_ru, item.unlock_level, item.fish_species_count) == ("Озеро Комариное", 2, 20)
|
||||
assert db.scalar(select(Waterbody).where(Waterbody.name_ru == "оз. Комариное")) is None
|
||||
|
||||
|
||||
def test_waterbody_catalog_rejects_untrusted_source(db: Session) -> None:
|
||||
with pytest.raises(CommunityImportError, match="source_url"):
|
||||
upsert_waterbody_catalog(db, [waterbody_row(source_url="https://example.test/map")])
|
||||
|
||||
|
||||
def test_waterbody_detail_updates_only_imported_identity_without_media_roles(db: Session) -> None:
|
||||
upsert_waterbody_catalog(db, [waterbody_row()])
|
||||
assert update_waterbody_detail(db, {
|
||||
"source_system": "rf4db",
|
||||
"source_external_id": "level_001_mosquito",
|
||||
"source_url": "https://rf4db.com/ru/maps/level_001_mosquito",
|
||||
"name": "оз. Комариное",
|
||||
"description": "Каменистые берега.",
|
||||
"aliases": ["Комариное", "Комариное"],
|
||||
"fish_species": ["Щука", "Окунь"],
|
||||
"image_urls": ["https://oss.rf4db.com/map.webp"],
|
||||
"point_urls": ["https://rf4db.com/ru/maps/level_001_mosquito/spots/12-34"],
|
||||
}) is True
|
||||
item = db.scalar(select(Waterbody).where(Waterbody.source_external_id == "level_001_mosquito"))
|
||||
assert item is not None
|
||||
assert item.source_aliases == ["Комариное"]
|
||||
assert item.source_fish_species == ["Щука", "Окунь"]
|
||||
assert item.source_image_urls == ["https://oss.rf4db.com/map.webp"]
|
||||
|
||||
|
||||
def test_waterbody_detail_accepts_authorized_download_subdomain(db: Session) -> None:
|
||||
upsert_waterbody_catalog(db, [waterbody_row()])
|
||||
assert update_waterbody_detail(db, {
|
||||
"source_system": "rf4db",
|
||||
"source_external_id": "level_001_mosquito",
|
||||
"source_url": "https://download.rf4db.com/ru/maps/level_001_mosquito",
|
||||
"name": "оз. Комариное",
|
||||
"description": None,
|
||||
"aliases": [],
|
||||
"fish_species": ["Щука"],
|
||||
"image_urls": [],
|
||||
"point_urls": [],
|
||||
}) is True
|
||||
|
||||
|
||||
def test_waterbody_detail_batch_validates_before_writing(db: Session) -> None:
|
||||
upsert_waterbody_catalog(db, [waterbody_row()])
|
||||
valid = {
|
||||
"source_system": "rf4db", "source_external_id": "level_001_mosquito",
|
||||
"source_url": "https://rf4db.com/ru/maps/level_001_mosquito", "name": "оз. Комариное",
|
||||
"description": "Описание", "aliases": [], "fish_species": ["Щука"],
|
||||
"image_urls": [], "point_urls": [],
|
||||
}
|
||||
invalid = valid | {"source_external_id": "unknown", "source_url": "https://example.test/map"}
|
||||
with pytest.raises(CommunityImportError, match="source_url"):
|
||||
update_waterbody_details(db, [valid, invalid])
|
||||
item = db.scalar(select(Waterbody).where(Waterbody.source_external_id == "level_001_mosquito"))
|
||||
assert item is not None and item.description is None
|
||||
|
||||
|
||||
def test_external_ids_are_isolated_by_source(db: Session) -> None:
|
||||
created, updated = stage_observations(db, [record("rf4db"), record("rf4stat-fishing")])
|
||||
|
||||
@@ -103,14 +189,28 @@ def test_complete_observation_with_reviewed_aliases_is_published(db: Session) ->
|
||||
assert item.catch_report is not None
|
||||
assert item.catch_report.fish_id == fish.id
|
||||
assert item.catch_report.waterbody_id == waterbody.id
|
||||
assert db.scalar(select(func.count()).select_from(CatchReport)) == 1
|
||||
|
||||
assert stage_observations(db, [record() | {"weight_g": 5_000}]) == (0, 1)
|
||||
db.refresh(item)
|
||||
assert item.status == "published"
|
||||
assert db.scalar(select(func.count()).select_from(CatchReport)) == 1
|
||||
|
||||
|
||||
def test_observation_preserves_coordinate_text_and_precision(db: Session) -> None:
|
||||
stage_observations(db, [record() | {
|
||||
"source_external_id": "coordinate-area",
|
||||
"x": None, "y": None, "coordinate_raw": "северная бухта",
|
||||
"coordinate_precision": "area", "weight_g": None,
|
||||
}])
|
||||
item = db.scalar(select(ExternalObservation).where(ExternalObservation.source_external_id == "coordinate-area"))
|
||||
assert item is not None
|
||||
assert (item.coordinate_raw, item.coordinate_precision, item.x, item.y) == ("северная бухта", "area", None, None)
|
||||
|
||||
|
||||
def test_coordinate_precision_rejects_unknown_value(db: Session) -> None:
|
||||
with pytest.raises(CommunityImportError, match="invalid coordinate_precision"):
|
||||
stage_observations(db, [record() | {"coordinate_precision": "guess"}])
|
||||
|
||||
|
||||
def test_changed_published_record_requires_review_and_reuses_report(db: Session) -> None:
|
||||
fish = Fish(slug="pike", name_ru="Щука")
|
||||
water = Waterbody(slug="test-lake", name_ru="Тестовое озеро")
|
||||
@@ -128,6 +228,8 @@ def test_changed_published_record_requires_review_and_reuses_report(db: Session)
|
||||
assert report.moderation_status.value == "pending"
|
||||
assert report.weight_g == 5000
|
||||
assert item.weight_g == 6000
|
||||
assert item.moderation_version == 1
|
||||
assert item.reviewed_at is not None
|
||||
stage_observations(db, [record() | {"weight_g": 6000}])
|
||||
assert item.status == "staged"
|
||||
with pytest.raises(ExternalReviewError):
|
||||
@@ -137,9 +239,83 @@ def test_changed_published_record_requires_review_and_reuses_report(db: Session)
|
||||
assert updated.id == report_id
|
||||
assert updated.weight_g == 6000
|
||||
assert updated.moderation_status.value == "approved"
|
||||
components = db.scalars(select(CatchTackleComponent).order_by(CatchTackleComponent.position)).all()
|
||||
assert [(component.position, component.raw_value) for component in components] == [(0, "Приманка")]
|
||||
assert db.scalar(select(func.count()).select_from(CatchReport)) == 1
|
||||
|
||||
|
||||
def test_missing_source_withdraws_published_record_until_manual_review(db: Session) -> None:
|
||||
fish = Fish(slug="pike", name_ru="Щука")
|
||||
water = Waterbody(slug="test-lake", name_ru="Тестовое озеро")
|
||||
db.add_all([fish, water])
|
||||
db.commit()
|
||||
seen = datetime(2026, 9, 13, 8, tzinfo=timezone.utc)
|
||||
checked = datetime(2026, 9, 13, 9, tzinfo=timezone.utc)
|
||||
stage_observations(db, [record() | {"weight_g": 5000}], fetched_at=seen)
|
||||
item = db.scalar(select(ExternalObservation))
|
||||
assert item is not None and item.catch_report is not None
|
||||
|
||||
record_source_check(db, item, "missing", checked_at=checked)
|
||||
|
||||
assert item.status == "withdrawn"
|
||||
assert item.source_check_status == "missing"
|
||||
assert item.source_checked_at.replace(tzinfo=timezone.utc) == checked
|
||||
assert item.catch_report.moderation_status.value == "pending"
|
||||
assert item.moderation_version == 1
|
||||
|
||||
stage_observations(db, [record() | {"weight_g": 5000}], fetched_at=checked)
|
||||
assert item.status == "staged"
|
||||
assert item.source_check_status == "available"
|
||||
assert item.catch_report.moderation_status.value == "pending"
|
||||
assert "reappeared" in (item.review_note or "")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("status", ["temporary_error", "blocked"])
|
||||
def test_non_authoritative_source_failures_do_not_withdraw(db: Session, status: str) -> None:
|
||||
fish = Fish(slug="pike", name_ru="Щука")
|
||||
water = Waterbody(slug="test-lake", name_ru="Тестовое озеро")
|
||||
db.add_all([fish, water])
|
||||
db.commit()
|
||||
stage_observations(db, [record() | {"weight_g": 5000}])
|
||||
item = db.scalar(select(ExternalObservation))
|
||||
assert item is not None and item.catch_report is not None
|
||||
|
||||
record_source_check(db, item, status) # type: ignore[arg-type]
|
||||
|
||||
assert item.status == "published"
|
||||
assert item.catch_report.moderation_status.value == "approved"
|
||||
assert item.source_check_status == status
|
||||
|
||||
|
||||
def test_scheduled_failure_only_affects_exact_source_url(db: Session) -> None:
|
||||
stage_observations(db, [
|
||||
record(external_id="matching"),
|
||||
record(external_id="other") | {"source_url": "https://rf4db.com/catches/other"},
|
||||
])
|
||||
|
||||
affected = record_scheduled_source_check(
|
||||
db,
|
||||
source_system="rf4db",
|
||||
source_url="https://rf4db.com/ru/catches/matching",
|
||||
status="missing",
|
||||
)
|
||||
items = {item.source_external_id: item for item in db.scalars(select(ExternalObservation))}
|
||||
|
||||
assert affected == 1
|
||||
assert items["matching"].status == "withdrawn"
|
||||
assert items["other"].status != "withdrawn"
|
||||
assert items["other"].source_check_status == "available"
|
||||
|
||||
record_scheduled_source_check(
|
||||
db,
|
||||
source_system="rf4db",
|
||||
source_url="https://rf4db.com/ru/catches/matching",
|
||||
status="available",
|
||||
)
|
||||
assert items["matching"].source_check_status == "available"
|
||||
assert items["matching"].status == "withdrawn"
|
||||
|
||||
|
||||
def test_auto_publication_requires_enabled_source(db: Session) -> None:
|
||||
source = DataSource(key="rf4db", name="RF4DB", base_url="https://rf4db.com", default_confidence=70, enabled=False)
|
||||
fish = Fish(slug="pike", name_ru="Щука")
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
from urllib.error import HTTPError
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from app.community_scheduler import MAX_BACKOFF_SECONDS, configured_sources, _static_registry, oldest_site_source, retry_delay
|
||||
from app.config import Settings
|
||||
from app.source_lifecycle import classify_source_failure
|
||||
|
||||
|
||||
def test_all_authorized_sources_are_scheduled() -> None:
|
||||
@@ -23,6 +25,16 @@ def test_failed_runs_back_off_but_success_resets_delay() -> None:
|
||||
assert retry_delay(["success", "failed"]) == 1800
|
||||
|
||||
|
||||
@pytest.mark.parametrize(("code", "expected"), [(404, "missing"), (410, "missing"), (403, "blocked"), (429, "blocked"), (500, "temporary_error")])
|
||||
def test_source_http_failure_classification(code: int, expected: str) -> None:
|
||||
error = HTTPError("https://rf4.example/source", code, "failure", {}, None)
|
||||
assert classify_source_failure(error) == expected
|
||||
|
||||
|
||||
def test_non_http_source_failure_is_temporary() -> None:
|
||||
assert classify_source_failure(TimeoutError("timeout")) == "temporary_error"
|
||||
|
||||
|
||||
def test_same_site_endpoints_rotate_by_oldest_attempt() -> None:
|
||||
now = datetime.now(timezone.utc)
|
||||
all_keys = {"rf4db", "rf4stat-fishing", "rf4stat-post", "rf4map", "rf4posts-spot"}
|
||||
|
||||
@@ -8,7 +8,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app.database import Base
|
||||
from app.importer import FetchResult, ImportAlreadyRunning, ImportSourceError, _lock_key, _official_import_lock, import_records, parse_html
|
||||
from app.models import CatchReport, ImportStatus, OfficialRecordImport, SourceType
|
||||
from app.models import CatchReport, CatchTackleComponent, ImportStatus, OfficialRecordImport, SourceType
|
||||
|
||||
|
||||
FIXTURE = Path(__file__).parents[3] / "tests" / "fixtures" / "records_ru_sample.html"
|
||||
@@ -50,6 +50,7 @@ def test_parser_and_import_are_idempotent() -> None:
|
||||
# A12: Second import of identical data creates no events (no fields changed)
|
||||
assert (second.rows_created, second.rows_updated) == (0, 0)
|
||||
assert db.scalar(select(func.count()).select_from(CatchReport).where(CatchReport.source_type == SourceType.official_record)) == 2
|
||||
assert db.scalar(select(func.count()).select_from(CatchTackleComponent)) == 2
|
||||
assert db.scalar(select(func.count()).select_from(OfficialRecordImport)) == 2
|
||||
|
||||
|
||||
|
||||
@@ -9,8 +9,51 @@ from sqlalchemy import create_engine, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.database import Base
|
||||
from app.admin_security import verify_admin
|
||||
from app.main import _check_rate_limit, _is_trusted_proxy
|
||||
from app.models import SubmissionAttempt
|
||||
from app.models import AdminAuthAttempt, SubmissionAttempt
|
||||
|
||||
|
||||
def _admin_config() -> MagicMock:
|
||||
config = MagicMock()
|
||||
config.admin_token = "correct-token"
|
||||
config.admin_auth_attempt_limit = 3
|
||||
config.admin_auth_window_seconds = 600
|
||||
config.rate_limit_secret = "test-secret-for-testing"
|
||||
config.trusted_proxy_cidrs = ["127.0.0.1/32"]
|
||||
return config
|
||||
|
||||
|
||||
def test_admin_auth_failures_are_hashed_and_limited() -> None:
|
||||
engine = create_engine("sqlite://")
|
||||
Base.metadata.create_all(engine)
|
||||
request = MagicMock()
|
||||
request.client.host = "203.0.113.42"
|
||||
request.headers.get.return_value = None
|
||||
with Session(engine) as db:
|
||||
for _ in range(3):
|
||||
with pytest.raises(HTTPException) as denied:
|
||||
verify_admin(request, db, "Bearer wrong", _admin_config())
|
||||
assert denied.value.status_code == 401
|
||||
with pytest.raises(HTTPException) as limited:
|
||||
verify_admin(request, db, "Bearer correct-token", _admin_config())
|
||||
assert limited.value.status_code == 429
|
||||
attempts = list(db.scalars(select(AdminAuthAttempt)))
|
||||
assert len(attempts) == 3
|
||||
assert all(item.client_hash != "203.0.113.42" and len(item.client_hash) == 64 for item in attempts)
|
||||
|
||||
|
||||
def test_successful_admin_auth_clears_previous_failures() -> None:
|
||||
engine = create_engine("sqlite://")
|
||||
Base.metadata.create_all(engine)
|
||||
request = MagicMock()
|
||||
request.client.host = "203.0.113.43"
|
||||
request.headers.get.return_value = None
|
||||
with Session(engine) as db:
|
||||
with pytest.raises(HTTPException):
|
||||
verify_admin(request, db, "Bearer wrong", _admin_config())
|
||||
assert verify_admin(request, db, "Bearer correct-token", _admin_config()) == "admin"
|
||||
assert list(db.scalars(select(AdminAuthAttempt))) == []
|
||||
|
||||
|
||||
def test_rate_limit_is_persistent_and_does_not_store_raw_client() -> None:
|
||||
|
||||
@@ -11,12 +11,13 @@ from app.readiness import readiness_report
|
||||
|
||||
|
||||
class AvailableStorage:
|
||||
def list_buckets(self) -> dict[str, list[object]]:
|
||||
return {"Buckets": []}
|
||||
def head_bucket(self, *, Bucket: str) -> dict[str, object]:
|
||||
assert Bucket
|
||||
return {}
|
||||
|
||||
|
||||
class UnavailableStorage:
|
||||
def list_buckets(self) -> None:
|
||||
def head_bucket(self, *, Bucket: str) -> None:
|
||||
raise ConnectionError("fixture unavailable")
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.database import Base
|
||||
from app.models import Rig, RigComponent, TackleItem
|
||||
from app.routers.catalog import rig_detail, tackle_item, tackle_items
|
||||
|
||||
|
||||
def test_tackle_catalog_filters_details_and_missing_fields() -> None:
|
||||
engine = create_engine("sqlite://")
|
||||
Base.metadata.create_all(engine)
|
||||
with Session(engine) as db:
|
||||
item = TackleItem(
|
||||
name="API тестовая блесна", normalized_name="api тестовая блесна",
|
||||
category="lure", subcategory="spinner", brand="RF4", family=None,
|
||||
unlock_level=0, source_system="fixture", source_external_id="api-lure-1",
|
||||
source_url="https://example.test/lure/api-lure-1",
|
||||
)
|
||||
rig = Rig(name="API тестовый монтаж", normalized_name="api тестовый монтаж", source_system="fixture")
|
||||
rig.components.append(RigComponent(role="lure", position=0, tackle_item=item, raw_value=item.name))
|
||||
db.add(rig)
|
||||
db.commit()
|
||||
|
||||
page = tackle_items(db, category="lure", brand="RF4", family=None, unlock_level=None, limit=10, offset=0)
|
||||
assert page.total == 1
|
||||
assert page.items[0].missing_fields == ["family", "source_checked_at"]
|
||||
assert tackle_item(item.id, db).name == item.name
|
||||
details = rig_detail(rig.id, db)
|
||||
assert details.components[0].raw_value == item.name
|
||||
assert details.missing_fields == ["source_url", "source_checked_at"]
|
||||
|
||||
engine.dispose()
|
||||
@@ -0,0 +1,35 @@
|
||||
import uuid
|
||||
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.database import Base
|
||||
from app.models import Rig, RigComponent, TackleItem
|
||||
|
||||
|
||||
def test_tackle_item_and_rig_components_keep_provenance_and_legacy_independence() -> None:
|
||||
engine = create_engine("sqlite:///:memory:")
|
||||
Base.metadata.create_all(engine)
|
||||
item_id = uuid.uuid4()
|
||||
rig_id = uuid.uuid4()
|
||||
with Session(engine) as session:
|
||||
item = TackleItem(
|
||||
id=item_id, name="Spiker #2", normalized_name="spiker #2",
|
||||
category="lure", subcategory="spinner", brand="RF4", family="spoon",
|
||||
unlock_level=0, source_system="rf4db", source_external_id="spiker-2",
|
||||
source_url="https://rf4db.com/ru/wiki/lures/spiker-2",
|
||||
raw_payload={"weight": {"state": "value", "value": 0}},
|
||||
)
|
||||
rig = Rig(
|
||||
id=rig_id, name="Method Popup", normalized_name="method popup",
|
||||
source_system="rf4db", source_external_id="method-popup",
|
||||
)
|
||||
rig.components.append(RigComponent(role="lure", position=0, tackle_item=item, raw_value="Spiker #2"))
|
||||
session.add(rig)
|
||||
session.commit()
|
||||
|
||||
saved = session.get(TackleItem, item_id)
|
||||
assert saved is not None
|
||||
assert saved.unlock_level == 0
|
||||
assert saved.raw_payload == {"weight": {"state": "value", "value": 0}}
|
||||
assert saved.rig_components[0].rig_id == rig_id
|
||||
@@ -7,4 +7,9 @@ export default defineConfig({
|
||||
output: "server",
|
||||
adapter: node({ mode: "standalone" }),
|
||||
server: { host: true, port: 4321 },
|
||||
vite: {
|
||||
// Keep executable scripts and compiled component styles in same-origin
|
||||
// assets so production CSP does not need broad inline allowances.
|
||||
build: { assetsInlineLimit: 0 },
|
||||
},
|
||||
});
|
||||
|
||||
|
Before Width: | Height: | Size: 57 KiB After Width: | Height: | Size: 6.2 KiB |
|
Before Width: | Height: | Size: 3.9 KiB After Width: | Height: | Size: 1.1 KiB |
|
Before Width: | Height: | Size: 60 KiB After Width: | Height: | Size: 6.4 KiB |
|
Before Width: | Height: | Size: 69 KiB After Width: | Height: | Size: 18 KiB |
|
After Width: | Height: | Size: 4.3 KiB |
|
After Width: | Height: | Size: 12 KiB |
@@ -0,0 +1,11 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64">
|
||||
<rect width="64" height="64" fill="#082226"/>
|
||||
<g stroke="#295158" stroke-width="1" opacity=".7">
|
||||
<path d="M16 8v48M32 8v48M48 8v48M8 16h48M8 32h48M8 48h48"/>
|
||||
</g>
|
||||
<path d="M11 46c8-5 14 5 22 0s14 5 21 0" fill="none" stroke="#6fc3c4" stroke-width="2.5" stroke-linecap="round"/>
|
||||
<path d="M37 8v21" stroke="#f4efe2" stroke-width="3" stroke-linecap="round"/>
|
||||
<path d="M32 17h10l-2 12h-6Z" fill="#c9f45b"/>
|
||||
<path d="M37 29v13c0 9-13 11-15 2-1-5 3-8 7-7" fill="none" stroke="#c9f45b" stroke-width="4" stroke-linecap="round"/>
|
||||
<circle cx="37" cy="13" r="2.5" fill="#ff785a"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 657 B |
@@ -6,10 +6,12 @@
|
||||
"start_url": "/",
|
||||
"scope": "/",
|
||||
"display": "standalone",
|
||||
"background_color": "#f2f5ee",
|
||||
"theme_color": "#082226",
|
||||
"background_color": "#071719",
|
||||
"theme_color": "#071719",
|
||||
"icons": [
|
||||
{ "src": "/icon-192.png", "sizes": "192x192", "type": "image/png", "purpose": "any maskable" },
|
||||
{ "src": "/icon-512.png", "sizes": "512x512", "type": "image/png", "purpose": "any maskable" }
|
||||
{ "src": "/icon-192.png", "sizes": "192x192", "type": "image/png", "purpose": "any" },
|
||||
{ "src": "/icon-512.png", "sizes": "512x512", "type": "image/png", "purpose": "any" },
|
||||
{ "src": "/icon-maskable-192.png", "sizes": "192x192", "type": "image/png", "purpose": "maskable" },
|
||||
{ "src": "/icon-maskable-512.png", "sizes": "512x512", "type": "image/png", "purpose": "maskable" }
|
||||
]
|
||||
}
|
||||
|
||||
@@ -4,9 +4,11 @@ import { activityLevel, ago, kg, plural, spotPath } from "../lib/api";
|
||||
import FishingIcon from "./FishingIcon.astro";
|
||||
import DataPassport from "./DataPassport.astro";
|
||||
import FishSilhouette from "./FishSilhouette.astro";
|
||||
import TackleGlyph from "./TackleGlyph.astro";
|
||||
const { item } = Astro.props as { item: Activity };
|
||||
const level = activityLevel(item.activity_score);
|
||||
const limited = item.catches < 3;
|
||||
const precision = { exact: "точные", approximate: "приблизительные", area: "район", missing: "не указаны" }[item.coordinate_precision];
|
||||
---
|
||||
<a class="spot-card" data-testid={`spot-${item.x}-${item.y}`} href={spotPath(item)}>
|
||||
<span class="spot-rank">{String(item.activity_score).padStart(2,"0")}</span>
|
||||
@@ -14,10 +16,10 @@ const limited = item.catches < 3;
|
||||
<div class="spot-topline"><span>{item.waterbody}</span><span class="activity-pill" data-activity-level={level.short}><i></i>{level.short}</span>{limited && <span class="data-quality">Данных мало</span>}</div>
|
||||
<h3>{item.fish}</h3>
|
||||
<FishSilhouette name={item.fish}/>
|
||||
<div class="spot-meta"><span><FishingIcon name="pin" size={14}/> {item.x}:{item.y}</span><span><FishingIcon name="clock" size={14}/> {ago(item.last_confirmed_at)}</span></div>
|
||||
<div class="spot-meta"><span><FishingIcon name="pin" size={14}/> {item.x}:{item.y} · {precision}</span><span><FishingIcon name="clock" size={14}/> {ago(item.last_confirmed_at)}</span></div>
|
||||
<p class="data-note">{item.explanation}</p>
|
||||
<DataPassport sources={item.sources} observedAt={item.last_confirmed_at} confidence={item.confidence_score}/>
|
||||
<div class="bait-line"><FishingIcon name="lure" size={25}/><div><span>Работает сейчас</span><strong>{item.best_bait ?? "не указана"}</strong></div></div>
|
||||
<div class="bait-line"><TackleGlyph name={item.best_bait}/><div><span>Работает сейчас</span><strong>{item.best_bait ?? "не указана"}</strong></div></div>
|
||||
</div>
|
||||
<div class="spot-stats"><div><strong>{item.catches}</strong><span>{plural(item.catches, ["улов", "улова", "уловов"])}</span></div><div><strong>{item.unique_players}</strong><span>{plural(item.unique_players, ["игрок", "игрока", "игроков"])}</span></div><div><strong>{kg(item.average_weight_g)}</strong><span>средний вес</span></div><div><strong>{item.confidence_score}%</strong><span>уверенность</span></div></div><span class="card-arrow"><FishingIcon name="arrow" size={22}/></span>
|
||||
</a>
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
---
|
||||
const { buckets } = Astro.props as { buckets: { start: string; end: string; count: number }[] };
|
||||
const max = Math.max(1, ...buckets.map(bucket => bucket.count));
|
||||
const meterLevel = (count: number) => Math.round((count / max * 100) / 5) * 5;
|
||||
---
|
||||
<section class="activity-timeline" aria-labelledby="timeline-title">
|
||||
<header><div><span class="overline">Последние 72 часа</span><h2 id="timeline-title">Леска активности</h2></div><p>Все одобренные записи · по времени поступления · шаг 12 часов</p></header>
|
||||
<div class="timeline-chart">
|
||||
{buckets.map((bucket, index) => <div class="timeline-slot"><span aria-hidden="true" class="timeline-line" style={`--height:${bucket.count / max * 100}%`}>{bucket.count > 0 && <i></i>}</span><strong>{bucket.count}</strong><small>{`${(6-index)*12}–${(5-index)*12} ч назад`}</small></div>)}
|
||||
{buckets.map((bucket, index) => <div class="timeline-slot"><span aria-hidden="true" class:list={["timeline-line", `meter-level-${meterLevel(bucket.count)}`]}>{bucket.count > 0 && <i></i>}</span><strong>{bucket.count}</strong><small>{`${(6-index)*12}–${(5-index)*12} ч назад`}</small></div>)}
|
||||
</div>
|
||||
</section>
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
---
|
||||
const path = Astro.url.pathname;
|
||||
const links = [
|
||||
["/admin", "Обзор"],
|
||||
["/admin/moderation", "Уловы"],
|
||||
["/admin/external-sources", "Источники"],
|
||||
["/admin/media", "Медиа"],
|
||||
] as const;
|
||||
---
|
||||
<nav class="admin-nav" aria-label="Разделы админ-панели">
|
||||
{links.map(([href, label]) => <a class={path === href ? "active" : undefined} aria-current={path === href ? "page" : undefined} href={href}>{label}</a>)}
|
||||
</nav>
|
||||
@@ -1,4 +1,4 @@
|
||||
<aside class="alpha-banner" aria-label="Информация об открытой альфе">
|
||||
<div><strong><i></i>Открытая альфа</strong><span>Данные могут быть неполными — источник и качество всегда указаны.</span></div>
|
||||
<nav aria-label="Ссылки открытой альфы"><a href="/status">Статус данных</a><a href="/rules">Как это работает</a><a class="alpha-banner__cta" href="/report">Добавить улов</a></nav>
|
||||
<nav aria-label="Ссылки открытой альфы"><a href="/status">Статус данных</a><a href="/rules">Как это работает</a><a class="alpha-banner__cta" data-action="inverse" href="/report">Добавить улов</a></nav>
|
||||
</aside>
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
---
|
||||
type Item = { label: string; href?: string };
|
||||
const { items } = Astro.props as { items: Item[] };
|
||||
---
|
||||
|
||||
<nav class="atlas-breadcrumbs content-grid" aria-label="Навигационная цепочка">
|
||||
<ol>
|
||||
{items.map((item, index) => <li>
|
||||
<span class="atlas-breadcrumbs__knot" aria-hidden="true"></span>
|
||||
{item.href ? <a href={item.href}>{item.label}</a> : <span aria-current="page">{item.label}</span>}
|
||||
{index < items.length - 1 && <span class="atlas-breadcrumbs__line" aria-hidden="true"></span>}
|
||||
</li>)}
|
||||
</ol>
|
||||
</nav>
|
||||
|
||||
<style>
|
||||
.atlas-breadcrumbs { padding-top: 34px; }
|
||||
ol { display: flex; align-items: center; gap: 0; margin: 0; padding: 0; list-style: none; overflow-x: auto; scrollbar-width: none; }
|
||||
ol::-webkit-scrollbar { display: none; }
|
||||
li { display: flex; align-items: center; flex: 0 0 auto; color: var(--text-muted); font-size: 12px; }
|
||||
a, li > span[aria-current] { padding: 8px 9px; color: inherit; text-decoration: none; white-space: nowrap; }
|
||||
a:hover { color: var(--text-primary); text-decoration: underline; text-underline-offset: 4px; }
|
||||
li > span[aria-current] { color: var(--text-primary); font-weight: 750; }
|
||||
.atlas-breadcrumbs__knot { width: 7px; height: 7px; flex: 0 0 auto; border: 1px solid var(--border-strong); border-radius: 50%; background: var(--paper); }
|
||||
li:last-child .atlas-breadcrumbs__knot { border-color: var(--accent-muted); background: var(--lime); box-shadow: 0 0 0 4px color-mix(in srgb,var(--lime) 18%,transparent); }
|
||||
.atlas-breadcrumbs__line { width: clamp(18px, 3vw, 42px); height: 1px; background: repeating-linear-gradient(90deg,var(--border-strong) 0 5px,transparent 5px 8px); }
|
||||
@media (max-width: 720px) { .atlas-breadcrumbs { width: calc(100% - 28px); padding-top: 22px; } a, li > span[aria-current] { padding-inline: 7px; } }
|
||||
</style>
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
import FishSilhouette from "./FishSilhouette.astro";
|
||||
import WaterbodyMark from "./WaterbodyMark.astro";
|
||||
const { href, label, kind, identity } = Astro.props as { href: string; label: string; kind: "fish" | "water"; identity: string };
|
||||
---
|
||||
<a class="atlas-entity-link" href={href}>
|
||||
<span class="atlas-entity-link__mark" aria-hidden="true">{kind === "fish" ? <FishSilhouette name={identity} size={45} /> : <WaterbodyMark identity={identity} size={48} />}</span>
|
||||
<span>{label}</span><b aria-hidden="true">→</b>
|
||||
</a>
|
||||
<style>
|
||||
.atlas-entity-link { display:grid;grid-template-columns:52px minmax(0,1fr) auto;align-items:center;gap:9px;min-height:54px;padding:5px 2px;border-bottom:1px solid var(--border-soft);color:var(--text-muted);font-size:13px;text-decoration:none }
|
||||
.atlas-entity-link__mark { width:48px;height:38px;display:grid;place-items:center;color:var(--decorative-water) }
|
||||
.atlas-entity-link__mark :global(.fish-silhouette),.atlas-entity-link__mark :global(.waterbody-mark) { position:static;opacity:.68 }
|
||||
.atlas-entity-link__mark :global(.waterbody-mark) { stroke:currentColor }.atlas-entity-link__mark :global(.waterbody-mark text) { fill:currentColor }
|
||||
b { color:var(--accent-muted);font-size:15px;transition:transform var(--motion-fast) var(--ease-out) }.atlas-entity-link:hover { color:var(--text-primary) }.atlas-entity-link:hover b { transform:translateX(2px) }
|
||||
</style>
|
||||
@@ -0,0 +1,25 @@
|
||||
---
|
||||
import FishSilhouette from "./FishSilhouette.astro";
|
||||
import WaterbodyMark from "./WaterbodyMark.astro";
|
||||
|
||||
const { waterbody, fish } = Astro.props as { waterbody: string; fish: string };
|
||||
---
|
||||
|
||||
<span class="atlas-pair-mark" aria-hidden="true">
|
||||
<span class="atlas-pair-mark__water"><WaterbodyMark identity={waterbody} size={104} /></span>
|
||||
<span class="atlas-pair-mark__fish"><FishSilhouette name={fish} size={78} /></span>
|
||||
<i>×</i>
|
||||
</span>
|
||||
|
||||
<style>
|
||||
.atlas-pair-mark { position: relative; display: block; width: 128px; height: 74px; }
|
||||
.atlas-pair-mark__water { position: absolute; inset: 15px auto auto 0; color: var(--lime); }
|
||||
.atlas-pair-mark__water :global(.waterbody-mark) { position: static; opacity: .72; stroke: currentColor; }
|
||||
.atlas-pair-mark__water :global(.waterbody-mark text) { fill: var(--white); opacity: .8; }
|
||||
.atlas-pair-mark__fish { position: absolute; right: -2px; top: -6px; color: var(--white); filter: drop-shadow(0 3px 7px #06191c); }
|
||||
.atlas-pair-mark__fish :global(.fish-silhouette) { position: static; opacity: .92; }
|
||||
i { position: absolute; right: 2px; bottom: 0; width: 18px; height: 18px; display: grid; place-items: center; border-radius: 50%; background: var(--lime); color: var(--deep); font: 800 11px/1 Inter, sans-serif; }
|
||||
@media (max-width: 720px) {
|
||||
.atlas-pair-mark { width: 94px; height: 55px; transform: scale(.73); transform-origin: center; }
|
||||
}
|
||||
</style>
|
||||
@@ -1,17 +1,18 @@
|
||||
---
|
||||
import SourceBadge from "./SourceBadge.astro";
|
||||
import { ago, kg, type Catch } from "../lib/api";
|
||||
import TackleGlyph from "./TackleGlyph.astro";
|
||||
const { catches } = Astro.props as { catches: Catch[] };
|
||||
---
|
||||
<div class="catch-list">
|
||||
{catches.map(item => {
|
||||
const timestamp = item.caught_at ?? item.reported_at;
|
||||
return <article>
|
||||
<div><strong>{item.fish}</strong><span>{item.bait ?? "Приманка не указана"}</span><SourceBadge source={item.source_system} href={item.source_url}/></div>
|
||||
<div><strong>{item.fish}</strong><span class="tackle-label"><TackleGlyph name={item.bait} size={24}/><span>{item.bait ?? "Приманка не указана"}</span></span><SourceBadge source={item.source_system} href={item.source_url}/></div>
|
||||
<div><strong>{kg(item.weight_g)}</strong><span>{item.player_name ?? "Анонимно"}</span><span>{item.caught_at ? "Время улова" : "Получено · время улова неизвестно"}</span><time datetime={timestamp}>{ago(timestamp)} · {new Date(timestamp).toLocaleString("ru-RU", { timeZone: "UTC" })} UTC</time></div>
|
||||
</article>;
|
||||
})}
|
||||
</div>
|
||||
<style>
|
||||
time{margin-top:7px;color:#496357;font-size:10px;font-weight:750}
|
||||
time{margin-top:7px;color:var(--text-secondary);font-size:10px;font-weight:750}
|
||||
</style>
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
---
|
||||
import SourceBadge from "./SourceBadge.astro";
|
||||
import type { MediaAsset } from "../lib/api";
|
||||
const { asset, compact = false, sourceLink = false } = Astro.props as { asset: MediaAsset; compact?: boolean; sourceLink?: boolean };
|
||||
---
|
||||
<figure class:list={["entity-media", { "entity-media--compact": compact }]}>
|
||||
<span class="entity-media__frame"><picture>
|
||||
{(["avif", "webp"] as const).map(format => {
|
||||
const variants = (asset.variants ?? []).filter(item => item.format === format);
|
||||
return variants.length ? <source type={`image/${format}`} srcset={variants.map(item => `${item.url} ${item.width}w`).join(", ")} sizes={compact ? "180px" : "(max-width: 720px) 100vw, 720px"} /> : null;
|
||||
})}
|
||||
<img src={asset.image_url} alt={asset.label ?? "Иллюстрация RF4"} width={asset.width} height={asset.height} loading="lazy" decoding="async" />
|
||||
</picture></span>
|
||||
<figcaption><SourceBadge source={asset.source_system} href={sourceLink ? asset.source_url : undefined} /><span>{asset.label ?? "Справочный материал"}</span></figcaption>
|
||||
</figure>
|
||||
@@ -1,9 +1,16 @@
|
||||
---
|
||||
const { name, size = 92 } = Astro.props as { name: string; size?: number };
|
||||
const variant = [...name].reduce((sum, char) => sum + (char.codePointAt(0) ?? 0), 0) % 3;
|
||||
import { fishVisualFamily, type FishVisualFamily } from "../lib/fish-visuals";
|
||||
const { name, size = 92, family: familyOverride } = Astro.props as { name: string; size?: number; family?: FishVisualFamily };
|
||||
const family = familyOverride ?? fishVisualFamily(name);
|
||||
---
|
||||
<svg class="fish-silhouette" width={size} height={Math.round(size * .48)} viewBox="0 0 120 58" aria-hidden="true">
|
||||
{variant === 0 && <><path d="M22 29C38 7 78 8 101 29 78 50 38 51 22 29Z"/><path d="m25 29-22-18v36Z"/><path d="M55 14 69 2l8 15M57 44l15 11 8-16"/><circle cx="91" cy="25" r="2.4" class="fish-eye"/></>}
|
||||
{variant === 1 && <><path d="M19 30C39 14 80 13 104 29 80 45 39 47 19 30Z"/><path d="M22 30 3 17v28Z"/><path d="m48 17 11-14 13 13M47 44l12 11 13-12"/><path d="M101 29h15"/><circle cx="94" cy="25" r="2.4" class="fish-eye"/></>}
|
||||
{variant === 2 && <><path d="M20 30C34 3 79 4 103 29 79 54 34 55 20 30Z"/><path d="M23 30 2 8v44Z"/><path d="m52 10 8-9 14 10M52 49l9 8 14-10"/><circle cx="92" cy="24" r="2.4" class="fish-eye"/></>}
|
||||
<svg class="fish-silhouette" data-family={family} width={size} height={Math.round(size * .48)} viewBox="0 0 120 58" aria-hidden="true">
|
||||
{family === "pike" && <><path d="M19 30C35 16 70 15 94 24l22 1-7 9-15 1C70 45 35 44 19 30Z"/><path d="M22 30 2 15v30Z"/><path d="m49 18 12-14 15 12M49 42l13 12 15-13"/><circle cx="105" cy="28" r="2.2" class="fish-eye"/></>}
|
||||
{family === "salmonid" && <><path d="M20 30C39 12 79 12 103 28 81 46 40 47 20 30Z"/><path d="M23 30 3 13v34Z"/><path d="m50 16 11-13 15 12M51 43l11 12 14-12"/><path d="m80 17 7-7 5 9"/><circle cx="94" cy="25" r="2.2" class="fish-eye"/></>}
|
||||
{family === "cyprinid" && <><path d="M22 29C38 5 77 5 101 29 77 53 38 53 22 29Z"/><path d="M25 29 3 9v40Z"/><path d="M52 10 68 1l12 13M53 48l16 9 12-14"/><circle cx="91" cy="24" r="2.2" class="fish-eye"/></>}
|
||||
{family === "perch" && <><path d="M20 31C39 13 79 14 103 29 80 47 39 49 20 31Z"/><path d="M23 31 3 16v31Z"/><path d="m42 18 7-15 7 13 8-13 8 14 8-10 5 13M52 45l13 11 15-13"/><circle cx="94" cy="25" r="2.2" class="fish-eye"/></>}
|
||||
{family === "catfish" && <><path d="M17 31C37 17 78 17 104 29 81 45 38 47 17 31Z"/><path d="M21 31 2 18v28Z"/><path d="M47 20 65 7l13 13M48 43l17 11 14-13"/><path class="fish-detail" d="M99 31c9 1 13 5 18 10M99 33c8 4 9 9 12 15"/><circle cx="94" cy="27" r="2.2" class="fish-eye"/></>}
|
||||
{family === "eel" && <><path d="M5 35c20-29 42 14 69-10 17-15 33-7 42 1-20-4-24 17-43 17C43 43 30 22 5 45Z"/><path d="m105 24 12-9-2 13"/><circle cx="106" cy="26" r="1.8" class="fish-eye"/></>}
|
||||
{family === "flatfish" && <><path d="M18 31C37 4 80 6 108 29 80 52 37 54 18 31Z"/><path d="M21 31 3 17v29Z"/><path d="m53 12 13-10 13 11M54 49l13 8 13-10"/><circle cx="91" cy="23" r="2.1" class="fish-eye"/><circle cx="97" cy="26" r="1.7" class="fish-eye"/></>}
|
||||
{family === "marine" && <><path d="M18 30C40 14 78 15 101 28l16-5-6 10 5 8-16-5C76 45 39 45 18 30Z"/><path d="M22 30 2 12v36Z"/><path d="m54 17 13-13 12 13M54 43l13 11 12-12"/><circle cx="94" cy="26" r="2.1" class="fish-eye"/></>}
|
||||
{family === "generic" && <><path d="M21 29C38 9 78 10 103 29 78 48 38 49 21 29Z"/><path d="m24 29-21-17v35Z"/><path d="M54 15 68 3l10 14M55 44l14 11 10-15"/><circle cx="93" cy="25" r="2.2" class="fish-eye"/></>}
|
||||
</svg>
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
---
|
||||
import FishSilhouette from "./FishSilhouette.astro";
|
||||
import WaterbodyMark from "./WaterbodyMark.astro";
|
||||
import AtlasPairMark from "./AtlasPairMark.astro";
|
||||
const { eyebrow, title, description, variant, count, identity = title, secondaryIdentity = "waterbody" } = Astro.props as { eyebrow: string; title: string; description?: string; variant?: "water" | "fish" | "pair"; count?: number; identity?: string; secondaryIdentity?: string };
|
||||
---
|
||||
<section class:list={["catalog-hero content-grid", { "catalog-hero--illustrated": variant }]}>
|
||||
<div class="catalog-hero__copy"><span class="overline">{eyebrow}</span><h1>{title}</h1>{description && <p>{description}</p>}</div>
|
||||
{variant && <div class:list={["catalog-hero__seal", { "catalog-hero__seal--pair": variant === "pair" }]} aria-hidden="true"><span>{variant === "water" ? <WaterbodyMark identity={identity} size={94} /> : variant === "pair" ? <AtlasPairMark waterbody={secondaryIdentity} fish={identity} /> : <FishSilhouette name={identity} size={92} />}</span>{typeof count === "number" && <><strong>{String(count).padStart(2,"0")}</strong><small>{variant === "water" ? "водоёмов" : "видов рыб"}</small></>}</div>}
|
||||
</section>
|
||||
@@ -0,0 +1,27 @@
|
||||
---
|
||||
import { pageHref, pageWindow } from "../lib/pagination";
|
||||
|
||||
interface Props {
|
||||
path: string;
|
||||
params: URLSearchParams;
|
||||
total: number;
|
||||
limit: number;
|
||||
offset: number;
|
||||
anchor?: string;
|
||||
itemLabel?: string;
|
||||
}
|
||||
|
||||
const { path, params, total, limit, offset, anchor = "", itemLabel = "записей" } = Astro.props;
|
||||
const state = pageWindow(total, limit, offset);
|
||||
---
|
||||
{total > limit && <nav class="pagination" aria-label="Пагинация">
|
||||
<span>{state.start}–{state.end} из {total} {itemLabel} · страница {state.page} из {state.pages}</span>
|
||||
<div>
|
||||
{state.previousOffset !== null
|
||||
? <a data-action="secondary" rel="prev" href={pageHref(path, params, state.previousOffset, anchor)}>← Предыдущая</a>
|
||||
: <span class="pagination__disabled" aria-disabled="true">← Предыдущая</span>}
|
||||
{state.nextOffset !== null
|
||||
? <a data-action="secondary" rel="next" href={pageHref(path, params, state.nextOffset, anchor)}>Следующая →</a>
|
||||
: <span class="pagination__disabled" aria-disabled="true">Следующая →</span>}
|
||||
</div>
|
||||
</nav>}
|
||||
@@ -0,0 +1,18 @@
|
||||
---
|
||||
const { eyebrow, title, description, count, id, editorial = false } = Astro.props as {
|
||||
eyebrow?: string;
|
||||
title: string;
|
||||
description?: string;
|
||||
count?: string;
|
||||
id?: string;
|
||||
editorial?: boolean;
|
||||
};
|
||||
---
|
||||
<div class:list={["section-heading", { "section-heading--editorial": editorial }]}>
|
||||
<div>
|
||||
{eyebrow && <span class="overline">{eyebrow}</span>}
|
||||
<h2 id={id}>{title}</h2>
|
||||
{description && <p>{description}</p>}
|
||||
</div>
|
||||
{count && <span class="result-count">{count}</span>}
|
||||
</div>
|
||||
@@ -1,5 +1,6 @@
|
||||
---
|
||||
import SourceBadge from "./SourceBadge.astro";
|
||||
import SectionHeading from "./SectionHeading.astro";
|
||||
import { ago, kg, type PublicObservation } from "../lib/api";
|
||||
const { signals } = Astro.props as { signals: PublicObservation[] };
|
||||
const groups = [...signals.reduce((map, signal) => {
|
||||
@@ -17,7 +18,7 @@ const groups = [...signals.reduce((map, signal) => {
|
||||
}, new Map<string, PublicObservation & { observations: PublicObservation[] }>()).values()];
|
||||
---
|
||||
<section class="signal-section content-grid" aria-labelledby="signals-title">
|
||||
<div class="signal-heading"><div><span class="overline">Сырые данные источников</span><h2 id="signals-title">Полевые сигналы</h2><p>Показываем сразу, даже если часть полей отсутствует. Эти карточки не участвуют в расчёте активности до полного подтверждения.</p></div><span class="signal-count">{signals.length} сигналов · {groups.length} сюжетов</span></div>
|
||||
<SectionHeading editorial eyebrow="Сырые данные источников" title="Полевые сигналы" id="signals-title" description="Показываем сразу, даже если часть полей отсутствует. Эти карточки не участвуют в расчёте активности до полного подтверждения." count={`${signals.length} сигналов · ${groups.length} сюжетов`} />
|
||||
<div class="signal-grid">{groups.map(signal => {
|
||||
const provenance = [...new Map(signal.observations.map(item => [`${item.source_system}|${item.source_url}`, item])).values()];
|
||||
return <article class="signal-card" data-quality={signal.quality}>
|
||||
@@ -30,6 +31,6 @@ const groups = [...signals.reduce((map, signal) => {
|
||||
})}</div>
|
||||
</section>
|
||||
<style>
|
||||
.repeat-note{position:relative;z-index:1;margin:-8px 0 10px!important;padding:6px 9px;border-left:2px solid #6b8d74;color:#496357!important;font-size:10px!important}
|
||||
.repeat-note{position:relative;z-index:1;margin:-8px 0 10px!important;padding:6px 9px;border-left:2px solid var(--border-strong);color:var(--text-secondary)!important;font-size:10px!important}
|
||||
.signal-card__top .source-strip{margin:0}
|
||||
</style>
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
---
|
||||
const { title, description, actionHref, actionLabel, tone = "empty", compact = false, contained = true } = Astro.props as { title: string; description?: string; actionHref?: string; actionLabel?: string; tone?: "empty" | "error" | "unavailable"; compact?: boolean; contained?: boolean };
|
||||
---
|
||||
<div class:list={["state", { "content-grid": contained, "compact-state": compact, "error-state": tone === "error", "unavailable-state": tone === "unavailable" }]} role={tone === "empty" ? "status" : "alert"}>
|
||||
<h2>{title}</h2>
|
||||
{description && <p>{description}</p>}
|
||||
{actionHref && actionLabel && <a data-action="secondary" href={actionHref}>{actionLabel}</a>}
|
||||
</div>
|
||||
@@ -0,0 +1,15 @@
|
||||
---
|
||||
import { tackleVisualKind, tackleVisualTone, type TackleVisualKind } from "../lib/tackle-visuals";
|
||||
const { name, size = 28, kind: kindOverride } = Astro.props as { name?: string | null; size?: number; kind?: TackleVisualKind };
|
||||
const kind = kindOverride ?? tackleVisualKind(name);
|
||||
const tone = tackleVisualTone(name);
|
||||
---
|
||||
<svg class="tackle-glyph" data-kind={kind} data-tone={tone} width={size} height={size} viewBox="0 0 32 32" fill="none" aria-hidden="true">
|
||||
{kind === "spinner" && <><path d="M17 4v7"/><path class="tackle-glyph__fill" d="M17 10c7 1 8 7 2 11-5 3-9-2-7-6 1-3 3-4 5-5Z"/><path d="M18 21v4c0 4-5 4-5 0"/></>}
|
||||
{kind === "wobbler" && <><path class="tackle-glyph__fill" d="M5 15c5-6 14-7 21-2-4 8-13 9-21 2Z"/><circle cx="22" cy="13" r="1"/><path d="m8 18-2 5m11-3-1 5m-1 0c0 3-4 3-4 0m10-2c0 3-4 3-4 0"/></>}
|
||||
{kind === "soft" && <><path class="tackle-glyph__fill" d="M4 17c6-7 14-7 20-2l5-5-1 10-5-3c-7 5-14 5-19 0Z"/><circle cx="8" cy="15" r="1"/></>}
|
||||
{kind === "boilie" && <><circle class="tackle-glyph__fill" cx="11" cy="16" r="6"/><circle class="tackle-glyph__fill" cx="22" cy="16" r="6"/><path d="M5 8c8 3 14 3 22 0"/></>}
|
||||
{kind === "worm" && <><path class="tackle-glyph__stroke" d="M5 22c0-11 8-14 12-7s10 3 10-5"/><path d="m23 7 4 3-4 2"/></>}
|
||||
{kind === "rig" && <><path d="M16 3v17c0 7-10 8-11 1-1-4 3-6 6-4"/><path d="m8 17 3 0-1 3"/><circle class="tackle-glyph__fill" cx="16" cy="7" r="3"/></>}
|
||||
{kind === "unknown" && <><path class="tackle-glyph__fill" d="M16 4c5 5 8 10 7 16-1 5-5 8-9 6-5-2-6-7-3-11 2-3 4-6 5-11Z"/><path d="M15 8c4 4 5 8 3 12"/></>}
|
||||
</svg>
|
||||
@@ -0,0 +1,29 @@
|
||||
---
|
||||
import { waterbodyVisual } from "../lib/waterbody-visuals";
|
||||
const { identity, size = 112 } = Astro.props as { identity: string; size?: number };
|
||||
const visual = waterbodyVisual(identity);
|
||||
const shores = [
|
||||
"M8 57C22 42 34 48 47 34S75 20 91 34s16 7 25-3",
|
||||
"M7 48c17-20 34 5 50-10s28-15 38 1 13 14 22 9",
|
||||
"M5 54c14-4 20-24 39-18s27-9 40-14 20 11 33 10",
|
||||
"M7 42c14 8 25-17 42-8s23 16 35 2 20-13 34-6",
|
||||
"M6 51c16-24 31 7 47-8s32-17 41 3 14 11 24 4",
|
||||
"M8 38c19-8 26 19 46 8s30-25 42-8 14 11 22 7",
|
||||
"M5 56c11-19 25-20 39-5s25-1 37-16 25-10 37 2",
|
||||
"M6 47c18 15 31-18 50-5s27 9 37-5 17-15 27-5",
|
||||
];
|
||||
---
|
||||
<svg class="waterbody-mark" width={size} height={Math.round(size * .62)} viewBox="0 0 124 72" aria-hidden="true">
|
||||
<path class="waterbody-mark__shore" d={shores[visual.shore]} />
|
||||
{visual.waves >= 1 && <path d="M12 59c18-7 31 6 49-2s31-6 51 0" />}
|
||||
{visual.waves >= 2 && <path d="M20 65c14-5 27 4 42-2s27-4 43 0" />}
|
||||
{visual.waves >= 3 && <path d="M31 70c11-3 21 2 32-1s20-2 30 0" />}
|
||||
<circle cx={visual.markerX} cy={visual.markerY} r="3" />
|
||||
<text x="111" y="15" text-anchor="end">{visual.code}</text>
|
||||
</svg>
|
||||
|
||||
<style>
|
||||
text { fill: currentColor; stroke: none; font: 700 8px Inter, sans-serif; letter-spacing: .08em; }
|
||||
:global(.catalog-hero__seal) .waterbody-mark { position: static; right: auto; bottom: auto; opacity: 1; color: var(--lime); stroke: currentColor; }
|
||||
:global(.catalog-hero__seal) text { fill: var(--white); }
|
||||
</style>
|
||||
@@ -7,10 +7,15 @@ import "../styles/data-legend.css";
|
||||
import "../styles/coordinate-radar.css";
|
||||
import "../styles/activity-timeline.css";
|
||||
import "../styles/fish-silhouette.css";
|
||||
import "../styles/tackle-glyph.css";
|
||||
import "../styles/empty-states.css";
|
||||
import "../styles/alpha-banner.css";
|
||||
import "../styles/signal-pagination.css";
|
||||
import "../styles/pagination.css";
|
||||
import "../styles/dashboard-polish.css";
|
||||
import "../styles/loading-states.css";
|
||||
import "../styles/theme.css";
|
||||
import "../styles/media-catalog.css";
|
||||
import FishingIcon from "../components/FishingIcon.astro";
|
||||
import AlphaBanner from "../components/AlphaBanner.astro";
|
||||
const {
|
||||
@@ -22,10 +27,17 @@ const {
|
||||
errorPage = false,
|
||||
} = Astro.props;
|
||||
const path = Astro.url.pathname;
|
||||
const storedTheme = Astro.cookies.get("rf4-theme")?.value;
|
||||
const theme = storedTheme === "light" || storedTheme === "dark" ? storedTheme : "system";
|
||||
const siteUrl = import.meta.env.PUBLIC_SITE_URL || "https://rf4spotter.ru";
|
||||
const canonical = new URL(path, siteUrl).toString();
|
||||
const socialImage = new URL(image, siteUrl).toString();
|
||||
const preventIndexing = noindex || path.startsWith("/admin/");
|
||||
const isAdminPath = path === "/admin" || path.startsWith("/admin/");
|
||||
const preventIndexing = noindex || isAdminPath;
|
||||
if (isAdminPath) {
|
||||
Astro.response.headers.set("Cache-Control", "private, no-store");
|
||||
Astro.response.headers.set("X-Robots-Tag", "noindex, nofollow");
|
||||
}
|
||||
const websiteJsonLd = { "@type": "WebSite", name: "RF4 Spotter", url: siteUrl, inLanguage: "ru" };
|
||||
// A08: Skip structuredData on error pages (explicit errorPage prop)
|
||||
// Don't infer error from noindex alone — main page can have noindex on 422
|
||||
@@ -34,9 +46,46 @@ const jsonLd = JSON.stringify({
|
||||
"@context": "https://schema.org",
|
||||
"@graph": jsonLdGraph,
|
||||
}).replaceAll("<", "\\u003c");
|
||||
const configuredFilesDomain = process.env.FILES_DOMAIN || "files.rf4spotter.ru";
|
||||
const filesDomain = /^(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?)(?:\.[a-z0-9](?:[a-z0-9-]*[a-z0-9])?)+$/i.test(configuredFilesDomain)
|
||||
? configuredFilesDomain
|
||||
: "files.rf4spotter.ru";
|
||||
const isLoopback = (hostname: string) => hostname === "localhost" || hostname === "127.0.0.1" || hostname === "[::1]";
|
||||
const safeOrigin = (value: string | undefined, fallback: string) => {
|
||||
try {
|
||||
const url = new URL(value || fallback);
|
||||
return url.protocol === "https:" || (url.protocol === "http:" && isLoopback(url.hostname)) ? url.origin : fallback;
|
||||
} catch {
|
||||
return fallback;
|
||||
}
|
||||
};
|
||||
const apiOrigin = safeOrigin(import.meta.env.PUBLIC_API_URL, siteUrl);
|
||||
const filesFallback = `https://${filesDomain}`;
|
||||
const filesOrigin = (() => {
|
||||
try {
|
||||
const url = new URL(process.env.FILES_ORIGIN || filesFallback);
|
||||
if ((url.protocol === "https:" && url.hostname === filesDomain) || (url.protocol === "http:" && isLoopback(url.hostname))) {
|
||||
return url.origin;
|
||||
}
|
||||
} catch {
|
||||
// Invalid deployment input falls back to the validated production hostname.
|
||||
}
|
||||
return filesFallback;
|
||||
})();
|
||||
const localOrigins = [apiOrigin, filesOrigin].filter((origin) => origin.startsWith("http://"));
|
||||
const cspNonce = crypto.randomUUID().replaceAll("-", "");
|
||||
Astro.response.headers.set("Content-Security-Policy", [
|
||||
"default-src 'self'", "base-uri 'self'", "object-src 'none'", "frame-ancestors 'none'",
|
||||
"form-action 'self'", `connect-src 'self'${apiOrigin === siteUrl ? "" : ` ${apiOrigin}`}`,
|
||||
`img-src 'self' data: ${filesOrigin}`,
|
||||
"font-src 'self'", "media-src 'self'", "manifest-src 'self'",
|
||||
`script-src 'self' 'nonce-${cspNonce}'`, "script-src-attr 'none'",
|
||||
"style-src 'self'", "style-src-attr 'none'",
|
||||
...(localOrigins.length ? [] : ["upgrade-insecure-requests"]),
|
||||
].join("; "));
|
||||
---
|
||||
<!doctype html>
|
||||
<html lang="ru">
|
||||
<html lang="ru" data-theme={theme === "system" ? undefined : theme}>
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width" />
|
||||
@@ -47,7 +96,8 @@ const jsonLd = JSON.stringify({
|
||||
<link rel="icon" href="/favicon-32.png" sizes="32x32" type="image/png" />
|
||||
<link rel="apple-touch-icon" href="/apple-touch-icon.png" />
|
||||
<link rel="manifest" href="/site.webmanifest" />
|
||||
<meta name="theme-color" content="#082226" />
|
||||
<meta name="theme-color" content="#f2f5ee" media={theme === "system" ? "(prefers-color-scheme: light)" : theme === "light" ? "all" : "not all"} data-theme-color="light" />
|
||||
<meta name="theme-color" content="#071719" media={theme === "system" ? "(prefers-color-scheme: dark)" : theme === "dark" ? "all" : "not all"} data-theme-color="dark" />
|
||||
<meta property="og:locale" content="ru_RU" />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:site_name" content="RF4 Spotter" />
|
||||
@@ -64,18 +114,50 @@ const jsonLd = JSON.stringify({
|
||||
<meta name="twitter:description" content={description} />
|
||||
<meta name="twitter:image" content={socialImage} />
|
||||
<meta name="twitter:image:alt" content="Лаймовый поплавок на тёмном озере с координатной сеткой" />
|
||||
<script type="application/ld+json" set:html={jsonLd} is:inline />
|
||||
<script type="application/ld+json" nonce={cspNonce} set:html={jsonLd} is:inline />
|
||||
<title>{title}</title>
|
||||
</head>
|
||||
<body>
|
||||
<a class="skip-link" href="#main-content">Перейти к содержимому</a>
|
||||
<header class="topbar">
|
||||
<a href="/" class="brand" aria-label="Ни хвоста, ни чешуи"><span class="brand-mark" aria-hidden="true"><FishingIcon name="hook" size={24}/></span><span class="brand-name"><span>Ни хвоста,</span><strong>ни чешуи</strong></span></a>
|
||||
<nav aria-label="Разделы сайта"><a class:list={{active:path === "/"}} aria-current={path === "/" ? "page" : undefined} href="/"><FishingIcon name="float"/> <span>Сейчас клюёт</span></a><a class:list={{active:path.startsWith("/waterbodies") || path.startsWith("/fish")}} aria-current={path.startsWith("/waterbodies") || path.startsWith("/fish") ? "page" : undefined} href="/waterbodies"><FishingIcon name="ripple"/> <span>Каталог</span></a><a class:list={{active:path.startsWith("/records")}} aria-current={path.startsWith("/records") ? "page" : undefined} href="/records"><FishingIcon name="trophy"/> <span>Рекорды</span></a><a class:list={{active:path.startsWith("/report")}} aria-current={path.startsWith("/report") ? "page" : undefined} href="/report"><FishingIcon name="plus"/> <span>Добавить улов</span></a></nav>
|
||||
<p class="live-badge"><span></span> Свежие данные и честная оценка</p>
|
||||
<a href="/" class="brand"><span class="brand-mark" aria-hidden="true"><FishingIcon name="hook" size={24}/></span><span class="brand-name"><strong>RF4 Spotter</strong><span>Ни хвоста, ни чешуи</span></span></a>
|
||||
<nav aria-label="Разделы сайта"><a class:list={{active:path === "/"}} aria-current={path === "/" ? "page" : undefined} href="/"><FishingIcon name="float"/> <span>Сейчас клюёт</span></a><a class:list={{active:path.startsWith("/waterbodies") || path.startsWith("/fish")}} aria-current={path.startsWith("/waterbodies") || path.startsWith("/fish") ? "page" : undefined} href="/waterbodies"><FishingIcon name="ripple"/> <span>Каталог</span></a><a class:list={{active:path.startsWith("/media") || path.startsWith("/admin/media")}} aria-current={path.startsWith("/media") || path.startsWith("/admin/media") ? "page" : undefined} href="/media"><FishingIcon name="lure"/> <span>Медиатека</span></a><a class:list={{active:path.startsWith("/records")}} aria-current={path.startsWith("/records") ? "page" : undefined} href="/records"><FishingIcon name="trophy"/> <span>Рекорды</span></a><a class:list={{active:path.startsWith("/report")}} aria-current={path.startsWith("/report") ? "page" : undefined} href="/report"><FishingIcon name="plus"/> <span>Добавить улов</span></a></nav>
|
||||
<div class="header-tools">
|
||||
<p class="live-badge"><span></span> Свежие данные и честная оценка</p>
|
||||
<div class="theme-switcher" role="group" aria-label="Цветовая тема">
|
||||
<button type="button" data-theme-option="system" aria-pressed={theme === "system"} title="Использовать системную тему"><span aria-hidden="true">◐</span><b>Системная</b></button>
|
||||
<button type="button" data-theme-option="light" aria-pressed={theme === "light"} title="Включить светлую тему"><span aria-hidden="true">☀</span><b>Светлая</b></button>
|
||||
<button type="button" data-theme-option="dark" aria-pressed={theme === "dark"} title="Включить тёмную тему"><span aria-hidden="true">●</span><b>Тёмная</b></button>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
<AlphaBanner />
|
||||
<main id="main-content" tabindex="-1"><slot /></main>
|
||||
<footer><a href="/" class="brand"><span class="brand-mark"><FishingIcon name="hook" size={24}/></span><span class="brand-name"><span>Ни хвоста,</span><strong>ни чешуи</strong></span></a><p>Неофициальный проект для игроков Russian Fishing 4. <a href="/status">Статус</a> · <a href="/rules">Правила</a> · <a href="/privacy">Конфиденциальность</a></p><span>НХНЧ · 2026</span></footer>
|
||||
<footer><a href="/" class="brand"><span class="brand-mark" aria-hidden="true"><FishingIcon name="hook" size={24}/></span><span class="brand-name"><strong>RF4 Spotter</strong><span>Ни хвоста, ни чешуи</span></span></a><p>Неофициальный проект для игроков Russian Fishing 4. <a href="/status">Статус</a> · <a href="/rules">Правила</a> · <a href="/privacy">Конфиденциальность</a></p><span>RF4S · 2026</span></footer>
|
||||
</body>
|
||||
</html>
|
||||
<script>
|
||||
const root = document.documentElement;
|
||||
const themeButtons = document.querySelectorAll<HTMLButtonElement>("[data-theme-option]");
|
||||
const themeColors = document.querySelectorAll<HTMLMetaElement>("[data-theme-color]");
|
||||
const allowedThemes = new Set(["system", "light", "dark"]);
|
||||
|
||||
const applyTheme = (value: string) => {
|
||||
const theme = allowedThemes.has(value) ? value : "system";
|
||||
if (theme === "system") delete root.dataset.theme;
|
||||
else root.dataset.theme = theme;
|
||||
themeButtons.forEach((button) => {
|
||||
button.setAttribute("aria-pressed", String(button.dataset.themeOption === theme));
|
||||
});
|
||||
themeColors.forEach((meta) => {
|
||||
const color = meta.dataset.themeColor;
|
||||
meta.media = theme === "system" ? `(prefers-color-scheme: ${color})` : color === theme ? "all" : "not all";
|
||||
});
|
||||
const secure = location.protocol === "https:" ? "; Secure" : "";
|
||||
document.cookie = `rf4-theme=${theme}; Max-Age=31536000; Path=/; SameSite=Lax${secure}`;
|
||||
};
|
||||
|
||||
themeButtons.forEach((button) => {
|
||||
button.addEventListener("click", () => applyTheme(button.dataset.themeOption ?? "system"));
|
||||
});
|
||||
</script>
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
export function adminErrorMessage(status: number, fallback: string): string {
|
||||
if (status === 401) return "Неверный или истёкший административный токен.";
|
||||
if (status === 409) return "Операция конфликтует с изменением в другой вкладке.";
|
||||
if (status === 429) return "Слишком много попыток. Повторите позже.";
|
||||
if (status >= 500) return "Сервис временно недоступен. Проверьте состояние и повторите позже.";
|
||||
return fallback;
|
||||
}
|
||||
|
||||
export function adminEndsSession(status: number): boolean {
|
||||
return status === 401 || status === 429;
|
||||
}
|
||||
@@ -4,6 +4,7 @@ export type Activity = {
|
||||
unique_players: number; average_weight_g: number; max_weight_g: number;
|
||||
last_confirmed_at: string; activity_score: number; confidence_score: number;
|
||||
explanation: string; sources: string[];
|
||||
coordinate_precision: "exact" | "approximate" | "area" | "missing"; coordinate_sources: string[];
|
||||
};
|
||||
|
||||
export type PaginatedActivity = {
|
||||
@@ -13,9 +14,11 @@ export type PaginatedActivity = {
|
||||
offset: number;
|
||||
};
|
||||
|
||||
export type Spot = { id: string; waterbody_slug: string; waterbody: string; x: number; y: number; description: string | null; catches_24h: number; catches_3d: number; catches_7d: number; top_baits: string[] };
|
||||
export type Spot = { id: string; waterbody_slug: string; waterbody: string; x: number; y: number; description: string | null; catches_24h: number; catches_3d: number; catches_7d: number; top_baits: string[]; coordinate_precision: "exact" | "approximate" | "area" | "missing"; coordinate_sources: string[] };
|
||||
export type Catch = { id: string; fish: string; weight_g: number; bait: string | null; player_name: string | null; caught_at: string | null; reported_at: string; retrieve_method: string | null; retrieve_speed: number | null; source_system: string; source_url: string | null };
|
||||
export type DictionaryItem = { id: string; slug: string; name_ru: string };
|
||||
export type DictionaryItem = { id: string; slug: string; name_ru: string; unlock_level?: number | null; fish_species_count?: number | null; source_system?: string | null; source_external_id?: string | null; source_url?: string | null; description?: string | null; source_aliases?: string[] | null; source_fish_species?: string[] | null; source_image_urls?: string[] | null; source_point_urls?: string[] | null; source_checked_at?: string | null };
|
||||
export type TackleItem = { id: string; name: string; category: string; subcategory: string | null; brand: string | null; family: string | null; unlock_level: number | null; source_system: string | null; source_external_id: string | null; source_url: string | null; source_checked_at: string | null; missing_fields: string[] };
|
||||
export type PaginatedTackleItems = { items: TackleItem[]; total: number; limit: number; offset: number };
|
||||
export type OfficialRecord = { id: string; fish: string; weight_g: number; waterbody: string; bait: string | null; player_name: string | null; record_date: string | null; category: string | null; region: string | null; source_url: string | null; source_system: string };
|
||||
export type PaginatedOfficialRecord = {
|
||||
items: OfficialRecord[];
|
||||
@@ -26,6 +29,8 @@ export type PaginatedOfficialRecord = {
|
||||
export type PublicObservation = { id: string; source_system: string; source_name: string; source_url: string; fish_name: string; waterbody_name: string; x: number | null; y: number | null; weight_g: number | null; last_seen_at: string; missing_fields: string[]; quality: "incomplete" | "unverified" };
|
||||
export type ImportRun = { id: string; started_at: string; finished_at: string | null; status: string; source_url: string; rows_seen: number; rows_created: number; rows_updated: number; error_summary: string | null };
|
||||
export type SourceStatus = { source_system: string; name: string; status: "healthy" | "stale" | "temporarily_limited" | "source_changed" | "waiting" | "disabled"; last_started_at: string | null; last_success_at: string | null; observations: number };
|
||||
export type MediaVariant = { role: "card" | "detail"; format: "webp" | "avif"; width: number; height: number; url: string };
|
||||
export type MediaAsset = { id: string; entity_type: "fish" | "waterbody" | "tackle" | "reference"; entity_key: string; label: string | null; width: number; height: number; content_type: string; image_url: string; source_system: string; source_url: string; variants?: MediaVariant[] };
|
||||
|
||||
export const spotPath = (item: Pick<Activity, "waterbody_slug" | "x" | "y">) => `/spots/${item.waterbody_slug}-${item.x}x${item.y}`;
|
||||
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
export type FishVisualFamily = "pike" | "salmonid" | "cyprinid" | "perch" | "catfish" | "eel" | "flatfish" | "marine" | "generic";
|
||||
|
||||
const FAMILY_TERMS: Array<[FishVisualFamily, RegExp]> = [
|
||||
["pike", /щук|судак|pike|pickerel|zander|hecht/i],
|
||||
["salmonid", /форел|лосос|сёмг|голец|таймен|хариус|trout|salmon|char|grayling|forelle|lachs|saibling/i],
|
||||
["cyprinid", /карп|карась|лещ|плотв|линь|язь|жерех|голав|усач|амур|толстолоб|carp|bream|roach|tench|barbel|karpfen|brasse/i],
|
||||
["perch", /окун|ёрш|perch|ruff|barsch/i],
|
||||
["catfish", /сом|catfish|wels/i],
|
||||
["eel", /угор|миног|eel|lamprey|aal/i],
|
||||
["flatfish", /камбал|палтус|скат|flounder|halibut|ray|rochen/i],
|
||||
["marine", /сельд|треск|скумбр|тунец|акул|сардин|herring|cod|mackerel|tuna|shark|hering|kabeljau/i],
|
||||
];
|
||||
|
||||
export function fishVisualFamily(name: string): FishVisualFamily {
|
||||
return FAMILY_TERMS.find(([, pattern]) => pattern.test(name))?.[0] ?? "generic";
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
import type { MediaAsset } from "./api";
|
||||
|
||||
const tokens = (value: string) => value
|
||||
.toLocaleLowerCase("ru")
|
||||
.replaceAll("ё", "е")
|
||||
.replace(/^(оз\.|р\.)\s*/, "")
|
||||
.replace(/[^a-zа-я0-9]+/giu, " ")
|
||||
.trim()
|
||||
.split(/\s+/)
|
||||
.filter(Boolean);
|
||||
|
||||
export const findMediaByLabel = (assets: MediaAsset[], label: string): MediaAsset | undefined => {
|
||||
const wanted = tokens(label);
|
||||
const exact = assets.filter((asset) => tokens(asset.label ?? "").join(" ") === wanted.join(" "));
|
||||
if (exact.length === 1) return exact[0];
|
||||
|
||||
const wantedSet = new Set(wanted);
|
||||
const candidates = assets.filter((asset) => {
|
||||
const available = new Set(tokens(asset.label ?? ""));
|
||||
return wanted.every((token) => available.has(token)) || [...available].every((token) => wantedSet.has(token));
|
||||
});
|
||||
return candidates.length === 1 ? candidates[0] : undefined;
|
||||
};
|
||||
@@ -0,0 +1,31 @@
|
||||
export type PageWindow = {
|
||||
start: number;
|
||||
end: number;
|
||||
page: number;
|
||||
pages: number;
|
||||
previousOffset: number | null;
|
||||
nextOffset: number | null;
|
||||
};
|
||||
|
||||
export const pageWindow = (total: number, limit: number, offset: number): PageWindow => {
|
||||
const safeTotal = Math.max(0, Math.trunc(total));
|
||||
const safeLimit = Math.max(1, Math.trunc(limit));
|
||||
const safeOffset = Math.max(0, Math.trunc(offset));
|
||||
const end = Math.min(safeOffset + safeLimit, safeTotal);
|
||||
return {
|
||||
start: safeTotal > 0 && safeOffset < safeTotal ? safeOffset + 1 : 0,
|
||||
end,
|
||||
page: Math.floor(safeOffset / safeLimit) + 1,
|
||||
pages: Math.max(1, Math.ceil(safeTotal / safeLimit)),
|
||||
previousOffset: safeOffset > 0 ? Math.max(0, safeOffset - safeLimit) : null,
|
||||
nextOffset: end < safeTotal ? safeOffset + safeLimit : null,
|
||||
};
|
||||
};
|
||||
|
||||
export const pageHref = (path: string, search: URLSearchParams, offset: number, anchor = "") => {
|
||||
const params = new URLSearchParams(search);
|
||||
if (offset > 0) params.set("offset", String(offset));
|
||||
else params.delete("offset");
|
||||
const query = params.toString();
|
||||
return `${path}${query ? `?${query}` : ""}${anchor}`;
|
||||
};
|
||||
@@ -0,0 +1,20 @@
|
||||
export type TackleVisualKind = "spinner" | "wobbler" | "soft" | "boilie" | "worm" | "rig" | "unknown";
|
||||
|
||||
const KIND_TERMS: Array<[TackleVisualKind, RegExp]> = [
|
||||
["spinner", /spinner|spiker|spoon|блесн|вращал|колеб|вертуш|spinner|blinker/i],
|
||||
["wobbler", /wobbler|воблер|minnow|crank|popper|jerk|walker|plug/i],
|
||||
["soft", /shad|twister|твистер|силикон|soft|vibro|виброхвост/i],
|
||||
["boilie", /бойл|boilie|pellet|пеллет|pop[- ]?up/i],
|
||||
["worm", /черв|worm|мотыл|опарыш|личин|maggot|bloodworm/i],
|
||||
["rig", /оснаст|монтаж|rig|hook|крюч|leader|повод/i],
|
||||
];
|
||||
|
||||
export function tackleVisualKind(name?: string | null): TackleVisualKind {
|
||||
if (!name?.trim()) return "unknown";
|
||||
return KIND_TERMS.find(([, pattern]) => pattern.test(name))?.[0] ?? "unknown";
|
||||
}
|
||||
|
||||
export function tackleVisualTone(name?: string | null): number {
|
||||
return [...(name?.trim().toLocaleLowerCase("ru") || "unknown")]
|
||||
.reduce((sum, char) => (sum + (char.codePointAt(0) ?? 0)) % 5, 0);
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
export type WaterbodyVisual = {
|
||||
shore: number;
|
||||
waves: number;
|
||||
markerX: number;
|
||||
markerY: number;
|
||||
code: string;
|
||||
};
|
||||
|
||||
export function stableVisualHash(identity: string): number {
|
||||
return [...identity.trim().toLocaleLowerCase("ru")]
|
||||
.reduce((hash, char) => Math.imul(hash ^ (char.codePointAt(0) ?? 0), 16777619) >>> 0, 2166136261);
|
||||
}
|
||||
|
||||
export function waterbodyVisual(identity: string): WaterbodyVisual {
|
||||
const hash = stableVisualHash(identity || "waterbody");
|
||||
return {
|
||||
shore: hash % 8,
|
||||
waves: 1 + ((hash >>> 4) % 3),
|
||||
markerX: 32 + ((hash >>> 8) % 59),
|
||||
markerY: 18 + ((hash >>> 15) % 23),
|
||||
code: hash.toString(36).toUpperCase().padStart(2, "0").slice(-2),
|
||||
};
|
||||
}
|
||||
@@ -1,39 +1,65 @@
|
||||
---
|
||||
import Layout from "../../layouts/Layout.astro";
|
||||
import AdminNav from "../../components/AdminNav.astro";
|
||||
const apiUrl = import.meta.env.PUBLIC_API_URL || "http://localhost:8000";
|
||||
---
|
||||
<Layout title="Внешние источники — RF4 Spotter">
|
||||
<section class="form-hero"><div><span class="eyebrow"><b>ADMIN</b> Staging</span><h1>Внешние<br/><em>наблюдения</em></h1></div><p>Сопоставьте названия с каноническим каталогом. Опубликовать можно только запись с координатами и весом.</p></section>
|
||||
<section class="moderation-app" data-api-url={apiUrl}>
|
||||
<form class="admin-login" autocomplete="off"><label>Административный токен<input name="token" type="password" required autocomplete="off" /></label><button type="submit">Открыть очередь</button></form>
|
||||
<AdminNav />
|
||||
<form class="admin-login" autocomplete="off"><label>Административный токен<input name="token" type="password" required autocomplete="off" /></label><button data-action="primary" type="submit">Открыть очередь</button></form>
|
||||
<p class="privacy">Токен остаётся только в памяти страницы. Исходная ссылка и происхождение сохраняются при публикации.</p>
|
||||
<div class="notice error" data-admin-error role="alert" hidden></div><div class="moderation-list" data-external-list aria-live="polite"></div>
|
||||
<div class="admin-session-bar" hidden><span>Административная сессия активна</span><button data-action="secondary" type="button" data-admin-logout>Выйти</button></div>
|
||||
<form class="admin-queue-filters" hidden>
|
||||
<label>Поиск<input name="q" type="search" maxlength="100" placeholder="Рыба или водоём" /></label>
|
||||
<label>Источник<select name="source"><option value="">Все источники</option></select></label>
|
||||
<label>Полнота<select name="completeness"><option value="all">Все записи</option><option value="complete">Только полные</option><option value="incomplete">Только неполные</option></select></label>
|
||||
<label>Порядок<select name="order"><option value="risk">Сначала требуют внимания</option><option value="newest">Сначала новые</option><option value="oldest">Сначала старые</option></select></label>
|
||||
<button data-action="primary" type="submit">Применить</button>
|
||||
</form>
|
||||
<div class="notice error" data-admin-error role="alert" hidden></div><div class="notice success" data-admin-status role="status" hidden></div><div class="moderation-list" data-external-list aria-live="polite"></div>
|
||||
<nav data-pages aria-label="Страницы очереди" hidden>
|
||||
<button type="button" data-previous>Предыдущая</button>
|
||||
<button data-action="secondary" type="button" data-previous>Предыдущая</button>
|
||||
<span data-page-number aria-live="polite"></span>
|
||||
<button type="button" data-next>Следующая</button>
|
||||
<button data-action="secondary" type="button" data-next>Следующая</button><button data-action="secondary" type="button" data-refresh>Обновить</button>
|
||||
</nav>
|
||||
<p class="admin-shortcuts"><kbd>S</kbd> подсказать · <kbd>M</kbd> сопоставить · <kbd>P</kbd> опубликовать карточку с фокусом</p>
|
||||
</section>
|
||||
<script>
|
||||
import { adminEndsSession, adminErrorMessage } from "../../lib/admin-errors";
|
||||
const root = document.querySelector<HTMLElement>(".moderation-app");
|
||||
const login = document.querySelector<HTMLFormElement>(".admin-login");
|
||||
const list = document.querySelector<HTMLElement>("[data-external-list]");
|
||||
const error = document.querySelector<HTMLElement>("[data-admin-error]");
|
||||
const status = document.querySelector<HTMLElement>("[data-admin-status]");
|
||||
const pages = document.querySelector<HTMLElement>("[data-pages]");
|
||||
const previous = document.querySelector<HTMLButtonElement>("[data-previous]");
|
||||
const next = document.querySelector<HTMLButtonElement>("[data-next]");
|
||||
const refresh = document.querySelector<HTMLButtonElement>("[data-refresh]");
|
||||
const pageNumber = document.querySelector<HTMLElement>("[data-page-number]");
|
||||
const sessionBar = document.querySelector<HTMLElement>(".admin-session-bar");
|
||||
const logout = document.querySelector<HTMLButtonElement>("[data-admin-logout]");
|
||||
const filters = document.querySelector<HTMLFormElement>(".admin-queue-filters");
|
||||
let offset = 0;
|
||||
let token = ""; let fishes: Record<string, string>[] = []; let waters: Record<string, string>[] = [];
|
||||
let sessionTimer: ReturnType<typeof setTimeout> | undefined;
|
||||
const loadingCards = () => `<div class="loading-grid" aria-hidden="true">${Array.from({length:2}, () => '<div class="loading-card"><span class="loading-line loading-line--label"></span><span class="loading-line loading-line--title"></span><span class="loading-line"></span><span class="loading-line loading-line--short"></span></div>').join("")}</div><span class="sr-only">Загружаем внешние наблюдения</span>`;
|
||||
const setLoading = (loading: boolean) => list?.setAttribute("aria-busy", String(loading));
|
||||
const esc = (value: unknown) => String(value ?? "—").replace(/[&<>'"]/g, char => ({"&":"&","<":"<",">":">","'":"'",'"':"""}[char] ?? char));
|
||||
const fail = (message: string) => { if (error) { error.textContent = message; error.hidden = false; } };
|
||||
const safeHttpUrl = (value: unknown) => { try { const url = new URL(String(value)); return url.protocol === "http:" || url.protocol === "https:" ? esc(url.href) : ""; } catch { return ""; } };
|
||||
const fail = (message: string) => { if (status) status.hidden = true; if (error) { error.textContent = message; error.hidden = false; } };
|
||||
const succeed = (message: string) => { if (error) error.hidden = true; if (status) { status.textContent = message; status.hidden = false; } };
|
||||
const endSession = (message?: string) => { token = ""; if (sessionTimer) clearTimeout(sessionTimer); sessionTimer = undefined; if (login) { login.hidden = false; login.reset(); } if (sessionBar) sessionBar.hidden = true; if (filters) filters.hidden = true; if (list) list.innerHTML = ""; pages?.setAttribute("hidden", ""); if (message) fail(message); };
|
||||
const keepSession = () => { if (sessionTimer) clearTimeout(sessionTimer); sessionTimer = setTimeout(() => endSession("Сессия завершена после 15 минут бездействия. Введите токен снова."), 15 * 60 * 1000); };
|
||||
const options = (items: Record<string, string>[], selected?: unknown) => items.map(item => `<option value="${esc(item.slug)}" ${item.slug === selected ? "selected" : ""}>${esc(item.name_ru)}</option>`).join("");
|
||||
async function json(url: string, init: RequestInit = {}) { const response = await fetch(url, init); if (!response.ok) throw new Error(response.status === 401 ? "Неверный административный токен." : `Запрос завершился ошибкой ${response.status}.`); return response.json(); }
|
||||
async function json(url: string, init: RequestInit = {}) { const response = await fetch(url, init); if (response.status === 409) { await loadQueue(); throw new Error("Запись уже изменена в другой вкладке. Очередь обновлена."); } if (!response.ok) { if (adminEndsSession(response.status)) endSession(); throw new Error(adminErrorMessage(response.status, `Запрос завершился ошибкой ${response.status}.`)); } if ((init.headers as Record<string, string> | undefined)?.Authorization) keepSession(); return response.json(); }
|
||||
async function loadQueue() {
|
||||
if (!root || !list) return; error?.setAttribute("hidden", ""); list.innerHTML = '<div class="state"><h2>Загружаем staging…</h2></div>';
|
||||
[fishes, waters] = await Promise.all([json(`${root.dataset.apiUrl}/api/v1/fishes`), json(`${root.dataset.apiUrl}/api/v1/waterbodies`)]);
|
||||
if (!root || !list) return; error?.setAttribute("hidden", ""); setLoading(true); list.innerHTML = loadingCards();
|
||||
if (!fishes.length || !waters.length) { const [fishRows, waterRows, sources] = await Promise.all([json(`${root.dataset.apiUrl}/api/v1/fishes`), json(`${root.dataset.apiUrl}/api/v1/waterbodies`), json(`${root.dataset.apiUrl}/api/v1/source-status`)]); fishes = fishRows; waters = waterRows; const sourceSelect = filters?.querySelector<HTMLSelectElement>('[name="source"]'); if (sourceSelect) sourceSelect.innerHTML = '<option value="">Все источники</option>' + (sources as Record<string, unknown>[]).map(source => `<option value="${esc(source.source_system)}">${esc(source.name)}</option>`).join(""); }
|
||||
pages?.setAttribute("hidden", "");
|
||||
const rows = await json(`${root.dataset.apiUrl}/api/v1/admin/external-observations?status=review&limit=51&offset=${offset}`, {headers:{Authorization:`Bearer ${token}`}});
|
||||
const values = new FormData(filters ?? undefined); const params = new URLSearchParams({status:"review",limit:"51",offset:String(offset),completeness:String(values.get("completeness") || "all"),order:String(values.get("order") || "newest")}); const source = String(values.get("source") || ""); const query = String(values.get("q") || "").trim(); if (source) params.set("source_system", source); if (query) params.set("q", query);
|
||||
const rows = await json(`${root.dataset.apiUrl}/api/v1/admin/external-observations?${params}`, {headers:{Authorization:`Bearer ${token}`}});
|
||||
setLoading(false);
|
||||
if (!rows.length && offset > 0) { offset = 0; return loadQueue(); }
|
||||
const pending = rows.slice(0, 50);
|
||||
if (pages) pages.hidden = !pending.length;
|
||||
@@ -41,7 +67,21 @@ const apiUrl = import.meta.env.PUBLIC_API_URL || "http://localhost:8000";
|
||||
if (next) next.disabled = rows.length <= 50;
|
||||
if (pageNumber) pageNumber.textContent = `Страница ${offset / 50 + 1}`;
|
||||
if (!pending.length) { list.innerHTML = '<div class="state"><h2>Очередь пуста</h2><p>Все внешние записи обработаны.</p></div>'; return; }
|
||||
list.innerHTML = pending.map((row: Record<string, unknown>) => { const complete = row.x != null && row.y != null && row.weight_g != null; return `<article class="moderation-card" data-observation-id="${esc(row.id)}"><div class="moderation-summary"><span class="activity-pill"><i></i>${esc(row.source_system)} · ${esc(row.status)}</span><h2>${esc(row.fish_name)}</h2><p>${esc(row.waterbody_name)}</p><dl><div><dt>Координаты</dt><dd>${row.x == null || row.y == null ? "нет" : `${esc(row.x)}:${esc(row.y)}`}</dd></div><div><dt>Вес</dt><dd>${row.weight_g == null ? "нет" : `${esc(row.weight_g)} г`}</dd></div><div><dt>ID источника</dt><dd>${esc(row.source_external_id)}</dd></div><div><dt>Состояние</dt><dd>${complete ? "полная запись" : "неполная"}</dd></div></dl><a href="${esc(row.source_url)}" target="_blank" rel="noreferrer">Открыть первоисточник</a></div><div class="moderation-actions"><label>Каноническая рыба<select name="fish" required><option value="">Выберите…</option>${options(fishes, row.fish_slug)}</select></label><label>Канонический водоём<select name="waterbody" required><option value="">Выберите…</option>${options(waters, row.waterbody_slug)}</select></label><label>Примечание<textarea name="note" rows="2" maxlength="1000">${esc(row.review_note ?? "")}</textarea></label><p data-alias-message role="status"></p><div><button type="button" data-suggest>Подсказать соответствия</button><button type="button" data-map>Сопоставить</button><button type="button" data-publish ${complete && row.status === "ready" ? "" : "disabled"}>Опубликовать</button><button class="reject" type="button" data-reject>Отклонить</button></div></div></article>`; }).join("");
|
||||
list.innerHTML = pending.map((row: Record<string, unknown>) => { const complete = row.x != null && row.y != null && row.weight_g != null; const sourceUrl = safeHttpUrl(row.source_url); return `<article class="moderation-card" data-observation-id="${esc(row.id)}" data-version="${esc(row.moderation_version)}"><div class="moderation-summary"><span class="activity-pill"><i></i>${esc(row.source_system)} · ${esc(row.status)}</span><h2>${esc(row.fish_name)}</h2><p>${esc(row.waterbody_name)}</p><dl><div><dt>Координаты</dt><dd>${row.x == null || row.y == null ? "нет" : `${esc(row.x)}:${esc(row.y)}`}</dd></div><div><dt>Вес</dt><dd>${row.weight_g == null ? "нет" : `${esc(row.weight_g)} г`}</dd></div><div><dt>ID источника</dt><dd>${esc(row.source_external_id)}</dd></div><div><dt>Состояние</dt><dd>${complete ? "полная запись" : "неполная"}</dd></div></dl>${sourceUrl ? `<a href="${sourceUrl}" target="_blank" rel="noreferrer">Открыть первоисточник</a>` : '<span class="privacy">Ссылка источника имеет небезопасный формат</span>'}</div><div class="moderation-actions"><label>Каноническая рыба<select name="fish" required><option value="">Выберите…</option>${options(fishes, row.fish_slug)}</select></label><label>Канонический водоём<select name="waterbody" required><option value="">Выберите…</option>${options(waters, row.waterbody_slug)}</select></label><label>Примечание<textarea name="note" rows="2" maxlength="1000">${esc(row.review_note ?? "")}</textarea></label><p data-alias-message role="status"></p><div><button data-action="secondary" type="button" data-suggest>Подсказать соответствия</button><button data-action="secondary" type="button" data-map>Сопоставить</button><button data-action="primary" type="button" data-publish ${complete && row.status === "ready" ? "" : "disabled"}>Опубликовать</button><button data-action="danger" type="button" data-reject>Отклонить</button></div></div></article>`; }).join("");
|
||||
const payloadLabels: Record<string, string> = {bait:"Приманка",fishing_method:"Метод ловли",rig_type:"Оснастка",retrieve_method:"Проводка",retrieve_speed:"Скорость проводки",player_name:"Игрок",published_at:"Опубликовано источником",region:"Регион",category:"Категория"};
|
||||
const missingLabels: Record<string, string> = {coordinates:"координаты",weight_g:"вес"};
|
||||
pending.forEach((row: Record<string, unknown>, index: number) => {
|
||||
const card = list.children[index] as HTMLElement | undefined;
|
||||
const summary = card?.querySelector<HTMLElement>(".moderation-summary");
|
||||
if (!summary) return;
|
||||
const payload = Object.entries((row.source_payload as Record<string, unknown>) ?? {});
|
||||
const missing = ((row.missing_fields as string[]) ?? []).map(key => missingLabels[key] ?? key);
|
||||
const details = document.createElement("details");
|
||||
details.className = "moderation-provenance";
|
||||
const sourceCheckLabels: Record<string, string> = {available:"доступен",missing:"не найден",temporary_error:"временная ошибка",blocked:"доступ ограничен"};
|
||||
details.innerHTML = `<summary>Происхождение и исходные поля</summary><dl><div><dt>Впервые замечено</dt><dd>${esc(new Date(String(row.first_seen_at)).toLocaleString("ru-RU"))}</dd></div><div><dt>Последний раз</dt><dd>${esc(new Date(String(row.last_seen_at)).toLocaleString("ru-RU"))}</dd></div><div><dt>Решение модератора</dt><dd>${row.reviewed_at ? esc(new Date(String(row.reviewed_at)).toLocaleString("ru-RU")) : "ещё нет"}</dd></div><div><dt>Проверка источника</dt><dd>${row.source_check_status ? esc(sourceCheckLabels[String(row.source_check_status)] ?? row.source_check_status) : "ещё не выполнялась"}${row.source_checked_at ? ` · ${esc(new Date(String(row.source_checked_at)).toLocaleString("ru-RU"))}` : ""}</dd></div><div><dt>Не хватает</dt><dd>${missing.length ? esc(missing.join(", ")) : "ничего"}</dd></div>${payload.map(([key,value]) => `<div><dt>${esc(payloadLabels[key] ?? key)}</dt><dd>${esc(value)}</dd></div>`).join("")}</dl><p>Показаны только разрешённые поля, сохранённые парсером. Перед публикацией сверьте их с первоисточником.</p>`;
|
||||
summary.append(details);
|
||||
});
|
||||
}
|
||||
async function changePage(delta: number) {
|
||||
const oldOffset = offset;
|
||||
@@ -49,12 +89,15 @@ const apiUrl = import.meta.env.PUBLIC_API_URL || "http://localhost:8000";
|
||||
if (previous) previous.disabled = true;
|
||||
if (next) next.disabled = true;
|
||||
try { await loadQueue(); }
|
||||
catch { offset = oldOffset; fail("Не удалось загрузить страницу. Повторите вход в очередь."); if (login) login.hidden = false; }
|
||||
catch { offset = oldOffset; setLoading(false); if (list) list.innerHTML = ""; fail("Не удалось загрузить страницу. Повторите вход в очередь."); if (login) login.hidden = false; }
|
||||
}
|
||||
previous?.addEventListener("click", () => changePage(-50));
|
||||
next?.addEventListener("click", () => changePage(50));
|
||||
login?.addEventListener("submit", async event => { event.preventDefault(); offset = 0; token = String(new FormData(login).get("token") || ""); try { await loadQueue(); login.hidden = true; } catch (cause) { fail(cause instanceof Error ? cause.message : "Ошибка загрузки."); } });
|
||||
list?.addEventListener("click", async event => { const button = (event.target as HTMLElement).closest<HTMLButtonElement>("button"); const card = button?.closest<HTMLElement>("[data-observation-id]"); if (!button || !card || !root) return; button.disabled = true; const base = `${root.dataset.apiUrl}/api/v1/admin/external-observations/${card.dataset.observationId}`; const headers = {Authorization:`Bearer ${token}`,"Content-Type":"application/json"}; try {
|
||||
refresh?.addEventListener("click", async () => { refresh.disabled = true; try { await loadQueue(); succeed("Очередь обновлена."); } catch (cause) { fail(cause instanceof Error ? cause.message : "Не удалось обновить очередь."); } finally { refresh.disabled = false; } });
|
||||
login?.addEventListener("submit", async event => { event.preventDefault(); offset = 0; token = String(new FormData(login).get("token") || ""); try { await loadQueue(); login.hidden = true; if (sessionBar) sessionBar.hidden = false; if (filters) filters.hidden = false; } catch (cause) { setLoading(false); if (list) list.innerHTML = ""; fail(cause instanceof Error ? cause.message : "Ошибка загрузки."); } });
|
||||
filters?.addEventListener("submit", async event => { event.preventDefault(); offset = 0; try { await loadQueue(); } catch (cause) { setLoading(false); fail(cause instanceof Error ? cause.message : "Ошибка фильтрации."); } });
|
||||
logout?.addEventListener("click", () => endSession("Вы вышли из административной панели."));
|
||||
list?.addEventListener("click", async event => { const button = (event.target as HTMLElement).closest<HTMLButtonElement>("button"); const card = button?.closest<HTMLElement>("[data-observation-id]"); if (!button || !card || !root) return; const cardButtons = card.querySelectorAll<HTMLButtonElement>("button"); cardButtons.forEach(item => item.disabled = true); const base = `${root.dataset.apiUrl}/api/v1/admin/external-observations/${card.dataset.observationId}`; const headers = {Authorization:`Bearer ${token}`,"Content-Type":"application/json"}; try {
|
||||
if (button.hasAttribute("data-suggest")) {
|
||||
const suggestion = await json(`${base}/alias-suggestions`, {headers});
|
||||
const fish = fishes.find(item => item.slug === suggestion.fish_slug);
|
||||
@@ -63,9 +106,10 @@ const apiUrl = import.meta.env.PUBLIC_API_URL || "http://localhost:8000";
|
||||
if (message) message.textContent = fish || water
|
||||
? `Ранее подтверждено: рыба — ${fish?.name_ru ?? "нет соответствия"}, водоём — ${water?.name_ru ?? "нет соответствия"}. Проверьте и выберите значения перед сопоставлением.`
|
||||
: "Для этого источника подтверждённых соответствий пока нет.";
|
||||
button.disabled = false;
|
||||
cardButtons.forEach(item => item.disabled = false);
|
||||
return;
|
||||
}
|
||||
if (button.hasAttribute("data-map")) { const fish_slug = card.querySelector<HTMLSelectElement>('[name="fish"]')?.value; const waterbody_slug = card.querySelector<HTMLSelectElement>('[name="waterbody"]')?.value; if (!fish_slug || !waterbody_slug) throw new Error("Выберите рыбу и водоём."); await json(`${base}/mapping`, {method:"PATCH",headers,body:JSON.stringify({fish_slug,waterbody_slug,note:card.querySelector<HTMLTextAreaElement>('[name="note"]')?.value || null})}); } else if (button.hasAttribute("data-publish")) { await json(`${base}/publish`, {method:"POST",headers}); } else if (button.hasAttribute("data-reject")) { const reason = card.querySelector<HTMLTextAreaElement>('[name="note"]')?.value.trim(); if (!reason) throw new Error("Укажите причину отклонения."); await json(`${base}/reject`, {method:"PATCH",headers,body:JSON.stringify({reason})}); } await loadQueue(); } catch (cause) { button.disabled = false; fail(cause instanceof Error ? cause.message : "Ошибка сохранения."); } });
|
||||
let message = "Наблюдение обновлено."; if (button.hasAttribute("data-map")) { const fish_slug = card.querySelector<HTMLSelectElement>('[name="fish"]')?.value; const waterbody_slug = card.querySelector<HTMLSelectElement>('[name="waterbody"]')?.value; if (!fish_slug || !waterbody_slug) throw new Error("Выберите рыбу и водоём."); await json(`${base}/mapping`, {method:"PATCH",headers,body:JSON.stringify({fish_slug,waterbody_slug,note:card.querySelector<HTMLTextAreaElement>('[name="note"]')?.value || null,expected_version:Number(card.dataset.version)})}); message = "Соответствия сохранены."; } else if (button.hasAttribute("data-publish")) { await json(`${base}/publish`, {method:"POST",headers,body:JSON.stringify({expected_version:Number(card.dataset.version)})}); message = "Наблюдение опубликовано."; } else if (button.hasAttribute("data-reject")) { const reason = card.querySelector<HTMLTextAreaElement>('[name="note"]')?.value.trim(); if (!reason) throw new Error("Укажите причину отклонения."); await json(`${base}/reject`, {method:"PATCH",headers,body:JSON.stringify({reason,expected_version:Number(card.dataset.version)})}); message = "Наблюдение отклонено."; } await loadQueue(); succeed(message); list.querySelector<HTMLButtonElement>("button")?.focus(); } catch (cause) { cardButtons.forEach(item => item.disabled = false); fail(cause instanceof Error ? cause.message : "Ошибка сохранения."); } });
|
||||
document.addEventListener("keydown", event => { const target = event.target as HTMLElement; if (target.matches("input,textarea,select") || event.ctrlKey || event.metaKey || event.altKey) return; const card = target.closest<HTMLElement>("[data-observation-id]"); const shortcuts: Record<string, string> = {s:"[data-suggest]",m:"[data-map]",p:"[data-publish]"}; const button = card?.querySelector<HTMLButtonElement>(shortcuts[event.key.toLowerCase()] ?? "[data-no-shortcut]"); if (button && !button.disabled) { event.preventDefault(); button.click(); } });
|
||||
</script>
|
||||
</Layout>
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
import Layout from "../../layouts/Layout.astro";
|
||||
import AdminNav from "../../components/AdminNav.astro";
|
||||
const apiUrl = import.meta.env.PUBLIC_API_URL || "http://localhost:8000";
|
||||
---
|
||||
<Layout title="Административная панель — RF4 Spotter">
|
||||
<section class="form-hero"><div><span class="eyebrow"><b>ADMIN</b> Центр управления</span><h1>Панель<br/><em>модератора</em></h1></div><p>Очереди, состояние источников и последние импорты в одном безопасном обзоре.</p></section>
|
||||
<section class="admin-dashboard" data-api-url={apiUrl}>
|
||||
<AdminNav />
|
||||
<form class="admin-login" autocomplete="off"><label>Административный токен<input name="token" type="password" required autocomplete="off" /></label><button data-action="primary" type="submit">Открыть панель</button></form>
|
||||
<p class="privacy">Токен существует только в памяти вкладки. Сессия завершится после 15 минут бездействия.</p>
|
||||
<div class="admin-session-bar" hidden><span>Административная сессия активна</span><button data-action="secondary" type="button" data-admin-logout>Выйти</button></div>
|
||||
<div class="notice error" data-admin-error role="alert" hidden></div>
|
||||
<div class="notice success" data-admin-status role="status" hidden></div>
|
||||
<section class="admin-dashboard-content" aria-live="polite"></section>
|
||||
</section>
|
||||
<script>
|
||||
import { adminEndsSession, adminErrorMessage } from "../../lib/admin-errors";
|
||||
const root = document.querySelector<HTMLElement>(".admin-dashboard");
|
||||
const login = document.querySelector<HTMLFormElement>(".admin-login");
|
||||
const content = document.querySelector<HTMLElement>(".admin-dashboard-content");
|
||||
const error = document.querySelector<HTMLElement>("[data-admin-error]");
|
||||
const status = document.querySelector<HTMLElement>("[data-admin-status]");
|
||||
const sessionBar = document.querySelector<HTMLElement>(".admin-session-bar");
|
||||
const logout = document.querySelector<HTMLButtonElement>("[data-admin-logout]");
|
||||
let token = "";
|
||||
let sessionTimer: ReturnType<typeof setTimeout> | undefined;
|
||||
const esc = (value: unknown) => String(value ?? "—").replace(/[&<>'"]/g, char => ({"&":"&","<":"<",">":">","'":"'",'"':"""}[char] ?? char));
|
||||
const fail = (message: string) => { if (error) { error.textContent = message; error.hidden = false; } };
|
||||
const succeed = (message: string) => { if (error) error.hidden = true; if (status) { status.textContent = message; status.hidden = false; } };
|
||||
const endSession = (message?: string) => { token = ""; if (sessionTimer) clearTimeout(sessionTimer); sessionTimer = undefined; if (login) { login.hidden = false; login.reset(); } if (sessionBar) sessionBar.hidden = true; if (content) content.innerHTML = ""; if (message) fail(message); };
|
||||
const keepSession = () => { if (sessionTimer) clearTimeout(sessionTimer); sessionTimer = setTimeout(() => endSession("Сессия завершена после 15 минут бездействия. Введите токен снова."), 15 * 60 * 1000); };
|
||||
async function authorizedJson(path: string) { const response = await fetch(`${root?.dataset.apiUrl}${path}`, {headers:{Authorization:`Bearer ${token}`}}); if (!response.ok) { if (adminEndsSession(response.status)) endSession(); throw new Error(adminErrorMessage(response.status, "Не удалось загрузить административные данные.")); } keepSession(); return response.json(); }
|
||||
async function loadDashboard() {
|
||||
if (!content) return;
|
||||
error?.setAttribute("hidden", ""); status?.setAttribute("hidden", ""); content.setAttribute("aria-busy", "true"); content.innerHTML = '<div class="loading-grid" aria-hidden="true"><div class="loading-card"></div><div class="loading-card"></div></div>';
|
||||
const diagnostics = await authorizedJson("/api/v1/admin/diagnostics");
|
||||
const [imports, sources, history] = await Promise.all([authorizedJson("/api/v1/admin/imports?limit=5"), authorizedJson("/api/v1/admin/source-status"), authorizedJson("/api/v1/admin/moderation-history?limit=8")]);
|
||||
const reports = diagnostics.counts?.catch_reports ?? {}; const observations = diagnostics.counts?.external_observations ?? {};
|
||||
const sourceLabels: Record<string, string> = {healthy:"Работает",waiting:"Ожидает",stale:"Устарел",disabled:"Отключён",source_changed:"Изменился",temporarily_limited:"Временно недоступен"};
|
||||
const sourceRows = (sources as Record<string, unknown>[]).map(source => { const state = String(source.status); const safeState = Object.hasOwn(sourceLabels, state) ? state : "waiting"; const cooldown = Number(source.cooldown_seconds ?? 0); const detail = cooldown > 0 ? ` · cooldown ${Math.ceil(cooldown / 60)} мин` : source.backoff_recommended ? " · backoff рекомендован" : ""; const success = source.last_success_at ? ` · успех ${new Date(String(source.last_success_at)).toLocaleString("ru-RU")}` : " · успешных запусков нет"; return `<li><span><i class="status-dot ${safeState}"></i>${esc(source.name)}<small>${esc(detail + success)}</small></span><strong>${sourceLabels[safeState]}</strong></li>`; }).join("");
|
||||
const importLabels: Record<string, string> = {running:"Выполняется",success:"Успешно",partial:"Частично",failed:"Ошибка"};
|
||||
const importRows = (imports as Record<string, unknown>[]).map(run => { const status = String(run.status); const rows = Number(run.rows_seen ?? 0); const created = Number(run.rows_created ?? 0); const updated = Number(run.rows_updated ?? 0); const result = rows ? ` · ${rows} строк · +${created}/↻${updated}` : ""; const finished = run.finished_at ? ` · завершён ${new Date(String(run.finished_at)).toLocaleString("ru-RU")}` : ""; return `<li><span>${esc(importLabels[status] ?? status)}<small>${esc(result)}</small></span><time>${esc(new Date(String(run.started_at)).toLocaleString("ru-RU"))}<small>${esc(finished)}</small></time></li>`; }).join("");
|
||||
const actionLabels: Record<string, string> = {approved:"Одобрено",rejected:"Отклонено",pending:"Возвращено на проверку",published:"Опубликовано",mapped:"Сопоставлено",ready:"Готово"};
|
||||
const typeLabels: Record<string, string> = {catch_report:"Улов",external_observation:"Внешнее наблюдение"};
|
||||
const historyRows = (history as Record<string, unknown>[]).map(event => { const action = String(event.action); const type = String(event.entity_type); return `<li><span><b>${esc(typeLabels[type] ?? "Запись")}</b> · ${esc(actionLabels[action] ?? action)}${event.reason ? `<small>${esc(event.reason)}</small>` : ""}</span><time>${esc(new Date(String(event.decided_at)).toLocaleString("ru-RU"))}</time></li>`; }).join("");
|
||||
content.removeAttribute("aria-busy"); content.innerHTML = `<div class="admin-kpis"><a href="/admin/moderation"><span>Уловы на проверке</span><strong>${esc(reports.pending ?? 0)}</strong><small>Открыть очередь →</small></a><a href="/admin/external-sources"><span>Наблюдения в staging</span><strong>${esc((observations.staged ?? 0) + (observations.mapped ?? 0) + (observations.ready ?? 0))}</strong><small>Проверить источники →</small></a><article><span>Одобрено уловов</span><strong>${esc(reports.approved ?? 0)}</strong><small>Участвуют в статистике</small></article><article><span>Источников включено</span><strong>${esc(diagnostics.counts?.enabled_data_sources ?? 0)}</strong><small>из ${esc(diagnostics.counts?.data_sources ?? 0)}</small></article></div><div class="admin-dashboard-grid"><section><div class="admin-section-head"><h2>Состояние источников</h2><button type="button" data-action="secondary" data-refresh>Обновить</button></div><ul>${sourceRows || "<li>Нет данных</li>"}</ul><a href="/status">Публичная страница состояния →</a><br /><a href="/admin/media">Проверить медиа →</a></section><section><div class="admin-section-head"><h2>Последние импорты</h2><button type="button" data-action="secondary" data-official-import>Запустить импорт</button></div><ul>${importRows || "<li>Запусков пока нет</li>"}</ul><p class="privacy">Импорт обращается к официальному источнику и соблюдает cooldown.</p></section><section class="admin-history"><div class="admin-section-head"><h2>Последние решения</h2><button type="button" data-action="secondary" data-history-export>Экспорт JSON</button></div><ul>${historyRows || "<li>Решений пока нет</li>"}</ul><p class="privacy">Экспорт обезличен: без UUID, модератора, причин и исходных данных.</p></section></div>`;
|
||||
}
|
||||
login?.addEventListener("submit", async event => { event.preventDefault(); token = String(new FormData(login).get("token") || ""); try { await loadDashboard(); login.hidden = true; if (sessionBar) sessionBar.hidden = false; } catch (cause) { if (content) { content.removeAttribute("aria-busy"); content.innerHTML = ""; } fail(cause instanceof Error ? cause.message : "Ошибка загрузки."); } });
|
||||
logout?.addEventListener("click", () => endSession("Вы вышли из административной панели."));
|
||||
content?.addEventListener("click", async event => {
|
||||
const button = (event.target as HTMLElement).closest<HTMLButtonElement>("[data-history-export],[data-official-import],[data-refresh]"); if (!button) return;
|
||||
button.disabled = true;
|
||||
try { if (button.hasAttribute("data-refresh")) { await loadDashboard(); succeed("Данные обновлены."); } else if (button.hasAttribute("data-official-import")) { const response = await fetch(`${root?.dataset.apiUrl}/api/v1/admin/imports/official-records`, {method:"POST",headers:{Authorization:`Bearer ${token}`}}); if (!response.ok) { if (adminEndsSession(response.status)) endSession(); throw new Error(response.status === 502 ? "Официальный источник временно недоступен. Старые данные сохранены." : adminErrorMessage(response.status, "Не удалось запустить импорт.")); } keepSession(); await loadDashboard(); succeed("Импорт запущен. Список запусков обновлён."); } else { const response = await fetch(`${root?.dataset.apiUrl}/api/v1/admin/moderation-history-export`, {headers:{Authorization:`Bearer ${token}`}}); if (!response.ok) { if (adminEndsSession(response.status)) endSession(); throw new Error(adminErrorMessage(response.status, "Не удалось выгрузить журнал решений.")); } const blob = await response.blob(); const link = document.createElement("a"); link.href = URL.createObjectURL(blob); link.download = "rf4spotter-moderation-history.json"; link.click(); URL.revokeObjectURL(link.href); keepSession(); } }
|
||||
catch (cause) { fail(cause instanceof Error ? cause.message : "Операция не выполнена."); }
|
||||
finally { button.disabled = false; }
|
||||
});
|
||||
</script>
|
||||
</Layout>
|
||||