Bug: 'choices=configured_sources()' in argparse opened DB session at
import time, causing --help to fail when DB was unavailable.
Fix:
- Added STATIC_SOURCE_CHOICES list with known source keys
- argparse uses static choices — no DB required for --help
- fetch-community command now checks enabled status at runtime
- Disabled sources return error: 'source X is disabled or not configured'
- run_source still handles locked/cooling down state
Verification:
- CLI --help works without DB
- fetch-community --help shows all known sources
- Disabled sources are rejected at runtime with clear error
- 124/124 Python tests pass (1 skipped)
Bug: 'observation.fish_external_id is None' in early return prevented
auto-publishing observations that only have fish_name (no external_id),
even when name-based fallback matching was available.
Fix:
- Removed fish_external_id check from early return condition
- Auto-publish now tries external_id first, falls back to name match
- review_note now describes actual matching method:
'Auto-matched: fish via external_id/name, waterbody via external_id/name'
- Previous note 'Automatically matched by previously reviewed source aliases'
was misleading when name fallback was used
Verification:
- 14/14 community_importer tests pass
- 124/124 Python tests pass (1 skipped)
- Observations without fish_external_id can now auto-publish via name match
- review_note accurately describes matching method
Bug: has_any_success allowed one healthy source to give overall 'ready'
when another source was stale/not_started/running — masking failures.
Fix:
- Added has_any_stale and has_any_running tracking
- Overall status is 'degraded' if ANY source is failed/stale/running
- Overall status is 'ready' ONLY when ALL enabled sources are healthy
- 'not_started' when no sources are enabled
- readiness (ready flag) still NOT blocked by import health (A01 requirement)
Verification:
- 7/7 readiness tests pass
- 124/124 Python tests pass (1 skipped)
- Stale source now shows 'degraded' instead of 'ready'
- Failed source still shows 'degraded'
- All healthy sources show 'ready'
Bug: review_note was empty or contained arbitrary text, not explaining
how the observation was matched to fish/waterbody.
Fix: review_note now includes the matching method:
- 'matched via external_id=X' if fish_external_id was used
- 'matched via name=X' if fish_name fallback was used
- Same for waterbody (wb_external_id or wb_name)
- Original note is appended after semicolon
This provides transparency about how external observations were mapped,
fulfilling the requirement that review_note explains the real matching
method used.
Verification:
- 124/124 Python tests pass
- Existing tests still pass (review_note is optional parameter)
- New review_note format is machine-readable and human-friendly
Added 3 new tests for A06 proxy chain verification:
1. test_rate_limit_independent_limits_for_two_clients_through_proxy
- Two clients behind trusted proxy have independent rate limits
- Client 1 blocked after 5 requests, Client 2 still allowed
2. test_forged_xff_rejected_on_untrusted_port
- XFF from untrusted connection is ignored
- Real client IP used for rate limiting, not forged XFF
3. test_direct_access_without_xff_header
- Direct access without XFF uses real client IP
- Hash is of real IP, not empty string
Verification:
- 8/8 rate limit tests pass
- Docker network CIDR (172.17.0.0/16) tested
- Forged XFF properly rejected from untrusted sources
- Independent rate limits verified for multiple clients
Bug: community_scheduler always had status='ready' even when individual
sources were failed or stale. Success of one source masked failure of another.
Fix:
- Overall status is 'degraded' if any enabled source has failed
- Overall status is 'stale' if all sources are stale but none failed
- Overall status is 'ready' only when at least one source is healthy
- Overall status is 'not_started' when no sources are enabled
- Readiness (ready flag) still NOT blocked by import health (A01 requirement)
Verification:
- 7/7 readiness tests pass
- 121/121 Python tests pass (1 skipped)
- Failed/stale sources are now visible in JSON without blocking scheduler
- Infrastructure (DB/MinIO) blocks readiness; imports are diagnostic only
- Per-source community scheduler health with backoff detection
- Stale/failed imports never block /ready — scheduler can recover them
- Add 'blocking: false' to all import components
- 4 new tests: per-source health, backoff detection, stale/failed non-blocking
- 108 Python tests pass
- Add ImportRecordEvent model to track per-record import changes
- Log created/updated events for each official record import
- Add alembic migration 0014 for import_record_event table
- Enables audit trail for which import run modified which records
- Generate requirements-lock.txt and requirements-dev-lock.txt via pip-compile
- CI uses locked files for reproducible installs
- Add web unit tests to CI (npm run test:unit)
- Add dependency-audit job using pip-audit
- Add Makefile with lock/lock-dev targets for regeneration
D04: Add fish name-based fallback in _auto_publish (was external_id only)
D06: Cap confidence at 50% for 1 player, 65% for 2 players
D07: Set caught_at=None for community imports (not published_at)
D08: Already OK - activity_rows has no top-100 limit
- Add Fish import to community_importer.py
- Add 2 unit tests for D06 confidence caps
- Update test_community_importer.py for D04 name match behavior
- Add _is_trusted_proxy() to check client IP against trusted CIDRs
- Only use X-Forwarded-For if connection came from trusted proxy
- Add TRUSTED_PROXY_CIDRS config (default: 127.0.0.1/32, ::1/128)
- Add parse_comma_separated_lists for env var parsing
- Add 3 unit tests: trusted CIDR check, untrusted ignores forwarded, trusted uses forwarded
- Add 'ready = ready and healthy' for community_scheduler check
- Add 'ready = False' for community_scheduler exception path
- Add 3 unit tests: success=ready, stale=not_ready, failed=not_ready
- Monitoring now correctly reports community import health
ik