Repository Guidelines

Project Structure & Module Organization

This repository maintains a local SQLite index of WCX publications. Production code lives in scripts/: wcx_sync.py scrapes site metadata, import_site.py imports it into SQLite, and update_wcx.sh coordinates synchronization, import, and OCR. OCR is split across process_pending_ocr.py, check_ocr.py, ocr.sh, and parse_ocr.py.

scripts/schema.sql defines the database model. Generated databases belong in database/, while generated scraper output belongs in import/; both are ignored by Git. Historical source data is stored in migration/. Database design notes live in docs/. The main executable test/diagnostic script is scripts/test_duration_matching.py.

Architectural boundaries

This repository is the reference-data project. Filename matching and scripts/match_filenames.py have moved to the separate /storage/disk1/WCX-collection repository; this repository no longer contains or runs filename matching. The reference-data project owns the database schema, metadata, OCR, imports, history, and aliases, and it is the sole writer to the reference database.

WCX-collection is an external read-only consumer of the reference database and may open it only in strict read-only mode using SQLite mode=ro. Local file status, paths, matching decisions, and collection status must never be stored in the reference database. Any future file-management registry or database must be owned separately and may refer to movie.id as the stable external ID. Do not introduce a shared Python library or API without a concrete need.

Build, Test, and Development Commands

There is no build step or third-party Python package installation; scripts use Python 3's standard library.

scripts/update_wcx.sh --skip-ocr synchronizes and imports site data without requiring a Vision API key.
scripts/update_wcx.sh runs the complete update, including pending OCR work.
scripts/migrate_csv.py --dry-run validates legacy CSV data without modifying SQLite.
scripts/test_duration_matching.py exercises duration-matching behavior against the configured database and media paths.
python3 -m py_compile scripts/*.py performs a quick syntax check.

Use a copied test database for commands that can write data. Never recreate or overwrite database/wcx.db casually because it contains manually maintained metadata.

Coding Style & Naming Conventions

Use four-space indentation and snake_case for Python functions and variables; use PascalCase for dataclasses and other classes. Prefer type hints, pathlib.Path, explicit error handling, and small focused functions. Shell scripts should use Bash with set -Eeuo pipefail, uppercase configuration constants, quoted expansions, and clear failure messages. No formatter or linter is currently configured, so follow the surrounding style.

Testing Guidelines

The project has no formal test framework or coverage threshold. Validate changes with syntax checks, dry runs, and disposable database copies. Name future Python tests test_*.py and keep fixtures isolated from production data. OCR tests require GOOGLE_VISION_API_KEY; do not expose the key in logs or commits.

Working style

Work incrementally and conservatively.
Preserve existing behavior unless the requested change explicitly alters it.
Inspect the relevant files before proposing changes.
Present a short implementation plan before larger changes.
Do not modify unrelated files.
Prefer complete, coherent changes over broad refactoring.
Run relevant tests or validation commands after changes.
Show the resulting git diff and summarize what changed.
Never commit or push unless explicitly requested.

Commit & Pull Request Guidelines

Recent commits use short, descriptive subjects such as Add movie history for site updates and Organize sync script and clean migration data. Keep each commit focused and use an imperative, specific subject. Pull requests should explain the data flow affected, list validation commands, call out schema or migration implications, and include representative terminal output when matching or OCR behavior changes. Do not commit generated JSON, SQLite files, credentials, or Python caches.

4.2 KiB Raw Blame History