Testing Policy

KiCad MCP Pro requires automated validation for behavior changes, public tool contract changes, release workflow changes, and security-sensitive changes.

Required tests by change type

Change type	Required validation
Pure Python logic	Unit tests and type checks.
KiCad artifact parsing	Fixture tests with representative .kicad_* samples.
MCP tool contract	Metadata checks, generated docs, and tool-surface snapshot tests.
TypeScript integration	TypeScript typecheck and unit tests where available.
Filesystem or subprocess behavior	Negative tests for traversal, unsafe extensions, missing files, and failure modes.
Release or package workflow	Dry-run or metadata verification plus workflow-security checks.
Documentation-only change	Markdown/docs build and link-sensitive review.

Regression tests

Bug fixes should include a regression test when the defect is reproducible in unit, fixture, or integration scope. If a regression test cannot be added, the PR must explain why and include manual validation evidence.

Coverage policy

The Python coverage gate is configured in pyproject.toml with tool.coverage.report.fail_under = 83, which satisfies the OpenSSF Silver 80% statement-coverage target for the Python package. Coverage should continue to increase over time, but critical tool paths remain prioritized over raw percentage targets.

Continuous integration

CI runs on pull requests and pushes to main. Required jobs cover server behavior, npm/package wrappers, protocol schemas, security checks, CodeQL, docs, Gitleaks, and Scorecard.

Fuzzing

Atheris fuzz smoke testing covers shared KiCad S-expression helpers. Fuzzing should expand to additional parsers and file import boundaries as those surfaces grow.

Git subprocess isolation

Tests that initialize or mutate Git repositories must use test-owned temporary directories and an isolated process environment. Set HOME, XDG_CONFIG_HOME, GIT_CONFIG_GLOBAL, and GIT_CONFIG_SYSTEM to test-owned paths, disable system configuration, and use an empty GIT_TEMPLATE_DIR so contributor hooks and templates cannot enter fixtures.

Production Git helpers discard repository-local variables such as GIT_DIR, GIT_WORK_TREE, and GIT_INDEX_FILE before spawning Git. Regression tests must seed hostile caller values and prove the source checkout status, unstaged diff, and staged diff remain unchanged across normal and failure paths. Destructive operations must revalidate the resolved repository root immediately before mutation.

Coverage and failed-test reporting

The CI Tests / Coverage job runs the full Python suite with the repository's 83% pytest-cov gate, publishes coverage.xml, and emits JUnit XML for Codecov Test Analytics. Upload steps run after test failures, but the original pytest failure is propagated after reporting. See Codecov coverage and test analytics.

Local hook scope

Lefthook owns local Git-hook installation. Commit-time hooks stay fast and deterministic: Lefthook delegates staged files to the pinned .pre-commit-config.yaml FAST catalog for file hygiene, syntax validation, Ruff formatting/linting, and Gitleaks secret scanning only. The Lefthook pre-push hook evaluates merge-base(origin/main, HEAD)..HEAD through scripts/hook_pre_push.py, which keeps the selected checks stable across normal pushes, rebases, and force-pushes. It selects scoped mypy, matching unit tests, architecture, tool-contract, workflow, web, Cargo, metadata, or compatibility checks.

The remote CI Python changed-file quality job reuses the same hook_pre_push.py selector in --ruff-only mode against the pull-request or push diff. This keeps Ruff format/lint coverage for maintained src/, tests/, and scripts/ Python files aligned without requiring legacy unrelated scripts to be globally Ruff-clean. The job is internal to Required PR Gate; it is not a separate branch-protection context.

The repository-wide unit suite, full-project type checking, coverage, integration, E2E, package/docs builds, release checks, container scans, and CodeQL remain in CI or explicit manual commands such as task ci.