OpenParable
Skip to content
← Back to skills

MLOps Validation

Testing & Quality

MLOps Validation is an Agent Skill for testing & quality published in the fmind/mlops-python-package repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Guide to implement rigorous validation layers including static analysis, automated testing, structured logging, and security scanning.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/fmind/mlops-python-package/blob/HEAD/.gemini/skills/MLOps Validation/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/mlops-validation, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

MLOps Validation

Goal

To ensure software quality, reliability, and security through automated validation layers. This skill enforces Strict Typing (ty), Unified Linting (ruff), Comprehensive Testing (pytest), and Structured Logging (loguru).

Prerequisites

  • Language: Python
  • Manager: uv
  • Context: Ensuring code quality before merge/deploy.

Instructions

1. Static Analysis (Typing & Linting)

Catch errors before they run.

  1. Typing:
    • Tool: ty.
    • Rule: No Any (unless absolutely necessary). Fully typed function signatures.
    • DataFrames: Use pandera schemas to validate DataFrame structures/types.
    • Classes: Use pydantic for data modeling and runtime validation.
  2. Linting & Formatting:
    • Tool: ruff (replaces black, isort, pylint, flake8).
    • Rule: Zero tolerance for linter errors. Use noqa sparingly and with justification.
    • Config: Centralize in pyproject.toml.

2. Testing Strategy

Verify behavior and prevent regressions.

  1. Tool: pytest.

  2. Structure: Mirror src/ in tests/.

    src/pkg/mod.py -> tests/test_mod.py
    
  3. Fixtures: Use tests/conftest.py for shared setup (mock data, temp paths).

  4. Coverage: Aim for high coverage (>80%) on core business logic. Use pytest-cov.

  5. Pattern: Use Given-When-Then in comments.

    def test_pipeline_execution(input_data):
        # Given: Valid input data
        # When: The pipeline processes the data
        # Then: The output content matches expectations
    

3. Structured Logging

Enable observability and debugging.

  1. Tool: loguru (replacing stdlib logging).
  2. Format: Use structured logging (JSON) in production for queryability.
  3. Levels:
    • DEBUG: Low-level tracing (payloads, internal state).
    • INFO: Key business events (Job started, Model saved).
    • ERROR: Actionable failures (with stack traces).
  4. Context: Include context (Job ID, Model Version) in logs.

4. Security

Protect the supply chain and runtime.

  1. Dependencies: Use GitHub Dependabot to patch vulnerable packages.
  2. Code Scanning: Run bandit to detect hardcoded secrets or unsafe patterns (e.g., eval, yaml.load).
  3. Secrets: NEVER log secrets. Sanitize outputs.

Self-Correction Checklist

  • Type Safety: Does ty pass without errors?
  • Lint Cleanliness: Does ruff check pass?
  • Test Discovery: Does pytest successfully find modules in src/?
  • Log Format: Are production logs serializing to JSON?
  • Security: Has bandit scanned the codebase?