MLOps Validation
Testing & QualityMLOps Validation is an Agent Skill for testing & quality published in the fmind/mlops-python-package repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Guide to implement rigorous validation layers including static analysis, automated testing, structured logging, and security scanning.
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/fmind/mlops-python-package/blob/HEAD/.gemini/skills/MLOps Validation/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/mlops-validation, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
MLOps Validation
Goal
To ensure software quality, reliability, and security through automated validation layers. This skill enforces Strict Typing (ty), Unified Linting (ruff), Comprehensive Testing (pytest), and Structured Logging (loguru).
Prerequisites
- Language: Python
- Manager:
uv - Context: Ensuring code quality before merge/deploy.
Instructions
1. Static Analysis (Typing & Linting)
Catch errors before they run.
- Typing:
- Tool:
ty. - Rule: No
Any(unless absolutely necessary). Fully typed function signatures. - DataFrames: Use
panderaschemas to validate DataFrame structures/types. - Classes: Use
pydanticfor data modeling and runtime validation.
- Tool:
- Linting & Formatting:
- Tool:
ruff(replaces black, isort, pylint, flake8). - Rule: Zero tolerance for linter errors. Use
noqasparingly and with justification. - Config: Centralize in
pyproject.toml.
- Tool:
2. Testing Strategy
Verify behavior and prevent regressions.
-
Tool:
pytest. -
Structure: Mirror
src/intests/.src/pkg/mod.py -> tests/test_mod.py -
Fixtures: Use
tests/conftest.pyfor shared setup (mock data, temp paths). -
Coverage: Aim for high coverage (>80%) on core business logic. Use
pytest-cov. -
Pattern: Use Given-When-Then in comments.
def test_pipeline_execution(input_data): # Given: Valid input data # When: The pipeline processes the data # Then: The output content matches expectations
3. Structured Logging
Enable observability and debugging.
- Tool:
loguru(replacing stdliblogging). - Format: Use structured logging (JSON) in production for queryability.
- Levels:
DEBUG: Low-level tracing (payloads, internal state).INFO: Key business events (Job started, Model saved).ERROR: Actionable failures (with stack traces).
- Context: Include context (Job ID, Model Version) in logs.
4. Security
Protect the supply chain and runtime.
- Dependencies: Use
GitHub Dependabotto patch vulnerable packages. - Code Scanning: Run
banditto detect hardcoded secrets or unsafe patterns (e.g.,eval,yaml.load). - Secrets: NEVER log secrets. Sanitize outputs.
Self-Correction Checklist
- Type Safety: Does
typass without errors? - Lint Cleanliness: Does
ruff checkpass? - Test Discovery: Does
pytestsuccessfully find modules insrc/? - Log Format: Are production logs serializing to JSON?
- Security: Has
banditscanned the codebase?