Getitune Optimizing A Model
Developmentgetitune-optimizing-a-modelGetitune Optimizing A Model is an Agent Skill for development published in the open-edge-platform/geti repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Optimize an exported getitune model (the Geti training library) with post-training quantization. Use when a user wants to run `OVEngine.optimize()` / `engine.optimize()` to produce an INT8 model via NNCF, understands calibration-set requirements, or needs to re-validate and run inference with a quantized model versus the original FP32/FP16 model. Covers OpenVINO NNCF post-training quantization and the accuracy/size trade-off.
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/open-edge-platform/geti/blob/HEAD/skills/library/getitune-optimizing-a-model/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/getitune-optimizing-a-model, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
Optimizing (quantizing) a model with getitune
getitune applies post-training quantization (PTQ) via
NNCF to shrink an exported OpenVINO
model and speed up inference. Quantization runs on an OpenVINO model (an
exported .xml), producing an INT8 version.
Run everything from library/.
Workflow
from getitune.engine import create_engine
# Load an exported OpenVINO model, then quantize it
ov_engine = create_engine(
model="/path/to/exported_model.xml",
data="/path/to/dataset",
)
ov_engine.optimize() # INT8 post-training quantization via NNCF
int8_metrics = ov_engine.test() # validate the quantized model
predictions = ov_engine.predict() # run inference with the quantized model
- Start from an exported OpenVINO model (
.xml). If you only have a checkpoint, export it first with thegetitune-exporting-a-modelskill.- Done when:
create_engine(model="....xml", data=...)builds anOVEngine.
- Done when:
- Provide a calibration dataset. Calibration images are taken automatically
from the training subset; 200-500 images is the recommended calibration size.
- Done when:
optimize()runs without a "not enough calibration data" issue.
- Done when:
- Run
optimize(). This replaces the engine's model in place with the INT8 version.- Done when: the call completes and subsequent
test()/predict()use INT8.
- Done when: the call completes and subsequent
- Re-validate accuracy with
test()and compare against the FP32/FP16 baseline; a small accuracy drop is expected in exchange for size/latency.- Done when: the INT8 metric is within your acceptable tolerance of baseline.
Comparing against the original model
After optimize() the engine holds the INT8 model. To re-check the original
FP32/FP16 model, either pass the original .xml path directly to .test() /
.predict(), or create the engine again from the original .xml.
Notes
- Quantization is OpenVINO/NNCF-based and applies to exported IR models — it is not a training-time step.
- Only OpenVINO IR (
.xml) is supported. An ONNX model must be converted to OpenVINO IR first before it can be optimized. - In the Geti application this is exposed as the
quantizejob (application/backend/app/execution/quantization/); libraryoptimize()is the same capability without the job/queue wrapper.
Verify
# from library/
just lint
just test-unit -- -k optimize # when you touched optimization code
Related skills
getitune-exporting-a-model— produce the OpenVINO.xmlto quantize.getitune-running-inference— run inference with the quantized model.