OpenParable
Skip to content
← Back to skills

Daft Udf Tuning

Testing & Quality
daft-udf-tuning

Daft Udf Tuning is an Agent Skill for testing & quality published in the Eventual-Inc/Daft repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Optimize Daft UDF performance. Invoke when user needs GPU inference, encounters slow UDFs, or asks about async/batch processing.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/Eventual-Inc/Daft/blob/HEAD/.claude/skills/daft-udf-tuning/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/daft-udf-tuning, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

Daft UDF Tuning

Optimize User-Defined Functions for performance.

UDF Types

TypeDecoratorUse Case
Stateless@daft.funcSimple transforms. Use async for I/O-bound tasks.
Stateful@daft.clsExpensive init (e.g., loading models). Supports gpus=N.
Batch@daft.func.batchVectorized CPU/GPU ops (NumPy/PyTorch). Faster.

Quick Recipes

1. Async I/O (Web APIs)

@daft.func
async def fetch(url: str):
    async with aiohttp.ClientSession() as s:
        return await s.get(url).text()

2. GPU Batch Inference (PyTorch/Models)

@daft.cls(gpus=1)
class Classifier:
    def __init__(self):
        self.model = load_model().cuda() # Run once per worker

    @daft.method.batch(batch_size=32)
    def predict(self, images):
        return self.model(images.to_pylist())

# Run with concurrency
df.with_column("preds", Classifier(max_concurrency=4).predict(df["img"]))

Tuning Keys

  • max_concurrency: Total parallel UDF instances.
  • gpus=N: GPU request per instance.
  • batch_size: Rows per call. Too small = overhead; too big = OOM.
  • into_batches(N): Pre-slice partitions if memory is tight.