OpenParable
Skip to content
← Back to skills

Vastai Rate Limits

Development
vastai-rate-limits

Vastai Rate Limits is an Agent Skill for development published in the Dicklesworthstone/pi_agent_rust repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Implement Vast.ai rate limiting, backoff, and idempotency patterns. Use when handling rate limit errors, implementing retry logic, or optimizing API request throughput for Vast.ai. Trigger with phrases like "vastai rate limit", "vastai throttling", "vastai 429", "vastai retry", "vastai backoff".

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/Dicklesworthstone/pi_agent_rust/blob/HEAD/tests/ext_conformance/artifacts/plugins-community/plugins/saas-packs/vastai-pack/skills/vastai-rate-limits/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/vastai-rate-limits, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

Vast.ai Rate Limits

Overview

Handle Vast.ai rate limits gracefully with exponential backoff and idempotency.

Prerequisites

  • Vast.ai SDK installed
  • Understanding of async/await patterns
  • Access to rate limit headers

Instructions

Step 1: Understand Rate Limit Tiers

TierRequests/minRequests/dayBurst
Free601,00010
Pro30010,00050
Enterprise1,000100,000200

Step 2: Implement Exponential Backoff with Jitter

async function withExponentialBackoff<T>(
  operation: () => Promise<T>,
  config = { maxRetries: 5, baseDelayMs: 1000, maxDelayMs: 32000, jitterMs: 500 }
): Promise<T> {
  for (let attempt = 0; attempt <= config.maxRetries; attempt++) {
    try {
      return await operation();
    } catch (error: any) {
      if (attempt === config.maxRetries) throw error;
      const status = error.status || error.response?.status;
      if (status !== 429 && (status < 500 || status >= 600)) throw error;

      // Exponential delay with jitter to prevent thundering herd
      const exponentialDelay = config.baseDelayMs * Math.pow(2, attempt);
      const jitter = Math.random() * config.jitterMs;
      const delay = Math.min(exponentialDelay + jitter, config.maxDelayMs);

      console.log(`Rate limited. Retrying in ${delay.toFixed(0)}ms...`);
      await new Promise(r => setTimeout(r, delay));
    }
  }
  throw new Error('Unreachable');
}

Step 3: Add Idempotency Keys

import { v4 as uuidv4 } from 'uuid';
import crypto from 'crypto';

// Generate deterministic key from operation params (for safe retries)
function generateIdempotencyKey(operation: string, params: Record<string, any>): string {
  const data = JSON.stringify({ operation, params });
  return crypto.createHash('sha256').update(data).digest('hex');
}

async function idempotentRequest<T>(
  client: Vast.aiClient,
  params: Record<string, any>,
  idempotencyKey?: string  // Pass existing key for retries
): Promise<T> {
  // Use provided key (for retries) or generate deterministic key from params
  const key = idempotencyKey || generateIdempotencyKey(params.method || 'POST', params);
  return client.request({
    ...params,
    headers: { 'Idempotency-Key': key, ...params.headers },
  });
}

Output

  • Reliable API calls with automatic retry
  • Idempotent requests preventing duplicates
  • Rate limit headers properly handled

Error Handling

HeaderDescriptionAction
X-RateLimit-LimitMax requestsMonitor usage
X-RateLimit-RemainingRemaining requestsThrottle if low
X-RateLimit-ResetReset timestampWait until reset
Retry-AfterSeconds to waitHonor this value

Examples

Queue-Based Rate Limiting

import PQueue from 'p-queue';

const queue = new PQueue({
  concurrency: 5,
  interval: 1000,
  intervalCap: 10,
});

async function queuedRequest<T>(operation: () => Promise<T>): Promise<T> {
  return queue.add(operation);
}

Monitor Rate Limit Usage

class RateLimitMonitor {
  private remaining: number = 60;
  private resetAt: Date = new Date();

  updateFromHeaders(headers: Headers) {
    this.remaining = parseInt(headers.get('X-RateLimit-Remaining') || '60');
    const resetTimestamp = headers.get('X-RateLimit-Reset');
    if (resetTimestamp) {
      this.resetAt = new Date(parseInt(resetTimestamp) * 1000);
    }
  }

  shouldThrottle(): boolean {
    // Only throttle if low remaining AND reset hasn't happened yet
    return this.remaining < 5 && new Date() < this.resetAt;
  }

  getWaitTime(): number {
    return Math.max(0, this.resetAt.getTime() - Date.now());
  }
}

Resources

Next Steps

For security configuration, see vastai-security-basics.