http-mocking

HTTP mocking in ellmer

Use this skill when writing or modifying tests that involve LLM provider APIs.

ellmer uses the vcr package to record and replay HTTP interactions. Tests that call provider APIs can run without credentials by replaying previously recorded responses.

When to use vcr

Most ellmer tests do not use vcr. Only use vcr when the test makes a real HTTP request to a provider API.

No cassette needed for tests that:

• Construct R objects or parse JSON (e.g. building turns, content types)
• Check request formatting or header construction
• Test internal logic without making HTTP requests

For chat-like behavior without HTTP, use MockedChat from helper-chat.R:

chat <- mocked_chat(c("response 1", "response 2"))

When you don't need a full HTTP round-trip, construct provider and response objects directly:

test_that("value_turn() prices cache writes at 1.25x", {
  provider <- ProviderAnthropic(
    name = "Anthropic",
    base_url = "https://api.anthropic.com/v1",
    model = "claude-sonnet-4-20250514",
    params = list(), extra_args = list(),
    extra_headers = character(), credentials = NULL,
    beta_headers = character(), cache = ""
  )
  result <- list(
    content = list(list(type = "text", text = "ok")),
    stop_reason = "end_turn",
    usage = list(
      input_tokens = 1000, output_tokens = 50,
      cache_creation_input_tokens = 400,
      cache_read_input_tokens = 200
    )
  )

  turn <- value_turn(provider, result)
  expect_equal(unname(turn@tokens), c(1000 + 400, 50, 200))
})

In practice, use one cassette to prove a feature works end-to-end with the real API (e.g. "can Anthropic do tool calling"), then test the details like edge cases, parsing logic, and error handling with constructed data.

Using vcr

The vcr::local_cassette() pattern

Call vcr::local_cassette() at the top of test_that(), before any code that triggers HTTP requests:

test_that("can make simple request", {
  vcr::local_cassette("anthropic-basic")

  chat <- chat_anthropic_test("Be as terse as possible; no punctuation")
  resp <- chat$chat("What is 1 + 1?")
  expect_match(resp, "2")
})

Provider _test() functions

When adding a new provider, create a chat_{provider}_test() function that sets echo = "none", temperature = 0, and defaults to a low-cost model. See chat_anthropic_test() in R/provider-claude.R for an example of the pattern.

Shared test helpers

tests/testthat/helper-provider.R defines standardized test_*() functions for common provider capabilities (tool calling, image input, data extraction, etc.). Check the file for the current list.

The helper-provider.R functions make HTTP requests internally, so they still need a cassette. Pass the _test() function (not a chat instance) so the helper can create its own chats:

test_that("supports tool calling", {
  vcr::local_cassette("anthropic-tool")
  chat_fun <- chat_anthropic_test

  test_tools_simple(chat_fun)
})

Credential handling

key_get() in R/utils.R enables dual-mode operation:

• API key set: runs the real request (and records a cassette if one is active).
• Replaying (VCR_IS_REPLAYING=TRUE): returns "" so the test doesn't error on a missing key.
• Testing without key: skips the test via testthat::skip().

Managing cassettes

Cassette naming and storage

Cassettes typically follow a {provider}-{feature} naming pattern (e.g., anthropic-tool.yml, openai-pdf.yml), though some use other descriptive names (e.g., chat-tools-callbacks.yml). Choose a name that clearly identifies the provider and feature being tested.

Cassettes are stored in:

• tests/testthat/_vcr/ for tests
• vignettes/_vcr/ for vignettes
• inst/_vcr/ for roxygen examples

Recording and managing cassettes

To record a cassette:

1. Set the provider's API key as an environment variable.
2. Run the test. If no cassette file exists, vcr records one automatically.
3. Commit the YAML cassette file alongside the test.

To re-record, delete the cassette file and run the test again with a valid API key.

Helper functions in tests/testthat/helpers-vcr.R:

• vcr_clean(url_prefix) -- deletes all cassettes whose first request URL matches a prefix. Useful when a provider changes its API URL.
• vcr_rebuild() -- re-records all cassettes by installing the package, rebuilding vignettes, and running all tests.

Adding tests for a new provider

1. Create a chat_{provider}_test() function (echo = "none", temperature = 0, low-cost model).
2. Write tests using vcr::local_cassette() and the shared helpers from helper-provider.R.
3. Record cassettes with a valid API key and commit the YAML files in tests/testthat/_vcr/.