Skip to content
Console(opens in a new tab)

Customize query generation

Pick the LLM provider, write your own provider or strategy, control the query shape cache, and read the token usage of each call.

The LLM providers snippet shows only what changes from the quickstart programs. Run the snippets in this guide explains how to run it.

TemplateStrategy and GenerativeStrategy call Anthropic’s API by default, with anthropic_api_key= or the ANTHROPIC_API_KEY environment variable; their model= argument applies only to that default. Pass llm_provider= to use another provider:

from enigma_api_client.strategies import (
AnthropicProvider, BedrockProvider, GenerativeStrategy, OpenRouterProvider, TemplateStrategy,
)
# Anthropic's API (ANTHROPIC_API_KEY)
TemplateStrategy(llm_provider=AnthropicProvider(model="claude-haiku-4-5-20251001"))
# OpenRouter (OPENROUTER_API_KEY). provider_routing is sent as OpenRouter's "provider"
# object and reasoning as its "reasoning" object; omit either to keep OpenRouter's default.
GenerativeStrategy(llm_provider=OpenRouterProvider(
model="anthropic/claude-haiku-4.5",
provider_routing={"order": ["anthropic"], "allow_fallbacks": False},
reasoning={"enabled": False},
))
# Claude on Amazon Bedrock (pip install "enigma-api-client[bedrock]"), signed with your AWS credentials.
# The default model is a US inference profile; pass model= outside US regions.
TemplateStrategy(llm_provider=BedrockProvider(region="us-east-1"))

To use another LLM, implement the LlmProvider protocol. Import LlmProvider, Completion, and ProviderHTTPError from enigma_api_client.strategies.

  • Implement async def complete(system, user, max_tokens) -> Completion.
  • Return Completion(text, truncated), where truncated is true when the model hit its output limit.
  • Pass the call’s token counts to Completion as input_tokens=, output_tokens=, cache_read_tokens=, and cache_write_tokens=, so that Token usage includes them. Leave cache reads and writes out of input_tokens; they have their own fields.
  • If your provider reports which host served the call, pass it as served_by=.
  • If your provider reports what it billed for the call, pass the amount in USD as a float, cost_usd=.
  • Raise these exceptions for failures:
Raise Reported as
ProviderHTTPError(status_code, message) with 408, 429, or 5xx TRANSIENT_PROVIDER_ERROR
ProviderHTTPError with any other HTTP status PROVIDER_ERROR
ProviderHTTPError with a value that isn’t an HTTP status TRANSIENT_PROVIDER_ERROR
ValueError, for a missing key or bad configuration GENERATE_QUERY_CALLBACK_NOT_CONFIGURED
RuntimeError, for any other provider failure TRANSIENT_PROVIDER_ERROR

Any other exception propagates unchanged.

TemplateStrategy and GenerativeStrategy keep an in-memory cache of query shapes learned from successful generations:

  • A shape is a generated query with its names and places left as blanks.
  • Once a shape has been confirmed across several different entities, a matching question is answered from the cache without an LLM call.
  • The cache is on by default and is shared by every caller of the same strategy instance at the same tier.
  • To turn it off, pass shape_cache=False, or set ENIGMA_SHAPE_CACHE_ENABLED=0 before you create the strategy.

After client.search(prompt=...) or generate_query returns or raises, enigma_api_client.strategies.GENERATION_USAGE.get() returns that call’s LLM token counts as an LlmUsage:

Field Holds
completions The number of LLM completions
input_tokens, cache_read_tokens, cache_write_tokens, output_tokens The token counts
unmetered_completions The number of completions whose provider left out the input or the output token count, so the sums miss their tokens
served_by The host each completion named as having served it, in call order, for providers that report one, such as OpenRouterProvider
cost_usd The amount in USD billed for the completions whose provider reported one, such as OpenRouterProvider; completions from other providers add nothing
  • Read it in the same task that awaited the call. asyncio.gather runs each call in a task of its own, so read the usage inside the coroutine you pass to it.
  • ServiceStrategy reports zero usage, and AgentStrategy reports its run’s token totals and no cost_usd.

To write your own strategy:

  1. Subclass QueryStrategy.
  2. Implement generate_query and aclose.
  3. At the start of generate_query, set GENERATION_USAGE to a fresh LlmUsage(), so Token usage reports this call, not an earlier one.

This strategy treats the whole question as a brand name:

import asyncio
import os
from enigma_api_client import DEFAULT_TIER, Client, ClientConfig, QueryStrategy
from enigma_api_client.strategies import GENERATION_USAGE, LlmUsage
from enigma_api_client.strategies.base import GenerateQueryResponse
QUERY = """
query Brand($name: String!) {
search(searchInput: { name: $name, entityType: BRAND, conditions: { limit: 1 } }) {
... on Brand { names(first: 1) { edges { node { name } } } }
}
}
"""
class BrandNameStrategy(QueryStrategy):
async def generate_query(self, prompt, *, tier=DEFAULT_TIER, prior_attempt=None):
GENERATION_USAGE.set(LlmUsage())
return GenerateQueryResponse(QUERY, {"name": prompt})
async def aclose(self):
pass
async def main():
config = ClientConfig(enigma_api_key=os.environ["ENIGMA_API_KEY"], query_strategy=BrandNameStrategy())
async with Client(config) as client:
response = await client.search(prompt="Chipotle")
print(response.data["search"][0]["names"]["edges"][0]["node"]["name"]) # CHIPOTLE MEXICAN GRILL
asyncio.run(main())