AI calls through the proxy

Replace or revoke a key

Open your project's Settings → Hereling keys to create a named Usage only key. Copy it when it appears; the full key is shown only once. For a planned change, deploy the new key to your app, confirm requests work, then revoke the old key. Both keys remain active until you revoke one.

If a key is exposed, use Revoke → Revoke now immediately, even if it is your last key. After revocation succeeds, new authentication checks reject it on the proxy, usage events and MCP. Requests already authenticated may finish. Create a replacement to restore access.

Setup

Change your AI client's base address and add three headers. That is the whole change.

Use a Usage only key for the proxy and usage events. It cannot read financial reports or change user rules. MCP read only and MCP management keys are separate and cannot send usage. A key with the wrong access returns 403 insufficient_key_permissions. Keep keys on your server, never in browser or mobile application code.

Whatever comes after the base address is passed to the provider untouched, so every endpoint your SDK already calls keeps working.

Keep sending your provider API key exactly as you do today, in Authorization: Bearer … or x-api-key. Hereling forwards it to the provider and never stores or logs it.

OpenAI SDK (Node)

const openai = new OpenAI({
  baseURL: "https://api.hereling.com/v1/openai",
  apiKey: process.env.OPENAI_API_KEY,
  defaultHeaders: {
    "x-hereling-key": process.env.HERELING_KEY,
    "x-hereling-user": userId,
    "x-hereling-feature": "chat",
  },
});

Anthropic SDK (Node)

const anthropic = new Anthropic({
  baseURL: "https://api.hereling.com/v1/anthropic",
  apiKey: process.env.ANTHROPIC_API_KEY,
  defaultHeaders: {
    "x-hereling-key": process.env.HERELING_KEY,
    "x-hereling-user": userId,
    "x-hereling-feature": "chat",
  },
});

Any language, plain HTTP

There is no Hereling SDK to install. If your app builds the request itself, the change is the address and the three extra headers.

curl https://api.hereling.com/v1/openai/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "x-hereling-key: $HERELING_KEY" \
  -H "x-hereling-user: user_42" \
  -H "x-hereling-feature: chat" \
  -H "content-type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}]}'

What Hereling changes in your request

Only for streamed /chat/completions calls to OpenAI, OpenRouter and DeepSeek, Hereling adds stream_options.include_usage (and on OpenRouter usage.include) so the provider reports token counts in its final chunk. Without it a streamed call could not be priced. Anything you already put in stream_options is kept. OpenAI Responses and Anthropic Messages already report usage; their request options are left unchanged.

Nothing else is touched. Headers, body, path and query reach the provider as they left you; the provider's status, headers and body come back as they are, streamed chunk by chunk. The only other exceptions are the two actions you switch on yourself, per user, on the Pro plan: the monthly limit and the model switch, both further down this page. Strict protection also setsservice_tier: "default" when omitted, so the request uses the standard tariff.

Supported usage measurement

Ordinary function calling and text tool results, including Anthropic thinking blocks in tool history, use normal token pricing. Images, audio, video, server tools such as web search or code execution, file uploads, embeddings, batches and background Responses are outside verified token pricing. Calls are still forwarded; an explicit OpenRouter cost is always retained, including zero and media/tool charges. Without a reported cost, unsupported usage remains unknown. Alternate service tiers, regional tariffs, conditional prices and Anthropic inputs over 200,000 tokens are not priced from the reference table. Unknown costs leave user and plan results and budget comparisons not yet calculated.

What gets recorded

One row per call: project, end user id, feature, provider, model, input tokens, output tokens, cost in USD, duration, HTTP status, time, separate cache token counts, and the reference price snapshot when used. There is no column for message content. Requests with no model field (listing models, uploading a file) are forwarded but not recorded.

OpenRouter's reported cost takes priority. Other supported calls use reference rates from the daily OpenRouter model list, with cache reads and writes priced separately. Each calculated row keeps the price source, refresh date, matched model and unit rates. Reference rates may differ from your provider invoice or negotiated price.

Missing usage, interrupted streams, missing rates and unknown cache write durations leave the cost unknown. Zero is used only when a measured call is free or a Hereling limit prevents it. Overview shows the recorded cost subtotal and the number of unpriced calls; profit and margin say “Not yet calculated”, and the daily cost line has gaps. The MCP summary and weekly email also disclose incomplete costs. Older records are not automatically recalculated.

Errors from Hereling itself

Hereling answers on its own only when it cannot forward your call. Those answers carry an x-hereling-error header and this body:

{ "error": { "message": "…", "type": "hereling_error", "code": "invalid_hereling_key" } }

Provider errors keep their original status and body and carry x-hereling-error-source: provider. Hereling errors carry x-hereling-error-source: hereling and x-hereling-error. Spending protection errors are listed below.

Traffic limits, timeouts and retries

Every project can start 120 proxy requests per 60-second window across all its keys and providers, on either plan. Usage events have a separate 120-request window and MCP has a 60-request window. These counters are shared across server instances. A separate service-wide ceiling accepts 3,000 key authentication attempts per window. Rejected requests receive HTTP 429 and a Retry-After value in seconds. These limits are separate from your monthly recording quota and provider limits.

Proxy request bodies are limited to 4 MiB; usage events and MCP accept 64 KiB. The limit is checked while reading, including uploads without a length header. Uploads must finish within 15 seconds. The proxy waits at most 30 seconds for provider response headers and allows 120 seconds total from the start of the provider call, including streaming. A timeout after headers have been sent ends the stream; it cannot change an already-sent HTTP status. Client disconnection cancels the upstream connection.

Settings → Connection health separates Hereling keys, recent provider responses, payment setup and recording evidence. A missing model, rejected provider key and provider quota show different guidance. The screen uses saved metadata; it never stores provider credentials or makes a billable test call.

The free plan limit

The free plan records 10,000 requests per project per calendar month, counting the proxy and the events endpoint together. Past that, Hereling keeps forwarding your requests and answering exactly as before; it only stops writing usage rows until the next month, and your project page shows a banner. Your app never slows down or fails because of this limit. The events endpoint says so in its reply ({"ok":true,"dropped":"monthly_request_limit"}); the proxy says nothing, on purpose.

Per-user monthly limits (Pro)

From a user's page in the dashboard you can set a monthly limit in USD and choose Keep running or Strict protection. Keep running checks recorded spend but continues if that spend lookup fails. Concurrent requests and unknown prices can exceed its limit; it is an availability choice, not a hard spending cap. Once that user's recorded AI cost in the current calendar month (UTC) reaches the amount, the proxy stops calling the provider for them and answers with your limit message (Settings → Limits) instead, shaped like a normal answer from the provider:

• chat/completions (OpenAI, OpenRouter, DeepSeek): a chat.completion with your message as the assistant's content, or the same as chat.completion.chunk events when the request asked to stream.
• Anthropic messages: a message with one text block, or the usual message_start … message_stop events when streaming.
• Any other model call (embeddings, audio, …): HTTP 429 with {"error":{"code":"usage_limit_reached", …}}.

The status is 200 for chat and messages, so your app shows the message like any other reply and the limit message can be shown like another reply. Strict protection can also return the errors below. Every limited answer carries the header x-hereling-limited: true, so you can show your own screen instead if you prefer. Limited requests are recorded with zero tokens and zero cost, marked as limited.

Strict protection: supported requests and recovery

Strict protection reserves capacity in the database before calling the provider, including calls already in flight. Each user can have 1–10 concurrent calls (default 2). Completion records the usage and reconciles the reservation in one transaction. Unknown historical costs pause new calls. This controls supported proxy calls against the recorded USD tariff; it is not a guarantee of the provider's invoice or of calls made directly to a provider or reported later through usage events.

Initial support: direct openai Chat Completions and Responses with gpt-4o-mini-2024-07-18, gpt-4o-2024-08-06, or gpt-4o-2024-11-20. Use plain text messages (or a string input for Responses), one completion, the default service tier, and an explicit max_completion_tokens / max_tokens(Chat) or max_output_tokens (Responses), at most 16,384. Streaming works on both APIs. Tools, images, audio, stored conversation references, background jobs, model aliases, other providers, beta headers, query parameters and unreviewed options are rejected. Keep running remains available for them.

The price must have been refreshed within 48 hours. The reservation covers the model's entire 128,000-token input window plus your output cap at undiscounted rates, rounded up to a microdollar. For example, GPT-4o mini with an output cap of 100 reserves at least $0.01926. Small requests can therefore be refused before their expected cost reaches the remaining budget. Completed calls use reported token counts with the tariff saved at admission, never below the reviewed OpenAI standard rates (including cached input). The saved price source identifies this conservative floor; negotiated discounts may make your invoice lower; no live price lookup can release capacity at a different tariff.

• 400 trusted_user_required: supply a trusted server-side user ID of 1–200 characters.
• 422 strict_request_unsupported: use a supported request and refresh stale or missing prices.
• 429 spend_concurrency_limit: wait for an admitted call to complete.
• 503 spend_cost_unknown: resolve an unknown historical cost or unfinished measurement.
• 503 spend_control_unavailable: the rules or reservation could not be verified; no provider call is made.

A disconnected, timed-out or unrecorded call keeps its reservation. Holds never expire automatically: the provider may still have charged. Unfinished holds stay counted across month boundaries; successfully reconciled calls count in the UTC month they were admitted. The user page shows held amounts and unresolved calls. If a hold remains after the call finished, contact support with the user, call time and x-hereling-reservation response header; an operator must verify the provider's usage before reconciling it. Do not delete a hold or assume a failed response cost zero.

Unreserved calls awaiting completion or recording also pause strict admissions for that user, including unresolved calls from an earlier month. Calls omitted by the recording quota pause admissions for their UTC month. Older receipts without a user label pause strict admissions across the project until resolved. A queued call resumes automatically once its accounting is recorded; a missing payload requires verified recovery.

Per-user model switch (Pro)

From the same page you can send one user to a cheaper model. Pick "Switch to a cheaper model", choose the provider and request type, and choose a supported target model (gpt-4o-mini, openai/gpt-4o-mini on OpenRouter, claude-3-5-haiku-20241022, …). From then on the proxy replaces the model field only for compatible text requests on that provider and request type. Other calls keep the requested model, and the provider's answer comes back unchanged.

• Every affected answer carries the header x-hereling-model with the model that actually answered.
• The usage row keeps both names, what your app asked for and what ran, and prices the call by the model that ran.
• If the user already asks for that model, nothing is changed.
• Model switching covers a conservative shared request size: at most 16 KiB of JSON, with any explicit output cap at most 4,096 tokens. Larger requests keep the requested model. Model-specific options such as thinking, reasoning settings, stored conversation references, structured output schemas and routing overrides also keep the requested model. Ordinary text and client function tools remain eligible.
• Provider and request type must match the saved rule. The source and target must both be supported text models. Media requests and unreviewed source models are not rewritten. Existing unscoped rules are paused until you choose a provider and request type and save again.

A user with both rules gets the limit message once over the limit; until then the model rule applies to compatible requests. If a project drops back to the free plan its rules stay saved but are not applied until it is Pro again.

Recording reliability and coverage

Before a model call starts, Hereling saves a unique recording receipt. If that cannot be saved,503 recording_unavailable is returned before calling the provider. Keep running governs spending checks; it does not bypass this recording prerequisite. After admission, your provider response continues unchanged, including when the recording quota is full.

Only token counts, costs and short labels enter the durable recording queue. Queued records retry every five minutes with backoff. Retries do not call the AI provider and do not add cost or consume quota twice. x-hereling-recording identifies the receipt for support; it is not a provider idempotency key.

Overview shows calls awaiting completion, queued records, failed attempts and quota gaps above the amounts. Missing records suppress the result and comparisons. If the process ends before saving final usage, the receipt remains unresolved: numbers cannot be reconstructed from a receipt alone. Support must verify usage with the provider. Never retry an AI call solely to repair its accounting record.