LLM throughput planner

What it does

A provider limits how fast your account may send requests and tokens. This tool works out how fast your product needs to send them at its busiest, and checks that against the limits you have.

• Your traffic. Active users a day and requests per user a day give the day's volume. The peak factor says how much busier your busiest minute is than the day's average minute; the burst factor adds headroom on top, for requests that arrive together.
• One request. Input and output tokens per request, and — if you know it from your logs — how long one request takes.
• Your limits. Enter the limits your provider shows for your account and model: requests per minute, tokens per minute, input or output tokens per minute, and requests in flight at once. Fill in the ones you have. The rate-limit headers decoder reads them from a response's headers.

It shows the requests and tokens per minute to plan for, the requests in flight at once, and, for every limit you entered, how much of it you would use, whether it fits, and how many active users it carries.

Everything runs in your browser. Nothing you enter is sent anywhere.

How it computes

• Average requests per minute are requests per day ÷ 1,440.
• The busiest minute is that average × the peak factor; the rate to plan for is the busiest minute × the burst factor.
• Tokens per minute are that rate × the tokens in one request — input and output apart, and together, because providers state limits in either form.
• Requests in flight follow Little's law: the number in a system equals the rate at which requests arrive times how long each stays. At 10 requests a second, each taking 3 seconds, 30 are in flight on average.
• How many users a limit carries. Every need grows in step with your users, so a limit carries your users × the limit ÷ the need.
• The arithmetic is exact: factors are held in tenths and the latency in milliseconds, so a need that is a whole number is that number, and a need equal to your limit fits it.
• Needs are rounded up for display, because a need rounded down is a limit that is not quite enough. A need fits exactly when the number shown is within your limit.

What it does not know

• Your limits. They differ by provider, account, tier and model, and change over time. The tool starts with none and never fills one in.
• Your latency. It depends on the model, the prompt and how long the answer is. The tool starts without one; until you enter yours, the requests in flight are unknown.
• How your provider measures a limit. A provider may count cached input, or the most output a request allows, towards a limit differently from the tokens actually used. Its own documentation says how yours is measured.
• Your traffic's shape. The peak and burst factors start at 1, which assumes an even day with no bursts. Your logs are the source for real ones.
• It does not compute cost. The AI unit economics calculator does, for the same volume.

FAQ

Which limit should I enter?

Every one your provider states for your account and model, each in its own field; leave the others empty. Some providers state one limit for all tokens, others separate input and output. The decoder shows which headers your provider sends.

Does Salyro add a limit?

Each Salyro API key carries its own limit on requests per minute — see API keys; the decoder reads it from Salyro's headers. Your requests reach the provider through your own provider account, so that account's limits apply as well. Enter the lower of the two as the requests-per-minute limit.

Why is "requests in flight" unknown?

Because you have not entered how long one request takes. The tool has no latency of its own to assume.

Can I share the plan?

Yes. The link holds your numbers, compressed after the #, which your browser never sends to a server.

Sources

The arithmetic is stated above. Every limit and the latency are your own figures; the tool holds no provider's numbers.