"maximum context length" error but tiktoken says it fits
tiktoken counts 73,878 tokens but the API says 249,114. Estimate the full request offline: system, tools, schema, history and reserved output.
This model's maximum context length is 128000 tokens. However, your messages resulted in 249114 tokens — in a real forum case the developer's own tiktoken count was 73,878 (thread). The two numbers usually measure different things.
[1] Check what the API counts. A forum reply says it is the total of the system message, tool and function definitions, the response-format schema and the chat history. The official function-calling guide says the same about tools:
Counting one user message with tiktoken will always come out low. A base64 image pasted into the text is counted as ordinary text, too.
[2] Pick the encoding for your model. Per the OpenAI cookbook table, gpt-4o and gpt-4o-mini use o200k_base; gpt-4-turbo, gpt-4 and gpt-3.5-turbo use cl100k_base. Ask tiktoken instead of memorising it.
$ python3 -m venv .venv && .venv/bin/pip install -q tiktoken$ .venv/bin/python -I -c "import tiktoken; print(tiktoken.encoding_for_model('gpt-4o').name)"o200k_base[3] Save a request shaped like the real one. Nothing is sent, so no API key is needed. This is a shortened example; dump your own messages, tools and response_format the same way. The script only counts messages whose values are strings; turn multi-part content or tool_calls into strings first, or it raises an error.
{ "model": "gpt-4o", "max_tokens": 4000, "messages": [ {"role": "system", "content": "You are a support assistant for Example Corp. Answer in the customer's language. Cite the order ID in every reply."}, {"role": "user", "content": "Where is my order 10234? It was supposed to arrive on Monday."}, {"role": "assistant", "content": "Let me check the shipping status for order 10234."}, {"role": "user", "content": "Please also tell me if I can change the delivery address."} ], "tools": [ {"type": "function", "function": { "name": "get_order_status", "description": "Look up the shipping status of an order by its numeric ID.", "parameters": {"type": "object", "properties": {"order_id": {"type": "integer", "description": "Numeric order ID"}}, "required": ["order_id"]}}} ], "response_format": {"type": "json_schema", "json_schema": {"name": "reply", "strict": true, "schema": {"type": "object", "properties": {"answer": {"type": "string"}, "order_id": {"type": "integer"}}, "required": ["answer", "order_id"], "additionalProperties": false}}}}[4] Write the counting script. The 3 tokens per message and 3 for reply priming follow the cookbook's num_tokens_from_messages. How the server turns tools and schemas into prompt text is not published, so those two lines count the JSON string — an approximation.
#!/usr/bin/env python3"""usage: count_budget.py request.json CONTEXT_WINDOW — estimate input + reserved output without calling the API"""import json, sysimport tiktoken req = json.load(open(sys.argv[1], encoding="utf-8"))window = int(sys.argv[2])enc = tiktoken.encoding_for_model(req["model"]) def count_messages(messages): # per-message overhead from the OpenAI cookbook (gpt-4o-2024-08-06 / gpt-4-0613 values) n = 0 for m in messages: n += 3 for k, v in m.items(): n += len(enc.encode(v)) if k == "name": n += 1 return n + 3 # reply priming def count_json(obj): # rough: the server's own serialisation of tools/schema is not published return len(enc.encode(json.dumps(obj, separators=(",", ":")))) body = sum(len(enc.encode(m["content"])) for m in req["messages"])msgs = count_messages(req["messages"])tools = count_json(req["tools"]) if "tools" in req else 0fmt = count_json(req["response_format"]) if "response_format" in req else 0reserve = req.get("max_tokens", 0)inp = msgs + tools + fmt print(f"encoding {enc.name}")print(f"message text {body}")print(f"messages (+fmt) {msgs}")print(f"tools (approx) {tools}")print(f"schema (approx) {fmt}")print(f"input estimate {inp}")print(f"max_tokens {reserve}")print(f"input + reserve {inp + reserve} / window {window}")print("input alone:", "OVER" if inp > window else "fits")print("with reserve:", "OVER" if inp + reserve > window else "fits", f"(headroom {window - inp - reserve})")[5] Run it with the model's context window as the second argument (take the number from the model's docs page). Here, 128,000 for gpt-4o.
$ .venv/bin/python -I count_budget.py request.json 128000encoding o200k_basemessage text 63messages (+fmt) 82tools (approx) 58schema (approx) 54input estimate 194max_tokens 4000input + reserve 4194 / window 128000input alone: fitswith reserve: fits (headroom 123806)Compare message text (63) with input estimate (194). Even this tiny request is nearly three times the message-only count.
[6] If you set a large max_tokens, check the reserve line too. Same request, pretending the window is 4,096:
$ .venv/bin/python -I count_budget.py request.json 4096 | tail -n 2input alone: fitswith reserve: OVER (headroom -98)The input fits alone but not together with the output you asked for. A forum reply says input and output share one window; no official sentence was found for it, so treat this line as headroom, not a rule. If it goes over, lower max_tokens or trim the history.
[7] When the script and the API disagree, the API is right. The cookbook says so itself: "Consider the counts from the function below an estimate, not a timeless guarantee." If the gap to your input estimate is large, look for an item you never counted — tools, schema or history.