Retry Gemini API 429 and 503 Errors with Backoff
429 RESOURCE_EXHAUSTED and 503 UNAVAILABLE need different handling. A Python retry function with backoff and jitter, tested on a fake server.
Retrying 429 RESOURCE_EXHAUSTED and 503 UNAVAILABLE the same number of times wastes calls on requests that hit a daily quota. Split the two and back off only on errors that clear with waiting. For responses that come back empty with SAFETY, see empty text and finishReason.
[1] Start from the errors the docs separate. This is the table on the API errors page.
| HTTP | Code | Documented action |
|---|---|---|
| 429 | rate_limit_exceeded · too_many_requests | Wait and retry with exponential backoff |
| 429 | quota_exceeded (daily quota) | Wait until the quota resets or request a quota increase |
| 500 | api_error | Retry the request |
| 503 | service_unavailable | Wait and retry with exponential backoff |
| 504 | deadline_exceeded | Remove or increase the client deadline |
| 400 | invalid_request · failed_precondition | Check the request or billing (do not retry) |
The troubleshooting page says to retry only on transient errors such as 429, 408 or 5xx. A 400 stays a 400 however often you resend it.
[2] Treat 503 as unrelated to quota. The staff answer on the official forum says "503 errors are unrelated to your quota": temporary overload. Raising your limit will not reduce it, so a few short retries fit.
[3] Use RetryInfo when a 429 body carries it. A user log on the forum shows "retryDelay": "2970.938289688s" next to QUOTA_EXHAUSTED. No official text says clients must follow that field, so the code treats it as a hint: use it if present, otherwise back off. If the hint exceeds max_wait (60 s), re-raise instead of sleeping.
[4] Write the function. It uses the standard library only. The wait is a random value between 0 and min(cap, base × 2^n) (full jitter), so parallel workers do not retry in lockstep. sleep is a parameter so the tests do not actually wait.
import json, random, re, time, urllib.request, urllib.error RETRY_STATUS = {429, 500, 503, 504} def retry_delay(body): """RetryInfo.retryDelay ('2s', '2970.9s') in the error body -> seconds, else None.""" try: for d in json.loads(body)["error"].get("details", []): if str(d.get("@type", "")).endswith("RetryInfo"): m = re.fullmatch(r"([0-9.]+)s", d["retryDelay"]) return float(m.group(1)) if m else None except (ValueError, KeyError, TypeError, AttributeError): pass return None def call(url, payload, *, tries=5, base=1.0, cap=30.0, max_wait=60.0, sleep=time.sleep): data = json.dumps(payload).encode() for n in range(tries): req = urllib.request.Request(url, data, {"Content-Type": "application/json"}) try: with urllib.request.urlopen(req, timeout=30) as r: return json.load(r) except urllib.error.HTTPError as e: body = e.read().decode("utf-8", "replace") if e.code not in RETRY_STATUS or n == tries - 1: raise hint = retry_delay(body) if hint is not None and hint > max_wait: raise # server says wait longer than we will: daily quota etc. wait = hint if hint is not None else random.uniform(0, min(cap, base * 2 ** n)) print(f"attempt {n+1}: HTTP {e.code}, sleeping {wait:.2f}s") sleep(wait)[5] Test it against a fake server. http.server returns scripted responses shaped like Google's error body (error.status, details[].retryDelay). The shape follows the forum log above; no real server is called.
import json, threading, urllib.errorfrom http.server import BaseHTTPRequestHandler, HTTPServerimport retrySCRIPT = []class H(BaseHTTPRequestHandler): def do_POST(self): self.rfile.read(int(self.headers["Content-Length"])) code, details = SCRIPT.pop(0) if code == 200: body = {"candidates": [{"content": {"parts": [{"text": "ok"}]}}]} else: body = {"error": {"code": code, "message": "fake", "status": {429: "RESOURCE_EXHAUSTED", 503: "UNAVAILABLE", 400: "INVALID_ARGUMENT"}[code], "details": details}} b = json.dumps(body).encode() self.send_response(code); self.send_header("Content-Length", str(len(b))); self.end_headers(); self.wfile.write(b) def log_message(self, *a): passsrv = HTTPServer(("127.0.0.1", 18429), H); threading.Thread(target=srv.serve_forever, daemon=True).start()url = "http://127.0.0.1:18429/v1beta/models/m:generateContent"ri = lambda s: [{"@type": "type.googleapis.com/google.rpc.RetryInfo", "retryDelay": s}]def run(name, script, **kw): SCRIPT[:] = script; sleeps = [] print("==", name) try: r = retry.call(url, {}, sleep=sleeps.append, **kw); print("result:", r["candidates"][0]["content"]["parts"][0]["text"]) except urllib.error.HTTPError as e: print("raised HTTP", e.code) print("calls left unused:", len(SCRIPT))run("503 x2 then 200", [(503, []), (503, []), (200, [])])run("429 with retryDelay 2s then 200", [(429, ri("2s")), (200, [])])run("429 retryDelay 2970.9s (give up)", [(429, ri("2970.938289688s")), (200, [])])run("400 not retried", [(400, []), (200, [])])run("503 forever, tries=3", [(503, [])]*5, tries=3)[6] Run it. The sleep function just records values, so the times shown are the times it would have slept.
$ python3 test.py== 503 x2 then 200attempt 1: HTTP 503, sleeping 0.73sattempt 2: HTTP 503, sleeping 0.70sresult: okcalls left unused: 0== 429 with retryDelay 2s then 200attempt 1: HTTP 429, sleeping 2.00sresult: okcalls left unused: 0== 429 retryDelay 2970.9s (give up)raised HTTP 429calls left unused: 1== 400 not retriedraised HTTP 400calls left unused: 1== 503 forever, tries=3attempt 1: HTTP 503, sleeping 0.24sattempt 2: HTTP 503, sleeping 1.85sraised HTTP 503calls left unused: 2Two 503s then success; the 429 follows the server's 2 s; 2970 s is refused; the 400 is sent once; endless 503 stops at tries. The sleeping times include random jitter and differ per run.
[7] To use it for real, change url and payload and add your API key header to the Request as the docs describe. If you use an official SDK, the forum answer points to its retry settings, so check those before rolling your own. For bulk work that does not need an immediate answer, the same answer suggests the Batch API.