Tell the cause of an Nginx 502 Bad Gateway from error.log
The access log only says 502. Match four error.log messages to their causes and the command that confirms each, and tell them apart from a 504.
The access log shows the 502 but not the reason; the reason is in the error.log message. The lines below are real logs from Nginx on port 18502 with deliberately broken backends. To keep them short, the leading timestamp and PID and the middle client:/server: fields are replaced with …. To count which paths return 5xx, see this post.
[1] Open error.log around the time of the 502. Match the access-log timestamp and the request: path. Find the file with nginx -T.
# nginx -T 2>/dev/null | grep error_log# tail -n 20 /var/log/nginx/error.log[2] connect() failed (111: Connection refused) while connecting to upstream means nothing listens on that address and port. Check the upstream: value with ss. A header with no rows means the backend is not running.
… [error] … *1 connect() failed (111: Connection refused) while connecting to upstream, … request: "GET /refused/ HTTP/1.1", upstream: "http://127.0.0.1:18591/", …$ curl -i http://127.0.0.1:18502/refused/HTTP/1.1 502 Bad Gateway# ss -tlnp 'sport = :18591'State Recv-Q Send-Q Local Address:Port Peer Address:Port Process[3] upstream prematurely closed connection while reading response header from upstream means the connection succeeded but the backend closed it before sending response headers. The port is open, so ss lists it. Look in the backend log for the same time: a crash, restart, or out-of-memory kill.
… [error] … *3 upstream prematurely closed connection while reading response header from upstream, … request: "GET /closed/ HTTP/1.1", upstream: "http://127.0.0.1:18593/", …# ss -tlnp 'sport = :18593'State Recv-Q Send-Q Local Address:Port Peer Address:Port ProcessLISTEN 0 5 127.0.0.1:18593 0.0.0.0:* users:(("python3",pid=48599,fd=3))[4] no live upstreams while connecting to upstream means every server in the upstream block is marked failed, so Nginx did not even try to connect. Below, the first request to a pool whose two servers are both down (one connect() failed per server), then the next request right after.
… [error] … *7 connect() failed (111: Connection refused) while connecting to upstream, … upstream: "http://127.0.0.1:18591/", …… [error] … *7 connect() failed (111: Connection refused) while connecting to upstream, … upstream: "http://127.0.0.1:18592/", …… [error] … *12 no live upstreams while connecting to upstream, … request: "GET /pool/ HTTP/1.1", upstream: "http://dead_pool/", …[5] upstream timed out (110: Connection timed out) is a 504 Gateway Time-out, not a 502. The backend is alive but slow, so find out why it is slow before raising proxy_read_timeout.
… [error] … *5 upstream timed out (110: Connection timed out) while reading response header from upstream, … request: "GET /slow/ HTTP/1.1", upstream: "http://127.0.0.1:18594/", …$ curl -i http://127.0.0.1:18502/slow/HTTP/1.1 504 Gateway Time-out[6] Using proxy_pass https:// against a plain-HTTP backend gives a 502 and the message below. Make the proxy_pass scheme match what the backend speaks.
… [error] … *10 SSL_do_handshake() failed (SSL: error:0A00010B:SSL routines::wrong version number) while SSL handshaking to upstream, … upstream: "https://127.0.0.1:18595/", …[7] Summary by message:
| Message | status | what to check |
|---|---|---|
connect() failed (111: Connection refused) | 502 | no row in ss -tlnp 'sport = :PORT' · proxy_pass address and port · backend service |
upstream prematurely closed connection | 502 | backend log for the same time · ss still lists it |
no live upstreams | 502 | per-server connect() failed lines · max_fails and fail_timeout |
SSL_do_handshake() failed … wrong version number | 502 | proxy_pass scheme (http:// or https://) |
upstream timed out (110: …) | 504 | slow backend response · proxy_read_timeout |
Analyze your Nginx log in the browser ↗ — nothing is uploaded.