A cluster of 403s that looked like an auth bug was actually a traffic-burst limiter

There's a specific kind of debugging session that makes you feel like you're losing your mind. You watch a credential that worked five minutes ago get rejected. You re-read your auth code for the third time. Everything looks fine. It IS fine. The bug is somewhere else entirely.

That's exactly what happened to me a few weeks ago, and I want to walk through it because the lesson is one I think most engineers only learn by getting burned.

The Setup: 403s That Made No Sense

I was load testing a client integration. Nothing exotic. Real credentials, real endpoints, a flow I'd already verified end-to-end multiple times at low volume.

Once concurrency climbed to around 25 simultaneous requests from the same client, a chunk of them started coming back 403. Not 429. Not 503. 403. Forbidden.

That's the "you're not allowed" status code. It means auth. Every instinct I've built over years of working with APIs said to go look at the auth layer.

So I did. I:

• Checked session token lifecycle and expiry logic
• Verified the credential rotation wasn't creating a race condition
• Re-read the authorization middleware twice
• Wrote a small harness to replay the exact failing requests one at a time

And one at a time? They all worked. Obviously.

What Actually Happened

Here's the thing about 403. The HTTP spec says it means the server understood your request and won't fulfill it. That's true. But it says nothing about WHICH server generated it, or where in the stack the decision was made.

There was an edge layer sitting in front of the application. A traffic burst protection mechanism. When it saw roughly 25 concurrent requests from the same origin arrive in a tight window, it flagged the traffic, blocked the requests, and returned 403.

The application never saw those requests. Never logged them. Never touched them. As far as my application code was concerned, those requests didn't exist.

The edge layer's own logs told a completely different story. Blocked. Burst threshold exceeded. 403 returned by the edge, not the app.

Two different systems. Same status code. Zero indication from the status code alone which one fired.

How I Actually Separated Them

A few things made the real cause visible once I stopped assuming the code I wrote was the code running.

The app logs had gaps. Requests I could see hitting the edge layer weren't showing up in the application logs at all. If it were an auth bug, the app would have at least seen the request and rejected it. Silence on the app side was the tell.

Response headers were different. The blocked responses from the edge had headers the application would never set. Small detail. Easy to miss if you're skimming. Worth looking at closely when something doesn't add up.

The failure pattern was load-shaped, not credential-shaped. Auth bugs tend to fail consistently or fail based on credential state. This one only appeared above a concurrency threshold. That shape doesn't fit an auth bug. It fits a rate limit.

The Fix Was Completely Different From What I Almost Did

If I'd kept chasing the auth hypothesis, I would have spent hours hardening session management and token handling that had nothing wrong with them. The fix had nothing to do with auth:

• Add client-side retry logic with exponential backoff so burst failures recover gracefully
• Batch or stagger concurrent requests to stay under the burst threshold
• And on the infrastructure side, evaluate whether the burst threshold itself needs adjusting for legitimate high-concurrency clients

None of those live in auth code. Not one.

The Actual Lesson

A status code names a category of failure. It does not name a location in your stack.

403 means "forbidden" at the layer that generated it. It does not mean your authorization middleware is broken. Your session logic is fine too. It means something, somewhere, decided not to fulfill the request. Finding out WHAT generated it is the first job. Not the second job. The first one.

The reflex of pattern-matching a status code directly to a layer of your own code is fast and usually fine. Until it isn't, and then it sends you in completely the wrong direction for a long time.

Check what actually returned the response before you decide what's broken. Your logs, your headers, and the gap between what hit the edge and what hit the app will usually tell you.