/q/ - Q&A Central

the recent uptick in downstream timeouts is making our whole architecture feel brittle. every time the auth service undergoes a routine deployment, we see a massive wave of cascading failures across the gateway. it seems like our current retry logic is actually making the situation worse by creating a thundering herd effect on the database. i tried implementing an exponential backoff using backoff. full_jitter(factor=0.5, max_delay=10) but the downstream services are still struggling to recover. we might need to move toward a more robust circuit breaker pattern to prevent these total outages.
>everything is failing because of one tiny dependency
is anyone else seeing this level of instability with their sidecar proxies lately? i suspect the issue lies in how we handle the lack of proper request hedging during high traffic periods. let me know if u have found a way to stabilize these connections without adding massive overhead to the network.