Very interesting read! But I want to point out a small correction - the DNS collapse issue at HAProxy, along with O(N^2), also had some O(N^3) code paths, which is just mind-blowing.
> Production Lesson: Code that "works fine" at small scale may still hide O(N²) or worse behavior. At hundreds or thousands of nodes, those costs stop being theoretical and start breaking production.
> Even an improvement of a few second per iteration can compound into hours of time savings, both for you and your fellow contributors
Completely agree with this. I’ve recently been dealing with building few CI workflows, and honestly, I was very frustrated by how much time everything was taking. A lot of that frustration came from having nothing to do while waiting for the CI to pass, and I could clearly see how much time all of it was costing me.
Looking back, I realize that I’ve wasted far more time during my normal development workflow because of small inefficiencies here and there.
This has led me to believe that the human mind is much more prone to thinking linearly than in a compounding way.
This type of possibility really worries me. Archive.is is much closer to actual history in many ways. If the data there starts getting corrupted or biased, there’s no way to know if what was truly there.
The idea that the permanent record of the internet could hinge on the ethics of one stranger behind a server rack is deeply unsettling.
Also, I believe this should be the correct GitHub issue link - https://github.com/haproxy/haproxy/issues/1404
> Production Lesson: Code that "works fine" at small scale may still hide O(N²) or worse behavior. At hundreds or thousands of nodes, those costs stop being theoretical and start breaking production.