Наш трекер потерял одну посылку из пятидесяти из-за сбоев DNS
DEV · 22.11.2026 · 09:14
Каждое утро наш трекер запрашивает статусы посылок у четырех перевозчиков. С увеличением объемов запросов начали возникать ошибки временного характера при разрешении имен.
Оригинал
Our tracking poller lost one parcel in fifty to DNS failures and the resolver was refusing packets we never saw
Every morning between six and eight our tracking poller asks four carriers about the status of every parcel in transit, around two hundred thousand of them. It is a Python service on a single large EC2 instance, with many workers, and until October it opened a new HTTPS connection for every request, which meant a new DNS lookup every time. In October volumes went up and about one poll in fifty began failing with socket.gaierror, Temporary failure in name resolution. Our dashboards put the failu