Distributed Web Crawler Engineer

Remotedxb in Dubai is seeking an experienced infrastructure engineer to design and maintain large-scale web crawlers and data pipelines. You will tackle high-throughput systems, handle anti-bot defenses, and optimize for performance and reliability.

You should be proficient in Go, Rust, Python, Java, or C++, familiar with HTTP, networking, and distributed systems, and able to work with big data and cloud or bare-metal infrastructure.