All jobs
Staff Software Engineer (Agentic Search) - Crawler
Nebius
Amsterdam · On-siteSenior (5–10 years)Full-timePosted 15 days ago
About the job
What you'll do
Design, build, and operate internet-scale crawling and ingestion systems that discover, fetch, prioritize, and refresh content from the open web and other data sources. Ensure reliable, cost-efficient, observable content acquisition and collaborate with indexing and machine learning teams to meet retrieval and ranking needs.
What they're looking for
Requires 8 or more years of backend or distributed systems experience, strong Go or C++ expertise, and experience with large-scale systems and production operations. Candidates should understand web protocols, crawling, scraping, content extraction, scalability, fault tolerance, and resource management.
Full description