Skip to content
Jobport

Member of Technical Staff (Site Reliability Engineer, Search Crawler)

Perplexity AI · London, United Kingdom 🇬🇧

On-siteFull-timeLead / ManagerSoftware & IT

Keeping a crawler running is not the same as keeping it productive. Queues can grow while servers look healthy, and a small change can become a fleet-wide bottleneck. On Perplexity's Crawler team, you'll improve throughput, plan capacity, and build operational reliability through safer deployments, better observability, and durable fixes.

Responsibilities

  • Operate and optimize Kubernetes workloads and supporting infrastructure for crawler services, including fetching and rendering fleets.

  • Build and maintain infrastructure as code and CI/CD pipelines, with safe rollouts, validation, and fast rollback.

  • Define SLOs and build monitoring, alerting, and observability for availability, latency, throughput, and resource saturation.

  • Own capacity planning and resource efficiency; diagnose queue backlogs, network failures, and resource bottlenecks with service owners.

  • Participate in the team's on-call rotation, lead incident response, and turn blameless postmortems into durable reliability improvements.

  • Write production software to automate diagnosis and routine remediation, with validation and approval gates for high-impact actions.

Qualifications

  • 5+ years of experience in SRE, production engineering, or infrastructure-heavy backend roles, with strong programming skills in Python, C++, Rust, or Go.

  • Hands-on experience with cloud infrastructure, production Kubernetes, infrastructure as code such as Terraform, and automated deployment workflows.

  • Strong Linux and networking fundamentals, including TCP/IP, DNS, TLS, HTTP, and load balancing.

  • Experience building observability with tools such as Datadog, Prometheus, or Grafana, and using SLOs to guide reliability improvements.

  • Proven experience diagnosing distributed-system failures, leading production incidents, and improving system performance or resource efficiency.

Nice to have

  • Experience operating web crawlers, search infrastructure, or browser fleets at scale.

  • Experience with distributed queues, backpressure, workload isolation, or multi-region recovery.

Posted 6d ago (13 Nov 2026) · Source: Perplexity AI via Ashby

Similar jobs in United Kingdom

ServiceNow · Staines, , United Kingdom 🇬🇧

On-siteFull-time

Company Description It all started when engineer Fred Luddy wrote code that automated a tedious task for his coworker, Phyllis. She cried tears of joy. That moment inspired Fred to build a company that could do that for everyone—freeing people from busywork so they could focus on

5h ago · ServiceNow via SmartRecruiters

n8n · London Office, United Kingdom 🇬🇧

RemoteFull-timeVisa sponsorship

The AI orchestration of your wildest imagination. n8n is the open workflow orchestration platform built for the new era of AI. We give technical teams the freedom of code with the speed of no-code, so they can automate faster, smarter, and without limits. Backed by a fiercely inv

5h ago · n8n via Ashby

Genomics England · London, England, United Kingdom 🇬🇧

HybridFull-time

Company Description Genomics England is a global leader in enabling genomic medicine and research, focused on creating a world where everyone benefits from genomic healthcare. Building on the 100,000 Genomes Project, we support the NHS’s world-first national whole genome sequenci

6h ago · Genomics England via SmartRecruiters

Exegy · Belfast, United Kingdom 🇬🇧

HybridFull-time

About Exegy Exegy is a global leader in intelligent market data, advanced trading systems, and future-proof technology. Exegy serves as a trusted partner to the complete ecosystem of the buy-side, sell-side, exchanges, and financial services technology firms around the globe. Hea

7h ago · Exegy via Ashby