Skip to content
Jobport

Member of Technical Staff (Index, Search Crawler)

Perplexity AI · Dublin, Ireland 🇮🇪

On-siteFull-timeLead / ManagerOther

Skills mentioned

AI answers are only as trustworthy as the information behind them. Perplexity grounds its answers in fresh, real-world sources, and the index is what makes that possible. We're looking for an engineer to work on the indexing and data processing layer of Perplexity's Search Crawler. You'll build systems that ingest, process, and organize large volumes of crawled content into efficient, queryable structures, working primarily on offline and nearline pipelines that keep data fresh, complete, and accessible to downstream retrieval. Throughput, scalability, and correctness are critical across datasets that change continuously.

Responsibilities

  • Design, implement, and operate large-scale indexing systems and data pipelines at the core of Perplexity's search infrastructure.

  • Develop and optimize indexing strategies that balance performance, freshness, and resource efficiency.

  • Work on storage formats, compaction, and update mechanisms that keep data accessible and current.

  • Keep pipelines reliable and predictable under high-throughput conditions.

  • Build well-tested components with clear responsibilities and interface contracts that stay flexible as the system evolves.

  • Define and implement observability across offline and near line pipelines, including structured logs, metrics, and data quality signals.

  • Monitor throughput, resource usage, and cost, and drive optimisations as business needs change.

  • Partner with retrieval, ranking, and ML teams to make sure indexing outputs meet their requirements.

  • Enable safe experimentation on indexing strategies and processing logic through controlled rollouts and clearly defined quality signals.

Qualifications

  • 5+ years of experience building production backend or data infrastructure systems.

  • Strong programming experience in Go, C++, or Rust.

  • Experience with high-throughput data processing systems and very large datasets.

  • Experience building or operating databases, storage systems, data planes, or indexing pipelines.

  • Strong understanding of distributed systems, fault tolerance, consistency, and scalability.

  • Experience running production systems and handling operational incidents.

  • A systems-thinking mindset and the ability to reason about end-to-end data flows.

Posted 2d ago (17 Nov 2026) · Source: Perplexity AI via Ashby

Similar jobs in Ireland

AECOM · Dublin, , Ireland 🇮🇪

HybridFull-time

Company Description Work with Us. Change the World. At AECOM, we're delivering a better world. Whether improving your commute, keeping the lights on, providing access to clean water, or transforming skylines, our work helps people and communities thrive. We are the world's truste

9h ago · AECOM via SmartRecruiters

Version 1 · Dublin, County Dublin, Ireland 🇮🇪

HybridFull-time

Company Description Version 1 has celebrated 30 years in business and continues to be trusted by global brands to deliver technology and transformation solutions that drive customer success. Our deep expertise enables our customers to

11h ago · Version 1 via SmartRecruiters

Charlotte Tilbury · Cork, County Cork, Ireland 🇮🇪

On-sitePart-time

Beauty Expert, Cork Airport (New Counter) Part Time available - 15 hours per week About Charlotte Tilbury Beauty Founded by British makeup artist and beauty entrepreneur Charlotte Tilbury MBE in 2013, Charlotte Tilbury Beauty has revolutionised the face of the global beauty indus

1d ago · Charlotte Tilbury via Workable

Charlotte Tilbury · Kildare, County Kildare, Ireland 🇮🇪

On-siteFull-time

Beauty Expert - Brown Thomas, Kildare Village Part Time - Permanent About Charlotte Tilbury Beauty Founded in 2013 by world-renowned makeup artist and beauty entrepreneur Charlotte Tilbury CBE, Charlotte Tilbury Beauty has transformed the global beauty industry with one mission:

1d ago · Charlotte Tilbury via Workable