A Processing Pipeline for High Volume Pulsar Candidate Data Streams

Robert Lyon, Benjamin Stappers, Lina Levin, Mitchell Mickaliger, Anna Scaife

    Research output: Contribution to journalArticlepeer-review

    187 Downloads (Pure)


    Pulsar candidate analysis pipelines have historically been comprised of bespoke software systems, supporting the off-line study of data. However modern data acquisition systems are making off-line analyses impractical, as they output candidates in such large volumes that they become prohibitively expensive to retain. To maintain processing capabilities when off-line analysis becomes infeasible due to cost, requires a shift to on-line data processing. This paper makes four contributions facilitating this shift relevant to the search for radio pulsars: i) it characterises for the modern era, the key components of a pulsar search candidate processing pipeline, ii) it examines the feasibility of implementing on-line candidate ltering via existing tools, iii) problems preventing an easy transition to on-line ltering are identied and explained, and nally iv) it presents a new prototype ltering pipeline capable of overcoming such problems. Realised using Commercial o-the-shelf (COTS) software components, the deployable system is open source, simple, scalable, and cheap to produce. It has the potential to achieve candidate ltering design requirements for the Square Kilometre Array (SKA), illustrated via testing under simulated SKA loads.
    Original languageEnglish
    JournalAstronomy and Computing
    Early online date12 Jun 2019
    Publication statusPublished - 1 Jul 2019


    • pulsars: general
    • methods: data analysis
    • methods: statistical
    • techniques: miscellaneous


    Dive into the research topics of 'A Processing Pipeline for High Volume Pulsar Candidate Data Streams'. Together they form a unique fingerprint.

    Cite this