Martlet: A scientific work-flow language for abstracted parallelisation

Daniel Goodman

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    12 Downloads (Pure)

    Abstract

    This paper describes a work-flow language ‘Martlet’ for the analysis of large quantities of distributed data. This work-flow language is fundamentally different to other languages as it implements a new programming model. Inspired by inductive constructs of functional programming this programming model allows it to abstract the complexities of data and processing distribution. This means the user is not required to have any knowledge of the underlying architecture or how to write distributed programs. As well as making distributed resources available to more people, this abstraction also reduces the potential for errors when writing distributed programs. While this abstraction places some restrictions on the user, it is descriptive enough to describe a large class of problems, including algorithms for solving Singular Value Decompositions and Least Squares problems. Currently this language runs on a stand-alone middleware. This middleware can however be adapted to run on top of a wide range of existing work-flow engines through the use of JIT compilers capable of producing other work-flow languages at run time. This makes this work applicable to a huge range of computing projects.This paper describes a work-flow language ‘Martlet’ for the analysis of large quantities of distributed data. This work-flow language is fundamentally different to other languages as it implements a new programming model. Inspired by inductive constructs of functional programming this programming model allows it to abstract the complexities of data and processing distribution. This means the user is not required to have any knowledge of the underlying architecture or how to write distributed programs. As well as making distributed resources available to more people, this abstraction also reduces the potential for errors when writing distributed programs. While this abstraction places some restrictions on the user, it is descriptive enough to describe a large class of problems, including algorithms for solving Singular Value Decompositions and Least Squares problems. Currently this language runs on a stand-alone middleware. This middleware can however be adapted to run on top of a wide range of existing work-flow engines through the use of JIT compilers capable of producing other work-flow languages at run time. This makes this work applicable to a huge range of computing projects.
    Original languageEnglish
    Title of host publicationProceedings of the UK eScience All Hands Meeting 2006
    Publication statusPublished - Sept 2006
    EventUK eScience All Hands Meeting - Nottingham
    Duration: 1 Jan 1824 → …

    Conference

    ConferenceUK eScience All Hands Meeting
    CityNottingham
    Period1/01/24 → …

    Fingerprint

    Dive into the research topics of 'Martlet: A scientific work-flow language for abstracted parallelisation'. Together they form a unique fingerprint.

    Cite this