Methods Included: Standardizing Computational Reuse and Portability with the Common Workflow Language

Michael R. Crusoe, Sanne Abeln, Alexandru Iosup, Peter Amstutz, John Chilton, Nebojša Tijanić, Hervé Ménager, Stian Soiland-Reyes, Bogdan Gavrilović, Carole A. Goble, The CWL Community (Collaborator)

Research output: Contribution to journalArticlepeer-review

172 Downloads (Pure)

Abstract

Computational Workflows are widely used in data analysis, enabling innovation and decision-making. In many domains (bioinformatics, image analysis, & radio astronomy) the analysis components are numerous and written in multiple different computer languages by third parties. However, many competing workflow systems exist, severely limiting portability of such workflows, thereby hindering the transfer of workflows between different systems, between different projects and different settings, leading to vendor lock-ins and limiting their generic re-usability. Here we present the Common Workflow Language (CWL) project which produces free and open standards for describing command-line tool based workflows. The CWL standards provide a common but reduced set of abstractions that are both used in practice and implemented in many popular workflow systems. The CWL language is declarative, which allows expressing computational workflows constructed from diverse software tools, executed each through their command-line interface. Being explicit about the runtime environment and any use of software containers enables portability and reuse. Workflows written according to the CWL standards are a reusable description of that analysis that are runnable on a diverse set of computing environments. These descriptions contain enough information for advanced optimization without additional input from workflow authors. The CWL standards support polylingual workflows, enabling portability and reuse of such workflows, easing for example scholarly publication, fulfilling regulatory requirements, collaboration in/between academic research and industry, while reducing implementation costs. CWL has been taken up by a wide variety of domains, and industries and support has been implemented in many major workflow systems.
Original languageEnglish
Pages (from-to)54–63
Number of pages10
JournalCommunications of the ACM
Volume65
Issue number6
DOIs
Publication statusPublished - 20 May 2022

Keywords

  • workflows
  • computational data analysis
  • CWL
  • scientific workflows
  • standards

Fingerprint

Dive into the research topics of 'Methods Included: Standardizing Computational Reuse and Portability with the Common Workflow Language'. Together they form a unique fingerprint.
  • Implementing FAIR Digital Objects in the EOSC-Life Workflow Collaboratory

    Goble, C., Soiland-Reyes, S., Bacall, F., Owen, S., Williams, A., Eguinoa, I., Droesbeke, B., Leo, S., Pireddu, L., Rodríguez-Navas, L., Fernández, J. M., Capella-Gutierrez, S., Ménager, H., Grüning, B., Serrano-Solano, B., Ewels, P. & Coppens, F., 1 Mar 2021, In: Zenodo.

    Research output: Contribution to journalArticle

    Open Access
  • CWLProv - Interoperable Retrospective Provenance capture and its challenges

    Khan, F. Z., Soiland-Reyes, S., Crusoe, M. R., Lonie, A. & Sinnott, R., 27 Mar 2018, (Submitted). 14 p.

    Research output: Contribution to conferencePaperpeer-review

    File
    29 Downloads (Pure)
  • common-workflow-language/cwltool

    Amstutz, P., Crusoe, M. R., Singh, M., Kumar, K., Chilton, J., [Unknown], B., Soiland-Reyes, S., Chapman, B., Kotliar, M., Leehr, D., Carrasco, G., Kartashov, A., Tijanic, N., Ménager, H., Safont, P. R., Porter, J. J., Molenaar, G., Yuen, D., Barrera, A. & Ivkovic, S. & 10 others, Spangler, R., [Unknown], P.-I., Tanjo, T., Vandewege, W., Randall, J. C., Kern, J., Bradley, J., Li, J., der Zwaan, J. V. & Connelly, A., 2017

    Research output: Non-textual formSoftware

    Open Access

Cite this