VADA: An Architecture for End User Informed Data Preparation

Nikolaos Konstantinou, Edward Abel, Luigi Bellomarini, Alex Bogatu, Cristina Civili, Endri Irfanie, Martin Koehler, Lacramioara Mazilu, Emanuel Sallinger, Alvaro Fernandes, Georg Gottlob, John Keane, Norman Paton

Research output: Contribution to journalArticlepeer-review


Background: Data scientists spend considerable amounts of time preparing data
for analysis. Data preparation is labour intensive because the data scientist
typically takes fine grained control over each aspect of each step in the process,
motivating the development of techniques that seek to reduce this burden.

Results: This paper presents an architecture in which the data scientist need
only describe the intended outcome of the data preparation process, leaving the
software to determine how best to bring about the outcome. Key wrangling
decisions on matching, mapping generation, mapping selection, format
transformation and data repair are taken by the system, and the user need only
provide: (i) the schema of the data target; (ii) partial representative instance data
aligned with the target; (iii) criteria to be prioritised when populating the target;
and (iv) feedback on candidate results. To support this, the proposed architecture dynamically orchestrates a collection of loosely coupled wrangling components, in which the orchestration is declaratively specied and includes self-tuning of component parameters.

Conclusion: This paper describes a data preparation architecture that has been
designed to reduce the cost of data preparation through the provision of a central role for automation. An empirical evaluation with deep web and open government data investigates the quality and suitability of the wrangling result, the cost-effectiveness of the approach, the impact of self-tuning, and scalability with respect to the numbers of sources.
Original languageEnglish
JournalJournal of Big Data
Early online date21 Aug 2019
Publication statusPublished - 2019


  • data preparation
  • data quality
  • data integration


Dive into the research topics of 'VADA: An Architecture for End User Informed Data Preparation'. Together they form a unique fingerprint.

Cite this