Targeted feedback collection applied to multi-criteria source Selection

Julio César Cortés Ríos, Norman W. Paton, Alvaro A.A. Fernandes, Edward Abel, John A. Keane

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    149 Downloads (Pure)


    A multi-criteria source selection (MCSS) scenario identifies, from a set of candidate data sources, the subset that best meets a user’s needs. These needs are expressed using several criteria, which are used to evaluate the candidate data sources. A MCSS problem can be solved using multi-dimensional optimisation techniques that trade-off the different objectives. Sometimes we may have uncertain knowledge regarding how well the candidate data sources meet the criteria. In order to overcome this uncertainty, we may rely on end users or crowds to annotate the data items produced by the sources in relation to the selection criteria. In this paper, we introduce an approach called Targeted Feedback Collection (TFC), which aims to identify those data items on which feedback should be collected, thereby providing evidence on how the sources satisfy the required criteria. TFC targets feedback by considering the confidence intervals around the estimated criteria values. The TFC strategy has been evaluated, with promising results, against other approaches to feedback collection, including active learning, using real-world data sets.

    Original languageEnglish
    Title of host publicationAdvances in Databases and Information Systems - 21st European Conference, ADBIS 2017, Proceedings
    PublisherSpringer Nature
    Number of pages15
    Volume10509 LNCS
    ISBN (Print)9783319669168
    Publication statusPublished - 2017
    Event21st European Conference on Advances in Databases and Information Systems, ADBIS 2017 - Nicosia, Cyprus
    Duration: 24 Sept 201727 Sept 2017

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume10509 LNCS
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349


    Conference21st European Conference on Advances in Databases and Information Systems, ADBIS 2017


    • Data integration
    • Feedback collection
    • Multi-objective optimisation
    • Pay-as-you-go
    • Source selection


    Dive into the research topics of 'Targeted feedback collection applied to multi-criteria source Selection'. Together they form a unique fingerprint.

    Cite this