Efficient Feedback Collection for Pay-as-you-go Source Selection

Julio César Cortés Ríos, Norman Paton, Alvaro Fernandes, Khalid Belhajjame

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    194 Downloads (Pure)


    Technical developments, such as the web of data and web
    data extraction, combined with policy developments such as
    those relating to open government or open science, are lead-
    ing to the availability of increasing numbers of data sources.
    Indeed, given these physical sources, it is then also possible
    to create further virtual sources that integrate, aggregate or
    summarise the data from the original sources. As a result,
    there is a plethora of data sources, from which a small subset
    may be able to provide the information required to support a
    task. The number and rate of change in the available sources
    is likely to make manual source selection and curation by
    experts impractical for many applications, leading to the
    need to pursue a pay-as-you-go approach, in which crowds
    or data consumers annotate results based on their correct-
    ness or suitability, with the resulting annotations used to
    inform, e.g., source selection algorithms. However, for pay-
    as-you-go feedback collection to be cost-eective, it may be
    necessary to select judiciously the data items on which feed-
    back is to be obtained. This paper describes OLBP (Order-
    ing and Labelling By Precision), a heuristics-based approach
    to the targeting of data items for feedback to support map-
    ping and source selection tasks, where users express their
    preferences in terms of the trade-o between precision and
    recall. The proposed approach is then evaluated on two
    dierent scenarios, mapping selection with synthetic data,
    and source selection with real data produced by web data
    extraction. The results demonstrate a signicant reduction
    in the amount of feedback required to reach user-provided
    objectives when using OLBP.
    Original languageEnglish
    Title of host publicationInternational Conference on Scientific and Statistical Database Management (SSDBM) July 18-20, 2016, Budapest, Hungary SSDBM ’16, July 18-20, 2016, Budapest, Hungary
    Publication statusPublished - 2016


    Dive into the research topics of 'Efficient Feedback Collection for Pay-as-you-go Source Selection'. Together they form a unique fingerprint.

    Cite this