Dealing with Under-reported Variables: An Information Theoretic Solution

Konstantinos Sechidis, Matthew Sperrin, Emily S Petherick, Mikel Lujan, Gavin Brown

Research output: Contribution to journalArticlepeer-review


Under-reporting occurs in survey data when there is a reason for participants to give a false negative response to a question, e.g. maternal smoking in epidemiological studies. Failing to correct this misreporting introduces biases and it may lead to misinformed decision making. Our work provides methods of correcting for this bias, by reinterpreting it as a missing data problem, and particularly learning from positive and unlabelled data. Focusing on information theoretic approaches we have three key contributions: (1) we provide a method to perform valid independence tests with known power by incorporating prior knowledge over misreporting; (2) we derive corrections for point/interval estimates of the mutual information that capture both relevance and redundancy; and finally, (3) we derive different ways for ranking under-reported risk factors. Furthermore, we show how to use our results in real-world problems and machine learning tasks.
Original languageEnglish
Pages (from-to)159-177
Number of pages19
JournalInternational Journal of Approximate Reasoning
Early online date7 Apr 2017
Publication statusPublished - Jun 2017


Dive into the research topics of 'Dealing with Under-reported Variables: An Information Theoretic Solution'. Together they form a unique fingerprint.

Cite this