Evaluating a meta-knowledge annotation scheme for bio-events

The correct interpretation of biomedical texts by text mining systems requires the recognition of a range of types of high-level information (or meta-knowledge) about the text. Examples include expressions of negation and speculation, as well as pragmatic/rhetorical intent (e.g. whether the information expressed represents a hypothesis, generally accepted knowledge, new experimental knowledge,etc.) Although such types of information have previously been annotated at the text-span level (most commonly sentences), annotation at the level of the event is currently quite sparse. In this paper, we focus on the evaluation of the multi-dimensional annotation scheme that we have developed specifically for enriching bio-events with meta-knowledge information. Our annotation scheme is intendedto be general enough to allow integration with different types of bio-event annotation, whilst being detailed enough to capture important subtleties in the nature of the metaknowledge expressed in the text. To our knowledge, our scheme is unique within the field with regards to the diversity of metaknowledgeaspects annotated for each event, whilst the evaluation results have confirmedits feasibility and soundness.