The BEL Information Extraction Workflow (BELIEF): updates and evaluation

Ever-increasing scientific literature enhances our understanding on how toxicants impact biological systems. In order to utilize this information in the growing field of systems biology and toxicology, the published knowledge must be transformed into a structured format highly efficient for modelling, reasoning, and ultimately high throughput data analysis and interpretation. Consequently, there is an increasing demand from systems biologists and toxicologists to access such knowledge in a computable format, here biological network models. The Biological Expression Language (BEL) is a machine- and human-readable language that represents molecular relationships and events as semantic triples: subject–relationship–object. These triples are called BEL statements. BEL statements associated with their evidence are encapsulated into the large BEL document.  The BEL document also captures additional information such as article references, PMID of the scientific article and a large annotation dataset that accurately defines the context of knowledge such as the organism, tissue and disease state. BEL statements can computationally be assembled to biological network models. To facilitate encoding and curation of biological findings in BEL, a BEL Information Extraction workFlow (BELIEF) was developed. BELIEF contains a text mining pipeline for the automatic generation of BEL compliant knowledge statements and a web-based curation interface – the BELIEF Dashboard – that facilitates manual curation of the automatically generated BEL statements. The text mining pipeline is UIMA-based and accommodates several named entity recognition processes and relationship extraction methods in order to detect concepts and BEL relationships from any text resource. The BELIEF Dashboard reuses the output of the BELIEF pipeline to create a web-based interface and facilitate the manual curation task. Although BEL itself enables curation given the human-readability of the syntax, BELIEF simplifies the curation process by highlighting named entities and disambiguating gene and protein names. In addition to the new features of BELIEF, we also present the competitive performance results based on the BioCreative V BEL track evaluation. The BELIEF pipeline automatically extracts normalized concepts with the best F-score of 76.8%. The detection of full relationships and entirely correct statements was achieved with the F-score of 43.1% (second-best) and 30.8% (highest). The participation at the Interactive task (IAT) track in BioCreative V revealed a Systems Usability Scale (SUS) of 67. Given the complexity of the task for untrained users this score certifies a high usability for BELIEF. In conclusion, BELIEF simplifies the curation process and facilitates the construction of biological network models that can be fully contextualized and used for the interpretation of systems biology and systems toxicology data. This workflow is currently being further developed to be used in an industrial setup for product risk assessment.