Identifying correct and incorrect statements is a problem that has been widely explored in the literature using various techniques ranging from pure information retrieval to crowdsourcing, and used in many applications such as fact-checking in journalism and the support of biomedical evidence from the literature.
We review below the major work related to our paper. Several approaches have been recently proposed to check the truthfulness of facts, particularly in the context of journalism [ 71 ].
Most of the methods proposed assume the availability of structured knowledge modeled as a graph, which is then used to check the consistency of facts.
Hence, recent works addressed this problem as a graph search problem [ 72 , 73 ], a link prediction problem [ 74 ], or a classification and ranking problem [ 75 ]. However, the aforementioned methods rely on the availability of a knowledge graph, which limits and reduces the scope and the impact of any method as: it requires a specific knowledge graph for each domain eventually sub-domain , and may fail in real-time assessment of facts, as it requires continuous update of the knowledge graph, a task generally done by domain-specialists.
BARC may overcome these two limitations as it uses a standard index, which can easily be updated to address new facts. In the context of biocuration, the information retrieval task that consists of finding relevant articles for the curation of a biological database is called triage [ 76 — 80 ]. Triage can be seen as a semi-automated method for assessing biological statements; it assists curators in the selection of relevant articles, which then must be manually reviewed to decide their correctness.
As mentioned in the Introduction, this triage process is time-consuming and expensive [ 4 , 5 ]; curation of a single protein may take up to a week and requires considerable human investment both in terms of knowledge and effort [ 6 ]. Rather, BARC is a fully automated method that directly helps biocurators to assess biological database assertions using the scientific literature. Also, in the biomedical context, Light et al. However, it is unclear how this method generalizes to relations that are mentioned in both speculative and non-speculative sentences.
Leach et al. This method combines and integrates multiple sources of data into a knowledge network through a reading and a reasoning component; as it requires complex mappings to identify and link entities in order to get a consistent knowledge graph, it may fail to scale to large datasets.
Zerva et al. The method is based on using a hybrid approach that combines rule induction and machine learning. This method focuses on biomedical pathways and uses features specifically designed to detect uncertain interactions. However, its generalisation for assessment other statements is unclear and is not discussed in the paper. Several other papers [ 84 , 85 ] have focused on extracting relations from text using distant supervision multi-instance learning to reduce the amount of manual effort for labeling.
Such work demonstrates that the approach can be successfully used to extract relations from literature about a biological process with little or no manually annotated corpus data. This method might be complementary to BARC; it may allow the extraction of new relevant features that characterize the relationships between entities.
Note that BARC is capable of overcoming all the drawbacks mentioned for these approaches, as it does not require any data integration or complex mappings and it has been designed in such a way to assess multiple types of statements. Crowdsourcing has attracted interest in the bioinformatics domain for data annotation [ 86 ]. It has been successfully applied in the biomedical and clinical domains [ 87 ].
This research has demonstrated that crowdsourcing is an inexpensive, fast, and practical approach for collecting high quality annotations for different BioIE tasks [ 88 ], including NER in clinical trial documents [ 89 ], disease mention annotation in PUBMED literature [ 90 ], relation extraction between clinical problems and medications [ 91 ], etc.
Different techniques have been explored to improve the quality and effectiveness of crowdsourcing, including probabilistic reasoning [ 92 ] to make sensible decisions on annotation candidates and gamification strategies [ 93 ] to motivate the continuous involvement of the expert crowd.
More recently, a method called CrowdTruth [ 94 ] was proposed for collecting medical ground truth through crowdsourcing, based on the observation that disagreement analysis on crowd annotations can compensate lack of medical expertise of the crowd. Experiments with the use of CrowdTruth for a medical relation extraction task show that the crowd performs just as well as medical experts in terms of quality and efficacy of annotation, and also indicate that at least 10 workers per sentence are needed to get the highest quality annotation for this task.
Crowdsourcing holds promise for biocuration tasks as well and could be combined with prioritisation methods such as BARC provides. In the context of information retrieval, despite the development of sophisticated techniques for helping users to express their needs, many queries still remain unanswered [ 95 ]. Hence, to tackle these issues, other types of question-answering QA systems have emerged to allow people to help each other to answer questions [ 96 ]. Community QA systems provide a means for answering several types of questions such as recommendation, opinion seeking, factual knowledge, or problem solving [ 96 ].
A challenge facing such systems is in the response time as well as the quality of the answers [ 95 , 97 — 99 ]. However, the time factor is critical in the context of biocuration as there is a huge quantity of information to be curated. Such waiting times to answer questions probably disqualify this body of work for their usage in biocuration. These methods that use unstructured knowledge that is, the literature for fact-checking and query answering address the problem as a pure information retrieval problem.
In other words, these methods rank documents or sentences relevant to a given information need, and it is up to the user to read and search for a support to a given assertion or question. In contrast, BARC allows spotting of potentially incorrect assertions, and presents them to the user with, in our experiments, a high classification accuracy. We have described BARC, a tool that aims to help biocurators in checking the correctness of biological relations using the scientific literature.
Specifically, given a new biological assertion represented as a relation, BARC first retrieves a list of documents that may be used to check the correctness of that relation.
This retrieval step is done using a set-based retrieval algorithm SaBRA. This list of documents is aggregated in order to compute features for the relation as a whole, which are used for a prediction task.
We evaluated BARC and retrieval algorithm SaBRA using publicly available datasets including the PubMed Central collection and two types of relational assertions, gene—disease relations, and protein—protein interactions. The results obtained showed that BARC substantially outperforms the best baselines, with an improvement of F-measure of 3. A limitation of this work is that it relies on the accuracy of the GNormPlus [ 31 ] and DNorm [ 32 ] entity recognition tools; these are automated tools and hence subject to error.
We note further that in this paper BARC has been evaluated on only two kinds of relations; we leave the analysis of its generalization to future work, such as drug-disease or drug-drug interactions. However, the results show that the methods are effective enough to be used in practice, and we believe they can provide a valuable tool for supporting biocurators. Atopy refers to the genetic tendency to develop allergic diseases such as allergic rhinitis, asthma and atopic dermatitis eczema.
We used word vectors induced from PMC by Pyysalo et al. Baxevanis AD, Bateman A. The importance of biological databases in biological discovery. Curr Protocol Bioinforma. Article Google Scholar. Bateman A. Curators of the world unite: the international society of biocuration.
Database resources of the national center for biotechnology information. Nucleic Acids Res. Expert curation in UniProtKB: a case study on dealing with conflicting and erroneous data.
The UniProt Consortium. UniProt: the universal protein knowledgebase. Biological databases for human research. Genom Proteomics Bioinforma. A classification of biological data artifacts.
Duplicates, redundancies and inconsistencies in the primary nucleotide databases: a descriptive study. Baumgartner Jr. WA, K. Manual curation is not sufficient for annotation of genomic databases. Ten simple rules for developing public biological databases. PLoS Comput Biol. Automated detection of records in biological sequence databases that are inconsistent with the literature.
J Biomed Inform. PubMed Article Google Scholar. Literature consistency of bioinformatics sequence databases is effective for assessing record quality. PubMed Central Google Scholar. Learning biological sequence types using the literature. New York: ACM: Google Scholar. Human genotype-phenotype databases: aims, challenges and opportunities. Nat Rev Genet. Prosite, a protein domain database for functional characterization and annotation. Binding moad, a high-quality protein—ligand database.
Stanley Fields, and Peer Bork. Comparative assessment of large-scale data sets of protein-protein interactions. Hu G, Agarwal P. Human disease-drug network based on genomic expression profiles. BMC Bioinformatics. Commun ACM. Okapi at trec In: TREC. Gaithersburg: NIST: Pivoted document length normalization.
Integrating co-occurrence statistics with information extraction for robust retrieval of protein interactions from medline.
In: Proceedings of the workshop on linking natural language processing and biology: towards deeper biological literature analysis.
Comparative analysis of five protein-protein interaction corpora. Evaluating similarity measures for emergent semantics of social tagging. Wang X, Zhai C. Mining term association patterns from search logs for effective query reformulation. Zhai C, Lafferty J.
A study of smoothing methods for language models applied to ad hoc information retrieval. Cortes C, Vapnik V. Support-vector networks. Mach Learn. Libsvm: A library for support vector machines. GNormPlus: An integrative approach for tagging genes, gene families, and protein domains. BioMed Res Int. DNorm: disease name normalization with pairwise learning to rank. Bouadjenek MR, Verspoor K.
Multi-field query expansion is effective for biomedical dataset retrieval. Diseases: Text mining and data integration of disease-gene associations.
Text mining of biomedical literature. BioGRID: a general repository for interaction datasets. Quinlan JR. Association for Computational Linguistics: Literature mining for the biologist: from information retrieval to biological discovery. Gene name ambiguity of eukaryotic nomenclatures. Toward information extraction: identifying protein names from biological papers. In: Pac symp biocomput, vol. Settles B. Abner: an open source tool for automatically tagging genes, proteins and other entity names in text.
Prominer: rule-based protein and gene entity recognition. Resolving abbreviations to their senses in medline. Relation Extraction: A Survey. ArXiv e-prints. Bach N, Badaskar S. A review of relation extraction.
Technical report: Carnegie Mellon University; Kambhatla N. Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations. Exploring various knowledge in relation extraction. Zhao S, Grishman R. Extracting relations with integrated information using kernel methods.
Stroudsburg: Association for Computational Linguistics: Simple algorithms for complex relation extraction with applications to biomedical ie. Subsequence kernels for relation extraction. Cambridge: MIT Press: Collins M, Duffy N. Convolution kernels for natural language. Kernel methods for relation extraction. J Mach Learn Res. Exploiting graph kernels for high performance biomedical relation extraction.
J Biomed Semant. Asm kernel: Graph kernel using approximate subgraph matching for relation extraction. Exploiting tree kernels for high performance chemical induced disease relation extraction. BioMed Central: Relation classification via convolutional deep neural network.
Nguyen TH, Grishman R. Relation extraction: Perspective from convolutional neural networks. Distant supervision for relation extraction via piecewise convolutional neural networks.
Neural relation extraction with selective attention over instances. Relation extraction with multi-instance multi-label convolutional neural networks. Incorporating relation paths in neural relation extraction. Huang Y, Wang WY. Deep residual learning for weakly-supervised relation extraction. Denmark: ACL: Miwa M, Bansal M. End-to-end relation extraction using lstms on sequences and tree structures.
End-to-end neural relation extraction with global optimization. Katiyar A, Cardie C. Going out on a limb: Joint extraction of entity mentions and relations without dependency trees. The ai2 system at semeval task 10 scienceie : semi-supervised end-to-end entity and relation extraction.
Combining neural networks and log-linear models to improve relation extraction. Learning local and global contexts using a convolutional recurrent network model for relation classification in biomedical text. Nguyen DQ, Verspoor K. Convolutional neural networks for chemical-disease relation extraction are improved with character-based word embeddings. Vlachos A, Riedel S. Fact checking: Task definition and dataset construction.
In: ACL Computational fact checking from knowledge networks. Shi B, Weninger T. Discriminative predicate path mining for fact checking in knowledge graphs. Knowl-Based Syst. Toward automated fact-checking: Detecting check-worthy factual claims by claimbuster. The ordinary business of the school term as well as changes due to the war now dictate life on campus, creating an atmosphere that is both serious and rigid.
As Gene hurries to report as new assistant manager at the Crew House, he thinks of Phineas' trick of balancing on a canoe and then tumbling headlong into the water. The thought pleases Gene, because it brings back the carefree image of his friend before his accident. Gene meets Cliff Quackenbush, the crew manager, who treats him with contempt.
Disgusted by Gene's inexperience and lack of motivation, Quackenbush calls him "maimed" — a remark that prompts Gene to hit Quackenbush in the face. In the struggle that follows, both boys end up in the water, and a drenched Gene leaves for his dormitory. On the way to his room, Gene meets Mr. Ludsbury, a strict Devon master who warns him that the wild antics of the summer will not be tolerated any longer.
Saddened by this stern lecture, Gene is only mildly curious when Mr. Ludsbury tells him he has a long-distance phone call. It turns out to be Phineas on the phone, calling from home. In a friendly conversation, Finny again dismisses Gene's confession and expresses relief that they will still be roommates.
The only conflict arises when Gene tells Finny about going out for assistant crew manager, a position usually taken by younger students with no athletic talents. Outraged that Gene would even consider such a position, Finny tells his friend that he must go out for sports.
Since Finny can no longer compete, Gene must take his place. With this pronouncement, Gene feels as if he is becoming part of Finny. This chapter emphasizes the changes in Devon and in Gene now that the Summer Session is over — brought to a close, symbolically, by Finny's fall.
The chapter begins with Finny's absence, but ends with him not only reasserting his presence, but also his influence over Gene. Without Finny, Gene notices, peace seems to have "deserted Devon. The "gypsy music" of the summer has also vanished, replaced by the drone of duty and tradition.
Note especially the title of the hymn, "Dear Lord and Father of Mankind, Forgive Our Foolish Ways" — an outright apology, it seems, for all the fun the boys had during the Summer Session. Within this atmosphere, Gene cannot help but feel responsibility for his part in Finny's fall.
On the other hand, how could he feel any responsibility, not having been there when Leper started going crazy, and after being a better friend to him than most of the boys at the school. Does Gene feel that he too is going crazy, which is why he doesn't want to hear it? Or is Gene simply being callous, and doesn't want to help Leper out any more?
Because the motivation for Gene flipping out and running away is anything but clear, his reaction doesn't have the same power that the prose clearly intends it to have. Gene says that he "didn't want to hear any more of it.
Ever" ; the repetition emphasizes his sentiments, but since it is hard to figure out why he is acting like this, it is impossible to empathize.
0コメント