Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling

Abstract

Most current statistical natural language processing models use only local features so as to permit dynamic programming in inference, but this makes them unable to fully account for the long distance structure that is prevalent in language use. We show how to solve this dilemma with Gibbs sam- pling, a simple Monte Carlo method used to perform approximate inference in factored probabilistic models. By using simulated annealing in place of Viterbi decoding in sequence models such as HMMs, CMMs, and CRFs, it is possible to incorporate non-local structure while preserving tractable inference. We use this technique to augment an existing CRF-based information extraction system with long-distance dependency models, enforcing label consistency and extraction template consistency constraints. This technique results in an error reduction of up to 9% over state-of-the-art systems on two established information extraction tasks

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 90,616

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

  • Only published works are available at libraries.

Similar books and articles

The Information Medium.Orlin Vakarelov - 2012 - Philosophy and Technology 25 (1):47-65.
The Construction of Meaning.Walter Kintsch & Praful Mangalath - 2011 - Topics in Cognitive Science 3 (2):346-370.
Notationality and the information processing mind.Vinod Goel - 1991 - Minds and Machines 1 (2):129-166.
Proper Names and Local Information.Osamu Kiritani - 2008 - Journal of Mind and Behavior 29 (3):281-284.
Local logics, non-monotonicity and defeasible argumentation.Gustavo A. Bodanza & Fernando A. Tohmé - 2004 - Journal of Logic, Language and Information 14 (1):1-12.

Analytics

Added to PP
2010-12-22

Downloads
26 (#524,588)

6 months
4 (#320,252)

Historical graph of downloads
How can I increase my downloads?