We demonstrate that an unlexicalized PCFG can parse much more accurately than previously shown, by making use of simple, linguistically motivated state splits, which break down false independence assumptions latent in a vanilla treebank grammar. Indeed, its performance of 86.36% (LP/LR F1) is better than that of early lexicalized PCFG models, and surprisingly close to the current state-of-theart. This result has potential uses beyond establishing a strong lower bound on the maximum possible accuracy of unlexicalized models: an unlexicalized PCFG is (...) much more compact, easier to replicate, and easier to interpret than more complex lexical models, and the parsing algorithms are simpler, more widely understood, of lower asymptotic complexity, and easier to optimize. (shrink)

Abstract Academic social scientists overwhelmingly vote Democratic, and the Democratic hegemony has increased significantly since 1970. Moreover, the policy preferences of a large sample of the members of the scholarly associations in anthropology, economics, history, legal and political philosophy, political science, and sociology generally bear out conjectures about the correspondence of partisan identification with left/right ideal types; although across the board, both Democratic and Republican academics favor government action more than the ideal types might suggest. Variations in policy views among (...) Democrats is smaller than among Republicans. Ideological diversity (as judged not only by voting behavior, but by policy views) is by far the greatest within economics. Social scientists who deviate from left?wing views are as likely to be libertarian as conservative. (shrink)

Analyzing the rate at which languages change can clarify whether similarities across languages are solely the result of cognitive biases or might be partially due to descent from a common ancestor. To demonstrate this approach, we use a simple model of language evolution to mathematically determine how long it should take for the distribution over languages to lose the influence of a common ancestor and converge to a form that is determined by constraints on language learning. We show that modeling (...) language learning as Bayesian inference of n binary parameters or the ordering of n constraints results in convergence in a number of generations that is on the order of n log n. We relax some of the simplifying assumptions of this model to explore how different assumptions about language evolution affect predictions about the time to convergence; in general, convergence time increases as the model becomes more realistic. This allows us to characterize the assumptions about language learning (given the models that we consider) that are sufficient for convergence to have taken place on a timescale that is consistent with the origin of human languages. These results clearly identify the consequences of a set of simple models of language evolution and show how analysis of convergence rates provides a tool that can be used to explore questions about the relationship between accounts of language learning and the origins of similarities across languages. (shrink)

This paper presents a novel approach to the unsupervised learning of syntactic analyses of natural language text. Most previous work has focused on maximizing likelihood according to generative PCFG models. In contrast, we employ a simpler probabilistic model over trees based directly on constituent identity and linear context, and use an EM-like iterative procedure to induce structure. This method produces much higher quality analyses, giving the best published results on the ATIS dataset.

We present a generative distributional model for the unsupervised induction of natural language syntax which explicitly models constituent yields and contexts. Parameter search with EM produces higher quality analyses than previously exhibited by unsupervised systems, giving the best published unsupervised parsing results on the ATIS corpus. Experiments on Penn treebank sentences of comparable length show an even higher F1 of 71% on nontrivial brackets. We compare distributionally induced and actual part-of-speech tags as input data, and examine extensions to the basic (...) model. We discuss errors made by the system, compare the system to previous models, and discuss upper bounds, lower bounds, and stability for this task. (shrink)

Abstract The party registration of tenure?track faculty at 11 California universities, ranging from small, private, religiously affiliated institutions to large, public, elite schools, shows that the ?one?party campus? conjecture does not extend to all institutions or all departments. At one end of the scale, U.C. Berkeley has an adjusted Democrat:Republican ratio of almost 9:1, while Pepperdine University has a ratio of nearly 1:1. Academic field also makes a tremendous difference, with the humanities averaging a 10:1 D:R ratio and business schools (...) averaging 1.3:1, and with departments ranging from sociology (44:1) to management (1.5:1). Across all departments and institutions, the D:R ratio is 5:1, while in the ?soft? liberal?arts fields, the ratio is higher than 8:1. These findings are generally in line with comparable previous studies. (shrink)

ABSTRACTAdam Smith infused the expression ‘impartial spectator’ with a plexus of related meanings, one of which is a super-being, which bears parallels to monotheistic ideas of God. As for any genuine, identified, human spectator, he can be deemed impartial only presumptively. Such presumptive impartiality as regards the incident does not of itself carry extensive implications about his intelligence, nor about his being aligned with benevolence towards any larger whole. We may posit, however, a being who is impartial and who holds (...) higher levels of intelligence and of benevolence, and then converse over what her sentiments would be about the matter under discussion. It is natural to conceive of a being who is unsurpassable in such qualities, who is morally supreme, and who naturally takes the definite article the without having been definitized by the writer. Signal passages, new to edition 6, suggest that Smith formulates the man within the breast as a representative of the always present and everywhere morally supreme impartial spectator. When Smith speaks of the man within the breast as ‘the supposed impartial spectator’, we interpret ‘supposed’ as sup-pos-ed, not sup-pos’d. (shrink)

This paper presents empirical studies and closely corresponding theoretical models of the performance of a chart parser exhaustively parsing the Penn Treebank with the Treebank’s own CFG grammar. We show how performance is dramatically affected by rule representation and tree transformations, but little by top-down vs. bottom-up strategies. We discuss grammatical saturation, including analysis of the strongly connected components of the phrasal nonterminals in the Treebank, and model how, as sentence length increases, the effective grammar rule size increases as regions (...) of the grammar are unlocked, yielding super-cubic observed time behavior in some configurations. (shrink)

We present a novel generative model for natural language tree structures in which semantic (lexical dependency) and syntactic (PCFG) structures are scored with separate models. This factorization provides conceptual simplicity, straightforward opportunities for separately improving the component models, and a level of performance comparable to similar, non-factored models. Most importantly, unlike other modern parsing models, the factored model admits an extremely effective A* parsing algorithm, which enables efficient, exact inference.

This paper separates conditional parameter estima- tion, which consistently raises test set accuracy on statistical NLP tasks, from conditional model struc- tures, such as the conditional Markov model used for maximum-entropy tagging, which tend to lower accuracy. Error analysis on part-of-speech tagging shows that the actual tagging errors made by the conditionally structured model derive not only from label bias, but also from other ways in which the independence assumptions of the conditional model structure are unsuited to linguistic sequences. The (...) paper presents new word-sense disambiguation and POS tagging experiments, and integrates apparently conflicting reports from other recent work. (shrink)

Unsupervised grammar induction systems commonly judge potential constituents on the basis of their effects on the likelihood of the data. Linguistic justifications of constituency, on the other hand, rely on notions such as substitutability and varying external contexts. We describe two systems for distributional grammar induction which operate on such principles, using part-of-speech tags as the contextual features. The advantages and disadvantages of these systems are examined, including precision/recall trade-offs, error analysis, and extensibility.

While Ç ´Ò¿ µ methods for parsing probabilistic context-free grammars (PCFGs) are well known, a tabular parsing framework for arbitrary PCFGs which allows for botton-up, topdown, and other parsing strategies, has not yet been provided. This paper presents such an algorithm, and shows its correctness and advantages over prior work. The paper finishes by bringing out the connections between the algorithm and work on hypergraphs, which permits us to extend the presented Viterbi (best parse) algorithm to an inside (total probability) (...) algorithm. (shrink)

At lunch one day a colleague and I had a friendly argument over occupational licensing. I attacked it for being anticompetitive, arguing that licensing boards raise occupational incomes by restricting entry, advertising, and commercialization. My colleague, while acknowledging anticompetitive aspects, affirmed the need for licensing on the grounds of protecting the consumer from frauds and quacks. In many areas of infrequent and specialized dealing, consumers are not able, ex ante or even ex post, to evaluate competence. I countered by suggesting (...) voluntary means by which reputational problems might be handled and by returning to the offensive. I said that in fact the impetus for licensing usually comes from the practitioners, not their customers, and that licensing boards seldom devote their time to ferreting out incompetence but rather simply to prosecuting unlicensed practitioners. I mentioned cross-sectional findings, such as those on state licensure, prices, and occupational incomes. Overall, I characterized the professional establishment as a group of dastardly operators, who set the standards, write the codes, and enforce behavior to enhance their own material wellbeing - in brief, as venal rent-seekers. (shrink)

Table at Dimitri's Taverna : on seeking a philosophy of old age -- Old Greek's olive trees : on Epicurus's philosophy of fulfillment -- Deserted terrace : on time and worry beads -- Tasso's rain-spattered photographs : on solitary reflection -- Sirocco of youth's beauty : on existential authenticity -- Tintinnabulation of sheep bells : on mellowing to metaphysics -- Iphigenia's guest : on stoicism and old old age -- Burning boat in Kamini Harbor : on the timeliness of spirituality (...) -- Returning home : on a mindful old age. (shrink)

OVERVIEWWe ambitiously reexamine Smith’s moral theory in relation to Hume’s. We regard Smith's developments as glorious and important. We also see them as quite fully agreeable to Hume, as enhancement, not departure. But Smith represents matters otherwise! Why would Smith overstate disagreement with his best friend?One aspect of Smith’s enhancement, an aspect he makes very conspicuous, is that between moral approval and beneficialness there is another phase, namely, the moral judge's sense of propriety. With that phase now finding formulation, Smith, (...) if only implicitly, generates a spiral of beneficialness and propriety, a spiral shown in Figure 7 in the present paper. We consider Figure 7, illustrating the spiral, to be the most important arrival point in the present paper; it highlights the non-foundationalism of Smith's ethics. But to arrive at the spiral, we must engage in extensive exegesis.In Part IV of The Theory of Moral Sentiments, Smith presents a foil against which he develops his own theory, a foil supposedly representing Hume. According to the foil, moral approval derives from ‘utility.’ But, in multiple ways, the foil is misleading. We provide an interpretation of Hume, notably his four-factor account of moral approval, before examining Smith's representation of Hume.One twist is that Smith used the words utility and useful differently than Hume did – Smith quietly stretched them to include species of agreeableness, thereby obscuring the importance of agreeableness in Hume’s theory.Another, more significant problem is that Smith allows the impression that in Hume moral approval derives quite determinately from beneficialness. In fact Hume conveys the interpretive and sentimental spaciousness of the operations that generate moral approval; here, Hume even speaks repeatedly of ‘proper sentiments’, thus almost using the term propriety himself.But the propriety phase in Smith opens up to a key facet of Smith’s development on Hume: He poeticizes a locus of sympathy not emphasized in Hume – namely, that between the moral judge and her own man within the breast; that locus enters the theory in addition to the sympathies emphasized in Hume, not in lieu of them. We distinguish lateral sympathy, which is important in Hume’s thought, and vertical sympathy, which is especially characteristic of Smith’s more inner-directed and allegorical thought. Smith embraces Hume’s lateral sympathy and enhances moral theory by adding formulations that elaborate vertical sympathy.Next, we come to something of a twist in the whole matter: We show that – as Smith well knew all along! – propriety is a species of agreeableness! Smith’s propriety phase represents another dimension within which such agreeableness lives: Smith’s vertical dimension thus gives rise to a spiral representing the diachronic development of the judge herself. It is a spiral of beneficialness and propriety: Each propriety phase in the next loop of the spiral engenders a species of agreeableness now as a part of beneficialness.Smith's developments on Hume, then, involve the following three facets: formulation of the propriety phase; the poetic elaboration of vertical sympathy; the diachronic spiral of propriety and beneficialness.The three facets come together, especially in Ed. 6 of The Theory of Moral Sentiments. The whole development goes beyond Hume, but, really, is agreeable to Hume – though Smith himself portrays his developments as disagreeing with features of Hume's moral theory.We speculate that Smith was more or less aware of all that that we say, including the absense of any really substantive disagreement. Why, in that case, would Smith have proceeded as he did? We address that question at the end of the piece. Our speculations suggest a method in the madness. (shrink)

While symbolic parsers can be viewed as deduction systems, this view is less natural for probabilistic parsers. We present a view of parsing as directed hypergraph analysis which naturally covers both symbolic and probabilistic parsing. We illustrate the approach by showing how a dynamic extension of Dijkstra’s algorithm can be used to construct a probabilistic chart parser with an Ç´Ò¿µ time bound for arbitrary PCFGs, while preserving as much of the flexibility of symbolic chart parsers as allowed by the inherent (...) ordering of probabilistic dependencies. (shrink)

We discuss two named-entity recognition models which use characters and character n-grams either exclusively or as an important part of their data representation. The first model is a character-level HMM with minimal context information, and the second model is a maximum-entropy conditional markov model with substantially richer context features. Our best model achieves an overall F1 of 86.07% on the English test data (92.31% on the development data). This number represents a 25% error reduction over the same model without word-internal (...) (substring) features. (shrink)

We present an improved method for clustering in the presence of very limited supervisory information, given as pairwise instance constraints. By allowing instance-level constraints to have spacelevel inductive implications, we are able to successfully incorporate constraints for a wide range of data set types. Our method greatly improves on the previously studied constrained -means algorithm, generally requiring less than half as many constraints to achieve a given accuracy on a range of real-world data, while also being more robust when over-constrained. (...) We additionally discuss an active learning algorithm which increases the value of constraints even further. (shrink)

Thomas Schelling has described how each of us is made up of conflicting impulses. The art of managing these impulses Schelling dubs ?egonomics?. The idea of egonomic calamity underlies paternalism (or, breaking convention, what I call ?parentalism'). The paper argues for laissez?faire in matters egonomic. The rationalizations I give for this libertarian sentiment are old ones, such as accentuating the dignity of the individual and letting the individual learn from example and from his own experience. Also I note, as H. (...) L. Mencken did, that parentalist measures often have unhappy unintended consequences, sometimes are counter?productive, and rarely are borne of sincere good?will. (shrink)

Knowledge consists of the triad: information, interpretation, and judgement. Much of modern political economy has miscarried by proceeding as though knowledge were merely “information” that is, as though interpretation were symmetric and final. Economic prosperity depends greatly on new knowledge or “discovery” of profit opportunities that translate into social betterment. These discoveries are often a transcending of the working interpretation, not merely the acquisition of new information. The evolution of interpretations is key to appreciating voluntarism as a maxim for policy. (...) Smith, Hayek, and Kirzner taught that bad public policy is partly the result of not knowing better, not of evil or greed. Economists may justify their efforts and their salaries chiefly by teaching citizens to know better. Such is the economist’s calling.La connaissance consiste en une triade : information, interprétation, et jugement. Une bonne part de l’économie politique moderne a échoué en assimilant la connaissance à l’information, ce qui revenait à dire que l’interprétation était symétrique et ultime. La prospérité économique repose dans une grande mesure sur l’émergence de nouvelles connaissances ou la “découverte” d’occasions de profit porteuses d’améliorations sociales. Souvent ces découvertes transcendent l’interprétation courante et non seulement l’acquisition de nouvelles informations. L’évolution des interprétations est essentielle pour évaluer le volontarisme en tant que maxime pour la politique. Smith, Hayek et Kirzner nous ont enseigné que des politiques publiques inappropriées résultent en partie d’une déficience de connaissances plutôt que de mauvaises intentions ou d’avidité. Les économistes peuvent justifier leurs efforts et leurs salaires essentiellement en apprenant aux citoyens à s’enrichir de connaissances. Telle est la vocation de l’économiste. (shrink)

This paper discusses ensembles of simple but heterogeneous classifiers for word-sense disambiguation, examining the Stanford-CS224N system entered in the SENSEVAL-2 English lexical sample task. First-order classifiers are combined by a second-order classifier, which variously uses majority voting, weighted voting, or a maximum entropy model. While individual first-order classifiers perform comparably to middle-scoring teams’ systems, the combination achieves high performance. We discuss trade-offs and empirical performance. Finally, we present an analysis of the combination, examining how ensemble performance depends on error independence (...) and task difficulty. (shrink)

A* PCFG parsing can dramatically reduce the time required to find the exact Viterbi parse by conservatively estimating outside Viterbi probabilities. We discuss various estimates and give efficient algorithms for computing them. On Penn treebank sentences, our most detailed estimate reduces the total number of edges processed to less than 3% of that required by exhaustive parsing, and even a simpler estimate which can be pre-computed in under a minute still reduces the work by a factor of 5. The algorithm (...) extends the classic A* graph search procedure to a certain hypergraph associated with parsing. Unlike bestfirst and finite-beam methods for achieving this kind of speed-up, the A* parser is guaranteed to return the most likely parse, not just an approximation. The algorithm is also correct for a wide range of parser control strategies and maintains a worst-case cubic time bound. (shrink)

erative clustering. First, we show formally that the common heuristic agglomerative clustering algorithms – Ward’s method, single-link, complete-link, and a variant of group-average – are each equivalent to a hierarchical model-based method. This interpretation gives a theoretical explanation of the empirical behavior of these algorithms, as well as a principled approach to resolving practical issues, such as number of clusters or the choice of method. Second, we show how a model-based viewpoint can suggest variations on these basic agglomerative algorithms. We (...) introduce adjusted complete-link, Mahalanobis-link, and line-link as variants, and demonstrate their utility. (shrink)

Method by subtracting off the error along several nonprincipal eigenvectors from the current iterate of the Power Method, making use of known nonprincipal eigenvalues of the Web hyperlink matrix. Empirically, we show that using Power Extrapolation speeds up PageRank computation by 30% on a Web graph of 80 million nodes in realistic scenarios over the standard power method, in a way that is simple to understand and implement.