Generative Prior Knowledge for Discriminative Classification
The authors present a novel framework for integrating prior knowledge into discriminative classifiers. Their framework allows discriminative classifiers such as Support Vector Machines (SVMs) to utilize prior knowledge specified in the generative setting. The dual objective of fitting the data and respecting prior knowledge is formulated as a bilevel program, which is solved (approximately) via iterative application of second-order cone programming. To test their approach, they consider the problem of using WordNet (a semantic database of English language) to improve low-sample classification accuracy of newsgroup categorization. WordNet is viewed as an approximate, but readily available source of background knowledge, and their framework is capable of utilizing it in a flexible way.