Data Management

Exploiting Wiktionary for Lightweight Part-of-Speech Tagging for Machine Learning Tasks

Download Now Free registration required

Executive Summary

Part-of-speech (PoS) tagging is a crucial part in many natural language machine learning tasks. Current state-of-the-art PoS taggers exhibit excellent qualitative performance, but also contribute heavily to the total runtime of text preprocessing and feature generation, which makes feature engineering a time-consuming task. The authors propose a lightweight dictionary and heuristics based PoS tagger that exploits Wiktionary as its information source. They demonstrate that its application to natural language machine learning tasks considerably decreases the feature generation runtime, while not degrading the overall performance on these tasks. They compare the lightweight tagger to a state-of-the-art maximum entropy based PoS tagger in clustering and classification tasks and evaluate its performance on the Brown Corpus.

  • Format: PDF
  • Size: 206.79 KB