Inspector Gadget: A Framework for Custom Monitoring and Debugging of Distributed Dataflows

The authors consider how to monitor and debug query processing dataflows, in distributed environments such as Pig/Hadoop. Their work is motivated by a series of informal user interviews, which revealed that monitoring and debugging needs are both pressing and diverse. In response to these interviews, they created a framework for custom dataflow instrumentation, called Inspector Gadget (IG). IG makes it easy to write a wide variety of monitoring and debugging behaviors, and attaches seamlessly to an existing, unmodified dataflow environment such as Pig.

Provided by: VLD Digital Topic: Data Management Date Added: Sep 2011 Format: PDF

Find By Topic