Hadoop : Tools for visualizing key value data and files for development

Question

Are there any simple, easily launched java tools for visualizing key/value data from map reduce job directories ?

Specifically i want to browse a 20-job map reduce workflow , cliicking on individual files and looking at the data, and maybe even see a histogram of file sizes.

• There are some caveats here, for example -

  • Some files have serialized data (not just text)
  • Obviously, this system would probably be silly to use at "cloud scale" , rather its a dev tool.

Nevertheless, such a tool would be useful developing and locally debugging large, connected m/r pipelines.

This is for development purposes (im not trying to visualize the distributed key/value hadoop data in a real cluster).

Answer 1

Checkout the KarmaSphere Studio

Monitor Job Execution Step-by-Step
- Workflow with Results: Shows the resulting output at each step of the MapReduce job.
- Hadoop Logs from the Desktop: Accesses Hadoop logs easily from the desktop.
- Job Failure Options: Allows the specification of job failure options such as automatic invocation of a specified script upon job failure for EMR.