Denny Lee

A funky way to do Hive and Hadoop … on Azure

Doctor Who Amys Choice

Funny how you can say something in your head and it sounds fine

Doctor Who (Matt Smith as The Doctor in “Amy’s Choice”)

This post was published January 21st, 2012 and the content may be obsolete. Thus, most of these links are no longer active but I am keeping this post for posterity.

Recently I had posted the wiki article: Hadoop on Azure Scenario: Query a web log via HiveQL.

In it described how to analyze a sample web log using HiveQL on the Hadoop on Azure CTP (HadoopOnAzure.com) using

  • Interactive Hive console,
  • Interactive Javascript console,
  • Secure FTP using curl to transfer data to HDFS
  • Creating an EXTERNAL table against compressed log files
  • Executing some simple HiveQL queries.
Hadoop on Azure Interactive Hive console executing a HiveQL query
Hadoop on Azure Interactive Hive console executing a HiveQL query

A bunch of people had followed up to me directly on the article with the question:

Gee – do I have to do all of these steps to do work with Hadoop on Azure?!

The quick answer is: No, the purpose of the wiki article was to showcase how one can interact with Hadoop on Azure – which is very different from the traditional command line interface (CLI).  A great blog post on how the design of Hadoop on Azure was conceived, jump over to Dave Vronay’s blog post: The Design of the Portal for HadoopOnAzure.com.

The wiki entry is a “scenarios” article – so that way you can use all of the different functionality available to you on the Hadoop on Azure portal – like using the interactive consoles, running HiveQL queries, etc.

Hadoop on Azure portal where you interact with live tiles to open ports, remote desktop in, and use an interactive JavaScript or Hive console to run your queries.
Hadoop on Azure portal where you interact with live tiles to open ports, Remote Desktop in, and use an interactive JavaScript or Hive console to run your queries

Most importantly – because of the cool design, try out the scenarios not only on your desktop/laptop but also on your mobile devices, eh?!

Enjoy!

Leave a Reply

%d bloggers like this: