Updated HDInsight on Azure ASV paths for multiple storage accounts

If you’ve joined the HDInsight Preview – you will notice many new changes including the tight integration with Windows Azure and that HDInsight defaults to ASV.  As noted in Why use Blob Storage with HDInsight on Azure, there are some interesting technical (performance) and business reasons for utilizing Azure storage accounts. But if you had been playing with the HadoopOnAzure.com beta and switched over to the Windows Azure HDInsight Service Preview – you’ll may have noticed a quick change in the way asv paths work.  Here’s a quick cheat sheet for you. In general, to access ASV sources #ls asv://$container$@$storage_account$.blob.core.windows.net/$path$…

Rate this:

Why use Blob Storage with HDInsight on Azure

By Brad Sarsfield and Denny Lee One of the questions we are commonly asked concerning HDInsight, Azure, and Azure Blob Storage is why one should store their data into Azure Blob Storage instead of HDFS on the HDInsight Azure Compute nodes.  After all, Hadoop is all about moving compute to data vs. traditionally moving data to compute as noted in Moving data to compute or compute to data? That is the Big Data question.  The network is often the bottleneck and making it performant can be expensive.  Yet the practice for HDInsight on Azure is to place the data into…

Rate this:

Hadoop on Azure: HiveQL query against Azure Blob Storage

The posting Setup Azure Blob Store for Hadoop on Azure CTP provides a quick way to upload files to your Azure Blob storage account and connect Hadoop on Azure CTP to it.  Now that you have done that, one of the first things you may want to do is to interact with the data. To do this, let’s create a Hive table within Hadoop on Azure CTP that is connected to the files you uploaded to your Azure Blob storage account and query it.  We will be referencing the scenario noted at: Hadoop on Azure Scenario: Query a web log…

Rate this:

Setup Azure Blob Store for Hadoop on Azure CTP

One of the cool ways to run Hadoop on Azure is to have it connect to Azure Blob storage via your Windows Azure Storage account.  To setup your Azure storage account, please refer to http://windows.azure.com. The tasks below will allow you to setup your Hadoop on Azure CTP account to connect to an existing Azure Blob Storage account using the asv protocol.  For example, within Hadoop, you normally would get a listing of files within HDFS using the command line interface: hadoop fs –ls / In the case of accessing files within Azure Blob storage, you can run the command:…

Rate this: