Oh where, oh where did my S3N go? (in Windows Azure HDInsight) Oh where, Oh where, can it be?!
As noted in my previous post Connecting Hadoop on Azure to your Amazon S3 Blob storage, you could easily setup HDInsight Azure to go against your Amazon S3 / S3N storage. With the updates to HDInsight, you’ll notice that Manage Cluster dialog no longer includes the quick access to Set up S3.
![image_thumb[4]](http://dennyglee.files.wordpress.com/2012/10/image_thumb41.png "image_thumb[4]")
Yet, there are times where you may want to connect your HDInsight cluster to access your S3 storage. Note, this can be a tad expensive due to transfer costs.
To get S3 setup on your Hadoop cluster, from the HDInsight dashboard click on the Remote Desktop tile so you can log onto the name node.
![image_thumb[8]](http://dennyglee.files.wordpress.com/2012/10/image_thumb8.png "image_thumb[8]")
Once you are logged in, open up the Hadoop Command Line Interface link from the desktop.
![image_thumb[10]](http://dennyglee.files.wordpress.com/2012/10/image_thumb10.png "image_thumb[10]")
From here, switch to the c:\apps\dist\Hadoop[]\conf folder and edit the core-site.xml file. The code to add is noted below.
<property>
<name>fs.s3n.awsAccessKeyId</name>
<value>[Access Key ID]</value>
</property>
<property>
<name>fs.s3n.awsSecretAccessKey</name>
<value>[Secret Access Key]</value>
</property>
Once this is setup, you will be able to access your S3 account from your Hadoop cluster.
Enjoy!
About dennyglee
Hi, Denny,
Think you have a couple of extra “Oh”s in your title. It’s from “Where, O where has my little dog gone. Where, o where can he be?’
–Cheers
LOL – fair enough, but I still prefer the extra O’s