While I’m a big fan of CDH5 and Hue – sometimes I will see some funkiness that’s a tad irritating. Specifically, there is a database with a name similar to cloudera_manager_metastore_canary_test_db_hive_hivemetastore_$guid$_2014_10_06_11_20_41 Even more irritating there is a table called cm_test_table which cannot be deleted (or renamed or even described). hive> describe cm_test_table; FAILED: SemanticException [Error 10001]: Table not found cm_test_table hive> alter table cm_test_table RENAME to cm_test_table2; FAILED: SemanticException [Error 10001]: Table not found cm_test_table hive> drop table cm_test_table; FAILED: SemanticException [Error 10001]: Table not found cm_test_table To work around this problem, its a matter of using the CASCADE reference to…
Category: Tech Tips Tuesday
Quick Tip for Compressing Many Small Text Files within HDFS via Pig
One of the good (or bad depending on your point of view) habits when working with Hadoop is that you can push your files into the Hadoop cluster and worry about making sense of this data at a later time. One of the many issues with this approach is that you may rapidly run out of disk space on your cluster or your cloud storage. A good way to alleviate this issue (outside of deleting the data) is to compress the data within HDFS. More information on how the script works are embedded within the comments. /* ** Pig Script:…
Hive and Windows Auth – the curse of the backslash
Captain Avery: Put down the sword. A sword could kill us all, girl. Amy: Yeah. Thanks. That’s actually why I’m pointing it at you. — from “Doctor Who: The Curse of the Black Spot” Background Typically when you get Hive / Hadoop up and running, everything runs pretty smoothly especially if you use one of the demo VMs (e.g. I’m currently using the Cloudera QuickStart VM). But if you are in production and you want to secure login access to your environment, you may have Windows authentication turned on for access to one of the boxes on your Hadoop cluster…
Lazy Geek Hack: Mismatch of Ubuntu and Microsoft Keyboards
Lt. Felix Gaeta: Sir, I’m running every diagnostic we’ve got. Checking each line of code could take days. Col. Saul Tigh: I am not interested in excuses. Fix it. Lt. Felix Gaeta: [shouting] It’s not an excuse, sir. It’s a fracking fact! — Though I may be a walking Apple advertisement, I am still a fanboy of Microsoft products (yeah, I know that some times this isn’t all that obvious). Today’s lazy geek hack is in reference to the awesomely comfortable and ergonomic Microsoft keyboards and Ubuntu OS. Today’s keyboard of choice is the Microsoft Sculpt Comfort keyboard though this…
Don’t forget to register you device when upgrading to iOS7
In my haste to get up and running on iOS7 for my iPhone 5, I forgot to register my device onto the developer portal – doh! I did the pre-requiste steps of: – Registering as an iOS developer (well, re-register for me but that’s besides the point) – Download iOS7 – Backup my iPhone 5 – Click on option-Restore iPhone in iTunes, selected the iOS7 ipsw – and then I let it rip! Yet because I forgot to register my phone got stuck on the activation phase – doh! . How to get the UUID if I was patient If I…
Thursday TechTips: Hadoop 1.01 and Compression Codecs
I have been playing around with the compression codecs with Hadoop 1.01 over the last few months and wanted to provide quick tech tips on compression codecs and Hadoop. The key piece of advice is for you to get Tom White’s (@tom_e_white) Hadoop: The Definitive Guide. It is easily the must-have guide for Hadoop novices to experts. The key fundamentals concerning compression codecs is that not all codecs are immediately available within Hadoop. Some of them are native to Hadoop (one needs to remember to compile the native libraries) while others need to be extracted for their source and compiled…
Tech Tip Tuesday: HTML Donut Charts and Apache Web Server on Mac OSX
Quick tech tip for today, how to create HTML donut charts like the one below and use Apache Web Server on Mac OSX. Hats off to the BI Happy Blog and Coolest Guy on the Planet Tech blog. I wanted to quickly generate some mockup donut charts by using HTML – and being lazy, figured I would just borrow some code. For starters, I needed to get Apache Web server started on my Mac OSX Mountain Lion – fortunately, the instructions are already provided – so here’s the link: Install and configure Apache, MySQL, PHP and phpMyAdmin on OSX 10.8…
Denny Lee 