Aggressive optimism and a no enemies policy pays dividends

— Eric14 aka Eric Baldeschwieler (@jeric14) during ApacheCon 2011 North America Thursday Keynote.

After all of these years doing BI, why have I jumped on the bandwagon of Hadoop and Big Data? Well, if you read some of my previous blog posts like Revelations – rolling the Hard Six to SQL BI and Hadoop or At a Crossroads … from SSAS to BigData! – I never left the Big Data wagon in the first place. Whether it is Tier-1 BI, web analytics, or Big Data – it’s all about solving some really complex problems.

It’s not just your “Big Data” problems, it’s about your BIG “Data Problems”. – Alexander Stojanovic (@stojanovic), General Manager and Founder Hadoop on Azure / Windows Server, at ApacheCon North America 2011 Meetup

It’s about the openness of the Open Source community (apologies for the pun) that allows us to focus on solving the actual problem instead of trying to understand how a particular system works. It’s the rich interactions on the various Apache mailing lists to Facebook and Netflix providing details on how they solved their problems. Some of my favorites great examples include:

• Moving an Elephant: Large Scale Hadoop Data Migration at Facebook – Moving 30PB of data, this is just awesome!
• Migrating Netflix from Datacenter Oracle to Global Cassandra – the uber-example of the ultimate Cloud app!
• Facebook: Why our ‘next-gen’ comms ditched MySQL – which is even cooler since one of the founders of Cassandra (created at Facebook) led the movement to switch to HBase.

And that’s why I have an aggressive optimism for Hadoop – because it allows us to solve the data problem instead of needlessly focusing on the technical problem.

.