seattle spark large

To Spark … and Beyond!

One of the very exciting thing about Spark is that there is the potential to have one ubiquitous tool to solve my aggregate, machine learning, graph, and other statistical / analytics problems.  And while I am proud of my time with the SQL Server team and we had achieved some amazing lofty goals (e.g. Yahoo!…

Rate this:

d3js-logo

2014 Flight Departure Performance via d3.js Crossfilter

As part of some quick analysis of flight departure data, to more quickly understand the impact of distance, date, and time of day on departure delays – I forked the Square Crossfilter and incorporated data from RITA BTS Flight Departure Statistics and Great Circle Mapper to calculate airport distances. At the bottom is a nice…

Rate this:

IMG_0872

Big Data and Legos

I was recently asked the question – how to explain Big Data to an 8yo. So after realizing the 4 Vs of Big Data barely make sense to non-marketing (i.e. most of us) let alone to kids – I realized that the best construct would be to use Legos. When I was her age, the…

Rate this:

spark-at-twitter-seattle

Learnings from Running Spark at Twitter

As part of the Seattle Spark Meetup series, we had a great Learnings from Running Spark at Twitter session at the @TwitterSeattle Offices.  We (Seattle Spark Meetup organizers) want to thank Sriram Krishnan (@krishnansriram) and Benjamin Hindman (@benh) for presenting and Jeff Currier (@jeff_currier) and @TwitterSeattle for hosting us! As well, we had raffled off Paco Nathan’s…

Rate this:

spark logo

Seattle Spark Meetup Kicks Off with DataBricks

I am very excited to that announce that Matei Zaharia and Pat McDonough from DataBricks will be speaking at the Seattle Spark Meetup and we’ve increased the room size to accommodate more people! Seattle Spark Meetup Kick Off with DataBricks They will come up and join us for pizzas and to talk about Apache Spark!  I…

Rate this:

mysql

Quick Tips on Restoring a MySQL Full InnobackupEx Backup

As I recently started using MySQL (with many years of SQL Server under my belt), here are some quick tips on restoring an InnobackupEx backup.  There was a context switch for me  but I’m enjoying the experience. Introduction There are many ways to backup / restore database (s) for MySQL but the mechanism I’m referring…

Rate this:

SQL to Hadoop

Quick Tip for extracting SQL Server data to Hive

While I have documented various techniques to transfer data from Hadoop to SQL Server / Analysis Services (e.g. How Klout changed the landscape of social media with Hadoop and BI Slides Updated, SQL Server Analysis Services to Hive, etc.), this post calls out the reverse – how to quickly extract SQL Server data to Hadoop /…

Rate this: