Optimizing Joins running on HDInsight Hive on Azure at GFS

by dennyglee Categories: BigDataTags: · · 6 Comments

.“…to look at the stars always makes me dream, as simply as I dream over the black dots of a map representing towns and villages…” — Vincent Van Gogh Image Source: Vincent Van Gogh Painting Tilt Shifted: http://coolvibe.com/2011/16-van-gogh-paintings-tilt-shifted/tilt-shift-van-gogh-15/ . Introduction To analyze hardware utilization within their data centers, Microsoft’s Online Services Division – Global Foundation Services (GFS) is working with Hadoop / Hive via HDInsight on Azure.  A common scenarios is to perform joins between the various tables of data.  This quick blog post provides a little context on how we managed take a query from >2h to <10min and…

Rate this:

Healthcare Compliance with Big Data and BI

by dennyglee Categories: BigData · Healthcare InformaticsTags: · Leave a comment

Originally posted on Ayad Shammout's SQL & BI Blog:
Healthcare Compliance with Big Data and BI Over the past few years Denny Lee  (Technical Principal Program Manager within Microsoft’s SQL Business Intelligence Group) and I are always working on a very exciting SQL Server projects, earlier this month we presented “Big Data, BI, and Compliance in Healthcare” at PASS BA Conference in Chicago, IL. Few years ago, we implemented “Centralized Audit Framework” to manage and view the audits of entire SQL Server environment that will parse, load, and report all of audit logs. Expanding on the “Reaching Compliance: SQL…

Rate this:

Halo 4 and the Flood of Big Data at Big Data Camp / PASS BA Conference

by dennyglee Categories: BigDataTags: Leave a comment

One of the great thing about working with the folks at 343 Industries – Halo developer – is that I get to claim my that playing Halo 4 is part of my job – awesome! But in the middle of realizing how awful of a player I am, I was given the opportunity to work some very creative and smart folks at 343 on their HDInsight on Azure project.  You can find more information and various learnings at: – 343 Industries Gets New User Insights from Big Data in the Cloud – Using Avro with HDInsight on Azure at 343…

Rate this: