I have run the calculations a dozen times, the grid should be enough to bring about fusion and power the device. But, to be sure, I won’t know until I press the freaking button.

— Lucas, Continuum Season 1, Episode 2, “Fast Times”

—

As noted by Lucas and typical of most Big Data Analytics projects, the calculations may be right but that doesn’t mean its going to work.   Fortunately, that is NOT the case for running Apache Kafka 0.8 with Scala 2.9.2 on Mac OSX Mountain Lion.  In fact this jump start guide really points to some key links and pages to get yourself up and running in short order.

Learning about Kafka

There are a lot of great resources on Apache Kafka and as typical of most Apache projects, the Apache Kafka home page is a good start.  It’s also worth you time to review Neha Narkhede’s Tech Talk: Kafka, LinkedIn’s open-source distributed pub-sub messaging system.  Basically, if you want to know more about building a simple (this is good!) high throughput messaging system, jump in!
.

Building Kafka

Most of the instructions can be followed verbatim from the Apache Kafka 0.8 Quick Start Guide.  Some quick notes that may be helpful for you to get started:

– You can get the latest Apache Kafka bits right from GitHub: https://github.com/apache/kafka

– But if you are like me and are already using Scala 2.9.2 (in my case, because I also like working with Spark), then you can follow Michael Noll’s excellent blog post Running a Multi-Broker Apache Kafka 0.8 Cluster on a Single Node.   It also includes his patch to work Scala 2.9.2 which can be found at: https://github.com/miguno/kafka/tree/scala-2.9.2

.

Running Kafka

To get yourself up and running Apache Kafka and test out the roles of producers, multiple brokers, consumers, and replications – just continue executing on Michael Noll’s excellent post:  Running a Multi-Broker Apache Kafka 0.8 Cluster on a Single Node.  In the post, not only does he include the instructions on how to run Kafka, but it also includes easy explanations of the relationship between topics, replications, and brokers in Kafka.

In fact, if it wasn’t for the fact that I was adding quotes from Continuum and Neha Narkhede’s tech talk, this blog post would probably be just a re-blog of Michael Noll’s post :).

A note on Tree

A quick way to view hierarchy structure of the Kafka log files is to use the tree command.  As noted from this thread Mac OS X equivalent of the Ubuntu “tree” command, you can use Brew Doctor or MacPorts (or Fink) to quickly install it.

Enjoy!