Apache Storm [1] is a distributed real time stream processing system that has garnered a lot of attention in recent times due to its reputation as a scalable and high performing stream processing system.
It became an Apache incubator project recently and also had to switch its default transport from ZeroMQ to Netty. However, if you still want to use ZeroMQ as the transport, for old times sake, here are the steps to configure ZeroMQ.
1. Build zeromq and jzmq bindings as instructed in 'Installing native dependencies' in the old storm wiki of Nathan Marz [2]
2. Unzip the apache storm distribution to a directory of your choice.
3. Copy the 'zmq.jar' from the installation directory of jzmq into the STORM_HOME/lib directory. I found in my Ubuntu distro that you can locate it at /usr/local/share/java. The STORM_HOME directory refers to the installation home directory of Storm.
4. Build the storm-zeromq dependencies from the github repo of Taylor Goetz at [3]
5. Copy the created storm-0mq-0.x.x.jar from target of the above build into STORM_HOME/lib
6. Edit the STORM_HOME/conf/storm.yaml file and add the following line.
storm.messaging.transport: "backtype.storm.messaging.zmq"
7. Et voila! Now you are ready to fire up storm. Start zookeeper, nimbus and submit some topology to ensure that no errors crop up during streaming.
If you do encounter any issues, ask away at user@storm.incubator.apache.org. The folks over their are really willing to help and give you good guidance on Storm.
[1] http://storm.incubator.apache.org/
[2] https://github.com/nathanmarz/storm/wiki/Installing-native-dependencies
[3] https://github.com/ptgoetz/storm-0mq
No comments:
Post a Comment