Dors/Cluc 2014: Discussing open source for globally distributed systems

What’s the role of open source tools in globally distributed systems that deliver billions of professional SMS messages to 190 countries achieving a throughput of 8000+ SMS per second? Well, our devs had quite a lot to say on this topic at the recent Dors/Cluc 2014 in Zagreb.

One of the oldest and most prominent IT conventions in the region, Dors/Cluc 2014 already saw plenty of evangelists of the open source movement in the past. This time, three of our devs, with a helping hand from our Head of Marketing, exchanged knowledge and experience with colleagues from the field and introduced them to some of the open source tools that Infobip uses. 

Apache Storm and Apache Kafka are both open source software we use for conducting real time analytics. At a high level, Kafka is a distributed, persistent message broker and Storm is a real time computation system, but they can also be used together. Kafka can be used to store the source stream and feed off that stream to conduct complex computations with Storm.

Additionally, we presented some use cases for different types of NoSQL distributed databases that we put to work on our Push project i.e. Cassandra, MongoDB, and Redis. Because we’re a large company, manually partitioning data over a large number of MySQL machines can cause quite a lot of trouble. This can be resolved by using one of these tools to create a layer below the app and one above MySQL that will automatically distribute data. Last but certainly not least, one of our devspresented the benefits that can be gained from the Chef – a framework which allows us to automate our IT structure so it can be properly defined in code.

Visitors and participants of this 3-day conference were able to structure content by their own interests. In fact, the entire program consisted of a half-hour lectures and five minute introductions of open source projects and communities. Furthermore, there were workshops where visitors had a chance to learn more about using and handling specific open source tool as well as open code. 

We are a large company, but it all started as a garage-type project of two IT enthusiasts, the same kind of enthusiasts that are bringing open source tools to our doorstep. But even if open source philosophy wasn’t rooted deep down in our company’s core, we would still be using these tools because of their high availability, system virtualisation, monitoring, alerting, centralised log ins and numerous other benefits. It’s about what fits best for your organisation. For us, it’s a mix of technologies that does the trick!