From Riding on Rails to Riding Camels

Christopher Hazlett, Head of Back Office Engineering and Sean Smith, Lead Engineer, Distribution, at Gilt Group

Gilt Groupe, the US leader in flash sales, built our initial infrastructure like many fast growing start-ups – we used Ruby on Rails. Ruby on Rails did everything, from displaying products to managing the entire lifecycle of inventory, purchasing to fulfillment. This worked well when we had a single Distribution Center and a few orders. However, we grew quickly, adding Distribution Centers and a lot more code along the way…organically. Each DC not only had it’s own protocol for communication (FTP, SFTP, HTTPS) but they each also had distinct formats (CSV, Pipe Delimited, XML), and communications semantics. In this session, we will discuss how to move from a fragmented set of integration technologies built with Ruby on Rails to a simple, common approach to integration that can easily normalize many different types of incoming messages from partners and DCs into canonical data models, reformat and transmit them into any formats and protocols needed by any partner. It was built using Scala, Camel, MongoDB, and Apache Zookeeper. We will share lessons learned in the process of switching communications for a 24/7 international operation, and how Camel was used to eliminate technical debt that had grown over time.