You may have heard of BlueKai. They’re a mover-and-shaker in “big data”, giving marketers, agencies, and organizations of all types a SaaS platform for ‘activating data’. In other words, BlueKai is a team of very smart people running hyper-intelligent software that collects, consolidates, and analyzes customers’ business information to deliver new insights into patterns, strategy, and much more. Very cool (and complex) stuff.
BlueKai’s platform offers a range of applications targeted to specific types of customers and outcomes. As you can imagine, the development engine behind their platform is big, fast, and coding ’round the clock to deliver updates, new capabilities, and better service, non-stop. What’s more, BlueKai uses multiple, massive Hadoop clusters to crunch its customers’ data, further upping the ante for warp-speed compute performance.
When faced with an extreme development velocity need, how can Ops keep up?
That was what Chris Gianelloni, BlueKai’s Senior System Engineer, asked himself less than a year ago, in the midst of continued customer growth and increasing requests from the development team for easier, faster access to computing power.
BlueKai had already expanded its infrastructure beyond multiple physical data centers to include various Amazon Web Services deployments. However, new resources were often configured differently by different teams, with each responsible for implementing their own patching and security controls, causing time management challenges and potential for errors in the development pipeline.
Utilizing Chef, the infrastructure team was able to provide a common solution for these problems and empower Devs with faster, easier access to resources. Chris and the systems team at BlueKai built an automated configuration management and application deployment tool chain with open source components from GitHub, Vagrant, and Jenkins in combination with Hosted Chef.
“We wanted to help Devs get what they needed when they needed it, but there are security controls that have to be maintained for everyone’s benefit,” Chris explained. “The cloud helps on the resource front, but without a shared code blueprint that each developer can use to spin up servers with the right configs, you get into a ‘wild west’ scenario.”
This was BlueKai’s wish list:
1) Rock-solid code blueprint for both ops and dev to use in configuring both Hadoop clusters and standard compute resources
2) Fine-grained control of code testing and process
3) Hyper-speed development pipeline fueled by both cloud and physical resources
To start checking off these items, Chris and his team worked with some of our Chef consultants to establish a coded blueprint for their infrastructure. It looks like this:
1) Developer commits new Chef recipe or cookbook code to GitHub repository
2) GitHub sends API call for new code to Jenkins server
3) Jenkins validates code by testing it in a Vagrant-based VM
4) Jenkins uploads tested cookbook to Hosted Chef
5) Production servers get tested code from Hosted Chef
The end result is a central repository of tested and approved-for-production Chef cookbooks in GitHub that power an agile application development pipeline for multiple components of BlueKai’s SaaS platform. Chris and his team made their developers more self-reliant, ensuring everyone works off the same blueprint of secure, tested code.
“I hear things like, ‘that’s it?’ and ‘this is so much easier now,’ which is awesome,” Chris continued. “What’s especially cool is how easy Chef makes deploying Hadoop servers. Most configuration management and all code testing are automated now, which has sped up development and improved our control.”
Sweet.
Currently, Chris and his team have rolled this DevOps toolchain out for a few of BlueKai’s many Dev teams, with plans to continue expanding its reach over time.
“There’s a lot more we can – and would like to – do, but we’re feeling really good about how far we’ve come. Coding our infrastructure has been a game changer.”
BlueKai Finds Efficiency in Big Data Activation with Opscode
Pioneer in Data-Driven Marketing Intelligence Deploys Opscode Hosted Chef™ to Automate Hybrid Infrastructure, Accelerating Development Velocity and Improving Infrastructure Control
SEATTLE – August 5, 2013 – Opscode®, the foundation for the coded business, today announced that BlueKai, the world’s first and only complete enterprise data activation solution for intelligent marketing, is automating configuration management for both physical and Amazon Web Services (AWS) cloud resources, including multiple, large-scale Hadoop clusters, with Opscode Hosted Chef™. Using Hosted Chef, BlueKai’s Infrastructure team has automated the testing and deployment of resource configuration code, improving collaboration between developers and systems engineers, reducing development cycle times and operational risk within an agile application development pipeline.
BlueKai offers the only end-to-end SaaS solution for marketers and publishers looking to maximize their cross-channel marketing efforts and create a proprietary solution for unlocking reach, scale and efficiency using data. The company uses multiple physical data centers combined with AWS resources to power the Hadoop clusters that support over 5 trillion data points monthly. To accelerate application development cycles and ensure universal use of security controls by development teams, BlueKai’s Infrastructure group built an automated configuration management and application deployment tool chain combining the Hosted Chef platform with open source development tools from GitHub, Vagrant, and Jenkins.
“We wanted to help Devs get what they needed when they needed it, but there are security controls that have to be maintained for everyone’s benefit,” said Chris Gianelloni, Senior System Engineer, BlueKai. “Using Hosted Chef as part of our application development pipeline we’ve automated most configuration management, including a couple big Hadoop clusters, and all code testing, speeding up development and improving systems control.”
BlueKai worked with Opscode Consulting to establish a coded blueprint for their infrastructure in which Hosted Chef recipes and cookbooks of configuration code are tested in Vagrant-based development environments to ensure full alignment with production environments. The tested code is stored in GitHub for easy access by developers, who deploy the code via Jenkins to power an agile, efficient application development pipeline for multiple components of BlueKai’s SaaS platform.
“BlueKai is transforming how organizations learn about, and market to, audiences with an innovative approach to big data collection and analysis,” said Adam Jacob, Chief Customer Officer, Opscode. “By using Chef code to empower development and operations teams to work faster, smarter, and more collaboratively, they represent the best of what DevOps-minded organizations can achieve.”
###