Inside development: Google Cloud Adoption at OBI4wan
One of the most important elements of software engineering is testing, especially when you are developing software for thousands of customers with concurrent users like OBI Engage or OBI Brand Monitor. In this article Matt Chapman, Backend Software Engineer at OBI4wan gives you a sneak peek into our development processes and one of our most recent projects: improving our testing environments.
Can you explain a little bit about what you wanted to achieve with Google Cloud Adoption at OBI4wan?
Looking at our legacy, we have always had environments for integration and acceptance testing. What we wanted here were full reproductions of our production systems, isolated and reproducible so that the environments could be torn down and rebuilt as needed. Testing environments are vitally important for our teams, so they can safely test their work without impacting each other or our customers.
A staging environment is a place where we run our own software product, that is separate from the one that customers use. We make changes here first, before releasing them to our customers.
Without staging environments, we would be limited to running tests in the same place that customers are working and using our product. As you can imagine, mistakes could prove very disruptive – so it is in everyone’s interest to test risky or experimental changes in a place where it can’t possibly affect our customers, nor the customers of our customers. Having these staging environments gives our teams a place that they can experiment (and make mistakes!) safely.
How does adopting cloud technologies help OBI4wan?
Cloud computing is the on-demand availability of computer system resources, especially data storage and computing power, without direct active management by the user – Wikipedia “Cloud computing”
“The cloud” is really a technical term for someone else’s computer, if you think about it. OBI4wan maintains a huge amount of physical computing hardware to run it’s products, and all that maintenance costs a considerable amount of time for our engineering department.
What businesses like Google and Amazon Web Services provide is essentially on-demand, á la carte computing resources. They provide hardware, infrastructure and software to manage it all – up to a point. The software we run remains our responsibility, but we get to leverage the huge pre-existing expertise of these companies, who manage a large amount of the operations workload for us. This gives us a lot more time developing new features and responding to customer needs, and a lot less time managing things like checking and replacing hardware or keeping third party software up to date.
This is also beneficial for OBI4wan as we look to expand outside of the Benelux region. Cloud computing companies provide resources all over the world and on every continent, meaning that we can provide customers with a great experience, no matter where they are situated. For example, if a customer requested that their data was stored in a certain location due to auditing or legal requirements, then having our systems in “the cloud” would be a huge advantage to us in providing that.
OBI4wan has long been a Google Cloud Platform (GCP) customer, with our hugely successful OBI Bots platform already running on Google services and infrastructure. With a considerable amount of cloud native development on our roadmap, and the shifting of our on-premise workloads to the cloud, building new testing environments for OBI Engage (which has always been run on-premise) would be an excellent proving ground for these future projects. It made sense, therefore, that these new systems would also run on GCP.
How did you go about delivering this project?
Based on our experiences with our existing on-premise testing environments, we already had a good idea of how we wanted to build these new ones in Google Cloud Platform. Reproducibility was key – being able to rebuild when something breaks, or duplicate when we need more capacity would be hugely helpful. It was also necessary to make sure that deploying new software versions was as automated as possible – even our most junior developers should be able to push changes with just a few clicks.
To achieve this we partnered with Binx.io, a consultancy which specializes in helping businesses adopt cloud-based platforms and technologies. With their help, we were able to plan and build our new staging environments in Google Cloud Platform, and adopt Terraform to store all of our infrastructure as code. This means that all of our new environments are defined as code, rather than a list of instructions that need to be followed by a human – allowing us to automate changes across all of them at once, as well as subject infrastructure changes to the same review and version control processes as our other code. Instead of telling the systems what to do, we simply tell them what we want things to look like once they are done – and Google Cloud Platform handles the rest!
“Spinning up a whole new clean environment is now simply a case of a few lines of code – and a job that would take up to several days to properly complete is fully automated with minimal manual work required.”
Google Cloud Platform handles all of our access control and authentication needs to keep these systems secure, ensuring that only developers from OBI4wan can access and use them, or make changes to them. The fact that we utilise GSuite internally for employee accounts, means that we can use exactly those same credentials to control access to the data and code that is deployed.
How do developers use these new environments?
We leveraged our existing Azure Pipelines setup that is used for automated builds and testing, and extended it to allow one-click deployments for staging environments in GCP. New versions of our software are automatically generated and tested as developers put their changes up for review by their colleagues, and now these builds can be quickly and automatically deployed to any environment in under a minute. This can have a huge impact on the time cost for testing changes, and gives us more time to ensure that the quality of our work is as high as it possibly can be.
Once the new version of our software is in the right place, they can simply go to their browser to use it and check and share with colleagues
In the future, it would be excellent to expand this kind of “one click” functionality to our customer-facing systems – continuous delivery of software to customers is an ideal that we strive for. I want to get to a point where we are seamlessly delivering updates to our customers multiple times a day, 5 days a week, without any disruption!
What’s next on the roadmap?
As mentioned – this was only the first of many projects we have on our roadmap for this year. OBI4wan is going “all in” on cloud technologies – the benefits that leveraging these kinds of technologies can provide us are huge, and will help us deliver the products and features that our customers need, at a much accelerated rate.
There are a very large number of utilities and services that we run internally at OBI4wan, each serving a purpose – whether that is providing features like real-time notifications in your browser, search, or sending customer satisfaction surveys. By moving these services to the cloud, we can leverage the same benefits that we will by moving our staging environments to the cloud – faster delivery of changes, less operational workload, and as a result, a higher quality product.