Automate your New Relic configuration
Being a software engineer these days is not only about writing good code. It’s also about deploying your code to production and checking it works all the time.y
Ocado Technology uses New Relic to monitor its services, either across our e-commerce applications or inside our highly-automated warehouses. There are multiple environments for the code we create and deploy, and each of these requires a separate configuration for alerting policies. Some of these alerting policies are common between environments, while some are unique for production, testing, etc.
The process of defining alerting policies is manual. We started to struggle when we had to introduce the same configuration change for multiple services in New Relic. After a thousand and one clicks, we decided we had had enough and began to explore other possibilities.
The solution we came up with was to use New Relic REST API to create a tool tailored for our needs:
Create missing alerting policies
Update rules for existing policies
Keep configuration in repository and run synchronisation automatically when a change is introduced
Don’t repeat yourself: extract common configuration and use it when needed
Detect invalid configuration before pushing changes to New Relic
We concentrated our attention on developing the tool and implemented it straight away. It proved to be the perfect solution to our problem; we no longer needed to browse through New Relic pages and introduce changes manually, and whenever we had to introduce a change that affected all our services, we only modified the element responsible for common alerting rules - Git commit, push, and wait for pipeline to execute. The consequence of these modifications was that our processing time decreased significantly. This had impact on business stories as well - because declaring alerting rules now proved to be a simple task, we added additional layers of checks to our applications.
Now we would like to share our efforts with rest of the community.
We have published New Relic Alerts Configurator library on GitHub. Using this library, you can configure New Relic alerting policies via Java code.
The library covers most of the popular alerting policy REST APIs, but not all of them. We are looking to introduce new features like NRQL support, but we also encourage you to add new features to help others using New Relic.
We hope you find our tool useful.