Bikram Chatterjee
DevOps Solution Architect hacking with Elixir at Helvetia Insurance
Bikram with his years of experience in Embedded and Mobile Application has started his functional programming journey over a decade ago in building Beam VM based high performance and distributed Telecom solutions.
Starting in traditional messaging domain, where he implemented SMS/MMS proxy servers, a high performance Oracle DB driver, a CRM platform, a configuration synchronisation platform and a couple of other application-level solutions based on Erlang OTP.
Today he is a DevOps Solution Architect at Helvetia Insurance in Switzerland and working in a team of Elixir experts on API platform automation engineering, His team is using Elixir to orchestrate multiple complex infrastructure provisioning pipelines.
Talk:
Kafka Migration at Scale
In this talk, we’ll explore how the diverse and dynamic capabilities of the Elixir ecosystem effectively supported the migration of thousands of Kafka topics and clients from self-hosting to cloud at Helvetia on a large corporate scale.
Over this year-long migration, we encountered various technical roadblocks due to the volume and complexity involved. However, we successfully overcame these challenges with Elixir to ensure a non-disruptive transition experience for our hundreds of applications processing billions of messages per day.
We adopted several useful Elixir language features and patterns to build the required tooling to support this process: • Improving performance with concurrent design • Multiple data transformation and validation scripts • Directed dependency graphs • Excel reporting framework • Macros • Templating with EEx
Keywords: • Confluent Kafka Cloud • Terraform Templating • GitOps • Synchronization of multiple platforms • Credentials generation and distribution