Splunk Cluster, Apigee and Heroku Migrations

Challenges

Upon joining the client, Curt was faced with several critical challenges that needed immediate attention:

1. Splunk Cluster Management: The company heavily relied on Splunk for log management and analysis. However, the existing infrastructure was inefficient and struggled to handle the growing volume of logs. Curt needed to build and operate several Splunk clusters on Google Cloud Platform (GCP) to ensure scalability and reliability.

2. Apigee to Kong Migration: the client had been using Apigee for API management, but the solution was becoming expensive and lacked flexibility. Curt was tasked with migrating the entire API ecosystem from Apigee to Kong without causing any service outages or downtime.

3. Heroku to GCP Migration: To reduce operational costs and gain more control over their applications, the company decided to migrate 17 Ruby on Rails applications from Heroku to GCP. This was a complex endeavor that involved Dockerizing each application and ensuring they ran seamlessly in Google Kubernetes Engine (GKE) on GCP.

Solutions

Curt approached these challenges systematically and effectively, demonstrating his expertise in platform engineering:

1. Splunk Cluster Management: Curt leveraged Terraform and Jenkins to automate the deployment and management of Splunk clusters on GCP. This automation not only reduced the operational overhead but also improved the system's reliability and scalability. The new clusters were able to handle the increasing volume of logs efficiently.

2. Apigee to Kong Migration: To ensure a smooth migration, Curt meticulously planned and executed the transition from Apigee to Kong. He carefully mapped out the existing APIs, tested the new Kong-based architecture extensively, and executed the migration with zero service outages or downtime, ensuring minimal disruption for users.

3. Heroku to GCP Migration: Curt's expertise in containerization and Kubernetes was crucial for the successful migration of the Ruby on Rails applications. He Dockerized each application, addressed compatibility issues, and optimized the deployment process on GKE. As a result, the applications not only performed better but also allowed for easier scaling and maintenance.

Results

Curt's contributions had a significant impact on the client's operations and platform performance:

1. Improved Log Management: The automated Splunk clusters on GCP enabled the company to handle logs efficiently, leading to better insights and troubleshooting capabilities.

2. Cost Savings: The migration from Apigee to Kong and Heroku to GCP resulted in significant cost savings for the company, as it could now optimize its resource allocation more effectively.

3. Enhanced Platform Reliability: The platform became more reliable, with no outages or downtime during the migration processes, ensuring a seamless experience for users.

4. Scalability and Control: The GKE-based deployment of Ruby on Rails applications on GCP allowed for better scalability and control, setting the stage for future growth.

Conclusion

Curt's expertise as a Platform Engineer at the client played a crucial role in overcoming significant infrastructure challenges. Through automation, meticulous planning, and hands-on technical skills, he not only improved the company's platform performance but also saved costs and ensured a smooth transition during critical migrations. Curt's contributions continue to be a cornerstone of the company's success in providing innovative solutions to its clients.