Running Kubernetes in Production: Make Sure Your Routing Strategy Works

Managing Kubernetes deployments in production raises some rather complex challenges. The dynamic nature of network communication in containerized environments produces unique operational issues–capable of bewildering even experienced DevOps teams. To mitigate these issues, software-based routing components offer key advantages across a number of areas.

Here are a few particularly important routing strategies to consider within any production-grade Kubernetes environment.

Leverage Tracing and Monitoring

However thoroughly you test applications during development, a fresh round of issues will emerge in production. To help understand (and respond to) these new obstacles, tracing and monitoring tools provide developers with crucial visibility into their runtime Kubernetes environment. Choosing routing technologies that closely integrate with established monitoring and tracing backends will also help make things easier.

All inter-service traffic passes through software routing components. This enables strategic application designs that purposefully collect microservices tracing and monitoring data. For example, tracing tools can identify the source and call flows of microservice invocations (assuming that the application is designed to support traceability). Thus, teams can determine how to fully leverage any available microservice, even those developed by different teams. By designing applications to provide metrics, DevOps teams can also use software routing and monitoring tools to understand production issues. For instance, tracking resource usage metrics for a microservice can reveal if its subcomponents are prepared to handle load at greater scale. That data can simultaneously point out performance limits and aid in troubleshooting new feature implementations.

Secure Communications and Permissions

Security must be a top-of-mind focus in any production Kubernetes deployment. Permissions governing internal network communications need to be tightly controlled to defend against attacks. Sensitive data transmitted within internal and external traffic must be encrypted and secured. Many businesses require data encryption not just as a best practice, but also as a matter of regulatory compliance. Unfortunately, these safeguards are absent in many Kubernetes environments.

Routing technologies make it possible to enforce security policies using network segmentation. For example, only client services with justified business requirements should have access to microservices that handle sensitive data. Routing tools can also provide encryption. Service meshes can secure internal east-west traffic with secure TLS encryption. Edge routers can utilize provided certificates to encrypt all external north-south traffic. In order to fully automate the lifecycle management of trusted certificates, DevOps teams can also pair up routing technologies with services such as Let’s Encrypt. This automation makes it possible to continuously encrypt and secure the transport of sensitive data, with no interventions from human workers required.

Absorb Unexpected Load Spikes

Sudden popularity, DDoS attacks and other unexpected events can trigger surprise load spikes. As a technique, rate limiting enables operators to control the request rates to front-end services. This equips applications to absorb surprise load spikes and avoid failures. In this way, leveraging a routing tool to implement rate limiting techniques effectively limits downtime due to load events.

Resolve Communication Issues

Microservices can become unreachable due to errors by a container, host, network partition or short-term microservice interruption. Therefore, implementing mitigation strategies to keep these communication failures from affecting users is absolutely crucial.

For example, the load balancer can respond to instance failures by directing requests to viable instances (and resuming normal traffic when the instance is available). During microservice interruptions, it’s critical to halt repeated retry requests to conserve resources and avoid cascading failures. Teams should use client services to employ circuit breaking to stop requests, send errors and fallback to alternative procedures.

While it’s possible to implement these mitigation techniques using network-layer logic within each client, that approach is highly challenging and error-prone. Teams are better off utilizing routing technologies to control their applications’ mitigation responses.

Achieve High Availability

Compared to traditional hardware failover solutions, today’s software routing technologies offer a far more effective and cost-efficient means of introducing high availability to Kubernetes environments.

Software routing technology enables DevOps teams to leverage architecture via a separate, horizontally-scalable data plane and a fault-tolerant control plane. The data plane makes it possible to add all the instances necessary to achieve the requisite capacity and resilience. At the same time, the control plane effectively tolerates failures to preserve uptime and safeguard seamless experiences for an application’s user base. A smart routing strategy thus ensures that production Kubernetes deployments are highly available and incredibly resilient.

Deploy Across Multi-Cloud and Other Heterogeneous Environments

DevOps teams may leverage Kubernetes across multiple cloud environments, on-prem environments and even alongside other container orchestration solutions. A routing strategy should meet the portability required to effectively support deployments across these different environments and solutions. By doing so, developers can deploy a common routing layer model across all deployments. This reduces key challenges by enabling a concentrated focus on a single and familiar solution.

Selecting an appropriately advantageous software routing technology can transform the experience of deploying Kubernetes into production. Routing tools provide capabilities that can simplify or even fully eliminate many common challenges (and some uncommon ones). In this way, DevOps teams can take advantage of everything Kubernetes has to offer that much more easily and successfully, while achieving more robust and reliable applications.

To learn more about containerized infrastructure and cloud native technologies, consider coming to KubeCon + CloudNativeCon EU, in Amsterdam. The CNCF has made the decision to postpone the event (originally set for March 30 to April 2, 2020) to instead be held in July or August 2020.

— Manuel Zapf