Kubernetes Conditional Interception
In this newsletter, we will discuss how Crafting facilitates testing and debugging of services in a large-scale Kubernetes environment.
In previous newsletters, we explored the use of per-developer Kubernetes namespaces on Crafting and how to intercept traffic for fast iteration on the development version of a service. However, the traffic interception capability is not limited to the per-dev namespaces managed by Crafting.
For large tech companies, the Kubernetes environment may comprise hundreds or even thousands of services. Creating individual end-to-end environments for each developer becomes impractical. Even with a dozen services, per-dev namespaces can be wasteful when developers only need to modify a few services.
Crafting provides a solution by allowing developers to debug their services within a pre-existing shared Kubernetes deployment that includes the entire context, such as staging. Developers can dynamically "plug in" their development version of a service on-demand. Moreover, multiple developers can perform this action simultaneously while ensuring traffic isolation, preventing interference between their respective development versions.
This capability is made possible by Crafting's advanced Kubernetes Conditional Interception feature. It enables developers to swap a service with their development version running in the sandbox, exclusively for their own testing traffic.
As shown in the figure above, the left side sits the two developers and the right side is a shared Kubernetes cluster deployed with a complete deployment.
In this scenario, developer A utilizes conditional interception to replace the API service with their development version running in the sandbox. This interception only affects developer A's testing traffic (green arrows), while developer B's testing traffic (yellow arrows) continues to hit the original version of the API service in the shared deployment. Similarly, developer B can also perform conditional interception on the Cart service, affecting only their own traffic. It is even possible for multiple developers to conditionally intercept the same service and redirect their traffic streams to their respective sandboxes concurrently.
This approach allows multiple developers to test their services simultaneously in the shared Kubernetes environment, eliminating the need for individual clusters or namespaces. Consequently, it significantly reduces testing costs and simplifies the maintenance of the testing environment.
Identify and Splitting Traffic Streams
One key part of this solution is the ability to identify different traffic streams and trace them through many services. Many companies implemented distributed tracing for the observability of their services, and several monitoring tools like DataDog, NewRelic, etc., rely on the tracing headers to provide end-to-end analytics.
Put it simply, distributed tracing relies on a mechanism of header propagation, which adds a specific HTTP header from incoming request to the outgoing requests triggered by the incoming request. That way, all the service-to-service requests for handling one external API call will have the same value for the tracing header, effectively identifying a traffic stream.
Crafting can leverage the existing header propagation and allow user to specify specific header matching rules. Or to make things even simpler, Crafting can add additional endpoints, through which the standard headers like Baggage, Tracestate injected. That way, as long as the app uses a tracing library supporting the standard headers, the traffic going through these additional endpoints can be identified and the conditional interception on any service touched by the stream can be done successfully.
As shown above, you can configure an endpoint or specific pattern machine rules for conditional interception on Crafting
For example after start, the test endpoint is created send traffic to frontend service and the conditional interception on checkout service is only applied to the traffic stream sent through the test endpoint. All other developers' traffic still see the originally deployed version of the checkout service, not affected by it.
In future newsletters, we will discuss other new innovative ways we built for engineers to develop with Kubernetes with a great experience.
Apr 5, 2023
Subscribe to our Newsletter
Check Out Our Other Blogs
Kubernetes Debugging with Traffic Interception
In this newsletter, we continue the discussion on Crafting for Kubernetes, focusing on helping developers quickly iterate their service in Kubernetes environments.
APR 5, 2023
Crafting for Kubernetes
In this newsletter, we introduce Crafting for Kubernetes, the Crafting feature specifically designed to support Kubernetes development.
MAR 25, 2023