Envoy Proxy: Basic Routing and Clusters

Hello! Welcome to your next lesson.

Introduction

In our last lesson, we compared HAProxy and nginx, analyzing their architectural trade-offs, performance characteristics, and configuration styles. We concluded that HAProxy excels as a specialized, high-performance load balancer, while nginx shines as a versatile, all-in-one web front-end. Both tools, however, were born in an era of more static infrastructure.

Today, we transition to a proxy designed from the ground up for dynamic, cloud-native environments. We will tackle the learning outcome: Set up Envoy proxy with basic routing rules and cluster definitions.

Envoy is the foundational technology for many service meshes like Istio and is known for its powerful, API-driven configuration model. While we will focus on its static configuration today, this lesson will lay the groundwork for understanding its role in modern microservices architectures. We will dissect its core components, learn its specific terminology, and build a functional YAML configuration to perform basic HTTP path-based routing.

1. Envoy's Architecture and Core Concepts

Before diving into a configuration file, it's essential to understand Envoy's terminology, which is distinct from that of nginx or HAProxy. The key to grasping Envoy is understanding how it models the flow of traffic through a series of configurable components.

Let's start with a quick reading to define the main building blocks.

Now, let's watch a video that visualizes these concepts and explains the architecture in more detail.

To synthesize these resources, here is the fundamental request flow in Envoy:

1. Downstream (Client) -> Listener: A client sends a request to an IP address and port where an Envoy Listener is bound.
2. Listener -> Filter Chain: The Listener accepts the connection and passes it to a Filter Chain. A Listener can have multiple filter chains and will select the most specific one that matches the connection's properties.
3. Filter Chain Processing: The connection data flows through a series of Filters. These perform tasks like TLS termination, observability, rate limiting, and, most importantly, routing.
4. Router Filter -> Cluster: The final and most crucial filter is typically a router (like the HTTP Connection Manager). It inspects the request (e.g., the URL path) and decides which upstream Cluster to send it to.
5. Cluster -> Endpoint: A Cluster is a logical grouping of identical backend servers, known as Endpoints. The cluster's configuration defines the load balancing policy (e.g., round-robin) for selecting an endpoint.
6. Envoy -> Upstream (Endpoint): Envoy forwards the request to the selected endpoint.

This modular, filter-based architecture is what makes Envoy so extensible.

2. Static Configuration: Building the envoy.yaml

Envoy's configuration is managed through YAML files. While it's famously verbose, it is also explicit and highly structured. We will build a configuration for a common use case: routing HTTP requests to different backend services based on the URL path.

For this, we'll turn to a hands-on demonstration that builds the configuration from scratch.

Let's break down the structure of the envoy.yaml file shown in the video. The configuration is organized under a top-level static_resources key.

A. Defining Clusters

First, you define the upstream services Envoy will proxy to.

• clusters: A list of all backend service pools.
• name: A unique name for the cluster (e.g., api_service). This is used later in the routing rules.
• connect_timeout: Connection timeout for reaching an endpoint.
• type: How Envoy discovers the endpoints. STRICT_DNS means Envoy will resolve a DNS name to get the list of IPs.
• lb_policy: The load balancing algorithm, such as ROUND_ROBIN or LEAST_REQUEST.
• load_assignment: Statically defines the endpoints.
  • endpoints -> lb_endpoints -> endpoint: This nested structure specifies the actual backend server addresses (socket_address).

Here is a snippet for one cluster:

clusters:
  - name: api_cluster
    connect_timeout: 1s
    type: STRICT_DNS
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: api_cluster
      endpoints:
        - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: api_service_container # Docker service name
                    port_value: 8080

B. Defining the Listener and Routing Rules

Next, you configure the entry point and the logic that connects incoming requests to the clusters.

• listeners: A list of listeners.
• address: The socket_address (IP and port) Envoy will listen on.
• filter_chains -> filters: The list of network filters.
  • name: envoy.filters.network.http_connection_manager: This is the key filter for any L7 HTTP proxying.
  • typed_config: The specific configuration for the http_connection_manager.
    • @type: A long, explicit string that tells Envoy which configuration message type to use. This is a hallmark of Envoy's Protobuf-based configuration schema.
    • stat_prefix: A prefix for statistics emitted by this filter.
    • route_config: This is where the routing logic lives.
      • virtual_hosts: Groups routes by domain name (e.g., * for all domains).
      • routes: A list of routing rules, evaluated in order.
        • match: The condition for the rule to apply (e.g., prefix: "/api").
        • route: The action to take, typically cluster: api_cluster to forward to the specified cluster.
    • http_filters: A list of HTTP-specific filters. The most important one is envoy.filters.http.router, which must be the last filter in this list. It is the filter that actually dispatches the request to the chosen cluster.

This structure maps our conceptual flow directly into YAML:

3. A Complete Example

Let's put this all together into a complete, runnable example. The following envoy.yaml configures Envoy to listen on port 10000 and route requests starting with /service1 to one backend and /service2 to another.

# envoy.yaml
static_resources:
  listeners:
    - name: main_listener
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 10000
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_http
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: backend_services
                      domains: ["*"]
                      routes:
                        - match:
                            prefix: "/service1"
                          route:
                            cluster: service1
                            # This removes /service1 before forwarding
                            prefix_rewrite: "/" 
                        - match:
                            prefix: "/service2"
                          route:
                            cluster: service2
                            prefix_rewrite: "/"
                http_filters:
                  - name: envoy.filters.http.router
                    typed_config: {}

  clusters:
    - name: service1
      connect_timeout: 1s
      type: LOGICAL_DNS
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: service1
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: service1 # Resolvable name for the backend
                      port_value: 80
    - name: service2
      connect_timeout: 1s
      type: LOGICAL_DNS
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: service2
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: service2 # Resolvable name for the backend
                      port_value: 80

admin:
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 9901

To run this, you could use Docker Compose with two simple echo services. The address fields in the clusters (service1, service2) would correspond to the service names in your docker-compose.yml. Testing with curl http://localhost:10000/service1 would then route to the first service.

Conclusion

In this lesson, we took our first steps into the world of Envoy. We moved beyond the more traditional proxy models of nginx and HAProxy to a tool architected for the dynamism and complexity of modern distributed systems.

Key Takeaways:

• Envoy's Core Components: Traffic flows through a pipeline of Listeners, Filter Chains, and Filters, which route requests to upstream Clusters composed of Endpoints.
• Configuration is Explicit: Envoy's YAML configuration is verbose but highly structured. It requires you to explicitly define every component of the request path, from the listener to the final routing decision.
• HTTP Routing: The http_connection_manager filter is the workhorse for L7 proxying. Its route_config allows for powerful matching rules (e.g., path-based) to direct traffic to different backend clusters.
• Static vs. Dynamic: Today we focused on static configuration. This is the foundation, but Envoy's true power lies in its ability to discover its configuration dynamically via APIs (xDS), a topic for more advanced study.

Preview of the Next Lesson:

While we've just scratched the surface of Envoy, our next lesson will circle back to nginx to explore fundamental resilience patterns. We will learn how to configure proxy timeouts in nginx and analyze their effect on system resilience and user experience. Concepts like timeouts and retries are universal and apply to all proxies, including Envoy. By first implementing them in the familiar context of nginx, we can solidify our understanding before applying them in more complex, cloud-native scenarios.