External load balancing and Internal service-to-service load balancing

This is one of the biggest concepts to understand in modern microservices. Let’s separate them clearly.

---

1. External Load Balancing (Client → Cluster)

This load balancer receives requests coming from outside your system (browser, mobile app, third-party APIs).

Example:

             Internet
                 │
                 │
         External Load Balancer
        (AWS ALB / Nginx / Ingress)
                 │
                 │
          API Gateway Service
                 │
        -------------------
        Kubernetes Cluster

The job is to distribute traffic coming from users.

Example:

User 1 ────────┐
User 2 ────────┤
User 3 ────────┤
               ▼
      External Load Balancer
               │
     ┌─────────┴─────────┐
     ▼                   ▼
Gateway Pod 1       Gateway Pod 2

If Gateway Pod 1 dies,

External LB
      │
      ▼
Gateway Pod 2

Users never notice.

---

Who provides this?

Usually

• AWS ALB
• AWS NLB
• Azure Load Balancer
• Google Cloud Load Balancer
• Nginx Ingress
• HAProxy

---

2. Internal Service-to-Service Load Balancing

Now suppose API Gateway wants to call Payment Service.

Payment service has 4 replicas.

Payment Pod 1
Payment Pod 2
Payment Pod 3
Payment Pod 4

Gateway shouldn’t always hit Pod 1.

Instead,

Gateway
    │
    ▼
Payment Service
    │
 ┌──┼──┬──┐
 ▼  ▼  ▼  ▼
P1 P2 P3 P4

Kubernetes Service distributes traffic among all pods.

This is internal load balancing.

No internet involved.

---

Complete Flow

Browser
   │
   ▼
External Load Balancer
   │
   ▼
API Gateway
   │
   ▼
Accounts Service
   │
   ▼
Payment Service
   │
   ▼
Database

Notice

Only the first request comes through the external load balancer.

Everything inside the cluster uses Kubernetes Services.

---

Example

Suppose Payment has three pods.

payment-service

        │
        ▼
-----------------------
| Pod1              |
| Pod2              |
| Pod3              |
-----------------------

Accounts Service sends requests.

Request 1

Accounts
   │
   ▼
payment-service
   │
   ▼
Pod1

Request 2

Accounts
   │
   ▼
payment-service
   │
   ▼
Pod3

Request 3

Accounts
   │
   ▼
payment-service
   │
   ▼
Pod2

Accounts Service never knows which pod handled the request.

It only calls

http://payment-service

---

Before Kubernetes (Spring Cloud Netflix Era)

In older microservices architecture, we often used:

Accounts
      │
      ▼
Spring Cloud LoadBalancer
      │
      ▼
Eureka Server
      │
      ▼
Payment Instance List

Flow:

1. Payment instances register with Eureka.
2. Accounts asks Eureka for available instances.
3. Spring Cloud LoadBalancer chooses one instance.
4. Request is sent directly.

Accounts
    │
    ▼
Spring Cloud LoadBalancer
    │
    ▼
10.0.0.11
10.0.0.12
10.0.0.13

---

With Kubernetes

No Eureka is required.

Accounts
    │
    ▼
payment-service
    │
    ▼
Kubernetes Service
    │
 ┌──┼──┬──┐
 ▼  ▼  ▼  ▼
P1 P2 P3 P4

Kubernetes itself:

• Discovers pods
• Removes unhealthy pods
• Adds new pods automatically
• Balances requests

---

Modern Enterprise Architecture

                    Internet
                        │
                        ▼
          External Load Balancer
      (AWS ALB / Nginx Ingress)
                        │
                        ▼
                  API Gateway
                        │
         ┌──────────────┼──────────────┐
         ▼              ▼              ▼
    Accounts       Orders         Customer
         │              │
         │              ▼
         │         payment-service
         │              │
         │      Kubernetes Service
         │              │
         ▼      ┌───────┼────────┐
                ▼       ▼        ▼
             Payment1 Payment2 Payment3

• External load balancing: Handles traffic from clients into your cluster (e.g., to the API Gateway).
• Internal service-to-service load balancing: Happens inside the cluster. Services communicate using Kubernetes Service names (like http://payment-service), and Kubernetes automatically distributes requests across healthy pod replicas.

This separation is one of the key differences between traditional microservice deployments and Kubernetes-native architectures.