Search the docs, videos, blog, apps, teams, skills and release notes.
At a glance
What it is
Load balancing design. Algorithms (round-robin, least-connections, IP-hash, weighted), L4 vs L7, health checks, session persistence, SSL offloading, global load balancing, auto-scaling integration, connection draining.
A skill is a written procedure a bot loads when a job calls for it. This one is a single file, SKILL.md, and the whole file is on this page.
The whole file, exactly as it downloads. Read it before you rely on it.
SKILL.md
534 lines · 2,030 words
You are a load balancing design expert with deep knowledge of algorithms, layer 4 vs layer 7 balancing, health checking, session persistence, SSL offloading, global traffic management, and integration with auto-scaling systems.
Health before traffic - Never send traffic to unhealthy backends.
Gradual introduction - New backends should warm up before receiving full traffic.
Load balancing design. Algorithms (round-robin, least-connections, IP-hash, weighted), L4 vs L7, health checks, session persistence, SSL offloading, global load balancing, auto-scaling integration, connection draining.
Nothing. It ships with Murage, which is open source. Your bots run on the AI plan you already have.
Yes. Open it in Settings → Skills to read, edit or duplicate it. Skill Guard scans it again after every edit.
Download the app, connect the AI you already pay for, and tell your Chief of Staff what needs doing. Plan on about ten minutes from install to a working team.
No account needed · Runs on the AI plan you already pay for
Paste it into your agent's instructions (CLAUDE.md, AGENTS.md or a custom GPT), or add it to a bot in Murage.
Ask Ember for the bot, then switch the skill on in its settings.
Step 1
Send your Chief of Staff this message. It proposes the bot and its role first, and creates it when you say go.
Send this to Ember
“Create a bot that uses the Load Balancer skill, and show me the plan first.”
You're done when your Chief proposes the bot and its role, and the bot appears in your sidebar once you say go.
Step 2
You're done when you see the bot’s skills and the library.
Step 3
You're done when the skill shows as on for that bot.
Step 4
You're done when the bot takes the job and opens the skill when it needs it.
Prefer to do it yourself? Choose + in the sidebar, then New Bot.Read the full guide to skills and Skill Guard
What it teaches a bot, what it covers, and the whole file to read.
The principle
You are a load balancing design expert with deep knowledge of algorithms, layer 4 vs layer 7 balancing, health checking, session persistence, SSL offloading, global traffic management, and integration with auto-scaling systems.
What it covers
Core Principles
L4 vs L7 Load Balancing
Load Balancing Algorithms
Health Checks
Session Persistence (Sticky Sessions)
SSL Offloading
Global Load Balancing
Auto-Scaling Integration
HAProxy Configuration Example
Monitoring Load Balancers
Production Checklist
When to Use
Output Format
Example
Edge Cases
Graceful removal - Drain connections before removing a backend.
Appropriate algorithm - Match the algorithm to the workload characteristics.
Observe everything - Monitor active connections, latency, error rates per backend.
Operates at: TCP/UDP level
Sees: Source IP, destination IP, ports
Cannot see: HTTP headers, URLs, cookies, request body
How it works:
Client -> LB (TCP connection) -> Backend (new TCP connection)
Routing decision based on: IP + Port only
# ... (condensed) ...
- Non-HTTP protocols (SMTP, custom TCP)
- Maximum performance requirements
- SSL passthrough (client-to-backend encryption)Operates at: HTTP/HTTPS level
Sees: Full HTTP request (headers, URL, cookies, body)
How it works:
Client -> LB (HTTP request parsed) -> Backend (new request forwarded)
Routing decision based on: URL path, host header, cookies, headers, etc.
Pros:
# ... (condensed) ...
- Microservices with path-based routing
- SSL termination
- A/B testing, canary deploymentsIs it HTTP/HTTPS traffic?
YES -> Layer 7 (almost always the right choice for web)
NO -> Is it a well-known TCP protocol (database, SMTP)?
YES -> Layer 4
NO -> Is it a custom protocol?
YES -> Layer 4
NO -> Layer 4
# ... (condensed) ...
Do you need maximum throughput with minimal latency?
YES -> Layer 4Distributes requests sequentially across backends.
Backend A -> Backend B -> Backend C -> Backend A -> ...
Best for:
- Backends with identical capacity
- Stateless applications
- Uniform request cost
# ... (condensed) ...
server 10.0.1.11:8080;
server 10.0.1.12:8080;
}Like round robin, but backends receive traffic proportional to their weight.
Backend A (weight=5) gets 5x more traffic than Backend C (weight=1)
Best for:
- Backends with different capacities (different hardware)
- Gradual rollout (canary with low weight)
- Migrating between different instance types
# ... (condensed) ...
server 10.0.1.11:8080 weight=3; # 30% traffic
server 10.0.1.12:8080 weight=2; # 20% traffic
}Sends new requests to the backend with fewest active connections.
Best for:
- Requests with varying processing times
- Long-lived connections (WebSocket, gRPC streams)
- Backends with uneven load from other sources
Example (Nginx):
# ... (condensed) ...
server 10.0.1.11:8080;
server 10.0.1.12:8080;
}Combines least connections with weights. Considers both active connections
and backend capacity.
Score = active_connections / weight
Route to backend with lowest score.
Example (HAProxy):
backend app_servers
balance leastconn
server web1 10.0.1.10:8080 weight 5
server web2 10.0.1.11:8080 weight 3Routes requests from the same client IP to the same backend.
Uses a hash of the client IP to select backend.
Best for:
- Applications requiring session affinity without cookies
- When cookie-based persistence is not possible (non-HTTP)
- Simple sticky sessions
# ... (condensed) ...
server 10.0.1.11:8080;
server 10.0.1.12:8080;
}Like IP hash, but adding/removing backends only affects a small fraction
of the key space (minimal disruption).
Best for:
- Caching layers (maximize cache hit ratio)
- Stateful services that need sticky routing
- Adding/removing backends frequently (auto-scaling)
# ... (condensed) ...
server 10.0.1.11:8080;
server 10.0.1.12:8080;
}Active health checks:
LB periodically sends probe requests to backends.
If probe fails N times, backend is marked unhealthy.
If probe succeeds M times, backend is marked healthy again.
Passive health checks:
LB monitors actual traffic responses from backends.
If a backend returns too many errors, it is marked unhealthy.
No extra probe traffic, but slower to detect failure.
Best practice: Use BOTH active and passive health checks.# Nginx (active health checks - requires Nginx Plus or OpenResty)
upstream backend {
zone backend 64k;
server 10.0.1.10:8080;
server 10.0.1.11:8080;
server 10.0.1.12:8080;
}
# ... (condensed) ...
server 10.0.1.11:8080 max_fails=3 fail_timeout=30s;
server 10.0.1.12:8080 max_fails=3 fail_timeout=30s;
}# HAProxy health checks
backend app_servers
option httpchk GET /health
http-check expect status 200
server web1 10.0.1.10:8080 check inter 5s fall 3 rise 2
server web2 10.0.1.11:8080 check inter 5s fall 3 rise 2
server web3 10.0.1.12:8080 check inter 5s fall 3 rise 2
# inter: check interval
# fall: consecutive failures to mark unhealthy
# rise: consecutive successes to mark healthyLB sets a cookie on the first response, subsequent requests with that cookie
go to the same backend.
Pros: Precise, survives IP changes (mobile)
Cons: Requires cookie support (browsers), cookie overhead# HAProxy cookie persistence
backend app_servers
balance roundrobin
cookie SERVERID insert indirect nocache httponly secure
server web1 10.0.1.10:8080 cookie web1
server web2 10.0.1.11:8080 cookie web2
server web3 10.0.1.12:8080 cookie web3# Nginx Plus sticky cookie
upstream backend {
sticky cookie srv_id expires=1h domain=.example.com httponly secure path=/;
server 10.0.1.10:8080;
server 10.0.1.11:8080;
}Client --[HTTPS]--> Load Balancer --[HTTP]--> Backend
Benefits:
- Centralized certificate management
- Reduced CPU on backends
- Simplified backend configuration
- Load balancer can inspect and route based on HTTP content
# ... (condensed) ...
backend app_servers
server web1 10.0.1.10:8080 checkClient --[HTTPS]--> Load Balancer --[HTTPS]--> Backend
(LB does NOT decrypt; routes at TCP level)
Benefits:
- End-to-end encryption (LB cannot see content)
- Compliance requirements (data never decrypted in transit)
- Backend controls its own certificates
# ... (condensed) ...
backend app_servers_tcp
mode tcp
server web1 10.0.1.10:443 checkClient --[HTTPS]--> Load Balancer --[HTTPS]--> Backend
(LB decrypts, inspects, re-encrypts to backend)
Benefits:
- Content-based routing AND encryption to backend
- Defense in depth
Drawback:
- Double encryption overhead
- More complex certificate managementGlobal Load Balancing routes users to the nearest regional cluster.
┌──────────────┐
│ DNS-based │
User ------->│ Global LB │
│ (Route 53, │
│ Cloudflare) │
└──────┬───────┘
# ... (condensed) ...
Latency: Route based on measured latency to each region
Weighted: Route percentage of traffic to each region
Failover: Route to secondary if primary health check failsAWS:
- Global Accelerator: Anycast IPs, TCP/UDP, static IPs
- CloudFront: HTTP/S CDN with origin failover
- Route 53: DNS-based (geolocation, latency, weighted, failover)
GCP:
- Global HTTP(S) Load Balancer: Single anycast IP, global
- Global TCP/SSL Proxy: L4, anycast IP
# ... (condensed) ...
Cloudflare:
- Load Balancing: Global, DNS or proxy-based
- Anycast network: Automatic geographic routing1. Load increases -> Auto-scaler adds new instances
2. New instances register with load balancer (or LB discovers via service discovery)
3. Health check passes -> LB starts sending traffic
4. Slow start period -> Gradually increase traffic to new instance
5. Load decreases -> Auto-scaler wants to remove instances
6. LB stops sending NEW requests to instance being removed
7. Existing connections drain (connection draining period)
# ... (condensed) ...
- Health check grace period: Time for new instances to start before checking
- Slow start: Gradually ramp up traffic to new instances
- Deregistration delay: Time to drain connections before removalWhen removing a backend from the pool:
1. Stop sending NEW connections to the backend
2. Allow EXISTING connections to complete
3. Wait up to drain_timeout seconds
4. Force-close remaining connections after timeout
Configuration:
AWS ALB: Deregistration delay (default 300s, recommend 30-120s)
# ... (condensed) ...
echo "set server app_servers/web1 state drain" | socat stdio [system-path]
# Wait for connections to complete, then:
echo "set server app_servers/web1 state maint" | socat stdio [system-path]Gradually increase traffic to new backends over a warm-up period.
Prevents overwhelming a cold instance (cold caches, JIT not compiled, etc.)
AWS ALB: Slow start duration (30-900 seconds)
- New target starts at 0 traffic
- Linearly increases to full share over the duration
HAProxy:
# ... (condensed) ...
upstream backend {
server 10.0.1.10:8080 slow_start=30s;
}global
log stdout format raw local0
maxconn 50000
stats socket [system-path] mode 660 level admin
defaults
mode http
log global
# ... (condensed) ...
stats uri /stats
stats refresh 10s
stats admin if LOCALHOSTTraffic Metrics:
- Request rate (requests/second)
- Active connections (current)
- New connections (per second)
- Bandwidth (bytes in/out)
Health Metrics:
- Healthy backend count
# ... (condensed) ...
- Connection errors
- Timeout rate
- Rejected connections (capacity)CRITICAL:
- All backends unhealthy (zero healthy targets)
- Error rate > 10% for 5 minutes
- Response time P99 > 10 seconds for 5 minutes
WARNING:
- Healthy backend count < minimum threshold
- Error rate > 1% for 10 minutes
# ... (condensed) ...
- Backend added/removed from pool
- Traffic spike (> 2x normal)
- Connection draining startedCore Configuration:
[ ] Appropriate algorithm selected for workload
[ ] Health checks configured (active and passive)
[ ] Health check endpoint returns meaningful status
[ ] Connection timeouts set appropriately
[ ] Retries and redispatch configured
[ ] Connection limits set to prevent overload
# ... (condensed) ...
[ ] Access logs with request timing information
[ ] SSL certificate expiration monitoring
[ ] Capacity planning alerts (connection limits)Use this skill when:
Designing or implementing load balancer solutions
Reviewing or improving existing load balancer approaches
Making architectural or implementation decisions about load balancer
Learning load balancer patterns and best practices
Troubleshooting load balancer-related issues
Do NOT use this skill when:
The question is about a fundamentally different technology domain
A more specific sibling skill covers the exact topic needed
The user needs a complete hands-on tutorial rather than expert guidance
# Load Balancer Analysis
## Context Assessment
[Situation summary and constraints]
## Recommended Approach
[Primary recommendation with rationale]
## Implementation Steps
1. [Step with specific details]
2. [Step with specific details]
3. [Step with specific details]
## Trade-offs and Considerations
- [Key trade-off 1]
- [Key trade-off 2]
## Next Steps
- [Immediate action item]
- [Follow-up action item]Input: "Help me implement load balancer for a medium-scale production application"
Output: A structured analysis covering current state assessment, recommended load balancer approach with specific patterns, implementation roadmap with milestones, and risk mitigation strategies tailored to the application scale and constraints.
Legacy system integration: When load balancer must coexist with legacy approaches, provide a gradual migration path rather than a complete rewrite
Scale mismatch: When the solution complexity exceeds the project scale, recommend a simpler approach and note when to revisit
Team skill gaps: When the team lacks experience with the recommended approach, include learning resources and simpler alternatives
Conflicting requirements: When constraints conflict (e.g., performance vs. maintainability), explicitly state the trade-off and recommend based on stated priorities
SKILL.md
534 lines · 2,030 words
The text of SKILL.md is shown as it downloads, under the Apache-2.0 license, by Ferrox Labs. The license is the one in the file's own frontmatter.
Your new bot
Load Balancer bot
Waiting in your sidebar once you say go.
No account needed · Runs on the AI plan you already pay for