SOP-011: Rate Limiting Configuration

Document Control

Field	Value
Document ID	SOP-011
Version	1.0
Status	Active
Owner	Claude AI Operations
Last Updated	2024-12-01
Review Date	2025-03-01

Overview

This SOP defines procedures for implementing and managing rate limiting for Claude AI API usage. It covers quota management, throttling strategies, monitoring, and optimization techniques to ensure reliable service within API limits while maximizing efficiency.

Rate Limiting Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Rate Limiting Stack                      │
├─────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐        │
│  │   Client    │  │   Local     │  │   Remote    │        │
│  │ Rate Limits │  │ Throttling  │  │ API Limits  │        │
│  │             │  │             │  │             │        │
│  │ • Request   │  │ • Queue     │  │ • Anthropic │        │
│  │ • Token     │  │ • Backoff   │  │ • Model     │        │
│  │ • Time      │  │ • Circuit   │  │ • Tier      │        │
│  └─────────────┘  └─────────────┘  └─────────────┘        │
├─────────────────────────────────────────────────────────────┤
│                 Management Layer                            │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐        │
│  │ Monitoring  │  │  Alerting   │  │ Optimization│        │
│  │ & Metrics   │  │ & Response  │  │ & Tuning    │        │
│  └─────────────┘  └─────────────┘  └─────────────┘        │
└─────────────────────────────────────────────────────────────┘

Prerequisites

Understanding of Anthropic API rate limits
Access to API usage monitoring
Knowledge of rate limiting algorithms
Monitoring and alerting infrastructure

Procedure

Step 1: Rate Limit Configuration

1.1 API Limits Understanding

bash

# Create rate limit configuration directory
mkdir -p ~/.config/claude/rate-limits/{configs,monitoring,logs}

# Document current API limits
cat > ~/.config/claude/rate-limits/api-limits.yaml << 'EOF'
# Anthropic API Rate Limits (as of 2024-12-01)
# Verify current limits at: https://docs.anthropic.com/api/rate-limits

api_limits:
  tier_1:
    name: "Starter"
    requests_per_minute: 20
    tokens_per_minute: 40000
    tokens_per_day: 1000000
    models:
      - claude-3-haiku
      - claude-3-5-haiku
    
  tier_2:
    name: "Developer"
    requests_per_minute: 50
    tokens_per_minute: 100000
    tokens_per_day: 2500000
    models:
      - claude-3-haiku
      - claude-3-5-haiku
      - claude-3-sonnet
      - claude-3-5-sonnet
    
  tier_3:
    name: "Professional"
    requests_per_minute: 100
    tokens_per_minute: 200000
    tokens_per_day: 5000000
    models:
      - claude-3-haiku
      - claude-3-5-haiku
      - claude-3-sonnet
      - claude-3-5-sonnet
      - claude-3-opus
    
  tier_4:
    name: "Enterprise"
    requests_per_minute: 1000
    tokens_per_minute: 2000000
    tokens_per_day: 50000000
    models:
      - "all"

# Model-specific considerations
model_factors:
  claude-3-opus:
    complexity_multiplier: 3.0
    recommended_batch_size: 1
  
  claude-3-5-sonnet:
    complexity_multiplier: 2.0
    recommended_batch_size: 3
  
  claude-3-sonnet:
    complexity_multiplier: 2.0
    recommended_batch_size: 3
    
  claude-3-5-haiku:
    complexity_multiplier: 1.0
    recommended_batch_size: 10
    
  claude-3-haiku:
    complexity_multiplier: 1.0
    recommended_batch_size: 10

# Rate limiting strategies
strategies:
  fixed_window:
    description: "Fixed time window rate limiting"
    use_case: "Simple applications with predictable load"
    
  sliding_window:
    description: "Sliding window rate limiting"
    use_case: "Better distribution of requests over time"
    
  token_bucket:
    description: "Token bucket algorithm"
    use_case: "Burst handling with sustained rate control"
    
  exponential_backoff:
    description: "Exponential backoff on rate limit hits"
    use_case: "Graceful degradation under high load"
EOF

1.2 Local Rate Limiting Implementation

bash

cat > ~/.local/bin/claude-rate-limiter << 'EOF'
#!/bin/bash
# Claude Rate Limiter Implementation

set -euo pipefail

RATE_LIMIT_DIR="$HOME/.config/claude/rate-limits"
STATE_FILE="$RATE_LIMIT_DIR/state.json"
CONFIG_FILE="$RATE_LIMIT_DIR/config.yaml"
LOG_FILE="$RATE_LIMIT_DIR/logs/rate-limiter.log"

# Initialize state file if it doesn't exist
init_state() {
    if [ ! -f "$STATE_FILE" ]; then
        mkdir -p "$(dirname "$STATE_FILE")"
        cat > "$STATE_FILE" << 'EOF'
{
  "requests": {
    "minute": {"count": 0, "window_start": 0},
    "day": {"count": 0, "window_start": 0}
  },
  "tokens": {
    "minute": {"count": 0, "window_start": 0},
    "day": {"count": 0, "window_start": 0}
  },
  "last_request": 0,
  "backoff_until": 0
}
EOF
    fi
}

log_event() {
    local level="$1"
    local message="$2"
    local timestamp=$(date --iso-8601=seconds)
    echo "[$timestamp] [$level] $message" | tee -a "$LOG_FILE"
}

get_current_limits() {
    local tier="${CLAUDE_TIER:-tier_1}"
    
    # Load limits from config (simplified - use yq for YAML parsing)
    case "$tier" in
        "tier_1")
            echo "20:40000:1000000"  # requests/min:tokens/min:tokens/day
            ;;
        "tier_2")
            echo "50:100000:2500000"
            ;;
        "tier_3")
            echo "100:200000:5000000"
            ;;
        "tier_4")
            echo "1000:2000000:50000000"
            ;;
        *)
            echo "20:40000:1000000"  # Default to tier 1
            ;;
    esac
}

update_state() {
    local request_count="$1"
    local token_count="$2"
    local current_time="$3"
    
    # Read current state
    local state=$(cat "$STATE_FILE")
    
    # Update state with new request/token counts
    local updated_state=$(echo "$state" | jq \
        --arg req_count "$request_count" \
        --arg tok_count "$token_count" \
        --arg curr_time "$current_time" '
        .requests.minute.count = ($req_count | tonumber) |
        .requests.minute.window_start = ($curr_time | tonumber) |
        .tokens.minute.count = ($tok_count | tonumber) |
        .tokens.minute.window_start = ($curr_time | tonumber) |
        .last_request = ($curr_time | tonumber)
    ')
    
    echo "$updated_state" > "$STATE_FILE"
}

check_rate_limits() {
    local request_count="${1:-1}"
    local estimated_tokens="${2:-1000}"
    
    init_state
    
    local current_time=$(date +%s)
    local state=$(cat "$STATE_FILE")
    
    # Get current limits
    IFS=':' read -r req_limit_min tok_limit_min tok_limit_day <<< "$(get_current_limits)"
    
    # Check if we're in backoff period
    local backoff_until=$(echo "$state" | jq -r '.backoff_until')
    if [ "$backoff_until" -gt "$current_time" ]; then
        local wait_time=$((backoff_until - current_time))
        log_event "INFO" "In backoff period, waiting ${wait_time}s"
        return 1
    fi
    
    # Check minute windows
    local req_minute_start=$(echo "$state" | jq -r '.requests.minute.window_start')
    local req_minute_count=$(echo "$state" | jq -r '.requests.minute.count')
    local tok_minute_start=$(echo "$state" | jq -r '.tokens.minute.window_start')
    local tok_minute_count=$(echo "$state" | jq -r '.tokens.minute.count')
    
    # Reset minute counters if window expired
    if [ $((current_time - req_minute_start)) -gt 60 ]; then
        req_minute_count=0
    fi
    if [ $((current_time - tok_minute_start)) -gt 60 ]; then
        tok_minute_count=0
    fi
    
    # Check limits
    if [ $((req_minute_count + request_count)) -gt "$req_limit_min" ]; then
        log_event "WARN" "Request rate limit would be exceeded: $((req_minute_count + request_count))/$req_limit_min per minute"
        return 1
    fi
    
    if [ $((tok_minute_count + estimated_tokens)) -gt "$tok_limit_min" ]; then
        log_event "WARN" "Token rate limit would be exceeded: $((tok_minute_count + estimated_tokens))/$tok_limit_min per minute"
        return 1
    fi
    
    # Check daily token limit
    local tok_day_start=$(echo "$state" | jq -r '.tokens.day.window_start')
    local tok_day_count=$(echo "$state" | jq -r '.tokens.day.count')
    
    # Reset daily counter if window expired
    if [ $((current_time - tok_day_start)) -gt 86400 ]; then
        tok_day_count=0
    fi
    
    if [ $((tok_day_count + estimated_tokens)) -gt "$tok_limit_day" ]; then
        log_event "WARN" "Daily token limit would be exceeded: $((tok_day_count + estimated_tokens))/$tok_limit_day per day"
        return 1
    fi
    
    # Update counters
    update_state "$((req_minute_count + request_count))" "$((tok_minute_count + estimated_tokens))" "$current_time"
    
    log_event "INFO" "Rate limit check passed: ${request_count} requests, ${estimated_tokens} tokens"
    return 0
}

implement_backoff() {
    local retry_count="${1:-1}"
    local base_delay="${2:-1}"
    
    # Exponential backoff with jitter
    local delay=$(echo "scale=2; $base_delay * 2^($retry_count - 1) + $(shuf -i 1-100 -n 1) / 100" | bc)
    local max_delay=300  # 5 minutes max
    
    if (( $(echo "$delay > $max_delay" | bc -l) )); then
        delay=$max_delay
    fi
    
    local backoff_until=$(echo "$(date +%s) + $delay" | bc | cut -d. -f1)
    
    # Update state with backoff time
    local state=$(cat "$STATE_FILE")
    local updated_state=$(echo "$state" | jq --arg backoff "$backoff_until" '.backoff_until = ($backoff | tonumber)')
    echo "$updated_state" > "$STATE_FILE"
    
    log_event "INFO" "Implementing backoff: ${delay}s (retry ${retry_count})"
    
    echo "$delay"
}

wait_for_quota() {
    local required_requests="${1:-1}"
    local required_tokens="${2:-1000}"
    
    local retry_count=1
    local max_retries=10
    
    while [ $retry_count -le $max_retries ]; do
        if check_rate_limits "$required_requests" "$required_tokens"; then
            return 0
        fi
        
        local delay=$(implement_backoff "$retry_count")
        log_event "INFO" "Waiting for quota: ${delay}s (attempt ${retry_count}/${max_retries})"
        
        sleep "$delay"
        ((retry_count++))
    done
    
    log_event "ERROR" "Max retries exceeded waiting for quota"
    return 1
}

show_status() {
    init_state
    
    echo "🚦 Claude Rate Limiter Status"
    echo "============================="
    
    local current_time=$(date +%s)
    local state=$(cat "$STATE_FILE")
    local tier="${CLAUDE_TIER:-tier_1}"
    
    IFS=':' read -r req_limit_min tok_limit_min tok_limit_day <<< "$(get_current_limits)"
    
    echo "Configuration:"
    echo "  Tier: $tier"
    echo "  Limits: ${req_limit_min} req/min, ${tok_limit_min} tokens/min, ${tok_limit_day} tokens/day"
    echo
    
    # Current usage
    local req_minute_count=$(echo "$state" | jq -r '.requests.minute.count')
    local req_minute_start=$(echo "$state" | jq -r '.requests.minute.window_start')
    local tok_minute_count=$(echo "$state" | jq -r '.tokens.minute.count')
    local tok_minute_start=$(echo "$state" | jq -r '.tokens.minute.window_start')
    local tok_day_count=$(echo "$state" | jq -r '.tokens.day.count')
    local tok_day_start=$(echo "$state" | jq -r '.tokens.day.window_start')
    
    # Reset counters if windows expired
    if [ $((current_time - req_minute_start)) -gt 60 ]; then
        req_minute_count=0
    fi
    if [ $((current_time - tok_minute_start)) -gt 60 ]; then
        tok_minute_count=0
    fi
    if [ $((current_time - tok_day_start)) -gt 86400 ]; then
        tok_day_count=0
    fi
    
    echo "Current usage:"
    echo "  Requests/minute: ${req_minute_count}/${req_limit_min} ($(echo "scale=1; $req_minute_count * 100 / $req_limit_min" | bc)%)"
    echo "  Tokens/minute: ${tok_minute_count}/${tok_limit_min} ($(echo "scale=1; $tok_minute_count * 100 / $tok_limit_min" | bc)%)"
    echo "  Tokens/day: ${tok_day_count}/${tok_limit_day} ($(echo "scale=1; $tok_day_count * 100 / $tok_limit_day" | bc)%)"
    
    # Backoff status
    local backoff_until=$(echo "$state" | jq -r '.backoff_until')
    if [ "$backoff_until" -gt "$current_time" ]; then
        local wait_time=$((backoff_until - current_time))
        echo "  Status: In backoff (${wait_time}s remaining)"
    else
        echo "  Status: Active"
    fi
}

reset_state() {
    local force="${1:-false}"
    
    if [ "$force" = "true" ] || read -p "Reset rate limiter state? (y/N): " -n 1 -r && [[ $REPLY =~ ^[Yy]$ ]]; then
        echo
        rm -f "$STATE_FILE"
        init_state
        log_event "INFO" "Rate limiter state reset"
        echo "✅ State reset successfully"
    else
        echo "❌ Reset cancelled"
    fi
}

show_help() {
    cat << 'EOF'
Claude Rate Limiter

Usage: claude-rate-limiter [COMMAND] [OPTIONS]

Commands:
    check <requests> <tokens>    Check if request would exceed limits
    wait <requests> <tokens>     Wait for quota availability
    status                       Show current rate limit status
    reset [--force]             Reset rate limiter state
    help                        Show this help

Environment Variables:
    CLAUDE_TIER                 Set your API tier (tier_1, tier_2, tier_3, tier_4)

Examples:
    claude-rate-limiter check 1 1500
    claude-rate-limiter wait 1 2000
    claude-rate-limiter status
    export CLAUDE_TIER=tier_3 && claude-rate-limiter status
EOF
}

case "${1:-help}" in
    "check")
        if [ -z "${2:-}" ] || [ -z "${3:-}" ]; then
            echo "❌ Request count and token count required"
            exit 1
        fi
        if check_rate_limits "$2" "$3"; then
            echo "✅ Request within rate limits"
            exit 0
        else
            echo "❌ Request would exceed rate limits"
            exit 1
        fi
        ;;
    "wait")
        if [ -z "${2:-}" ] || [ -z "${3:-}" ]; then
            echo "❌ Request count and token count required"
            exit 1
        fi
        if wait_for_quota "$2" "$3"; then
            echo "✅ Quota available"
            exit 0
        else
            echo "❌ Unable to obtain quota"
            exit 1
        fi
        ;;
    "status")
        show_status
        ;;
    "reset")
        reset_state "${2:-false}"
        ;;
    "help"|*)
        show_help
        ;;
esac
EOF

chmod +x ~/.local/bin/claude-rate-limiter

Step 2: Advanced Rate Limiting Strategies

2.1 Token Bucket Implementation

bash

cat > ~/.local/bin/claude-token-bucket << 'EOF'
#!/bin/bash
# Claude Token Bucket Rate Limiter

set -euo pipefail

BUCKET_DIR="$HOME/.config/claude/rate-limits/buckets"
mkdir -p "$BUCKET_DIR"

create_bucket() {
    local bucket_name="$1"
    local capacity="$2"
    local refill_rate="$3"
    local refill_period="${4:-60}"  # seconds
    
    local bucket_file="$BUCKET_DIR/${bucket_name}.json"
    
    cat > "$bucket_file" << EOF
{
  "name": "$bucket_name",
  "capacity": $capacity,
  "current_tokens": $capacity,
  "refill_rate": $refill_rate,
  "refill_period": $refill_period,
  "last_refill": $(date +%s),
  "created": "$(date --iso-8601=seconds)"
}
EOF
    
    echo "✅ Created token bucket: $bucket_name (capacity: $capacity, refill: $refill_rate/$refill_period)"
}

refill_bucket() {
    local bucket_file="$1"
    local current_time=$(date +%s)
    
    if [ ! -f "$bucket_file" ]; then
        return 1
    fi
    
    local bucket_data=$(cat "$bucket_file")
    local capacity=$(echo "$bucket_data" | jq -r '.capacity')
    local current_tokens=$(echo "$bucket_data" | jq -r '.current_tokens')
    local refill_rate=$(echo "$bucket_data" | jq -r '.refill_rate')
    local refill_period=$(echo "$bucket_data" | jq -r '.refill_period')
    local last_refill=$(echo "$bucket_data" | jq -r '.last_refill')
    
    # Calculate tokens to add
    local time_passed=$((current_time - last_refill))
    local tokens_to_add=$(echo "scale=0; $time_passed * $refill_rate / $refill_period" | bc)
    
    # Update current tokens (don't exceed capacity)
    local new_token_count=$((current_tokens + tokens_to_add))
    if [ "$new_token_count" -gt "$capacity" ]; then
        new_token_count="$capacity"
    fi
    
    # Update bucket
    local updated_bucket=$(echo "$bucket_data" | jq \
        --arg new_tokens "$new_token_count" \
        --arg refill_time "$current_time" '
        .current_tokens = ($new_tokens | tonumber) |
        .last_refill = ($refill_time | tonumber)
    ')
    
    echo "$updated_bucket" > "$bucket_file"
    
    echo "$new_token_count"
}

consume_tokens() {
    local bucket_name="$1"
    local tokens_requested="$2"
    
    local bucket_file="$BUCKET_DIR/${bucket_name}.json"
    
    if [ ! -f "$bucket_file" ]; then
        echo "❌ Bucket not found: $bucket_name"
        return 1
    fi
    
    # Refill bucket first
    local current_tokens=$(refill_bucket "$bucket_file")
    
    # Check if we have enough tokens
    if [ "$current_tokens" -lt "$tokens_requested" ]; then
        echo "❌ Insufficient tokens: $current_tokens available, $tokens_requested requested"
        return 1
    fi
    
    # Consume tokens
    local remaining_tokens=$((current_tokens - tokens_requested))
    local bucket_data=$(cat "$bucket_file")
    local updated_bucket=$(echo "$bucket_data" | jq \
        --arg remaining "$remaining_tokens" '
        .current_tokens = ($remaining | tonumber)
    ')
    
    echo "$updated_bucket" > "$bucket_file"
    
    echo "✅ Consumed $tokens_requested tokens, $remaining_tokens remaining"
    return 0
}

show_bucket_status() {
    local bucket_name="$1"
    local bucket_file="$BUCKET_DIR/${bucket_name}.json"
    
    if [ ! -f "$bucket_file" ]; then
        echo "❌ Bucket not found: $bucket_name"
        return 1
    fi
    
    # Refill bucket to get current status
    local current_tokens=$(refill_bucket "$bucket_file")
    
    local bucket_data=$(cat "$bucket_file")
    local capacity=$(echo "$bucket_data" | jq -r '.capacity')
    local refill_rate=$(echo "$bucket_data" | jq -r '.refill_rate')
    local refill_period=$(echo "$bucket_data" | jq -r '.refill_period')
    local fill_percentage=$(echo "scale=1; $current_tokens * 100 / $capacity" | bc)
    
    echo "🪣 Token Bucket: $bucket_name"
    echo "================================"
    echo "Capacity: $capacity tokens"
    echo "Current: $current_tokens tokens (${fill_percentage}%)"
    echo "Refill rate: $refill_rate tokens per ${refill_period}s"
    
    # Calculate time to full
    if [ "$current_tokens" -lt "$capacity" ]; then
        local tokens_needed=$((capacity - current_tokens))
        local time_to_full=$(echo "scale=1; $tokens_needed * $refill_period / $refill_rate" | bc)
        echo "Time to full: ${time_to_full}s"
    else
        echo "Status: Full"
    fi
}

setup_default_buckets() {
    echo "🔧 Setting up default token buckets..."
    
    # Request bucket - matches API request limits
    create_bucket "requests" 100 50 60  # 50 requests per minute
    
    # Token bucket - matches API token limits  
    create_bucket "tokens" 200000 100000 60  # 100k tokens per minute
    
    # Burst bucket - for handling short bursts
    create_bucket "burst" 10 5 30  # 5 requests per 30 seconds
    
    echo "✅ Default buckets created"
}

show_help() {
    cat << 'EOF'
Claude Token Bucket Rate Limiter

Usage: claude-token-bucket [COMMAND] [OPTIONS]

Commands:
    create <name> <capacity> <refill_rate> [refill_period]
        Create new token bucket
        
    consume <bucket> <tokens>
        Consume tokens from bucket
        
    status <bucket>
        Show bucket status
        
    list
        List all buckets
        
    setup
        Setup default buckets
        
    help
        Show this help

Examples:
    claude-token-bucket create requests 100 50 60
    claude-token-bucket consume requests 1
    claude-token-bucket status requests
    claude-token-bucket setup
EOF
}

case "${1:-help}" in
    "create")
        if [ -z "${2:-}" ] || [ -z "${3:-}" ] || [ -z "${4:-}" ]; then
            echo "❌ Bucket name, capacity, and refill rate required"
            exit 1
        fi
        create_bucket "$2" "$3" "$4" "${5:-60}"
        ;;
    "consume")
        if [ -z "${2:-}" ] || [ -z "${3:-}" ]; then
            echo "❌ Bucket name and token count required"
            exit 1
        fi
        consume_tokens "$2" "$3"
        ;;
    "status")
        if [ -z "${2:-}" ]; then
            echo "❌ Bucket name required"
            exit 1
        fi
        show_bucket_status "$2"
        ;;
    "list")
        echo "📋 Token Buckets:"
        for bucket in "$BUCKET_DIR"/*.json; do
            if [ -f "$bucket" ]; then
                local name=$(basename "$bucket" .json)
                local data=$(cat "$bucket")
                local capacity=$(echo "$data" | jq -r '.capacity')
                local current=$(refill_bucket "$bucket")
                echo "  $name: ${current}/${capacity} tokens"
            fi
        done
        ;;
    "setup")
        setup_default_buckets
        ;;
    "help"|*)
        show_help
        ;;
esac
EOF

chmod +x ~/.local/bin/claude-token-bucket

2.2 Request Queue Manager

bash

cat > ~/.local/bin/claude-request-queue << 'EOF'
#!/bin/bash
# Claude Request Queue Manager

set -euo pipefail

QUEUE_DIR="$HOME/.config/claude/rate-limits/queue"
QUEUE_FILE="$QUEUE_DIR/pending.json"
PROCESSING_FILE="$QUEUE_DIR/processing.json"
COMPLETED_FILE="$QUEUE_DIR/completed.json"
LOG_FILE="$QUEUE_DIR/queue.log"

mkdir -p "$QUEUE_DIR"

log_queue_event() {
    local level="$1"
    local message="$2"
    local timestamp=$(date --iso-8601=seconds)
    echo "[$timestamp] [$level] $message" | tee -a "$LOG_FILE"
}

init_queue() {
    if [ ! -f "$QUEUE_FILE" ]; then
        echo "[]" > "$QUEUE_FILE"
    fi
    if [ ! -f "$PROCESSING_FILE" ]; then
        echo "[]" > "$PROCESSING_FILE"
    fi
    if [ ! -f "$COMPLETED_FILE" ]; then
        echo "[]" > "$COMPLETED_FILE"
    fi
}

add_request() {
    local request_id="$1"
    local model="$2"
    local estimated_tokens="$3"
    local priority="${4:-5}"  # 1 (high) to 10 (low)
    local payload="$5"
    
    init_queue
    
    local request_data='{
        "id": "'$request_id'",
        "model": "'$model'",
        "estimated_tokens": '$estimated_tokens',
        "priority": '$priority',
        "payload": "'$(echo "$payload" | base64 -w 0)'",
        "created_at": "'"$(date --iso-8601=seconds)"'",
        "status": "pending"
    }'
    
    # Add to queue and sort by priority
    local current_queue=$(cat "$QUEUE_FILE")
    local updated_queue=$(echo "$current_queue" | jq --argjson req "$request_data" '. + [$req] | sort_by(.priority)')
    
    echo "$updated_queue" > "$QUEUE_FILE"
    
    log_queue_event "INFO" "Request queued: $request_id (priority: $priority, tokens: $estimated_tokens)"
    
    echo "✅ Request added to queue: $request_id"
}

process_queue() {
    local max_concurrent="${1:-3}"
    local dry_run="${2:-false}"
    
    init_queue
    
    echo "🔄 Processing request queue (max concurrent: $max_concurrent)"
    
    while true; do
        # Get pending requests
        local pending_requests=$(cat "$QUEUE_FILE")
        local pending_count=$(echo "$pending_requests" | jq 'length')
        
        if [ "$pending_count" -eq 0 ]; then
            echo "📭 No pending requests"
            break
        fi
        
        # Get currently processing requests
        local processing_requests=$(cat "$PROCESSING_FILE")
        local processing_count=$(echo "$processing_requests" | jq 'length')
        
        if [ "$processing_count" -ge "$max_concurrent" ]; then
            echo "⏳ Max concurrent requests reached ($processing_count/$max_concurrent)"
            sleep 5
            continue
        fi
        
        # Get next request (highest priority)
        local next_request=$(echo "$pending_requests" | jq '.[0]')
        local request_id=$(echo "$next_request" | jq -r '.id')
        local estimated_tokens=$(echo "$next_request" | jq -r '.estimated_tokens')
        
        # Check rate limits
        if ! claude-rate-limiter check 1 "$estimated_tokens" >/dev/null 2>&1; then
            log_queue_event "INFO" "Rate limit hit, waiting..."
            sleep 10
            continue
        fi
        
        # Move request to processing
        local updated_pending=$(echo "$pending_requests" | jq '.[1:]')
        local updated_processing=$(echo "$processing_requests" | jq --argjson req "$next_request" '. + [$req]')
        
        echo "$updated_pending" > "$QUEUE_FILE"
        echo "$updated_processing" > "$PROCESSING_FILE"
        
        log_queue_event "INFO" "Processing request: $request_id"
        
        if [ "$dry_run" = "true" ]; then
            echo "DRY RUN: Would process request $request_id"
            # Simulate processing time
            sleep 2
            complete_request "$request_id" "dry-run-success"
        else
            # Process request in background
            process_request "$request_id" &
        fi
        
        sleep 1  # Brief pause between dispatching requests
    done
    
    echo "✅ Queue processing completed"
}

process_request() {
    local request_id="$1"
    
    log_queue_event "INFO" "Starting processing: $request_id"
    
    # Get request details from processing queue
    local processing_requests=$(cat "$PROCESSING_FILE")
    local request_data=$(echo "$processing_requests" | jq --arg id "$request_id" '.[] | select(.id == $id)')
    
    if [ -z "$request_data" ] || [ "$request_data" = "null" ]; then
        log_queue_event "ERROR" "Request not found in processing queue: $request_id"
        return 1
    fi
    
    local model=$(echo "$request_data" | jq -r '.model')
    local payload=$(echo "$request_data" | jq -r '.payload' | base64 -d)
    
    # Simulate API call (replace with actual Claude API call)
    log_queue_event "INFO" "Making API call for: $request_id"
    
    # Placeholder for actual API call
    local start_time=$(date +%s)
    sleep $((RANDOM % 10 + 5))  # Simulate variable processing time
    local end_time=$(date +%s)
    local duration=$((end_time - start_time))
    
    local response="Simulated response for request $request_id using $model"
    
    # Complete the request
    complete_request "$request_id" "$response" "$duration"
}

complete_request() {
    local request_id="$1"
    local response="$2"
    local duration="${3:-0}"
    
    # Move from processing to completed
    local processing_requests=$(cat "$PROCESSING_FILE")
    local completed_requests=$(cat "$COMPLETED_FILE")
    
    local request_data=$(echo "$processing_requests" | jq --arg id "$request_id" '.[] | select(.id == $id)')
    local updated_processing=$(echo "$processing_requests" | jq --arg id "$request_id" '[.[] | select(.id != $id)]')
    
    # Add completion info
    local completed_request=$(echo "$request_data" | jq \
        --arg response "$(echo "$response" | base64 -w 0)" \
        --arg duration "$duration" \
        --arg completed_at "$(date --iso-8601=seconds)" '
        .response = $response |
        .duration = ($duration | tonumber) |
        .completed_at = $completed_at |
        .status = "completed"
    ')
    
    local updated_completed=$(echo "$completed_requests" | jq --argjson req "$completed_request" '. + [$req]')
    
    echo "$updated_processing" > "$PROCESSING_FILE"
    echo "$updated_completed" > "$COMPLETED_FILE"
    
    log_queue_event "INFO" "Request completed: $request_id (duration: ${duration}s)"
}

show_queue_status() {
    init_queue
    
    echo "📊 Request Queue Status"
    echo "======================="
    
    local pending_count=$(cat "$QUEUE_FILE" | jq 'length')
    local processing_count=$(cat "$PROCESSING_FILE" | jq 'length')
    local completed_count=$(cat "$COMPLETED_FILE" | jq 'length')
    
    echo "Pending requests: $pending_count"
    echo "Processing requests: $processing_count"
    echo "Completed requests: $completed_count"
    
    if [ "$pending_count" -gt 0 ]; then
        echo
        echo "Next 5 pending requests:"
        cat "$QUEUE_FILE" | jq -r '.[:5][] | "  \(.id) (priority: \(.priority), tokens: \(.estimated_tokens))"'
    fi
    
    if [ "$processing_count" -gt 0 ]; then
        echo
        echo "Currently processing:"
        cat "$PROCESSING_FILE" | jq -r '.[] | "  \(.id) (started: \(.created_at))"'
    fi
}

clear_completed() {
    local older_than="${1:-86400}"  # Default: 24 hours
    local current_time=$(date +%s)
    
    local completed_requests=$(cat "$COMPLETED_FILE")
    local filtered_requests=$(echo "$completed_requests" | jq \
        --arg cutoff_time "$((current_time - older_than))" '
        [.[] | select((.completed_at | strptime("%Y-%m-%dT%H:%M:%S%z") | mktime) > ($cutoff_time | tonumber))]
    ')
    
    echo "$filtered_requests" > "$COMPLETED_FILE"
    
    local removed_count=$(($(echo "$completed_requests" | jq 'length') - $(echo "$filtered_requests" | jq 'length')))
    echo "✅ Removed $removed_count completed requests older than $((older_than / 3600)) hours"
}

show_help() {
    cat << 'EOF'
Claude Request Queue Manager

Usage: claude-request-queue [COMMAND] [OPTIONS]

Commands:
    add <id> <model> <tokens> [priority] [payload]
        Add request to queue
        
    process [max_concurrent] [--dry-run]
        Process queue requests
        
    status
        Show queue status
        
    clear [hours]
        Clear completed requests older than hours
        
    help
        Show this help

Examples:
    claude-request-queue add "req-001" "claude-3-sonnet" 1500 3 "Hello world"
    claude-request-queue process 3
    claude-request-queue status
    claude-request-queue clear 24
EOF
}

case "${1:-help}" in
    "add")
        if [ -z "${2:-}" ] || [ -z "${3:-}" ] || [ -z "${4:-}" ]; then
            echo "❌ Request ID, model, and estimated tokens required"
            exit 1
        fi
        add_request "$2" "$3" "$4" "${5:-5}" "${6:-empty}"
        ;;
    "process")
        process_queue "${2:-3}" "${3:-false}"
        ;;
    "status")
        show_queue_status
        ;;
    "clear")
        clear_completed "$((${2:-24} * 3600))"
        ;;
    "help"|*)
        show_help
        ;;
esac
EOF

chmod +x ~/.local/bin/claude-request-queue

Step 3: Monitoring and Alerting

3.1 Rate Limit Monitor

bash

cat > ~/.local/bin/claude-rate-monitor << 'EOF'
#!/bin/bash
# Claude Rate Limit Monitor

set -euo pipefail

MONITOR_DIR="$HOME/.config/claude/rate-limits/monitoring"
METRICS_FILE="$MONITOR_DIR/metrics.json"
ALERTS_FILE="$MONITOR_DIR/alerts.json"
THRESHOLDS_FILE="$MONITOR_DIR/thresholds.yaml"

mkdir -p "$MONITOR_DIR"

setup_monitoring() {
    echo "🔧 Setting up rate limit monitoring..."
    
    # Create thresholds configuration
    cat > "$THRESHOLDS_FILE" << 'EOF'
# Rate Limit Monitoring Thresholds
thresholds:
  request_rate:
    warning: 80    # % of limit
    critical: 95
    
  token_rate:
    warning: 75
    critical: 90
    
  daily_tokens:
    warning: 85
    critical: 95
    
  queue_size:
    warning: 100   # number of pending requests
    critical: 500
    
  response_time:
    warning: 5000  # milliseconds
    critical: 10000

alerts:
  enabled: true
  cooldown: 300    # seconds between same alert type
  channels:
    - email
    - log
    # - slack
    # - webhook
EOF
    
    # Initialize metrics file
    if [ ! -f "$METRICS_FILE" ]; then
        echo "[]" > "$METRICS_FILE"
    fi
    
    # Initialize alerts file
    if [ ! -f "$ALERTS_FILE" ]; then
        echo "[]" > "$ALERTS_FILE"
    fi
    
    echo "✅ Monitoring setup completed"
}

collect_metrics() {
    local timestamp=$(date +%s)
    
    # Get rate limiter status
    local rate_status=$(claude-rate-limiter status | grep -E "(Requests/minute|Tokens/minute|Tokens/day)" | sed 's/.*: //')
    
    # Parse current usage
    local req_usage=$(echo "$rate_status" | head -1 | cut -d'/' -f1)
    local req_limit=$(echo "$rate_status" | head -1 | cut -d'/' -f2 | cut -d' ' -f1)
    local req_percent=$(echo "$rate_status" | head -1 | grep -o '([0-9.]*%)' | tr -d '()%')
    
    local tok_min_usage=$(echo "$rate_status" | head -2 | tail -1 | cut -d'/' -f1)
    local tok_min_limit=$(echo "$rate_status" | head -2 | tail -1 | cut -d'/' -f2 | cut -d' ' -f1)
    local tok_min_percent=$(echo "$rate_status" | head -2 | tail -1 | grep -o '([0-9.]*%)' | tr -d '()%')
    
    local tok_day_usage=$(echo "$rate_status" | tail -1 | cut -d'/' -f1)
    local tok_day_limit=$(echo "$rate_status" | tail -1 | cut -d'/' -f2 | cut -d' ' -f1)
    local tok_day_percent=$(echo "$rate_status" | tail -1 | grep -o '([0-9.]*%)' | tr -d '()%')
    
    # Get queue metrics
    local queue_metrics=""
    if command -v claude-request-queue >/dev/null 2>&1; then
        queue_metrics=$(claude-request-queue status | grep -E "(Pending|Processing)")
        local pending_count=$(echo "$queue_metrics" | grep "Pending" | awk '{print $3}')
        local processing_count=$(echo "$queue_metrics" | grep "Processing" | awk '{print $3}')
    else
        local pending_count=0
        local processing_count=0
    fi
    
    # Create metrics entry
    local metrics_entry='{
        "timestamp": '$timestamp',
        "datetime": "'"$(date --iso-8601=seconds)"'",
        "requests_per_minute": {
            "usage": '${req_usage:-0}',
            "limit": '${req_limit:-0}',
            "percentage": '${req_percent:-0}'
        },
        "tokens_per_minute": {
            "usage": '${tok_min_usage:-0}',
            "limit": '${tok_min_limit:-0}',
            "percentage": '${tok_min_percent:-0}'
        },
        "tokens_per_day": {
            "usage": '${tok_day_usage:-0}',
            "limit": '${tok_day_limit:-0}',
            "percentage": '${tok_day_percent:-0}'
        },
        "queue": {
            "pending": '${pending_count:-0}',
            "processing": '${processing_count:-0}'
        }
    }'
    
    # Append to metrics file and keep last 1000 entries
    local current_metrics=$(cat "$METRICS_FILE")
    local updated_metrics=$(echo "$current_metrics" | jq --argjson entry "$metrics_entry" '. + [$entry] | .[-1000:]')
    
    echo "$updated_metrics" > "$METRICS_FILE"
    
    echo "📊 Metrics collected at $(date)"
}

check_thresholds() {
    local latest_metrics=$(cat "$METRICS_FILE" | jq '.[-1]')
    
    if [ "$latest_metrics" = "null" ]; then
        echo "No metrics available"
        return
    fi
    
    # Load thresholds (simplified - use yq for YAML parsing)
    local req_warning=80
    local req_critical=95
    local tok_warning=75
    local tok_critical=90
    local day_warning=85
    local day_critical=95
    local queue_warning=100
    local queue_critical=500
    
    # Check request rate
    local req_percent=$(echo "$latest_metrics" | jq -r '.requests_per_minute.percentage')
    if (( $(echo "$req_percent >= $req_critical" | bc -l) )); then
        send_alert "CRITICAL" "Request rate at ${req_percent}% (critical threshold: ${req_critical}%)"
    elif (( $(echo "$req_percent >= $req_warning" | bc -l) )); then
        send_alert "WARNING" "Request rate at ${req_percent}% (warning threshold: ${req_warning}%)"
    fi
    
    # Check token rate  
    local tok_percent=$(echo "$latest_metrics" | jq -r '.tokens_per_minute.percentage')
    if (( $(echo "$tok_percent >= $tok_critical" | bc -l) )); then
        send_alert "CRITICAL" "Token rate at ${tok_percent}% (critical threshold: ${tok_critical}%)"
    elif (( $(echo "$tok_percent >= $tok_warning" | bc -l) )); then
        send_alert "WARNING" "Token rate at ${tok_percent}% (warning threshold: ${tok_warning}%)"
    fi
    
    # Check daily tokens
    local day_percent=$(echo "$latest_metrics" | jq -r '.tokens_per_day.percentage')
    if (( $(echo "$day_percent >= $day_critical" | bc -l) )); then
        send_alert "CRITICAL" "Daily token usage at ${day_percent}% (critical threshold: ${day_critical}%)"
    elif (( $(echo "$day_percent >= $day_warning" | bc -l) )); then
        send_alert "WARNING" "Daily token usage at ${day_percent}% (warning threshold: ${day_warning}%)"
    fi
    
    # Check queue size
    local pending_count=$(echo "$latest_metrics" | jq -r '.queue.pending')
    if [ "$pending_count" -ge "$queue_critical" ]; then
        send_alert "CRITICAL" "Queue size at $pending_count requests (critical threshold: $queue_critical)"
    elif [ "$pending_count" -ge "$queue_warning" ]; then
        send_alert "WARNING" "Queue size at $pending_count requests (warning threshold: $queue_warning)"
    fi
}

send_alert() {
    local level="$1"
    local message="$2"
    local current_time=$(date +%s)
    
    # Check cooldown
    local recent_alerts=$(cat "$ALERTS_FILE" | jq --arg level "$level" --arg cutoff "$((current_time - 300))" '
        [.[] | select(.level == $level and (.timestamp | tonumber) > ($cutoff | tonumber))]
    ')
    
    if [ "$(echo "$recent_alerts" | jq 'length')" -gt 0 ]; then
        echo "🔇 Alert in cooldown period: $level - $message"
        return
    fi
    
    # Create alert entry
    local alert_entry='{
        "timestamp": '$current_time',
        "datetime": "'"$(date --iso-8601=seconds)"'",
        "level": "'$level'",
        "message": "'$message'"
    }'
    
    # Add to alerts file
    local current_alerts=$(cat "$ALERTS_FILE")
    local updated_alerts=$(echo "$current_alerts" | jq --argjson alert "$alert_entry" '. + [$alert] | .[-100:]')
    
    echo "$updated_alerts" > "$ALERTS_FILE"
    
    # Send alert via configured channels
    echo "🚨 $level ALERT: $message"
    
    # Log to syslog
    logger -t claude-rate-monitor "$level: $message"
    
    # Email alert (if configured)
    if command -v mail >/dev/null 2>&1; then
        echo "$message" | mail -s "Claude Rate Limit Alert: $level" "${ALERT_EMAIL:-admin@localhost}" 2>/dev/null || true
    fi
}

generate_report() {
    local hours="${1:-24}"
    local cutoff_time=$(($(date +%s) - hours * 3600))
    
    echo "📈 Rate Limit Usage Report (Last $hours hours)"
    echo "=============================================="
    
    # Get metrics for time period
    local period_metrics=$(cat "$METRICS_FILE" | jq --arg cutoff "$cutoff_time" '[.[] | select(.timestamp > ($cutoff | tonumber))]')
    
    if [ "$(echo "$period_metrics" | jq 'length')" -eq 0 ]; then
        echo "No metrics available for the specified period"
        return
    fi
    
    # Calculate statistics
    local avg_req_percent=$(echo "$period_metrics" | jq '[.[].requests_per_minute.percentage] | add / length')
    local max_req_percent=$(echo "$period_metrics" | jq '[.[].requests_per_minute.percentage] | max')
    
    local avg_tok_percent=$(echo "$period_metrics" | jq '[.[].tokens_per_minute.percentage] | add / length')
    local max_tok_percent=$(echo "$period_metrics" | jq '[.[].tokens_per_minute.percentage] | max')
    
    local avg_day_percent=$(echo "$period_metrics" | jq '[.[].tokens_per_day.percentage] | add / length')
    local max_day_percent=$(echo "$period_metrics" | jq '[.[].tokens_per_day.percentage] | max')
    
    local avg_pending=$(echo "$period_metrics" | jq '[.[].queue.pending] | add / length')
    local max_pending=$(echo "$period_metrics" | jq '[.[].queue.pending] | max')
    
    echo "Request Rate Usage:"
    printf "  Average: %.1f%%\n" "$avg_req_percent"
    printf "  Peak: %.1f%%\n" "$max_req_percent"
    
    echo
    echo "Token Rate Usage:"
    printf "  Average: %.1f%%\n" "$avg_tok_percent"
    printf "  Peak: %.1f%%\n" "$max_tok_percent"
    
    echo
    echo "Daily Token Usage:"
    printf "  Average: %.1f%%\n" "$avg_day_percent"
    printf "  Peak: %.1f%%\n" "$max_day_percent"
    
    echo
    echo "Queue Statistics:"
    printf "  Average pending: %.0f requests\n" "$avg_pending"
    printf "  Peak pending: %.0f requests\n" "$max_pending"
    
    # Recent alerts
    local recent_alerts=$(cat "$ALERTS_FILE" | jq --arg cutoff "$cutoff_time" '[.[] | select(.timestamp > ($cutoff | tonumber))]')
    local alert_count=$(echo "$recent_alerts" | jq 'length')
    
    echo
    echo "Alerts in period: $alert_count"
    if [ "$alert_count" -gt 0 ]; then
        echo "$recent_alerts" | jq -r '.[] | "  \(.datetime) - \(.level): \(.message)"'
    fi
}

start_monitoring() {
    local interval="${1:-60}"
    
    echo "🔄 Starting continuous monitoring (interval: ${interval}s)"
    
    while true; do
        collect_metrics
        check_thresholds
        sleep "$interval"
    done
}

show_help() {
    cat << 'EOF'
Claude Rate Limit Monitor

Usage: claude-rate-monitor [COMMAND] [OPTIONS]

Commands:
    setup                   Setup monitoring configuration
    collect                 Collect current metrics
    check                   Check thresholds and send alerts
    report [hours]          Generate usage report
    start [interval]        Start continuous monitoring
    help                    Show this help

Examples:
    claude-rate-monitor setup
    claude-rate-monitor collect
    claude-rate-monitor report 24
    claude-rate-monitor start 30
EOF
}

case "${1:-help}" in
    "setup")
        setup_monitoring
        ;;
    "collect")
        collect_metrics
        ;;
    "check")
        check_thresholds
        ;;
    "report")
        generate_report "${2:-24}"
        ;;
    "start")
        start_monitoring "${2:-60}"
        ;;
    "help"|*)
        show_help
        ;;
esac
EOF

chmod +x ~/.local/bin/claude-rate-monitor

Verification

Rate Limiting Testing

bash

# Test basic rate limiting
claude-rate-limiter status
claude-rate-limiter check 1 1000
claude-rate-limiter wait 1 2000

# Test token bucket
claude-token-bucket setup
claude-token-bucket status requests
claude-token-bucket consume requests 1

# Test request queue
claude-request-queue add "test-001" "claude-3-sonnet" 1500 5 "test payload"
claude-request-queue status
claude-request-queue process 1 --dry-run

Monitoring Setup

bash

# Setup monitoring
claude-rate-monitor setup
claude-rate-monitor collect
claude-rate-monitor check

# Generate report
claude-rate-monitor report 1

# Test continuous monitoring (run for a short time)
timeout 60 claude-rate-monitor start 10

Troubleshooting

Common Issues

Issue	Symptoms	Resolution
Rate limits exceeded	429 API errors	Implement proper backoff and queue management
Inaccurate limit tracking	Wrong usage calculations	Reset rate limiter state and recalibrate
Queue backing up	Requests not processing	Check API connectivity and increase concurrent workers
False alerts	Incorrect threshold alerts	Adjust thresholds in monitoring configuration

Debug Commands

bash

# Debug rate limiter state
cat ~/.config/claude/rate-limits/state.json | jq .

# Check token bucket status
claude-token-bucket list

# Monitor queue processing
claude-request-queue status

# Review monitoring metrics
tail -f ~/.config/claude/rate-limits/monitoring/metrics.json

Recovery Procedures

bash

# Reset rate limiter
claude-rate-limiter reset --force

# Clear stuck queue
rm -f ~/.config/claude/rate-limits/queue/*.json
claude-request-queue status

# Restart monitoring
pkill -f claude-rate-monitor
claude-rate-monitor start &

SOP-011: Rate Limiting Configuration ​

Document Control ​

Overview ​

Rate Limiting Architecture ​

Prerequisites ​

Procedure ​

Step 1: Rate Limit Configuration ​

1.1 API Limits Understanding ​

1.2 Local Rate Limiting Implementation ​

Step 2: Advanced Rate Limiting Strategies ​

2.1 Token Bucket Implementation ​

2.2 Request Queue Manager ​

Step 3: Monitoring and Alerting ​

3.1 Rate Limit Monitor ​

Verification ​

Rate Limiting Testing ​

Monitoring Setup ​

Troubleshooting ​

Common Issues ​

Debug Commands ​

Recovery Procedures ​

See Also ​

SOP-011: Rate Limiting Configuration

Document Control

Overview

Rate Limiting Architecture

Prerequisites

Procedure

Step 1: Rate Limit Configuration

1.1 API Limits Understanding

1.2 Local Rate Limiting Implementation

Step 2: Advanced Rate Limiting Strategies

2.1 Token Bucket Implementation

2.2 Request Queue Manager

Step 3: Monitoring and Alerting

3.1 Rate Limit Monitor

Verification

Rate Limiting Testing

Monitoring Setup

Troubleshooting

Common Issues

Debug Commands

Recovery Procedures

See Also