Guided Project 2: Distributed Caching with Rate Limiting

This project combines two fundamental Redis use cases crucial for scalable web applications:

1. Distributed Caching: Storing frequently accessed data in Redis to reduce the load on primary databases and speed up response times.
2. Rate Limiting: Preventing abuse of APIs or services by restricting the number of requests a user or client can make within a given time window.

We’ll build a simplified API-like service that uses Redis for both caching and rate limiting, demonstrated with Node.js and Python.

Project Objective

Create a mock “Product API” where:

• Product details are fetched from a “slow” backend (simulated with a delay).
• Product details are cached in Redis for faster subsequent requests.
• API requests are rate-limited per client IP address.

Prerequisites

• A running Redis server.
• Node.js and ioredis installed.
• Python and redis-py installed.
• A basic understanding of web server frameworks (Express for Node.js, Flask for Python) is helpful but not strictly required, as we’ll keep the server part minimal.

Step 1: Initialize the Project and Redis Client

Create a new directory for your project.

Node.js:

mkdir redis-cache-ratelimit-nodejs
cd redis-cache-ratelimit-nodejs
npm init -y
npm install ioredis express # express for basic API simulation

Then create a file app.js.

Python:

mkdir redis-cache-ratelimit-python
cd redis-cache-ratelimit-python
python3 -m venv venv
source venv/bin/activate
pip install redis flask # flask for basic API simulation

Then create a file app.py.

Step 2: Implement the Rate Limiter

We’ll implement a “fixed window counter” rate limiter. For each IP address, we’ll store a counter in Redis that expires after a specific time window. Each request increments the counter. If the counter exceeds a threshold, the request is denied.

Rate Limiter Logic:

1. Generate a unique key for the rate limit (e.g., rate_limit:<IP_Address>:<Window_Start_Timestamp>).
2. INCR the counter for this key.
3. If the key is new, EXPIRE it to the end of the current time window.
4. Check if the counter exceeds the MAX_REQUESTS.

Node.js (app.js):

// redis-cache-ratelimit-nodejs/app.js
const express = require('express');
const Redis = require('ioredis');
const app = express();
const redis = new Redis();

const RATE_LIMIT_WINDOW_SECONDS = 60; // 1 minute window
const MAX_REQUESTS_PER_WINDOW = 5;    // Max 5 requests per minute

/**
 * Checks if a client is rate-limited.
 * @param {string} ip - The client's IP address.
 * @returns {Promise<boolean>} - True if rate-limited, false otherwise.
 */
async function isRateLimited(ip) {
  const now = Math.floor(Date.now() / 1000); // Current timestamp in seconds
  // Key format: rate_limit:<IP_Address>:<Window_Start_Timestamp>
  // We use current minute as the window identifier.
  const windowStartTimestamp = Math.floor(now / RATE_LIMIT_WINDOW_SECONDS) * RATE_LIMIT_WINDOW_SECONDS;
  const key = `rate_limit:${ip}:${windowStartTimestamp}`;

  try {
    // Increment the counter for the current window.
    // PX: Set expiration in milliseconds (PEXPIRE)
    // NX: Only set expiration if key doesn't exist (SETNX)
    const requests = await redis.incr(key);

    if (requests === 1) {
      // If this is the first request in the window, set its expiration
      await redis.expire(key, RATE_LIMIT_WINDOW_SECONDS);
    }

    if (requests > MAX_REQUESTS_PER_WINDOW) {
      console.log(`IP ${ip} is rate-limited. Requests: ${requests}`);
      return true;
    }
    console.log(`IP ${ip} request count: ${requests}`);
    return false;
  } catch (error) {
    console.error(`Rate limiting error for IP ${ip}:`, error);
    // In case of Redis error, fail open (don't rate limit) to avoid disrupting service
    return false;
  }
}

// Middleware for rate limiting
app.use(async (req, res, next) => {
  // Use a simplified IP for local testing, e.g., '127.0.0.1' or random
  const clientIp = req.ip || '127.0.0.1'; // In production, req.ip will be more reliable
  if (await isRateLimited(clientIp)) {
    return res.status(429).send('Too Many Requests');
  }
  next();
});

// ... (API routes and server setup later)

Python (app.py):

# redis-cache-ratelimit-python/app.py
from flask import Flask, request, jsonify
import redis
import time
import json # For product data serialization

app = Flask(__name__)
r = redis.Redis(host='localhost', port=6379, db=0)

RATE_LIMIT_WINDOW_SECONDS = 60 # 1 minute window
MAX_REQUESTS_PER_WINDOW = 5    # Max 5 requests per minute

"""
Checks if a client is rate-limited.
:param ip: The client's IP address.
:returns: True if rate-limited, false otherwise.
"""
def is_rate_limited(ip):
    now = int(time.time())
    # Key format: rate_limit:<IP_Address>:<Window_Start_Timestamp>
    # We use current minute as the window identifier.
    window_start_timestamp = (now // RATE_LIMIT_WINDOW_SECONDS) * RATE_LIMIT_WINDOW_SECONDS
    key = f"rate_limit:{ip}:{window_start_timestamp}"

    try:
        # Increment the counter for the current window.
        # Use a pipeline for INCR and EXPIRE to make it atomic where possible,
        # especially for the initial setting of the expiration.
        
        # We can use r.incr and r.expire separately, but need to be careful if
        # expire fails. SET with EX option is better: SET key value EX seconds NX
        # However, INCR doesn't have an EX NX option directly.
        # For this simple example, we'll use a transaction for atomicity
        
        with r.pipeline() as pipe:
            pipe.incr(key)
            pipe.expire(key, RATE_LIMIT_WINDOW_SECONDS, nx=True) # Only set EX if key is new
            # If the key already existed, expire(nx=True) does nothing.
            # If it was new, incr set it to 1, then expire sets the TTL.
            results = pipe.execute()
            requests = results[0] # Result of INCR

        if requests > MAX_REQUESTS_PER_WINDOW:
            print(f"IP {ip} is rate-limited. Requests: {requests}")
            return True
        print(f"IP {ip} request count: {requests}")
        return False
    except Exception as e:
        print(f"Rate limiting error for IP {ip}: {e}")
        # In case of Redis error, fail open (don't rate limit)
        return False

# Decorator for rate limiting
@app.before_request
def rate_limit_middleware():
    client_ip = request.remote_addr or '127.0.0.1' # Use request.remote_addr in Flask
    if is_rate_limited(client_ip):
        return jsonify({'message': 'Too Many Requests'}), 429

# ... (API routes and server setup later)

Step 3: Implement the Caching Layer

We’ll cache product details. If a product is not in Redis, we’ll fetch it from a simulated “slow database” and then store it in Redis with a TTL.

Caching Logic:

1. Check Redis for the product:<ID> key.
2. If found, return the cached data immediately.
3. If not found, simulate fetching from a backend database (e.g., fetchFromDatabase(productId)).
4. Store the fetched data in Redis (e.g., SET product:<ID> JSON_data EX TTL_seconds).
5. Return the data.

Node.js (app.js):

// ... (previous code for rate limiter and Express setup)

const CACHE_TTL_SECONDS = 300; // Cache for 5 minutes

// Simulate a slow database fetch
async function fetchProductFromDatabase(productId) {
  console.log(`Fetching product ${productId} from slow database...`);
  await new Promise(resolve => setTimeout(resolve, 1000)); // Simulate 1 second delay
  return {
    id: productId,
    name: `Product ${productId}`,
    description: `This is a detailed description for product ${productId}.`,
    price: parseFloat((Math.random() * 1000).toFixed(2)),
    last_updated: new Date().toISOString()
  };
}

/**
 * Retrieves product details, first from cache, then from database.
 * @param {string} productId - The ID of the product.
 * @returns {Promise<object>} - Product details.
 */
async function getProductDetails(productId) {
  const cacheKey = `product:${productId}`;

  try {
    // 1. Try to fetch from Redis cache
    let cachedProduct = await redis.get(cacheKey);
    if (cachedProduct) {
      console.log(`Product ${productId} found in cache.`);
      return JSON.parse(cachedProduct);
    }

    // 2. If not in cache, fetch from the "database"
    console.log(`Product ${productId} not found in cache. Fetching from database.`);
    const productData = await fetchProductFromDatabase(productId);

    // 3. Store in cache with TTL
    await redis.setex(cacheKey, CACHE_TTL_SECONDS, JSON.stringify(productData));
    console.log(`Product ${productId} cached for ${CACHE_TTL_SECONDS} seconds.`);
    return productData;

  } catch (error) {
    console.error(`Error getting product details for ${productId}:`, error);
    // Fallback to directly returning from database if caching fails
    return fetchProductFromDatabase(productId);
  }
}

// ... (API routes and server setup later)

Python (app.py):

# ... (previous code for rate limiter and Flask setup)

CACHE_TTL_SECONDS = 300 # Cache for 5 minutes

# Simulate a slow database fetch
def fetch_product_from_database(product_id):
    print(f"Fetching product {product_id} from slow database...")
    time.sleep(1) # Simulate 1 second delay
    return {
        'id': product_id,
        'name': f"Product {product_id}",
        'description': f"This is a detailed description for product {product_id}.",
        'price': round(random.uniform(10.0, 1000.0), 2),
        'last_updated': time.time()
    }

"""
Retrieves product details, first from cache, then from database.
:param product_id: The ID of the product.
:returns: Product details.
"""
def get_product_details(product_id):
    cache_key = f"product:{product_id}"

    try:
        # 1. Try to fetch from Redis cache
        cached_product = r.get(cache_key)
        if cached_product:
            print(f"Product {product_id} found in cache.")
            return json.loads(cached_product)

        # 2. If not in cache, fetch from the "database"
        print(f"Product {product_id} not found in cache. Fetching from database.")
        product_data = fetch_product_from_database(product_id)

        # 3. Store in cache with TTL
        r.setex(cache_key, CACHE_TTL_SECONDS, json.dumps(product_data))
        print(f"Product {product_id} cached for {CACHE_TTL_SECONDS} seconds.")
        return product_data

    except Exception as e:
        print(f"Error getting product details for {product_id}: {e}")
        # Fallback to directly returning from database if caching fails
        return fetch_product_from_database(product_id)

# ... (API routes and server setup later)

Step 4: Create the API Endpoint and Server

Now, let’s wire up our rate limiter and caching logic to a simple API endpoint.

Node.js (app.js):

// ... (all previous code)

// API route to get product details
app.get('/products/:id', async (req, res) => {
  const productId = req.params.id;
  try {
    const product = await getProductDetails(productId);
    res.json(product);
  } catch (error) {
    console.error(`Failed to get product ${productId}:`, error);
    res.status(500).send('Internal Server Error');
  }
});

const PORT = 3000;
app.listen(PORT, () => {
  console.log(`\nAPI server listening on port ${PORT}`);
  console.log(`Rate Limit: ${MAX_REQUESTS_PER_WINDOW} requests per ${RATE_LIMIT_WINDOW_SECONDS} seconds.`);
  console.log(`Cache TTL: ${CACHE_TTL_SECONDS} seconds.`);
  console.log('Test with: curl http://localhost:3000/products/123');
  console.log('To simulate multiple IPs, use `--header "X-Forwarded-For: 192.168.1.X"` in curl');
});

// Graceful shutdown
process.on('SIGINT', async () => {
  console.log('\nShutting down Redis client...');
  await redis.quit();
  console.log('Redis client disconnected. Exiting.');
  process.exit(0);
});

Python (app.py):

# ... (all previous code)

# API route to get product details
@app.route('/products/<string:product_id>', methods=['GET'])
def products_route(product_id):
    try:
        product = get_product_details(product_id)
        return jsonify(product)
    except Exception as e:
        print(f"Failed to get product {product_id}: {e}")
        return jsonify({'message': 'Internal Server Error'}), 500

if __name__ == '__main__':
    print(f"\nAPI server listening on port 5000")
    print(f"Rate Limit: {MAX_REQUESTS_PER_WINDOW} requests per {RATE_LIMIT_WINDOW_SECONDS} seconds.")
    print(f"Cache TTL: {CACHE_TTL_SECONDS} seconds.")
    print("Test with: curl http://localhost:5000/products/abc")
    print("To simulate multiple IPs, use `--header \"X-Forwarded-For: 192.168.1.X\"` in curl")
    app.run(debug=False, port=5000)

Step 5: Run and Test the Application

1. Start your Redis server.

2. Start the API application:

   • Node.js: node app.js
   • Python: python app.py (or flask run if configured)

3. Test the caching and rate limiting using curl:

   • Initial Request (slow, then cached):

     curl http://localhost:3000/products/123 # For Node.js
     curl http://localhost:5000/products/abc # For Python

     You should see “Fetching product… from slow database” in the server logs, followed by the product JSON.

   • Subsequent Requests (fast, from cache): Immediately repeat the same curl command multiple times.

     curl http://localhost:3000/products/123

     You should now see “Product found in cache” and a much faster response.

   • Test Rate Limiting (same IP): Keep sending requests quickly. After 5 requests within the minute window, you should get a “Too Many Requests” (429) error.

     for i in $(seq 1 10); do curl http://localhost:3000/products/123; sleep 0.5; done

     The server logs will show “IP 127.0.0.1 is rate-limited.”

   • Test with a Different IP (simulated): Use the X-Forwarded-For header to simulate requests from a different IP address. This new IP should have its own rate limit.

     for i in $(seq 1 10); do curl --header "X-Forwarded-For: 192.168.1.10" http://localhost:3000/products/456; sleep 0.5; done

     (Adjust the port to 5000 for Python).

   • Wait for TTL/Rate Limit Reset: Wait for 60 seconds (for the rate limit to reset) or 300 seconds (for cache to expire), and observe the behavior again.

Further Challenges and Enhancements

1. Sliding Window Rate Limiter (Advanced):

   • The current rate limiter is a “fixed window.” Implement a “sliding window log” or “sliding window counter” rate limiter, which is generally more fair. This involves storing timestamps of individual requests in a Redis Sorted Set or using a more complex counter approach. (Hint: ZADD, ZREMRANGEBYSCORE, ZCARD).

2. Cache Invalidation:

   • What if a product’s price or description changes in the database? The cache would be stale for CACHE_TTL_SECONDS.
   • Implement an endpoint or a mechanism (e.g., another Redis Pub/Sub message) to explicitly DEL a product’s cache key when its data is updated in the backend.

3. Tiered Caching:

   • Implement a multi-level cache: an in-memory (application-local) cache for very hot items (e.g., using a Map/dictionary), falling back to Redis, and then to the database. Add a small TTL (e.g., 5 seconds) to the local cache.

4. Error Handling and Metrics:

   • Add more robust error handling for Redis connection issues.
   • Implement basic metrics (e.g., number of cache hits/misses, rate limit denials) and expose them (e.g., simple print statements or a dedicated /metrics endpoint).

5. Distributed Lock for Cache Population:

   • If many clients request an uncached item simultaneously, they could all hit the slow database (“thundering herd” problem).
   • Implement a Redis Distributed Lock (SET key value EX seconds NX) before calling fetchProductFromDatabase. Only one instance should fetch from the DB; others wait or fetch from cache if it becomes available.

By successfully building and testing this project, you’ve gained invaluable experience in using Redis for distributed caching and rate limiting – two critical components for building high-performance and resilient modern web applications.