3  How FastAPI Handles Many Requests at Once

3.1 Why One Slow Call Delays Everyone

Imagine a restaurant with one waiter handling every table. One table asks for a custom order that takes five minutes to confirm with the kitchen. If the waiter stands there waiting, no other table gets service. That is a blocking server (one server worker/thread handling requests sequentially).

In FastAPI terms, this happens during I/O-bound work: waiting on a database, file system, or external API. While that wait is happening, the server thread can’t move on, so other requests queue up.

sequenceDiagram
    participant C1 as Table 1 (Slow Request)
    participant C2 as Table 2 (Quick Request)
    participant S as Sync Server (One Waiter)

    C1->>S: POST /slow-query
    Note over S: Waiting on external I/O (5s)
    C2->>S: GET /health
    Note over C2: Waiting in line...
    S->>C1: Response
    S->>C2: Response (after C1)

With async, the waiter places Table 1’s request, then serves other tables while waiting. Same waiter, better flow.

Note“Synchronous” and “Asynchronous” are not the best terms

In everyday language, “synchronous” describes events that are “in sync”, i.e. they take the same steps together. It DOES NOT tell us if events happen sequentially or simultaneously, although it is often interpreted as “events happening simultaneously”.

And this is where the confusion arises. In programming, “synchronous” refers to “an event and the code that handles it happen simultaneously”. In other words, the code is “in sync” with the event and cannot handle other events until it finishes. This means that the code can only handle events sequentially, one at a time. So “synchronous” in programming actually means “sequential”, which is the opposite of what the term suggests in everyday language.

3.2 async and await: Pause One Request, Serve Another

Python uses two keywords to juggle multiple requests without blocking: async and await. - When you declare a function with async def, you are telling Python: “This function might pause itself to let others work.” - When you write await in front of a slow operation (like an API call), you are saying: “I am stepping aside; let the next customer in line while I wait for this to finish.”

Asynchronous operations are handled by an event loop (managed by Uvicorn), which keeps track of all the paused functions and wakes them up when their awaited operations complete.

Here is a simple async endpoint. We will use asyncio.sleep to simulate a slow operation:

import asyncio
from fastapi import FastAPI

app = FastAPI()

@app.get("/slow")
async def slow_endpoint():
    # This simulates a 5-second database query or LLM call
    await asyncio.sleep(5)
    return {"message": "Sorry for the wait, here is your data"}

If you hit this endpoint in a browser, it takes five seconds. If your friend hits the /health endpoint on the same server during those five seconds, they get an instant response. The server was not blocked.

Just remember the rule: if a function might take time, make it async def and await the slow parts.

3.3 Dependency Injection: Reusing Shared Tools Cleanly

Sometimes, your endpoint may need to call an external service (A database, a tool, etc.), so it needs a “client” to talk to that service (not to be confused with the “client” that calls your endpoint). A simple setup might look like this:

import httpx # for creating HTTP clients

@app.get("/fetch")
async def fetch_data():
    client = httpx.AsyncClient()
    response = await client.get("https://api.some-external-service.com")
    await client.aclose()
    return {"status": response.status_code} # return something

Notes: - httpx is a popular library for making HTTP requests in Python, and AsyncClient is its asynchronous version. It allows you to create a client object that can make non-blocking HTTP requests. You can use other packages or custom client object. - client.get() sends a GET request to the specified URL. - client.aclose() closes the client object and any open connections.

Creating a client object involves network setup (it needs to find the server, negotiate encryption, authenticate) and this takes time. If you do this inside the endpoint, every single request will repeat this expensive setup.

So can we just create the client object outside the endpoint and reuse it? Let’s try:

client = httpx.AsyncClient()
@app.get("/fetch")
async def fetch_data():
    response = await client.get("https://api.some-external-service.com")
    return {"status": response.status_code}

This approach will work, but it has a problem: you need to keep the client object connected for the entire lifetime of the server since it may be used by multiple requests. This is expensive and can lead to resource leaks. You also need to close the client object manually when the server shuts down.

Dependency Injection solves this: you define a function that creates the client object and FastAPI handles the rest. Here’s how it works:

Instead of creating the client object directly, you define a dependency function that yields the client object (see below), then you declare that your endpoint depends on that function using Depends(). When a request comes in, FastAPI runs the dependency function, gets the client object, and passes it to your endpoint. When the request is done, FastAPI resumes the dependency function to clean up (close connections):

from fastapi import Depends
import httpx

# The dependency function
async def get_http_client():
    async with httpx.AsyncClient() as client:
        yield client  # yield: hand client to the endpoint, then pause
                      # When the server shuts down, execution resumes here

@app.get("/fetch")
async def fetch_data(client: httpx.AsyncClient = Depends(get_http_client)):
    # FastAPI sees Depends(), runs get_http_client(), passes the yielded client 
    response = await client.get("https://api.github.com")
    return {"status": response.status_code}

Why yield instead of return? With return, the function ends immediately—no chance to run cleanup later. yield pauses the function, hands over the client, waits for the endpoint to finish, then resumes so the async with ... as block can close connections.

3.4 Keeping API Keys Safe

When you create a client object for an external service, you often need to provide an API key for authentication in addition to the service URL. This is a secret credential that should never be hardcoded in your source code. Instead, we store it in a .env file as an environment variable and use Pydantic to validate them.

Your .env file (never commit this to Git!) looks like:

OPENAI_API_KEY=... # If you are using OpenAI, for example
MODEL_NAME=...

Install the helper:

pip install pydantic-settings

Create a settings object that reads from your .env file automatically:

from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    openai_api_key: str
    model_name: str = "gpt-4"  # Default value if not in env
    
    class Config:
        env_file = ".env"

settings = Settings()

# Now use it in your dependency
def get_settings():
    return settings

You can now access settings via dependency injection: settings: Settings = Depends(get_settings).

3.5 Hands-On: Upgrade the Quote API to Async + Depends

Let us bring it all together by upgrading your Quote of the Day API from Part 2. We will convert it to async, add a fake “slow” quote lookup to prove non-blocking behavior, and inject a shared “database connection” (simulated with a simple class).

First, the dependency. We will create a fake database handler that takes time to “connect”:

import asyncio
from fastapi import FastAPI, Depends, HTTPException

app = FastAPI()

# Simulating an expensive-to-create resource
class QuoteDatabase:
    def __init__(self):
        # Imagine this opens a real Postgres connection pool
        self.quotes = {
            1: {"text": "Patience is a virtue", "author": "Unknown"},
            2: {"text": "FastAPI is fun", "author": "You"}
        }
    
    async def get_by_id(self, quote_id: int):
        # Simulate network latency to a real database
        await asyncio.sleep(2)  
        if quote_id not in self.quotes:
            raise HTTPException(status_code=404, detail="Quote not found")
        return self.quotes[quote_id]

# The dependency provider
async def get_db():
    db = QuoteDatabase()  # In production, you'd cache this properly
    try:
        yield db
    finally:
        # Cleanup would happen here (close connections)
        pass

Now, two endpoints to test concurrency. One is slow (uses the database), one is instant:

@app.get("/quotes/{quote_id}")
async def get_quote(quote_id: int, db: QuoteDatabase = Depends(get_db)):
    # This takes 2 seconds due to the simulated latency inside db.get_by_id
    quote = await db.get_by_id(quote_id)
    return quote

@app.get("/health")
async def health():
    # This responds instantly, even if /quotes/1 is grinding away
    return {"status": "ok", "server": "async and ready"}

Testing workflow: 1. Start the server: uvicorn main:app --reload 2. In one terminal (or browser tab), request the slow quote: curl http://localhost:8000/quotes/1 (This will hang for 2 seconds) 3. Immediately, in a second terminal, hit the health check: curl http://localhost:8000/health

You will see the health check return instantly, while the quote request takes its full two seconds. If you had written this synchronously (using time.sleep instead of await asyncio.sleep), the health check would have waited in line behind the slow quote, taking over two seconds itself.

3.6 Recap: Terms You Learned

Web Term	What it actually means in Python
Async (async def)	A marker telling Python this function can pause itself to let other work happen
Await (await)	The keyword that triggers the pause, specifically for I/O operations like network calls
Blocking I/O	Waiting for external resources (network, disk) in a way that freezes the entire thread
Event Loop	The invisible manager (run by Uvicorn) that juggles all your paused async functions
Dependency Injection	The pattern of providing pre-built tools (like database clients) to your functions via Depends()