Local Development Architecture

This page shows which services run when you start Open Library with Docker, and how they connect. The service definitions live in compose.yaml (base) and compose.override.yaml (local development additions).

How the Services Connect

---
config:
  flowchart:
    curve: basis
    nodeSpacing: 45
    rankSpacing: 70
---
flowchart LR
    accTitle: Open Library local development architecture
    accDescr: Browser requests reach the FastAPI application, which proxies unmatched requests to the web.py application; both connect to PostgreSQL, infobase, Solr, memcached, and local mock services. The Solr updater polls infobase and sends index updates to Solr.
    mc[("memcached")]
    mock["mockservices<br>fake archive.org APIs"]
    pg[("db<br>PostgreSQL")]
    ib["infobase :7000"]
    covers["covers :7075<br>book cover images"]

    subgraph apps["Applications"]
        direction TB
        fast["fast_web :8080<br>FastAPI application<br>primary entry point"]
        web["web<br>legacy web.py application"]
        fast -->|"fallback proxy (proxy.py)<br>for unmatched routes"| web
    end

    upd["solr-updater"]
    browser(["Your browser"])

    browser -->|"web pages and API requests"| apps
    browser -->|"localhost:7075"| covers

    apps -->|"search"| solr[("solr :8983")]
    apps -->|"cache"| mc
    apps -->|"IA API stubs"| mock
    apps -->|"reads / writes"| pg
    apps -->|"reads / writes"| ib
    ib -->|"reads / writes"| pg
    upd -->|"sends index updates"| solr
    upd -->|"polls the write log"| ib

fast_web is the primary entry point in local dev: it is published on localhost:8080 and serves every route FastAPI has. Any request without a matching FastAPI route is forwarded to web (the legacy web.py application) by the fallback proxy in openlibrary/fastapi/proxy.py. New endpoints should be built in fast_web, and over time more traffic moves from web to fast_web. (In production/staging the layout is the reverse: web is published on port 8080 and FastAPI runs on 18080 behind the site's proxy.)

Both applications connect to the data stores in the same way, so the diagram draws their connections as one group. They cache in memcached, read and write postgres and infobase, query solr, and call the fake archive.org APIs in local development. The catalog data itself flows through the hybrid connection type (see connection_type in conf/openlibrary.yml): catalog reads go directly to postgres, and catalog writes go through infobase, which reads from and writes to postgres and records every change in its write log. On top of that, the applications read and write app-specific postgres tables directly (beyond the infogami connection), and they also read from infobase over HTTP for requests the local postgres reader does not serve, such as the write log and account operations. Every arrow above is verified against the code and config files; the sections below cite where each one is defined.

Where each connection is defined

All arrows follow caller → callee direction: the initiator points at the service it talks to.

• Apps → memcached: memcache_servers: [memcached:11211] in conf/openlibrary.yml installs a cache middleware around the infobase connection.
• Apps → mockservices: the ia_*_api_url and recaptcha_url settings in conf/openlibrary.yml point at mockservices:8090; smtp_server: mockservices routes email to Mailpit.
• Apps → db (reads and writes): catalog reads go through HybridConnection's local postgres reader, and app-specific tables (bookshelves, ratings, booknotes, waitinglist, and friends) are read and written directly via db.get_db() in openlibrary/core/.
• Apps → infobase (reads and writes): catalog writes are routed to the remote infobase server by HybridConnection, and infobase also serves reads that bypass the local reader, such as the write log (/log, /_recentchanges) and account requests.
• Infobase → db (reads and writes): db_parameters in conf/infobase.yml point infobase's local connection at the db host; it reads catalog data to serve requests and writes (persists) every change back to postgres.
• Apps → solr: plugin_worksearch.solr_base_url in conf/openlibrary.yml points at http://solr:8983/solr/openlibrary.
• solr-updater → infobase: the updater polls /openlibrary.org/log on the infobase host directly (the same write log that records recent changes), not the applications.
• solr-updater → solr: index updates are posted to the solr base URL above.
• Browser → covers: compose.override.yaml publishes covers on localhost:7075 (also coverstore_public_url in conf/openlibrary.yml) so your browser loads cover images directly; all other services stay inside the Docker network by default. solr is additionally published on localhost:8983 for local debugging, but browsers never talk to it.

The home container also runs on both networks; it executes one-off setup and maintenance jobs (cloning the docs wiki, seeding languages, and building the solr index on first boot), so it has no fixed place in the request flow.

The Services

Service	Purpose	Local address
web	Legacy web.py application. Serves most pages today; reached via the FastAPI fallback proxy in local dev	internal (no host port)
fast_web	FastAPI application. Primary entry point in local dev; new endpoints go here	http://localhost:8080
db	PostgreSQL database. Stores all catalog and account data	internal to Docker only
infobase	Infogami data server. Handles wiki-style edits and revisions	internal, port 7000
memcached	Cache for frequent database queries	internal
solr	Search engine. Powers search results and autocomplete	http://localhost:8983
solr-updater	Polls infobase's recent-changes log and keeps the solr index current	none
covers	Coverstore. Serves book cover images to your browser directly	http://localhost:7075
mockservices	Stands in for archive.org APIs. Captures email with Mailpit	http://localhost:8025 (Mailpit)
home	Utility container for setup scripts and maintenance	none

Services without a local address listen only inside the Docker network. You reach them through the fast_web application, which proxies anything it doesn't handle to web. solr is exposed on the host for local debugging only; the applications always reach it as solr:8983 inside the network.

Related Guides

• Quick Start — start these services for the first time
• Lifecycle of a Network Request — what happens inside the web application after a request arrives
• Docker Guide — build details, volumes, and troubleshooting