---
url: /developers/misc/ab-testing.md
---
# A/B Testing Guide

This document describes how to use and test the A/B testing framework in Open Library.

## 1. Configuration

Active experiments are defined in `openlibrary/core/experiments.py` under the `ACTIVE_EXPERIMENTS` dictionary.

Every active experiment must declare its variants nested under a `"variants"` key, along with a targeting rule under the `"audience"` key:

```python

ACTIVE_EXPERIMENTS = {
    "AB_Testing": {
        "variants": {
            "control": 30,
            "a": 35,
            "b": 35,
        },
        "audience": Audience.ALL,  # Target everyone
    },
    "Another_Experiment": {
        "variants": {
            "control": 50,
            "treatment": 50,
        },
        "audience": Audience.LOGGED_IN,  # Logged-out users automatically fall back to control
    }
}
```

### Audience Targeting Options

The framework supports three audience categories imported from `openlibrary.core.experiments.Audience`:

* `Audience.ALL`: All requests are split according to the variant weights.
* `Audience.LOGGED_IN`: Only logged-in users are distributed into the variants; guest users automatically fall back to the `control` variant.
* `Audience.LOGGED_OUT`: Only guest (anonymous) users are distributed into the variants; logged-in users automatically fall back to the `control` variant.

## 2. Usage Examples

### Inside HTML Templates / Templetor Macros

To conditionally render markup based on a user's variant bucket:

```html
$code: variant = ctx.get('experiments', {}).get('AB_Testing') $if variant ==
'a':
<div class="layout-a">
  <!-- Variant A layout -->
</div>
$elif variant == 'b':
<div class="layout-b">
  <!-- Variant B layout -->
</div>
$else:
<div class="layout-control">
  <!-- Control layout -->
</div>
```

### Inside Python Controllers / Handlers

Open Library runs both legacy `web.py` and async `FastAPI` routes. The active experiments are made available in their respective request contexts:

#### In web.py Controllers

Access context-scoped experiments via `web.ctx`:

```python
import web

variant = web.ctx.get("experiments", {}).get("AB_Testing")
if variant == "a":
    # Execute Variant A logic
    pass
elif variant == "b":
    # Execute Variant B logic
    pass
else:
    # Execute Control logic
    pass
```

#### In FastAPI Routes

Access experiments using the request state context (`request.state`):

```python
from fastapi import Request

@app.get("/example")
async def read_example(request: Request):
    variant = request.state.experiments.get("AB_Testing")
    if variant == "a":
        # Execute Variant A logic
        pass
    elif variant == "b":
        # Execute Variant B logic
        pass
    else:
        # Execute Control logic
        pass
```

### Inside Frontend JS (jQuery, Vue, Lit)

To check variants client-side, query the global `window.getExperiment` helper or import it:

```javascript
// Anywhere in your scripts:
const variant = window.getExperiment("AB_Testing");
if (variant === "a") {
  // Enable variant A layout
} else if (variant === "b") {
  // Enable variant B layout
} else {
  // Enable control layout
}

// Or import as an ES module in modern Webpack / Vite bundles:
import { getExperiment } from "./experiments";
const variant = getExperiment("AB_Testing");
```

## 3. Testing & URL Overrides

For development, QA, and debugging, you can force a specific experiment variant using URL query parameters.

The format is: `?experiment_[experiment_name]=[variant]`

### Examples

* To force Variant `b` of the `AB_Testing` experiment: `http://localhost:8080/?experiment_AB_Testing=b`
* To force the control group: `http://localhost:8080/?experiment_AB_Testing=control`
* Overrides also work across multiple parameters: `http://localhost:8080/?experiment_AB_Testing=a&experiment_AnotherTest=control`

> \[!NOTE]
> URL overrides will only apply if the specified variant name exists in the configuration of the active experiment. Invalid override variants or incorrectly formatted parameters are safely ignored, falling back to the user's determinisnically assigned bucket.