Development Guide

[!IMPORTANT] Discord is the primary place to ask questions. Join at discord.observal.io.

#contributing, setup help, workflow questions, anything about the contribution process

#bug, discuss bugs before filing a GitHub issue

#feature-requests, pitch ideas and get early feedback before writing code

GitHub issues and PRs are for concrete, actionable items. Exploratory discussion belongs on Discord first.

Parts of this guide structure were inspired by the AnkiDroid Development Guide. Attribution given with thanks, check them out if you want another welcoming OSS project to contribute to.

Community Standards

We are a small, active community and we take the quality of interactions seriously.

[!WARNING] The following will result in a moderator warning. A second violation results in a temporary or permanent ban from the repository and Discord:

Pinging contributors, maintainers, or reviewers unnecessarily (outside of a direct reply on your own open PR or issue)

Submitting low-effort or unreviewed PRs (slop), including unreviewed AI output or autonomous agent submissions

Violating the Code of Conduct in any channel

Harassing reviewers over merge timelines

Maintainers volunteer their time. Treat them accordingly.

A note on autonomous coding agents

Autonomous coding agents (Devin, SWE-agent, OpenHands, etc.) are not permitted to submit PRs. This is a legal constraint, not a quality judgment. The US Copyright Office’s 2025 guidance confirms that purely AI-generated code has no copyright owner, which breaks our CLA and our ability to license and enforce the project. See the AI Policy for the full explanation.

Prerequisites

Tool	Version	Install
Docker	24+ with Compose v2	docs.docker.com
uv	latest	`curl -LsSf https://astral.sh/uv/install.sh \| sh`
Node.js	22.6+	Via nvm or mise
pnpm	10+	`npm install -g pnpm`
Git	2.28+	git-scm.com

Verify your Docker installation supports Compose v2:

docker compose version
# should print: Docker Compose version v2.x.x

If you see docker-compose: command not found or a v1 version, upgrade Docker Desktop or install the Compose plugin separately.

Git Setup

SSH authentication

Using SSH is strongly recommended, it avoids password prompts and is required if you enable 2FA on GitHub.

# Generate a key (skip if you already have one at ~/.ssh/id_ed25519)
ssh-keygen -t ed25519 -C "your@email.com"

# Copy the public key to your clipboard
cat ~/.ssh/id_ed25519.pub

Add the output to GitHub → Settings → SSH and GPG keys → New SSH key. Test it:

ssh -T git@github.com
# Hi your-username! You've successfully authenticated...

Git identity

Set your identity so it matches your GitHub account:

git config --global user.name "Your Name"
git config --global user.email "your@email.com"

If you want to sign commits with GPG as well (optional but appreciated):

git config --global commit.gpgsign true
git config --global user.signingkey YOUR_GPG_KEY_ID

Fork and clone

You only need to do this once. For every subsequent contribution, just create a new branch on your existing fork.

Fork the repository on GitHub (top-right Fork button).
Clone your fork using SSH:

git clone git@github.com:YOUR-USERNAME/Observal.git
cd Observal

Add the upstream remote so you can pull in changes from the main repo:

git remote add upstream https://github.com/Observal/Observal.git

Verify your remotes:

git remote -v
# origin    git@github.com:YOUR-USERNAME/Observal.git (fetch)
# origin    git@github.com:YOUR-USERNAME/Observal.git (push)
# upstream  https://github.com/Observal/Observal.git (fetch)
# upstream  https://github.com/Observal/Observal.git (push)

First-time Setup

Install pre-commit hooks

Do this before your first commit. The hooks run ruff, format checkers, secret scanners, SPDX header injection, and migration chain validation automatically.

make hooks

Start the Docker stack

cp .env.example .env
make up

The first build takes several minutes while Docker downloads and compiles images. Subsequent starts are fast. Watch progress with:

make logs

For code, dependency, and Compose changes, use:

make rebuild

make rebuild-fast is the same target under another name. It is not a separate dev profile and does not enable hot reload. It builds only the two app images that contain local code:

observal-api, shared by the API, init, and worker services
observal-web, used by the web service

Then it starts the full stack with the freshly built images. Wait until all services are healthy:

docker compose -f docker/docker-compose.yml ps
# All services should show "healthy" or "running"

Default endpoints:

Service	URL
LB (all traffic)	`http://localhost`
Web UI (direct)	`http://localhost:3000`
Prometheus, optional	`http://localhost:9090`
Grafana, optional	`http://localhost:3001`
ClickHouse HTTP	`http://localhost:8123`

Install the CLI

uv tool install --editable .

Log in

observal auth login

On a fresh server, this bootstraps the admin account automatically. Use super@demo.example / super-changeme for the super_admin account.

Demo accounts

Seeded automatically on first startup:

Email	Password	Role
`super@demo.example`	`super-changeme`	super_admin
`admin@demo.example`	`admin-changeme`	admin
`dev@demo.example`	`dev-changeme`	developer
`user@demo.example`	`user-changeme`	user

Make Targets Reference

make up            # start the full Docker stack
make down          # stop the stack
make rebuild       # rebuild app images and restart (alias: make rebuild-fast)
make logs          # tail logs from all services
make test          # run the test suite quickly
make test-v        # verbose test output
make lint          # ruff check + hadolint
make format        # ruff format (auto-fix)
make check         # pre-commit on all files
make hooks         # install pre-commit hooks
make reset         # nuke all volumes and rebuild from scratch (destructive)

Which rebuild target should I use?

Situation	Target	Notes
Any code, dependency, schema, init, worker, or Compose change	`make rebuild`	`make rebuild-fast` is the same target. Rebuilds the shared API image (API, init, worker) and the web image, then recreates changed services.
Cache looks stale or the stack behaves unexpectedly	`make rebuild-clean`	Rebuilds from scratch without the Docker cache and removes volumes.

make down stops containers started by make rebuild.

Architecture Overview

Observal is a monorepo:

observal-server/    FastAPI backend (Python)
observal_cli/       CLI and session exporters (Python)
web/                Vite 8 / React 19 frontend (TypeScript)
tests/              Shared test suite (~1500 tests, 96 files)
docker/             Docker Compose and Dockerfiles
docs/               Documentation
scripts/            Dev tooling scripts

Databases:

PostgreSQL, relational data (users, agents, registry, feedback)
ClickHouse, session events, aggregates, audit events, and security events

They are not interchangeable. Never write telemetry to Postgres or relational data to ClickHouse. Supporting services: Redis (pub/sub + arq job queue), arq worker, nginx reverse proxy. Prometheus and Grafana are optional. See AGENTS.md for a complete map of every important file and service.

Git Workflow

Making a new branch

Always branch from the latest main. Never commit directly to main.

git checkout main
git pull upstream main
git checkout -b feature/my-feature
# or: fix/my-bug, docs/my-doc

Keeping your branch up to date

If main has moved forward while you were working, rebase your branch onto it:

git checkout main
git pull upstream main
git checkout feature/my-feature
git rebase main

Rebasing keeps history linear and makes your PR easier to review than a merge commit would.

Dealing with merge conflicts

Conflicts arise when the same file was changed in both main and your branch. During git rebase main, git will pause and show you:

CONFLICT (content): Merge conflict in some/file.py

Open the file and look for the three Git conflict-marker lines: seven less-than signs followed by HEAD, seven equals signs, and seven greater-than signs followed by the other branch name. The text between them contains the competing changes. Edit the file to the correct final state (keeping whichever changes are right, or combining them), then:

git add some/file.py
git rebase --continue

If you get confused and want to start over:

git rebase --abort

After resolving and pushing, use --force-with-lease rather than --force, it’s safer:

git push origin feature/my-feature --force-with-lease

Submitting a pull request

Make sure your branch is rebased on the latest main (see above).
Push to your fork:
git push origin feature/my-feature
GitHub will show a banner on your fork offering to open a PR. Click it, or go to the Observal repository directly.
Fill in the PR template completely. PRs that do not follow the template will be closed.
Link the related issue if one exists (Fixes #123 in the PR body closes it automatically on merge).

After review feedback

If a reviewer requests changes, make them on the same branch and push again. Do not open a new PR. If your changes are small fixes to an existing commit, amend rather than adding new commits:

git add the-changed-file.py
git commit --amend --no-edit
git push origin feature/my-feature --force-with-lease

Working on the Backend

Running tests

Tests mock all external services. Docker does not need to be running.

make test
make test-v
make test-eval-completeness
make test-adversarial
make test-all
# or directly:
cd observal-server
uv run --with pytest --with pytest-asyncio --with hypothesis --with pyarrow pytest ../tests/ -q

Testing conventions

New Python tests should follow the Testing Guide. The current suite has mixed historical patterns, so do not rewrite old tests only for style. When adding or touching tests, prefer the clean pattern documented there: one behavior area per file, small local helper factories, hermetic CLI and API test setup, explicit async mocks, and behavior-focused assertions.

Running a single test

cd observal-server
uv run pytest ../tests/test_registry_types.py -q
# or a specific test function:
uv run pytest ../tests/test_registry_types.py::TestMcpRoutes::test_submit_mcp -v

Code coverage

Coverage is collected automatically on the 3.13 matrix run in CI. To generate it locally:

cd observal-server
uv run pytest ../tests/ --cov=../observal_cli --cov=. --cov-report=html -q
# open htmlcov/index.html in your browser

Adding a database migration

bash scripts/new_migration.sh "describe_your_change"

Edit the generated file in observal-server/alembic/versions/. Then verify the chain is intact:

python3 scripts/check_migrations.py

[!CAUTION] Never edit an existing migration file. Always create a new one. A broken migration chain blocks CI and prevents the server from starting.

Apply migrations to your local stack:

docker compose -f docker/docker-compose.yml run --rm init

Connecting to PostgreSQL directly

docker compose -f docker/docker-compose.yml exec db \
  psql -U observal -d observal

Useful for inspecting tables, running manual queries, or verifying migration results.

Connecting to ClickHouse directly

ClickHouse exposes an HTTP interface. Use the Play UI in your browser:

http://localhost:8123/play

Or query from the terminal:

curl -s "http://localhost:8123/?query=SELECT+count()+FROM+session_events" \
  -u "default:"

Or connect with the CLI client:

docker compose -f docker/docker-compose.yml exec clickhouse \
  clickhouse-client --user default

Debugging the API

The API server runs with --reload in development mode. Log output is available via:

docker compose -f docker/docker-compose.yml logs -f observal-api

To add a breakpoint, use breakpoint() in Python code, the debugger will pause in the container’s stdout. Or attach a remote debugger by adding debugpy to your dev dependencies and exposing a debug port. The OpenAPI docs are available at (API port, not through the LB which blocks these paths):

http://localhost:8000/docs (Swagger UI, requires direct API port access)
http://localhost:8000/redoc (ReDoc, requires direct API port access)

Note: The nginx LB blocks /docs, /redoc, and /openapi.json in production. For local dev, expose the API port directly or use docker compose exec.

Working on the Frontend

Running the frontend in isolation

cd web
pnpm install
pnpm dev

The Vite dev server proxies all /api calls to the backend at http://localhost:8000, configured in web/vite.config.ts. The Docker stack does not publish port 8000, so run the API locally or point the proxy target at http://localhost (the nginx LB).

Design system

Token	Value
Color space	OKLCH
Themes	`light`, `dark`, `midnight`, `forest`, `sunset`
Display font	Archivo
Body font	Albert Sans
Code font	JetBrains Mono
Spacing base	4pt
Components	shadcn/ui
Charts	Recharts
Data fetching	TanStack Query
Tables	TanStack Table

All themes are defined in web/src/app/globals.css as CSS custom properties. Semantic tokens (background, foreground, card, border, primary, secondary, accent, destructive, success, warning, info) map to OKLCH values per theme. When adding new UI, use the semantic tokens, never hardcode colors. Check all five themes look correct before submitting.

Adding a new API endpoint to the frontend

Add the TypeScript response type to web/src/lib/types.ts.
Add the fetch call to web/src/lib/api.ts.
Add the TanStack Query hook to web/src/hooks/use-api.ts.
Use the hook in your component.

Screenshots for UI changes

[!IMPORTANT] Any PR that touches the web frontend must include screenshots of all affected screens in the PR description. This is required regardless of how small the change is. Attach screenshots directly to the PR body, not as review comments.

Working on the CLI

Reinstalling after changes

The CLI is installed as a uv tool. Changes to source files are reflected immediately because it is installed in editable mode (--editable). If you add new entry points or change pyproject.toml, reinstall:

uv tool install --editable . --reinstall

Testing session delivery

Run a harness session with hooks installed, then reconcile and inspect exporter status:

observal reconcile --dry-run
observal auth status
observal ops telemetry status

Session records remain in the local outbox until the server acknowledges them.

Harness Recommendations

VS Code

Install these extensions for the best experience:

Python (ms-python.python), LSP, type checking
Pylance (ms-python.vscode-pylance), fast type inference
Ruff (charliermarsh.ruff), linting and formatting on save
ESLint (dbaeumer.vscode-eslint), JavaScript/TypeScript linting
Prettier (esbenp.prettier-vscode), TypeScript formatting
Docker (ms-azuretools.vscode-docker), container management

Recommended settings.json additions:

{
    "[python]": {
        "editor.defaultFormatter": "charliermarsh.ruff",
        "editor.formatOnSave": true
    },
    "[typescript]": {
        "editor.defaultFormatter": "esbenp.prettier-vscode",
        "editor.formatOnSave": true
    },
    "python.analysis.typeCheckingMode": "basic"
}

PyCharm / IntelliJ

Open the project root as a PyCharm project.
Set the Python interpreter to the uv-managed venv: observal-server/.venv/bin/python.
Mark observal-server as the sources root.
Install the Ruff plugin for in-editor linting.
Enable File Watchers to run ruff format on save.

Pre-commit Hooks

Install once with make hooks. The hooks run automatically on every git commit:

Hook	What it does
`ruff`	Lints Python, auto-fixes what it can
`ruff-format`	Formats Python in place
`trailing-whitespace`	Strips trailing whitespace
`end-of-file-fixer`	Ensures files end with a newline
`check-yaml` / `check-toml` / `check-json`	Validates config file syntax
`check-added-large-files`	Blocks files over 500KB
`check-merge-conflict`	Blocks leftover `<<<<<<<` markers
`detect-private-key`	Scans for private key material
`no-commit-to-branch`	Blocks direct commits to `main`
`check-secrets`	Scans staged content for API keys, tokens, `.env` files
`check-migrations`	Validates Alembic migration chain integrity
`spdx-update`	Adds your SPDX copyright line to staged files
`hadolint-docker`	Lints Dockerfiles

If a hook fails, fix the reported issue and commit again. To bypass in an emergency (not recommended):

git commit --no-verify -m "your message"

Pull Request Checklist

Before opening a PR:

Getting Help

If you are stuck, the best place to ask is #contributing on Discord. Search the channel history first, most common questions have been answered there already. For bugs use #bug. For feature ideas use #feature-requests.

[!NOTE] Do not open a GitHub issue just to ask a question. Issues are for confirmed bugs and accepted feature requests. Questions belong on Discord.

​Table of Contents

​Community Standards

​A note on autonomous coding agents

​Prerequisites

​Git Setup

​SSH authentication

​Git identity

​Fork and clone

​First-time Setup

​Install pre-commit hooks

​Start the Docker stack

​Install the CLI

​Log in

​Demo accounts

​Make Targets Reference

​Which rebuild target should I use?

​Architecture Overview

​Git Workflow

​Making a new branch

​Keeping your branch up to date

​Dealing with merge conflicts

​Submitting a pull request

​After review feedback

​Working on the Backend

​Running tests

​Testing conventions

​Running a single test

​Code coverage

​Adding a database migration

​Connecting to PostgreSQL directly

​Connecting to ClickHouse directly

​Debugging the API

​Working on the Frontend

​Running the frontend in isolation

​Design system

​Adding a new API endpoint to the frontend

​Screenshots for UI changes

​Working on the CLI

​Reinstalling after changes

​Testing session delivery

​Harness Recommendations

​VS Code

​PyCharm / IntelliJ

​Pre-commit Hooks

​Pull Request Checklist

​Getting Help

Table of Contents