Developer Guide

This guide helps developers set up DBCalm for local development and contribute to the project.

Note

This guide is for developers who want to contribute to DBCalm or build from source. If you just want to install and use DBCalm, see Installation instead.

Development vs Production

Development Setup: - Runs from source code with auto-reload - Uses sudo to run command services as different users - Requires manual setup with make dev-install - Python 3.11+ required

Production Setup (Package Installation): - Runs from compiled binaries - Systemd handles privilege separation (no sudo needed) - Automated setup via .deb/.rpm package - Python bundled in package

Development Setup

Prerequisites

  • Debian 12+ / Ubuntu 22.04+

  • Python 3.11+ (for development only)

  • MariaDB or MySQL server

  • Git

  • Root or sudo access

Quick Start

  1. Clone Repository

    git clone https://github.com/mschot/dbcalm-open-backend.git
cd dbcalm-open-backend

  2. Automated Setup

    make dev-install

    This script (dev/install.sh) automatically:

    • Creates dbcalm system user and group

    • Creates required directories (/etc/dbcalm, /var/lib/dbcalm, /var/log/dbcalm, /var/run/dbcalm)

    • Sets up file permissions

    • Configures sudo access for your user (only for development)

    • Generates self-signed SSL certificates

  3. Python Environment

    python3.11 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

  4. Database Setup

    Create MariaDB backup user:

    CREATE USER 'backupuser'@'localhost' IDENTIFIED BY 's0m3p455w0rd';
GRANT RELOAD, PROCESS, REPLICATION CLIENT ON *.* TO 'backupuser'@'localhost';
FLUSH PRIVILEGES;

    Configure credentials:

    sudo nano /etc/dbcalm/credentials.cnf

    [client-dbcalm]
user=backupuser
password=s0m3p455w0rd

    Set permissions:

    sudo chown mysql:dbcalm /etc/dbcalm/credentials.cnf
sudo chmod 640 /etc/dbcalm/credentials.cnf

  5. Create First User

    python dbcalm.py users add

SSL Certificates (Development)

Use mkcert for local development:

# Install mkcert
sudo apt install mkcert
mkcert -install

# Generate certificates
sudo mkdir -p /etc/dbcalm/ssl
mkcert -cert-file /tmp/fullchain-cert.pem \
       -key-file /tmp/private-key.pem \
       localhost 127.0.0.1

sudo mv /tmp/fullchain-cert.pem /etc/dbcalm/ssl/
sudo mv /tmp/private-key.pem /etc/dbcalm/ssl/
sudo chown dbcalm:dbcalm /etc/dbcalm/ssl/*

Configure MariaDB Backup User

CREATE USER 'backupuser'@'localhost' IDENTIFIED BY 's0m3p455w0rd';
GRANT RELOAD, PROCESS, REPLICATION CLIENT ON *.* TO 'backupuser'@'localhost';
FLUSH PRIVILEGES;

Create /etc/dbcalm/credentials.cnf:

[client-dbcalm]
user=backupuser
password=s0m3p455w0rd

sudo chown dbcalm:dbcalm /etc/dbcalm/credentials.cnf
sudo chmod 600 /etc/dbcalm/credentials.cnf

Running the Development Server

The development environment runs two command services and the API server with auto-reload:

make dev

This starts:

  1. MariaDB command service (as mysql user)

  2. Generic command service (as root)

  3. API server with auto-reload (as dbcalm user)

Accept SSL Certificate

Before you can access the API, you must accept the self-signed SSL certificate in your browser.

Open your browser and navigate to:

https://dbcalm.localhost:8335

You’ll see a certificate warning:

  1. Click “Advanced” (or “Show Details”)

  2. Click “Proceed to dbcalm.localhost” (or “Accept the Risk and Continue”)

This is a required one-time step for development setups using self-signed certificates.

Access the API

Now that you’ve accepted the certificate, access the API at: https://dbcalm.localhost:8335/docs

Project Structure

backend/
├── dbcalm/                      # Main application package
│   ├── api/                     # API models (request/response)
│   ├── cli/                     # CLI commands (server, users, clients)
│   ├── config/                  # Configuration management
│   ├── domain/                  # Domain models and logic
│   ├── errors/                  # Custom exceptions
│   ├── logger/                  # Logging configuration
│   ├── repository/              # Data access layer
│   └── routes/                  # FastAPI route handlers
├── dbcalm_cmd/                  # Generic command service
│   ├── adapter/                 # System command adapters
│   ├── command/                 # Command validator
│   └── process/                 # Process runner
├── dbcalm_cmd_client/           # Client library for cmd service
├── dbcalm_mariadb_cmd/          # MariaDB command service
│   ├── adapter/                 # MariaDB-specific adapters
│   ├── builder/                 # Command builders
│   └── command/                 # Backup/restore validator
├── dbcalm_mariadb_cmd_client/   # Client library for MariaDB service
├── tests/                       # Test suite
│   ├── unit/                    # Unit tests
│   └── e2e/                     # End-to-end tests
├── dbcalm.py                    # Main CLI entry point
├── dbcalm-cmd.py                # Generic command service entry
└── dbcalm-mariadb-cmd.py        # MariaDB command service entry

Architecture

Components

DBCalm consists of three separate programs:

  • dbcalm - Main CLI and API server

    • Entry point: dbcalm.py

    • Runs as: dbcalm user

    • Handles: HTTP requests, authentication, business logic

    • Communicates: Via Unix sockets to command services

  • dbcalm-mariadb-cmd - MariaDB backup command service

    • Entry point: dbcalm-mariadb-cmd.py

    • Runs as: mysql user

    • Handles: Mariabackup operations with MySQL data file access

    • Socket: /var/run/dbcalm/mariadb-cmd.sock

  • dbcalm-cmd - Generic system command service

    • Entry point: dbcalm-cmd.py

    • Runs as: root user

    • Handles: Crontab management for backup schedules

    • Socket: /var/run/dbcalm/cmd.sock

Privilege Separation

The three programs communicate via Unix domain sockets with whitelist-based command validation.

Each service runs with the minimum privileges needed for its specific tasks:

  • The API server runs as dbcalm and delegates privileged operations to the command services

  • The MariaDB service runs as mysql to access database files

  • The generic command service runs as root for crontab management

This separation ensures that even if the API server is compromised, attackers cannot directly execute system commands or access database files.

Repository Pattern

The codebase follows the repository pattern:

  • Domain Models: Business entities (Client, Backup, Schedule)

  • Repositories: Data access abstraction (SQLModel/SQLite)

  • Adapters: External system interfaces (mariabackup, systemctl)

  • Routes: FastAPI endpoints connecting everything together

Development Commands

Build & Run

# Run API server
python dbcalm.py server

# Or use development mode with auto-reload
make dev

User Management

# Add user
./dbcalm.py users add

# List users
./dbcalm.py users list

# Delete user
./dbcalm.py users delete

# Update password
./dbcalm.py users update-password

Client Management

# Add client
./dbcalm.py clients add

# List clients
./dbcalm.py clients list

# Update client
./dbcalm.py clients update

# Delete client
./dbcalm.py clients delete

Testing

# Run all tests
.venv/bin/python -m pytest tests/

# Run unit tests only
.venv/bin/python -m pytest tests/unit/

# Run e2e tests only
.venv/bin/python -m pytest tests/e2e/

Linting

# Check for issues
ruff check .

# Auto-fix issues
ruff check . --fix

Pre-commit Hook

The pre-commit hook automatically runs linting and tests. Install it:

cp hooks/pre-commit .git/hooks/
chmod +x .git/hooks/pre-commit

Building Binaries

Build standalone executables for production deployment:

# Build main API server binary (includes authentication dependencies)
pyinstaller --onefile --hidden-import passlib.handlers.bcrypt --clean \
  --workpath=./build/pyinstaller --distpath=dist/production dbcalm.py

# Build generic command service binary
pyinstaller --onefile --clean \
  --workpath=./build/pyinstaller --distpath=dist/production dbcalm-cmd.py

# Build MariaDB command service binary
pyinstaller --onefile --clean \
  --workpath=./build/pyinstaller --distpath=dist/production dbcalm-mariadb-cmd.py

Binaries will be created in dist/production/:

  • dist/production/dbcalm - Main API server

  • dist/production/dbcalm-cmd - Generic command service

  • dist/production/dbcalm-mariadb-cmd - MariaDB command service

Note: The --hidden-import passlib.handlers.bcrypt flag is required for dbcalm.py to include password hashing dependencies.

Code Style Guidelines

General Principles

  • Python 3.11+ required

  • Follow FastAPI framework conventions

  • Use type hints throughout

  • Domain-driven design with modular components

Imports

Group imports in this order:

  1. Standard library

  2. Third-party packages

  3. Local modules

import os
from pathlib import Path

from fastapi import FastAPI
from sqlmodel import Session

from dbcalm.domain.client import Client

Naming Conventions

  • snake_case for functions and variables

  • PascalCase for classes

  • UPPER_CASE for constants

# Good
def create_backup(client_id: int) -> Backup:
    pass

class BackupRepository:
    pass

MAX_RETRIES = 3

String Formatting

Use f-strings for string formatting and double quotes:

# Good
message = f"Backup created for client {client_id}"

# Avoid
message = "Backup created for client %s" % client_id

Path Handling

Use pathlib.Path instead of os operations:

from pathlib import Path

# Good
config_path = Path("/etc/dbcalm/config.json")
if config_path.exists():
    content = config_path.read_text()

# Avoid
import os
if os.path.exists("/etc/dbcalm/config.json"):
    with open("/etc/dbcalm/config.json") as f:
        content = f.read()

Error Handling

Use appropriate exceptions with descriptive messages:

from dbcalm.errors.validation_error import ValidationError

if not client_id:
    raise ValidationError("Client ID is required")

Documentation

Use docstrings for public interfaces:

def create_backup(client_id: int, full: bool = True) -> Backup:
    """Create a new backup for the specified client.

    Args:
        client_id: The ID of the client to backup
        full: Whether to create a full backup (default: True)

    Returns:
        The created backup instance

    Raises:
        ValidationError: If client_id is invalid
    """
    pass

TODOs

Mark TODOs with noqa: FIX002:

# TODO: Add support for incremental backups  # noqa: FIX002

Multiline Collections

Add comma after last item in multiline collections:

# Good
dependencies = [
    "fastapi",
    "sqlmodel",
    "uvicorn",
]

Contributing

Workflow

  1. Fork the repository

  2. Create a feature branch: git checkout -b feature/my-feature

  3. Make your changes following code style guidelines

  4. Run linter: ruff check . --fix

  5. Run tests: pytest tests/

  6. Commit your changes: git commit -m "Add my feature"

  7. Push to your fork: git push origin feature/my-feature

  8. Create a pull request

Pull Request Guidelines

  • Write clear, descriptive commit messages

  • Include tests for new features

  • Update documentation as needed

  • Ensure all tests pass and linter is clean

  • Keep PRs focused on a single feature/fix

Getting Help