Envoxy is a service platform framework + uWSGI‑based daemon that unifies messaging, persistence, background processing and an opinionated ORM layer. One install gives you structured modules, connectors, conventions and a packaged migration workflow.

• Quick Start
• Core Capabilities
• Installation
• Usage Examples
• ORM Conventions
• Migrations
• Docker Support
• Development
• Publishing Releases
• Documentation
• Contributing

# Install the framework and uWSGI server
pip install envoxy envoxyd

# Create a new project structure
mkdir my-service && cd my-service

# Create your application (run.py)
# See usage examples below

# Start the uWSGI server
envoxyd --http :8080 --set conf=/path/to/envoxy.json

pip install envoxy

This installs the Envoxy framework with all Python dependencies for building services.

pip install envoxyd

This installs:

• The envoxy framework (as a dependency)
• A pre-built envoxyd binary (uWSGI with dynamic Python 3.12 linking)
• All shared libraries bundled for portability

Note: envoxyd is Linux-only. For development on macOS/Windows, use the framework with your own WSGI server.

# Clone the repository
git clone https://github.com/habitio/envoxy.git
cd envoxy

# Create virtual environment
python3.12 -m venv venv
source venv/bin/activate

# Install in development mode
pip install -e .[dev]

# Install pre-commit hooks
pre-commit install

See CONTRIBUTING.md for detailed development setup.

from envoxy.db.orm import EnvoxyBase
from sqlalchemy import Column, String

class Product(EnvoxyBase):
    name = Column(String(255), nullable=False)

# Table name: aux_products
# Columns: id, name, created, updated, href

from envoxy import redisc

# Set value
redisc.set("server_key", "my_key", {"a": 1, "b": 2})

# Get value
val = redisc.get("server_key", "my_key")

# Direct client access
client = redisc.client('server_key')
client.hgetall('my_hash')

from envoxy import mqttc

# Publish
mqttc.publish('server_key', '/v3/topic/channel',
              {"data": "test"}, no_envelope=True)

# Subscribe
def callback(data):
    print(f"Received: {data}")

mqttc.subscribe('server_key', '/v3/topic/channels/#', callback)

from envoxy import couchdbc

# Find documents
docs = couchdbc.find(
    db="server_key.db_name",
    fields=["id", "field2"],
    params={"id": "1234", "field1__gt": "2345"}
)

# Get document
doc = couchdbc.get("005r9odyj...", db="server_key.db_name")

For local development and testing, a Docker Compose setup is available:

cd docker/dev
docker-compose up -d

This starts:

• PostgreSQL database
• Redis cache
• pgAdmin (PostgreSQL GUI)
• RedisInsight (Redis GUI)

Note: For production deployments, use the envoxyd wheel from PyPI instead of Docker images.

See docker/dev/README.md for development environment documentation.

# All tests
make test

# Unit tests only
make test-unit

# With coverage
make test-cov

# Lint and format
make lint
make format

# Type checking
make type-check

# Run all CI checks locally
make ci-local

# Build system documentation
cat docs/BUILD.md

# CI/CD pipeline documentation
cat docs/CI-CD.md

# Testing guide
cat tests/README.md

TestPyPI is used to test package distribution before publishing to production PyPI.

Update the version in both package files:

# Edit pyproject.toml for envoxy
vim pyproject.toml  # Update version = "0.6.10"

# Edit vendors/pyproject.toml for envoxyd
vim vendors/pyproject.toml  # Update version = "0.5.10"

git add pyproject.toml vendors/pyproject.toml
git commit -m "chore: Bump version to 0.6.10 / 0.5.10"
git push origin main

When you push to main branch, both workflows automatically trigger:

• envoxy-publish.yml: Builds pure Python wheel → publishes to TestPyPI
• envoxyd-manylinux.yml: Builds manylinux wheel → publishes to TestPyPI

No manual workflow trigger needed! Every push to main automatically publishes to TestPyPI for testing.

# Create a fresh virtual environment
python3.12 -m venv test-env
source test-env/bin/activate

# Install from TestPyPI (note: --extra-index-url needed for dependencies)
pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ envoxy
pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ envoxyd

# Verify installation
python -c "import envoxy; print(envoxy.__version__)"
envoxyd --version

# Test basic functionality
python -c "from envoxy.db.orm import EnvoxyBase; print('Import successful')"

Production releases require manual workflow dispatch to prevent accidental publishes.

Make sure pyproject.toml and vendors/pyproject.toml have the correct version numbers and are pushed to main.

# Create annotated tag (recommended for documentation)
git tag -a v0.6.10 -m "Release version 0.6.10

- Feature: Description of major features
- Fix: Description of bug fixes
- Chore: Infrastructure updates
"

# Push the tag to GitHub
git push origin v0.6.10

Note: Pushing a tag will trigger the workflows to build the packages, but NOT publish to production PyPI.

After pushing the tag and verifying the builds succeeded:

For envoxy (pure Python wheel):

1. Go to GitHub Actions
2. Select Build and publish envoxy (pure Python) workflow
3. Click "Run workflow"
4. Select the tag (e.g., v0.6.10) from the branch dropdown
5. Click "Run workflow"

This will publish to production PyPI at https://pypi.org/project/envoxy/

For envoxyd (manylinux binary):

1. Go to GitHub Actions
2. Select Build and publish envoxyd (manylinux) workflow
3. Click "Run workflow"
4. Select the tag (e.g., v0.6.10) from the branch dropdown
5. Click "Run workflow"

This will publish to production PyPI at https://pypi.org/project/envoxyd/

# Install from production PyPI
pip install --upgrade envoxy envoxyd

# Verify versions
pip show envoxy envoxyd

• Check PyPI pages: envoxy, envoxyd
• Verify GitHub Actions succeeded: Actions tab
• Check workflow logs for any errors

Follow Semantic Versioning:

• Major version (1.0.0): Breaking changes
• Minor version (0.6.0): New features, backward compatible
• Patch version (0.6.1): Bug fixes, backward compatible

Both envoxy and envoxyd should be versioned together to maintain compatibility:

# Example: Major release
envoxy: 1.0.0
envoxyd: 1.0.0

# Example: Feature release
envoxy: 0.7.0
envoxyd: 0.6.0  # Only if envoxyd has no changes

Build fails on GitHub Actions:

• Check workflow logs in Actions tab
• Verify all tests pass: make test
• Ensure dependencies are up to date

TestPyPI/PyPI upload fails:

• Verify TEST_PYPI_API_TOKEN and PYPI_API_TOKEN secrets are set in repository settings
• Check if version already exists (PyPI doesn't allow re-uploading same version)
• Ensure package name is available

envoxyd binary doesn't run:

• Check RPATH is correct: patchelf --print-rpath $(which envoxyd)
• Verify shared libraries are bundled: auditwheel show wheelhouse/*.whl
• Test in clean manylinux container: docker run -it quay.io/pypa/manylinux_2_28_x86_64 bash

• ZeroMQ / UPnP integration ("Zapata")
• MQTT / AMQP (RabbitMQ) messaging
• Celery task dispatch
• CouchDB & PostgreSQL connectors (direct + SQLAlchemy helpers)
• Redis cache / key‑value utilities
• Packaged Alembic migrations & CLI (envoxy-alembic)
• Opinionated ORM conventions (automatic naming, audit columns, index safety)

• EnvoxyBase declarative base (prefix + pluralization + audit fields)
• Idempotent mapper listeners (populate id/created/updated/href)
• Bundled Alembic config (alembic.ini + env.py) with model auto‑discovery
• Session helpers: session_scope, transactional

Reduce boilerplate per service, enforce naming consistency across a fleet and make migrations / messaging predictable and safe.

EnvoxyBase is the unified model base.

Automatic rules:

• Table prefix: aux_
• Pluralized class names (with curated exceptions)
• Audit columns injected: id, created, updated, href
• Index names rewritten with framework prefix
• Audit values populated by idempotent listeners

Why aux_? The core Envoxy platform is designed around a ZeroMQ data-layer for primary domain entities. The SQLAlchemy layer is intentionally a secondary/auxiliary persistence mechanism for sidecar tables: caches, denormalized projections, integration state, small feature flags, ephemeral join helpers—NOT the canonical domain records.

The aux_ prefix:

• Visibly segregates auxiliary tables from core platform data-layer storage.
• Prevents future naming collisions if a core table later lands in RDBMS form.
• Makes auditing / cleanup simpler (drop or archive all aux_ tables safely).
• Signals that schemas can evolve faster with fewer cross‑service guarantees.

Guidelines:

• Keep business‑critical source-of-truth entities in the primary data-layer.
• Use ORM aux_ tables for performance, enrichment, or transient coordination.
• Avoid back‑writing from aux_ tables into the core pipeline except via explicit integration processes.

Minimal model:

from envoxy.db.orm import EnvoxyBase
from sqlalchemy import Column, String

class Product(EnvoxyBase):
    name = Column(String(255), nullable=False)

metadata = EnvoxyBase.metadata  # for Alembic autogenerate

Benefits: consistent naming, fewer conflicts, less boilerplate. More: docs/HOWTO-migrations.md, docs/POSTGRES.md.

Run migrations without copying config.

CLI:

envoxy-alembic revision -m "create product table" --autogenerate
envoxy-alembic upgrade head

Module form:

python -m envoxy.tools.alembic.alembic current

Features: packaged config, model discovery, sqlite path normalization, baseline versions. CI helper: python -m envoxy.tools.check_migrations <versions_dir>. Docs: docs/HOWTO-migrations.md.

envoxyd boots service modules via embedded customized uWSGI.

Install (Make):

make install

Docker:

# Development: Quick start with docker-compose (recommended)
cd docker/dev && docker-compose up -d

# Or build builder images manually
docker build --build-arg UID=$(id -u) --build-arg GID=$(id -g) \
  -t envoxy-ubuntu:24.04 -f docker/builder/ubuntu-24.04.Dockerfile .

# Production runtime image
docker build -t envoxy:runtime -f docker/runtime/Dockerfile .

See docker/README.md for detailed Docker documentation.

Run:

envoxyd --http :8080 --set conf=/path/to/confs/envoxy.json

Build packages:

make packages

Manual (example):

# Using tools/build.sh (recommended)
./tools/build.sh packages

# Or manually with Python 3.12
python3.12 setup.py sdist bdist_wheel
cd vendors && python3.12 setup.py sdist bdist_wheel
twine upload dist/*

Details: docs/ENVOXYD.md.

envoxy-cli --create-project --name my-container

Docker (mount volumes):

docker run -it -d -p 8080:8080 \
  -v /path/to/project:/home/envoxy \
  -v /path/to/plugins:/usr/envoxy/plugins envoxy

Read-only queries:

from envoxy import pgsqlc
rows = pgsqlc.query("db_name", "select * from sample_table where id = 1;")

Writes: use the ORM (SQLAlchemy) via PgDispatcher.sa_manager() and models. Direct insert() on the raw client is no longer supported by design.

Find:

from envoxy import couchdbc
docs = couchdbc.find(
    db="server_key.db_name",
    fields=["id", "field2"],
    params={"id": "1234", "field1__gt": "2345"}
)

Get:

doc = couchdbc.get("005r9odyj...", db="server_key.db_name")

from envoxy import redisc
redisc.set("server_key", "my_key", {"a": 1, "b": 2})
val = redisc.get("server_key", "my_key")
client = redisc.client('server_key'); client.hgetall('my_hash')

Publish:

from envoxy import mqttc
mqttc.publish('server_key', '/v3/topic/channel', {"data": "test"}, no_envelope=True)

Subscribe:

from envoxy import mqttc
mqttc.subscribe('server_key', '/v3/topic/channels/#', callback)

on_event class:

from envoxy import on
from envoxy.decorators import log_event

@on(endpoint='/v3/topic/channels/#', protocols=['mqtt'], server='server_key')
class MqttViewCtrl(View):
    @log_event
    def on_event(self, data, **kw):
        do_stuff(data)

Low‑level client example: docs/MQTT.md.

• BUILD.md - Build system and tools documentation
• CI-CD.md - Continuous Integration/Deployment guide
• ENVOXYD.md - Daemon build & packaging
• POSTGRES.md - PostgreSQL (direct + ORM)
• COUCHDB.md - CouchDB usage & selectors
• MQTT.md - MQTT detailed usage

• HOWTO-migrations.md - Migrations & Alembic packaging
• HOWTO-shared-db.md - Shared DB guidance
• HOWTO-ci-tools.md - CI tooling helpers

• tests/README.md - Comprehensive testing guide
• docker/README.md - Docker build and deployment
• docker/MIGRATION.md - Docker migration guide

We welcome contributions! Please see CONTRIBUTING.md for:

• Development setup instructions
• Code style guidelines
• Testing requirements
• Pull request process
• Release workflow

See LICENSE for details.

• Repository: github.com/habitio/envoxy
• Issues: github.com/habitio/envoxy/issues
• Changelog: CHANGELOG

Made with ❤️ by the Envoxy team