The rzfz.ai Stack — the integrated open-source stack for local AI infrastructure — bundles
curated open-source modules as Docker Compose containers, behind Caddy as the single
externally reachable entry point and Authentik SSO, on the rzfz.ai Box, your own Ubuntu 26.04
servers, or in the cloud.
The Stack Map
From the reverse proxy to the agents — the entire rzfz.ai Stack, live from the data, grouped by layer. Click, focus, or tap a tile for details.
The agent manager is an rzfz.ai-developed management interface where users launch, monitor and stop their own persistent agent instances — Hermes, Moltis, Coding Agents. It provisions per-user containers and volumes, registers their routes with Caddy, and reconciles them automatically after a reboot. Currently included as an experimental module.
The MCP manager is an rzfz.ai-developed management interface where users set up their own MCP integrations with their personal credentials — encrypted in an AES-256-GCM vault and isolated in a per-user proxy container. The MCP manager wires the integration automatically into that user's own Hermes, Moltis or coding agents. Currently included as an experimental module.
Based on Hermes by Nous Research — packaged as a container for the stack by razzfazz.ai GmbH.
Apache-2.0
Hermes is a self-improving Python agent by Nous Research, which rzfz.ai packages as a container with persistent memory, more than 30 tools and an MCP client. In the stack, each user provisions their own Hermes instance through the agent manager, along with the Hermes Workspace UI for browsing memory and skills. Currently included as an experimental module.
Moltis is a personal agent server written in Rust with Matrix, Telegram and Discord gateways and sandboxed code execution, which rzfz.ai packages as a container. In the stack, each user provisions their own Moltis instance through the agent manager, with persistent memory. Currently included as an experimental module.
OpenHands is an autonomous coding agent that plans, writes, runs and tests code, spinning up per-task sandbox containers as it works. In the stack it integrates with Gitea for repositories and pull requests and with GPUStack for inference, and requires access to the Docker socket. Currently included as an experimental module.
Paperclip is an agent orchestrator that rzfz.ai runs as a container: it assigns roles and goals to teams of agents, distributes tasks and tracks budgets with an audit log. In the stack it integrates Gitea, Infisical and GPUStack, and is still in an early stage of development. Currently included as an experimental module.
Codex is an open-source terminal coding agent built by OpenAI. In the stack it runs as one of the sandboxed per-user containers in the persistent coding agents workspace, pre-wired against the local GPUStack model. Currently included as an experimental module.
OpenCode is an open-source AI coding agent for the terminal. In the stack it runs as one of the sandboxed per-user containers in the persistent coding agents workspace, pre-wired against the local GPUStack model. Currently included as an experimental module.
gsd-pi is an open-source tool for spec-driven development. It is no longer its own baked image, but an rzfz.ai-suggested user-defined install (`npm i -g @opengsd/gsd-pi`) inside the persistent coding agents workspace. Currently included as an experimental module.
Alongside Codex and OpenCode, the coding agents workspace also bundles a "user-defined" variant: its own rzfz.ai-developed container with a terminal and a protected sandbox, where users install any coding agent of their choice — such as Gemini CLI or Claude Code — and run it centrally around the clock, instead of a preconfigured tool. New in 2026.07-ga. Currently included as an experimental module.
Gotenberg converts HTML, Markdown and Office documents to PDF via Chromium and LibreOffice, behind a REST API. In the stack it serves as the conversion backend for Paperless-ngx and is also usable from Dify workflows; it runs internal-only.
Docling simplifies document processing: it parses diverse formats — including advanced PDF understanding — and integrates seamlessly with the generative-AI ecosystem. It supports input formats including PDF, DOCX, PPTX, XLSX, HTML and image formats, and exports to Markdown, HTML or JSON. For agentic AI, it ships plug-and-play integrations with LangChain, LlamaIndex, CrewAI and Haystack, and runs as an API or MCP server. Currently included as an experimental module.
Presidio is a PII-detection toolkit originally started by Microsoft and since transferred to the Data Privacy Stack organization: the Analyzer, Anonymizer and Image Redactor detect and redact more than 30 categories of personal data in text and images. In the stack it runs internal-only as a preprocessing step before documents are passed on to other modules. Currently included as an experimental module.
Apache Tika extracts text, metadata and language from over a thousand file formats. In the stack it runs internal-only as the extraction backend for Paperless-ngx. Currently included as an experimental module.
Stirling PDF is a powerful, open-source PDF editing platform that runs as a desktop app, in the browser, or on your own servers with a private API. It ships more than 50 PDF tools for editing, merging, splitting, signing, redacting, converting, OCR and compression, alongside enterprise features such as single sign-on and auditing. Currently included as an experimental module.
Crawl4AI is a web crawler built for LLMs that fetches page content via Chromium/Playwright and turns it into clean markdown or JSON. In the stack it distills web content for LightRAG, Cognee, Onyx and Dify to ingest — unlike SearXNG's plain URL discovery. Currently included as an experimental module.
Onyx is an enterprise search platform with semantic search and LLM answers across more than 40 connectors, running its own Vespa index and embedding-model servers. In the stack it can connect Gitea and Paperless-ngx as data sources and, at six containers, is the heaviest module in the stack. Currently included as an experimental module.
Paperless-ngx turns physical documents into a searchable online archive — so you can keep, well, less paper. It is the official successor to earlier Paperless versions and is developed in a community-driven way. Currently included as an experimental module.
Open WebUI is an extensible, feature-rich and user-friendly self-hosted AI platform that operates entirely offline. It supports LLM runners such as Ollama and OpenAI-compatible APIs and ships its own built-in RAG inference engine, plus granular access controls, real-time collaboration through channels, persistent memory, and voice and video capabilities — in the stack, rounded out by Speaches for voice input and output and SearXNG for web search.
Dify is an open-source platform for building LLM applications that combines several capabilities in one interface: a visual workflow editor, a Prompt IDE, RAG pipelines for document ingestion, agent capabilities with more than 50 built-in tools, and LLMOps for monitoring production applications. It supports hundreds of proprietary and open-source LLMs; in the stack, the self-hosted community edition runs against the local GPUStack backend.
SearXNG is a meta search engine that aggregates results from multiple search services without tracking or profiling users. In the stack, it powers web research — for example in Open WebUI and in Dify. SearXNG is open source and licensed under the AGPL-3.0.
Gitea is a self-hosted software development platform written in Go, built for an uncomplicated all-in-one development service. It bundles Git hosting, code review, issue tracking, a kanban board, a wiki, team collaboration, a package registry and CI/CD via Gitea Actions, which can reuse GitHub Actions workflows. Because it is written in Go, Gitea runs across many platforms and architectures, from Linux and macOS to Windows, FreeBSD and OpenBSD.
Matrix is an open network and an open standard for secure, decentralised and interoperable real-time communication. Synapse, the server implementation used in the stack, is developed and maintained by Element as an open-source Matrix homeserver. Currently included as an experimental module.
Vaultwarden is an alternative server implementation of the Bitwarden Client API, written in Rust and compatible with the official Bitwarden clients, built for self-hosted deployments. It supports nearly all major Bitwarden features, including personal vaults, organizations, two-factor authentication and administrative tools. The project is community-maintained and not officially affiliated with Bitwarden, Inc. Currently included as an experimental module.
Infisical is a developer-focused secrets management service with project/environment scoping, via CLI and REST API. In the stack, AI coding agents such as OpenHands use it to pull credentials at runtime, secured behind Authentik SSO. Currently included as an experimental module.
The backup service is built into a container by rzfz.ai, but the underlying tool — docker-volume-backup — comes from Offen. It runs nightly, automated, GPG-encrypted backups of every database and volume in the stack, takes a database dump ahead of the volume snapshot, and retains backups for a configurable retention period.
The config portal is the rzfz.ai-developed central interface for configuring the stack: turning modules on and off, managing settings and secrets, uploading TLS certificates, triggering backups, and running a factory reset. It writes changes directly into the Docker Compose configuration and applies them.
The help center is an rzfz.ai-developed, locally hosted documentation hub for the stack. It renders the bundled guides and reference pages so support content stays available even without an internet connection.
The start portal is an rzfz.ai-developed landing page shown after login: it reads the enabled modules from the Compose configuration and links directly to every unlocked application.
Autoheal watches every container's Docker health checks and restarts unhealthy ones automatically. In the stack it's opted in specifically for the GPUStack family and the Presidio analyzer, whose worker processes occasionally wedge.
Komodo is a dashboard for container health, logs and restart controls. In the stack it monitors every Docker Compose stack, backed by its own database separate from the main system.
GPUStack is an open-source GPU cluster manager for AI model serving and GPU instance provisioning. It orchestrates inference engines such as vLLM, SGLang and TensorRT-LLM across on-premises servers, Kubernetes and cloud environments, and covers a broad hardware ecosystem spanning NVIDIA and AMD GPUs to Ascend, Hygon and other accelerators. Its core capabilities include automated engine configuration, "day-0" support for newly released models, and monitoring and access controls. In the rzfz.ai stack, inference runs on a llama.cpp backend via Vulkan (GGUF models).
Speaches is a speech-to-text and text-to-speech service that transcribes and synthesizes audio locally behind an OpenAI-compatible API. In the stack it powers voice input and output for Open WebUI and runs internal-only.
Cognee builds a knowledge graph from documents and data sources and answers questions via GraphRAG — more precise than plain vector search. In the stack it serves as the central company brain that chat, workflows and agents access through the MCP registry. Currently included as an experimental module.
LightRAG is a dual-level GraphRAG framework that links entities and relationships for more precise retrieval answers. In the stack it uses GPUStack for embeddings and graph construction, serving as a lightweight alternative to Cognee for smaller knowledge bases. Currently included as an experimental module.
Observability (OpenLIT + ClickHouse) measures cost, latency, token usage and prompts for every LLM call that flows through the chat pipeline. In the stack, access is restricted to a dedicated Authentik group, and a PII filter redacts prompt and completion text by default. Currently included as an experimental module.
Caddy is the stack's reverse proxy and its single external entry point: all requests pass through ports 80/443, with TLS certificates managed automatically. A rate-limit plugin protects the login endpoints.
PostgreSQL with the pgvector extension is the stack's relational database — one instance with a separate database per module. It stores the data for Open WebUI, Dify, Authentik, GPUStack and other modules, and doubles as the vector store for RAG.
Authentik is the stack's identity provider: single sign-on, role-based access control and multi-factor authentication. It sits as a forward-auth gate behind Caddy in front of every module and decides, by group membership, who may reach which application.
Valkey is a Redis-compatible in-memory data structure store for caching and message queues. In the stack, several modules share one instance, each isolated on its own database index.
The SMTP relay is a Postfix mail server for outbound system messages. It runs internal-only and delivers notifications and password resets for modules like Authentik and Dify, either via an external provider or direct MX delivery.
The Docker socket proxy is a filtered access point to the Docker API. Modules that need to manage containers — such as the config portal, backup service, agent manager or Authentik — talk to Docker exclusively through this proxy, never through the unprotected host socket.
developed by razzfazz.ai GmbH
Experimental
Features
The rzfz.ai Stack — the integrated open-source stack for local AI infrastructure — covers
seven core scenarios end to end. Each feature combines several modules into a finished use
case, entirely on your own hardware.
Local Chat
ChatGPT comfort without a word leaving the building
Open WebUI is the central chat interface for every model on the box — combining retrieval-augmented generation, web search and speech in/out in one interface. In the background, GPUStack with llama.cpp handles inference on your own GPU, so every request is answered locally. Because the model, vector search and interface all run on the same infrastructure, no prompt and no answer ever leaves the building — with no external API dependency at all. In practice, Open WebUI serves as the front end for central knowledge bases and industry-specific RAG applications, from production control to foreign trade.
Automate business processes with AI — on your own hardware
Dify orchestrates business processes as visual workflows — from simple classification to multi-step document pipelines with approval steps. Every node in the workflow runs against the stack's local models, so automations can be assembled without programming knowledge and tested directly against real business data. Because orchestration and inference run on the same box, sensitive client and customer data stays in house throughout processing. In production, Dify already classifies support emails for an IT service provider, supports accounting processes at a tax advisory firm, and structures digitalization projects in consulting.
One central company brain for agents, chat and workflows
Cognee builds a knowledge graph across your documents and systems and answers questions via GraphRAG instead of plain keyword search — more precise for complex relationships and across document boundaries. In the stack it serves as shared memory that chat, workflows and agents access through the MCP registry, instead of every module keeping its own knowledge. Because the knowledge graph is built entirely on your own hardware, internal documents and process knowledge stay in house — Cognee is currently included as an experimental module and under active development. At one production company it already consolidates manuals, incident history and process documentation into one central knowledge base that staff query simply via chat.
Personal AI agents per employee — no Telegram, no WhatsApp needed
Every employee provisions their own personal agents through the agent manager — Hermes as a Python agent with persistent memory and more than 30 tools, Moltis as a Rust-based agent server with Matrix integration. They are reached the normal way, through Open WebUI, with no need for Telegram, WhatsApp or any other external messenger. Because the agent manager, Hermes and Moltis all run on the same box as the models, memory contents and conversations stay entirely local — all three are currently included as experimental building blocks of the stack. This makes it possible to build personal assistants that remember earlier requests and take on tasks independently, without data ever leaving your own infrastructure.
Modules: Agent Manager, Hermes, Moltis
Local Coding Agents
No code leaves the box
Coding agents run in a protected container terminal on the box or master — with access to the local Gitea and on-box coding models. They take on bounded jobs like raising test coverage, refactorings or spec-driven implementation and deliver results as merge requests. With models, code and git server on the same infrastructure, there are no external API keys and not a single line of code leaves the premises. One customer already runs an agent that raises unit-test coverage around the clock.
Docling, Tika and Gotenberg read, convert and interpret documents of every kind — from PDFs and Office files to scanned receipts with layout analysis. Vision models on your own GPU handle image recognition, while Stirling PDF is available as a toolbox for merging, splitting and converting. Because the entire pipeline — recognition, conversion and vision inference — runs on the same infrastructure, invoices, contracts and other sensitive documents never leave the building; Docling, Tika and Stirling PDF are currently included as experimental modules. In production, this pipeline already pre-processes receipts for a tax advisory firm and extracts inbound customer documents before sensitive data gets redacted.
Modules: Docling, Apache Tika, Gotenberg, Stirling PDF
Detect and redact personal data — in text and images
Microsoft Presidio detects and anonymizes personal data in document pipelines before it is passed on to other modules — more than 30 categories, in text as well as in images. The Analyzer identifies sensitive spots, the Anonymizer redacts or replaces them, and the Image Redactor does the same job for photos and scans. Presidio runs internal-only as a preprocessing step on the same box as the other modules — personal data gets redacted before it ever reaches a language model; the module is currently included as an experimental building block of the stack. It is already used to redact inbound customer documents and, in image analysis for patient care, to keep sensitive health data entirely within the facility.
"Nothing leaves the box unless a module is explicitly configured to reach out."
Enterprise
The rzfz.ai Stack — the integrated open-source stack for local AI infrastructure — is hardened
for enterprise use: identity, network and compliance are designed in from the start, not
bolted on afterward.
Identity & access
Authentik sits in front of every module — no service is reachable without signing in.
Role-based access control (RBAC) and multi-factor authentication (MFA) are the default,
not an add-on.
Network hardening
Caddy is the only externally reachable entry point — with rate limiting against abuse.
The Docker daemon is never directly exposed; a Docker socket proxy mediates every
access.
An SSRF proxy filters outbound HTTP calls from workflows before they leave the network.
Every service port is bound to localhost — nothing listens on a public interface.
Compliance
The security documentation includes mapping tables for NIS2 and ISO 27001, plus an
assessment against the OWASP LLM Top 10. Every release ships with an SBOM and a security
assessment.
Secrets & backups
Secret rotation tooling and GPG-encrypted backups are part of standard operations — no
plaintext credentials, no unencrypted backups.
Three tiers: Community, Subscription, Services
The rzfz.ai Stack's source code is public on Codeberg — if you run the stack privately or
for evaluation, all you need is community support via Codeberg Issues and the community
wiki (on Codeberg).
The rzfz.ai subscription is a license, not a service: it grants your company the right to
operate the current, patched version of the rzfz.ai Stack commercially. The source code is
public — commercial production use runs on the subscription.
The maintenance flat rate, standard support, single support requests and trainings are
standalone services and not part of the subscription — you book them separately, as
needed.
Documentation is split: the community wiki is the public community documentation, freely
accessible. docs.rzfz.ai is the Enterprise documentation, gated behind login for
subscription customers.
rzfz.ai Subscription
799 € per server/box/VM per year, net
Right to operate the current, patched stack commercially
At least 4 releases per year (currently monthly) incl. security updates
Access to the Enterprise documentation (docs.rzfz.ai)
One subscription per stack installation (server, VM or box)
The rzfz.ai Stack — the integrated open-source stack for local AI infrastructure — is
versioned with CalVer and currently ships on a monthly release cycle. The list below shows
every release, newest first.
v2026.07-ga
Open-core licensing goes public: Apache-2.0 Community, the rzfz.ai Subscription and coding-agent workspaces
Open-core licensing, delivered publicly — a free Apache-2.0 Community tier, a source-available tier (BUSL-1.1) under the rzfz.ai Subscription, a live license overview, a public Codeberg mirror and a published Community Wiki
Coding-agent workspaces — personal coding agents are now per-type sandboxed containers (opencode, Codex, user-defined) with live web-app preview plus per-user MCP integrations and memory
Dify 1.15.0 — single sign-on no longer asks you to log in twice
v2026.06-ga
One default model: qwen3.6 at 1M context for chat, coding and vision
qwen3.6 is the one default model for chat, coding, general tasks and vision at 1M context — no per-role model juggling
Document Q&A works out of the box — the reranker and chunking defaults that make RAG find the right facts ship pre-configured
Smoother upgrades — both the standard and the from-2026.04 upgrade paths are validated, with upgrade self-healing and journaling
Unattended USB appliance — a bootable USB image performs an unattended Ubuntu 26.04 install and prepares the stack for first boot
Audit-ready compliance tables — structured NIS2 (EU 2022/2555) and ISO/IEC 27001:2022 evidence in the security architecture
v2026.05-ga
Per-user personal agents: Hermes, Moltis and coding agents in the "My Agents" drawer
rzfz.ai start portal — tile launcher with per-user pinning
New Crawl4AI module and observability profile (OpenLIT + ClickHouse)
Open WebUI ↔ Dify manifold pipe
v2026.04-GA
One command from base install to fully configured stack: razzfazz-post-install.sh
razzfazz-post-install.sh automates GPUStack model deployment plus Open WebUI and Dify setup in one step
Two experimental RAG modules: LightRAG (graph-aware) and Cognee with FalkorDB
Encrypted .env snapshots before every configuration change; AMD GPU inference fix
The rzfz.ai Stack — the integrated open-source stack for local AI infrastructure — supports
only Ubuntu 26.04 LTS as an operating system. The same rule applies to every deployment
profile: We name the limits before you find them.
rzfz.ai Box
AMD Strix Halo (Ryzen AI MAX+), 128 GB unified memory
AMD-only inference — no NVIDIA/CUDA path on the box
vLLM currently unsupported on Strix Halo (production: llama.cpp via Vulkan)
The 32 GB container budget limits how many modules run concurrently
One intensive agent loop at a time — sizing rule ~1 box per 6–10 employees
Own server / VM without GPU
Any Ubuntu 26.04 host, ~16–24 GB RAM
Requirements
Ubuntu 26.04 LTS, kernel 7.x
Docker Engine + Compose v2
Limitations
CPU inference is slow — fine for light or batch workloads, not for interactive agents
As a control plane it delegates inference to GPU workers (boxes or cloud)
NVIDIA/CUDA server
Ubuntu 26.04 host with NVIDIA GPUs — own hardware or GPU cloud
Requirements
Ubuntu 26.04 LTS
CUDA-capable NVIDIA GPU(s)
Docker Engine + Compose v2
Limitations
Inference via llama.cpp on CUDA, models in GGUF format
The VRAM ceiling is a budget question, not an architectural one
Cloud instance
Any Ubuntu 26.04 cloud instance — with GPU as worker, without GPU as control plane
Requirements
Ubuntu 26.04 LTS
Docker Engine + Compose v2
Limitations
Data sovereignty depends on the cloud provider — for strictly local scenarios choose the box or own hardware
Mixed fleets
Many teams combine profiles instead of committing to one: a control plane without its own
GPU orchestrates several rzfz.ai boxes as local inference workers and, when needed, pulls in
additional cloud GPU workers for peak load. Core operations stay local while peak load
scales in elastically. Find matching combinations under Bundles.
rzfz.ai · stack · tty1 all data stays local
[ de ]