Deployment

A MagicAF deployment requires three local services. Here’s a reference Docker Compose configuration that starts all of them. Infrastructure Components Service Default Port Purpose Example Software Embedding server 8080 Dense vector embeddings text-embeddings-inference, llama.cpp, vLLM Vector database 6333 Similarity search Qdrant LLM server 8000 Chat completion / generation vLLM, llama.cpp, TGI, Ollama All three communicate via HTTP REST. No gRPC, no cloud SDK, no vendor library is required at the network level. ...

MagicAF is designed for air-gapped environments from the ground up. Every service runs locally, and all dependencies can be vendored for offline use. For more information about MagicAF’s security architecture and compliance features, see the About page. Overview An air-gapped deployment requires four preparation steps on an internet-connected machine, followed by transfer and build on the isolated host. Step 1 — Vendor Rust Dependencies On an internet-connected machine: # From the MagicAF workspace root cargo vendor > .cargo/config.toml This downloads all crate dependencies into a vendor/ directory and generates a .cargo/config.toml that redirects Cargo to use local sources. ...

MagicAF supports deployment on resource-constrained devices where running Qdrant and a full LLM server is impractical. Architecture ┌ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └ ─ ─ ─ ─ ─ ┌ │ │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ O E ( ─ ─ ─ ─ n m O ─ ─ ─ ─ - b N ─ ─ ─ ─ D e N ─ ─ ─ ─ e d X ┬ └ ┌ │ └ ┌ │ │ └ ─ ─ ─ v d / ─ ─ ─ ─ ─ ─ ─ ─ ─ i i C ─ ─ ─ R ─ ─ ─ ─ ─ ─ c n o ─ ─ ─ A ─ ─ R o ─ ─ ─ ─ e g r ─ ─ ─ G ─ ─ e r ─ ─ ─ ─ e ─ ─ ─ W ─ ─ m ─ ─ ─ ─ M ─ ─ ─ o ─ ─ o C ─ ─ ─ ─ L ─ ┬ ▼ r ┬ ▼ t a ─ ─ ─ ┐ ) ┘ ─ ─ k ─ ─ e c ─ ─ ─ │ │ │ ─ ─ f ─ ─ h ─ ─ ─ ─ ─ l ─ ─ L e ─ ─ ─ E ┌ └ ─ ─ o ─ ─ L d ─ ─ ─ d ─ │ │ │ ─ ─ ─ w ─ ─ M ─ ─ ─ g ─ ─ ─ ─ ─ ─ ─ ─ ─ e ─ I ─ ─ ┐ │ ┘ ┐ │ ┘ ─ ─ ─ n ( ─ ─ │ ─ ─ D ─ M m ─ ─ ─ ─ e ─ e a ─ ─ ← ← ─ ─ v ─ m g ─ ─ ─ ─ i ─ o i ─ ─ F W ─ ─ c ─ r c ─ ─ u h ─ ─ e ─ y a ─ ─ l e ─ ─ ─ V f ┬ ┘ l n ─ ─ ─ e - ─ ─ ─ ─ c c ─ M n ─ ─ ─ t o ─ a e ─ ─ ─ o r ─ g t ─ ─ ─ r e ─ i w ─ ─ ─ S ) ─ c o ─ ─ ─ t ─ A r ─ ─ ─ o ─ F k ─ ─ ─ r ─ ─ ─ ─ e ─ o a ─ ─ ┐ ┘ r v ─ ─ │ │ │ c a ─ ─ h i ─ ─ e l ─ ─ s a ─ ─ t b ─ ─ r l ─ ─ a e ─ ─ t ─ ─ i ─ ─ o ─ ─ n ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ ─ ┐ │ │ │ │ │ │ │ │ │ │ │ │ │ ┘ Component Availability Component Server Deployment Edge Deployment Vector Store Qdrant (magicaf-qdrant) InMemoryVectorStore (in magicaf-core) Embeddings llama.cpp / TEI server On-device (ONNX, CoreML, TFLite) LLM vLLM / llama.cpp Omitted, or remote when online Persistence Qdrant handles it InMemoryVectorStore::save() / load() InMemoryVectorStore The zero-dependency vector store that ships with magicaf-core: ...

MagicAF provides built-in observability through structured logging, health checks, and tracing instrumentation. Structured Logging MagicAF uses the tracing ecosystem. All public methods are annotated with #[instrument] and emit structured spans with contextual fields. Basic Setup tracing_subscriber::fmt() .json() .with_env_filter("info,magicaf=debug") .init(); This produces JSON-formatted log lines with fields like: { "timestamp": "2025-01-15T10:30:00Z", "level": "INFO", "target": "magicaf_core::rag", "message": "Searching vector store", "span": { "collection": "my_docs" }, "fields": { "top_k": 10 } } Structured Fields Span Fields RAGWorkflow::run collection VectorStore::search collection, limit VectorStore::index collection, count EmbeddingService::embed count (number of inputs) LlmService::chat model, messages (count) Production Configuration For production, attach a subscriber that ships spans to your observability stack: ...

MagicAF applications are stateless by design. Every component can be scaled independently. Scaling Strategy Component Strategy Notes Embedding Horizontal: N replicas behind a load balancer CPU or GPU instances Qdrant Distributed mode (built-in clustering) Sharding + replication LLM Horizontal: multiple GPU nodes behind a load balancer vLLM supports tensor parallelism Application Horizontal: stateless, scale freely Each instance gets its own RAGWorkflow Embedding Service Scaling Embedding is typically the most CPU/GPU-bound operation. Scale by running multiple replicas: ...

MagicAF is designed with security as a first principle. All services run locally, no data leaves the network boundary, and the framework makes no assumptions about data sensitivity. Learn more about MagicAF’s defense-grade architecture and compliance features. Security Architecture No outbound network — every service endpoint is configurable and defaults to localhost No secrets in structs — API keys are Option<String> and never serialized to logs No PII/PHI handling — the framework never inspects payload content; data classification is the application’s responsibility Domain-neutral — adapters control what data enters the pipeline and what leaves it Hardening Checklist All service endpoints bind to internal interfaces only (127.0.0.1 or VPC, not 0.0.0.0) API keys configured for Qdrant and LLM if exposed on a shared network TLS termination at load balancer or reverse proxy Log output does not contain raw payloads (configure log level accordingly) Model files are integrity-checked (SHA-256) after transfer to air-gapped hosts Vector store access is restricted to authorized application instances Docker images are scanned for vulnerabilities before deployment Rust dependencies are audited (cargo audit) Network Security Bind to Internal Interfaces # docker-compose.yml — bind to localhost only services: qdrant: ports: - "127.0.0.1:6333:6333" llm: ports: - "127.0.0.1:8000:8000" API Key Configuration # Set API keys for services on shared networks export QDRANT_API_KEY="your-secret-key" export LLM_API_KEY="your-secret-key" MagicAF passes these keys in request headers automatically. ...