Systems & Architecture Review
Local Document Intelligence That Never Leaves the Rack
By Ross Nesbitt
Every vendor currently flogging artificial intelligence to the enterprise is selling the exact same illusion: infinite intelligence in a black box, piped straight out of a multi-tenant cloud data center. For casual text generation or brainstorming session notes, that model works fine. But for the professional who holds other people's confidential material as a condition of doing business—lawyers, medical practices, accountants, and defense subcontractors—handing client files to a third-party API isn't a feature; it is an audit failure waiting to happen.
The market has plenty of tools measuring tokens per second. What it has been missing is custody.
Over the past year, I built the answer: a self-hosted, air-gapped document analysis appliance running entirely on hardware you own, in a room you control, with the cloud path severed at the image level rather than trusting a software policy.
Here is how it works, why a small model on modest iron is all you actually need for ninety percent of the workload, and how you can either build your own from these blueprints or have a pre-rolled unit shipped directly to your rack.
Anatomy of the Appliance: Custody Over Cloud
The core realization behind this architecture is simple: intelligence is cheap and getting cheaper, but control is non-negotiable. Firms handling sensitive contracts, HR records, and case notes cannot answer client questionnaires with a shrug and a link to a cloud vendor's fluctuating terms of service.
To solve this without requiring a venture-capital budget or a cluster of water-cooled accelerators, the system is segmented into practical hardware tiers:
- The Entry Tier (Z240 SFF, 8 GB RAM): Deliberately modest. Running a 1.5B model with no GPU overhead, it yields a first token in about 3 seconds and sustains roughly 23 tokens/sec (measured). It is built for classification, date and entity extraction, routing, and policy flagging. It is slow, private, and cheap enough to test on your own desk before committing real money.
- The Working Tier (Z230, 32 GB RAM, SSD): The sweet spot for full adversarial analysis using 7B–14B models, handling the deep analytical lifting where nuance actually matters.
- The Cluster Tier (12 × Z230 in a single rack): Scaled for heavy overnight corpus processing where interactive latency takes a back seat to raw batch throughput.
The Architecture: A Queue, Not a Load Balancer
When building out multi-node processing, traditional web architectures immediately reach for load balancers and round-robin DNS. For overnight corpus processing, where nothing is actively waiting on an interactive HTTP connection, a load balancer only adds a component, a failure mode, and a health-check overhead for zero benefit.
Instead, this architecture relies on a fundamental rule: Nodes pull. Nothing pushes.
A central head unit running Redis manages a single work queue. Consumer nodes run independently, pulling items via a robust `BRPOP` pipeline. A slow node naturally takes fewer items; a dead node takes none; a new node joins the topology instantly without registration or DNS gymnastics.
To make multi-node parallelization rock-solid without race conditions, the engine uses worker-scoped leases and per-item claims atomic with selection. This eliminates the double-processing trap and turns commodity desktop hardware into a synchronized, resilient processing swarm.
Build Your Own or Let Us Roll It For You
The complete architecture—from the pipeline scripts and model-tier routing logic down to the provisioning notes—is fully documented for engineers who want to raid eBay for used Z230 workstations, flash the images, and spin up their own local sovereign intelligence grid.
If you have the hardware and the time, clone the repository, set up your Redis queue, and run it.
If you want a turnkey solution—a pre-configured, hardened appliance delivered to your office with setup, baseline training, and ongoing support backed by decades of enterprise reliability engineering—get in touch.
Intelligence belongs to the models. Your data belongs to you.
Facts Only
* The system proposes a self-hosted, air-gapped document analysis appliance.
* The architecture aims to provide custody over client data by avoiding reliance on third-party cloud APIs for sensitive material processing.
* Hardware tiers are defined: Entry Tier (Z240 SFF, 8 GB RAM) for classification and extraction; Working Tier (Z230, 32 GB RAM, SSD) for deep adversarial analysis of 7B–14B models; and a Cluster Tier (12 × Z230s) for batch processing.
* The architecture utilizes a Redis-managed work queue where nodes pull tasks via BRPOP instead of using traditional load balancing.
* Processing synchronization relies on worker-scoped leases and atomic per-item claims to prevent double-processing.
* The system is designed to use commodity hardware rather than specialized accelerators for most workloads.
* Documentation is provided for building the architecture or acquiring a pre-rolled unit.
Executive Summary
The proposed system advocates for a self-hosted, air-gapped document analysis appliance to address the security and control concerns associated with using third-party cloud AI services for handling confidential enterprise data. The core argument is that while cloud models offer generalized intelligence, they compromise custody over sensitive materials. To solve this, the architecture segments processing into hardware tiers based on workload needs: an Entry Tier for lightweight classification tasks, a Working Tier for deep analysis of nuanced content using larger models (7B-14B), and a Cluster Tier for high-throughput batch processing.
The technical architecture rejects traditional load balancing in favor of a pull-based queue system managed by Redis, where nodes independently retrieve work items via pipelines rather than relying on centralized push mechanisms. This design aims to create a resilient swarm of processing nodes utilizing commodity hardware, emphasizing data sovereignty and control over the intelligence pipeline. The solution offers an option for users to build the system from provided blueprints or procure a turnkey appliance.
Full Take
The narrative centers on shifting the paradigm from outsourcing intelligence to establishing local, sovereign control over data processing infrastructure. The architectural choices—specifically the pull-based queuing system and hardware segmentation—reflect a deep skepticism regarding centralized vendor control inherent in current AI deployment models. The argument moves beyond mere performance metrics (tokens per second) to focus on ownership and operational resilience (custody). This taps into a foundational tension between the accessibility of generalized AI tools and the necessity of organizational security protocols for handling sensitive information.
The appeal lies in dismantling the "black box" structure by reintroducing physical control over the computation environment. The segmentation into tiers suggests an understanding that different analytical tasks require vastly different computational profiles, challenging the monolithic application of single-model approaches to enterprise data. A critical implication is whether this distributed, localized approach truly scales or introduces new points of systemic fragility compared to centralized cloud systems. Further inquiry must focus on the long-term operational costs and maintenance burden associated with maintaining an air-gapped infrastructure versus relying on established, albeit less sovereign, cloud governance models. What are the hidden externalities of achieving perfect custody?
Sentinel — Human
The text reads like an experienced engineer or architect presenting a technical solution rooted in practical constraints, rather than a synthesized summary of existing knowledge.