Launch HN: Magnitude (YC S25) – Self

Magnitude is an open-source inference engine designed to optimize itself for specific hardware by compiling and tuning kernels directly on the device. It enables running open models up to two times faster than llama.cpp through device-specific kernel tuning, achieving 92% faster decode on Metal and 19% faster decode on CUDA. The system connects existing agent software, including Pi, OpenCode, Hermes, Codex, and more, via a one-click connection process. It operates across Apple Silicon, NVIDIA, AMD, or standard CPUs. The application provides memory optimization by allowing agents to free memory when idle, offering 27% less memory per agent, and supports fast concurrent sessions through shared prefix caches. The software is free, private, and open source under the Apache 2.0 license, with no token costs.

The architecture of Magnitude centers on radical device-specific optimization, moving inference efficiency from generalized library implementations to hardware-specific kernel tuning executed locally. This approach challenges the reliance on generalist frameworks by embedding performance gains directly into the execution layer. The core pattern here is the shift from abstracting computation to localizing and customizing it for the physical constraints of the machine. The promise of "self-optimizing" kernels suggests a continuous feedback loop where software adapts precisely to hardware characteristics, which counters the typical pattern of deploying generalized solutions that require post-hoc optimization. The integration with existing agent ecosystems via an OpenAI-compatible API emphasizes interoperability as a structural advantage, positioning Magnitude not just as a tool for running models, but as an infrastructure layer connecting heterogeneous AI agents. The implication is that true efficiency in decentralized AI systems is realized when the inference pipeline intimately understands and manipulates the physical substrate upon which it runs. What are the long-term consequences of delegating such fine-grained control to end-users across diverse hardware landscapes? How does this level of deep, local optimization reshape the landscape for model distribution and competitive advantage in an increasingly fragmented computing environment?