Every gateway ships with a set of built-in policies. Authentication. Rate limiting. Request routing. Prompt guards. These cover most use cases. But what about the ones they don’t cover?
What if you need to add a custom header based on a database lookup? What if you need to transform a request body in a way no existing filter supports? What if your business has unique logic that no off-the-shelf ga...
This tutorial exemplifies the growing trend of customizing AI infrastructure to meet specific business needs, moving beyond off-the-shelf solutions. The use of Rust for extending Envoy highlights the demand for performance and safety in low-level gateway modifications, while Kubernetes deployment underscores the dominance of containerized, cloud-native architectures. The mock LLM approach is a pragmatic choice, reducing barriers to experimentation and avoiding vendor lock-in during development.
...