Composable middleware for LLM inference Optimization Passes
A multi-agent framework written in Rust that enables you to build, deploy, and coordinate multiple intelligent agents. Today I'm sharing a feature I haven't seen done well elsewhere: composable middleware layers for LLM inference pipelines.
LibraryOpen source
Reactions (1)
Similar products
by description similarity
LLM
github.coma Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)
Offeryn
github.comBuild Tools for LLMs in Rust, with Model Context Protocol
An educational Rust project for exporting and running inference on Qwen3 LLM family
GraphFlow
github.comA lightweight Rust framework for multi-agent orchestration
Agentpanel
github.comUniversal LLM API Written in Rust