Red Candle
Run LLMs Natively in Ruby with Rust. Ruby gem for running state-of-the-art language models locally. Access LLMs, embeddings, rerankers, and NER models directly from Ruby using Rust-powered Candle with Metal/CUDA acceleration. I've been working on Red Candle, a Ruby gem that runs LLMs (Llama, Mistral, Gemma, Phi) directly in your Ruby process through Rust bindings (based on the candle crate from Hugging Face).
Open source
Reactions (1)
Similar products
by description similarity
Modern LLM providers can stream their responses. This is great for user experience — instead of a loading spinner, users see the response being generated in real time. They can also call external functions (also called "tools"): search your database
Offeryn
github.comBuild Tools for LLMs in Rust, with Model Context Protocol
liteLLM Proxy Server
github.com50+ LLM Models, Error Handling, Caching
LLM
github.coma Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)
LLM Rescuer
github.comFixing the billion dollar mistake in Ruby