← Latest products

Spinf-serve

inference server for bulk scoring and decisions. Bulk classification and decision inference on one GPU: the content is read once, every question about it is scored in the same pass. OpenAI-compatible, Apache 2.0.

Self-hostedOpen source

20 stars · GitHubexternal site

Visit github.comexternal site

Reactions (1)

Similar products

by description similarity

  1. Cuts Long Horizon Inference Costs by 50% via external KV Cache Offload

    github.com

    OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

    Hacker News22 pts

  2. I built a tensor library from scratch in C++/CUDA

    github.com

    Tensor library & inference framework for machine learning

    Hacker News119 pts

  3. We made our own inference engine for Apple Silicon

    github.com

    A high-performance inference engine for AI models

    Hacker News186 pts

  4. Spice.ai OSS 1.0

    spiceai.org

    data query and AI-inference engine built in Rust

    Hacker News26 pts

  5. TXT OS

    github.com

    Open-Source AI Reasoning, One Plain-Text File at a Time

    Hacker News11 pts