← Latest products

Jlama

A fast Java inference engine for GPT and Llama models. Jlama is a modern LLM inference engine for Java.

Open source

1,310 stars · GitHubexternal site

Visit github.comexternal site

Reactions (1)

Similar products

by description similarity

  1. A fully open-source (Apache 2.0)implementation of llama

    github.com

    Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

    Hacker News158 pts

  2. Llmtop

    github.com

    Htop for LLM Inference Clusters (vLLM, SGLang, Ollama, llama)

    Hacker News5 pts

  3. Goku

    userfrom1995.github.io

    WASM (wllama)-powered LLM inference and model manager

    Hacker News9 pts

  4. We made our own inference engine for Apple Silicon

    github.com

    A high-performance inference engine for AI models

    Hacker News186 pts

  5. Timber

    github.com

    Ollama for classical ML models, 336x faster than Python

    Hacker News207 pts