Run Llama.cpp In-Process from Java with Project Panama FFM

Similar products

by description similarity

1. Run Llama 3.1 8B in the browser

   app.wiz.chat

   HNHacker News23 pts

2. Run LLM generated scripts in CLI.

   verbomate.com

   HNHacker News5 pts

3. Open-source load balancer for llama.cpp

   github.com

   Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.

   HNHacker News151 pts

4. Llama.go

   github.com

   port of llama.cpp to pure Go

   HNHacker News25 pts

5. Low-latency local LLM runner via OpenJDK Panama FFM (Java 22)

   github.com

   Unified native AI runtime for Java 17+, Java 22+ (Panama FFM), Android & C++. Run GGUF LLMs, Vision, Video, Whisper ASR, TTS & Embeddings in-process.

   HNHacker News38 pts