Run Llama.cpp In-Process from Java with Project Panama FFM

비슷한 프로덕트

설명이 비슷한 순

1. Run Llama 3.1 8B in the browser

   app.wiz.chat

   HNHacker News23점

2. Run LLM generated scripts in CLI.

   verbomate.com

   HNHacker News5점

3. Open-source load balancer for llama.cpp

   github.com

   Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.

   HNHacker News151점

4. Llama.go

   github.com

   port of llama.cpp to pure Go

   HNHacker News25점

5. Low-latency local LLM runner via OpenJDK Panama FFM (Java 22)

   github.com

   Unified native AI runtime for Java 17+, Java 22+ (Panama FFM), Android & C++. Run GGUF LLMs, Vision, Video, Whisper ASR, TTS & Embeddings in-process.

   HNHacker News38점