← Trending

Tangerine Index lists products launched since ChatGPT and people's reactions to them.

TensorRT-LLM

github.com

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

Open source

Visit github.comexternal siteGitHub15,046 starsexternal site

Install and connect

From the official MCP Registry, Homebrew, npm, PyPI, and crates.io. Matched by repository or domain once a day.

Reactions (1)

Similar products

by description similarity

  1. LLGTRT

    TensorRT-LLM+Rust server w/ OpenAI-compat and Structured Output

    Hacker News6 ptsGitHub73 starsgithub.comOpen source2 reactionslatest reaction

  2. Qwen-TensorRT-LLM

    GitHub618 starsgithub.comOpen sourcelatest reaction

  3. LLM

    a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)

    Hacker News45 ptsGitHub6,145 starsgithub.comCLIOpen source2 reactionslatest reaction

  4. TensorSharp

    A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability

    GitHub667 starsgithub.comOpen sourcelatest reaction

  5. LLM CLI tool now supports self-hosted language models via plugins

    LLM is my command-line utility and Python library for working with large language models such as GPT-4. I just released version 0.5 with a huge new feature: you can now …

    Hacker News6 ptssimonwillison.netCLISelf-hosted2 reactionslatest reaction