← Latest products

I built a tensor library from scratch in C++/CUDA

Tensor library & inference framework for machine learning. My main focus has been on getting the basics right, prioritizing a clean API, simplicity, and clear observability for running small LLMs locally.

LibraryOpen source

119 stars · GitHubexternal site

Visit github.comexternal site

Reactions (1)

Similar products

by description similarity

  1. Tiny-vLLM

    github.com

    high performance LLM inference engine in C++ and CUDA

    Hacker News205 pts

  2. Free Inference Engineer and Model Training Roadmap

    inferquest.org

    Free, gamified roadmaps for LLM engineering: an Inference Engineering path (KV caches, CUDA kernels, production vLLM serving) and a Model Training path (pretraining on a budget, scaling laws, SFT/DPO/GRPO) — 185 tasks with auto-verified milestones instead of a paper certificate.

    Hacker News16 pts

  3. TinyGPT

    github.com

    A Simple, Educational Deep Learning Library

    Hacker News13 pts

  4. Lightweight Llama3 Inference Engine

    github.com

    CUDA C

    Hacker News12 pts

  5. Open source ML programming language playground

    sw-ml-study.github.io

    After joining a ML Study group and trying Google Colab I decided to create my own array programming language to help me learn and visualize machine learning.

    Hacker News11 pts