← 최신 프로덕트

Llama 3.2 Interpretability with Sparse Autoencoders

A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and fully reproducible. I spent a lot of time and money on this rather big side project of mine that attempts to replicate the mechanistic interpretability research on proprietary LLMs that was quite popular this year and produced great research papers by Anthropic [1], OpenAI [2] and Deepmind [3].

오픈소스

641 스타 · GitHub외부 사이트보관됨

github.com 방문외부 사이트

반응 (1)

비슷한 프로덕트

설명이 비슷한 순

  1. Llama 3.3 70B Sparse Autoencoders with API access

    goodfire.ai

    Hacker News201점

  2. A fully open-source (Apache 2.0)implementation of llama

    github.com

    Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

    Hacker News158점

  3. Smolmodels

    github.com

    open-source tool to build ML models using natural language

    Hacker News37점

  4. LLM

    github.com

    a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)

    Hacker News45점

  5. Llama-8B Teaches Itself Baby Steps to Deep Research Using RL

    github.com

    Autonomously train research-agent LLMs on custom data using reinforcement learning and self-verification.

    Hacker News39점