β νΈλ λ©
Tangerine Indexλ ChatGPT μ΄ν λμ¨ νλ‘λνΈμ μ¬λλ€μ λ°μμ μ£μ΅λλ€.
ring-attention-pytorch
github.com
Implementation of π Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
μ€νμμ€
github.com λ°©λ¬ΈμΈλΆ μ¬μ΄νΈGitHub548 μ€νμΈλΆ μ¬μ΄νΈ
λΉμ·ν νλ‘λνΈ
μ€λͺ μ΄ λΉμ·ν μ
ring-flash-attention
Ring attention implementation with flash attention
native-sparse-attention-pytorch
Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper
FlashQLA
high-performance linear attention kernel library built on TileLang
attention-gym
Helpful tools and examples for working with flex-attention
cuLA
CUDA kernels for linear attention variants, written in CuTe DSL and CUTLASS C++.
