← νŠΈλ Œλ”©

Tangerine IndexλŠ” ChatGPT 이후 λ‚˜μ˜¨ ν”„λ‘œλ•νŠΈμ™€ μ‚¬λžŒλ“€μ˜ λ°˜μ‘μ„ μ‹£μŠ΅λ‹ˆλ‹€.

flash-linear-attention

github.com

πŸš€ Efficient implementations for emerging model architectures

μ˜€ν”ˆμ†ŒμŠ€

github.com λ°©λ¬Έμ™ΈλΆ€ μ‚¬μ΄νŠΈGitHub6,027 μŠ€νƒ€μ™ΈλΆ€ μ‚¬μ΄νŠΈ

λ°˜μ‘ (1)

λΉ„μŠ·ν•œ ν”„λ‘œλ•νŠΈ

μ„€λͺ…이 λΉ„μŠ·ν•œ 순

  1. FlashQLA

    high-performance linear attention kernel library built on TileLang

    GitHubμŠ€νƒ€ 750github.comλΌμ΄λΈŒλŸ¬λ¦¬μ˜€ν”ˆμ†ŒμŠ€μ΅œκ·Ό λ°˜μ‘

  2. flash-sparse-attention

    Trainable fast and memory-efficient sparse attention

    GitHubμŠ€νƒ€ 747github.comμ˜€ν”ˆμ†ŒμŠ€μ΅œκ·Ό λ°˜μ‘

  3. ring-flash-attention

    Ring attention implementation with flash attention

    GitHubμŠ€νƒ€ 1,067github.comμ˜€ν”ˆμ†ŒμŠ€μ΅œκ·Ό λ°˜μ‘

  4. cuLA

    CUDA kernels for linear attention variants, written in CuTe DSL and CUTLASS C++.

    GitHubμŠ€νƒ€ 560github.comμ˜€ν”ˆμ†ŒμŠ€μ΅œκ·Ό λ°˜μ‘

  5. mav

    Model Activity Visualiser

    GitHubμŠ€νƒ€ 521github.comμ˜€ν”ˆμ†ŒμŠ€μ΅œκ·Ό λ°˜μ‘