metadata
license: bsd-3-clause
tags:
- kernel
Warning: do not use yet! We are still ironing out the last few issues.
Flash Attention 3
Flash Attention is a fast and memory-efficient implementation of the attention mechanism, designed to work with large models and long sequences. This is a Hugging Face compliant kernel build of Flash Attention.
Original code here https://github.com/Dao-AILab/flash-attention.