lhallee commited on
Commit
9790c44
·
verified ·
1 Parent(s): 3c4a612

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -11,7 +11,7 @@ FastESM is a Huggingface compatible plug in version of ESM2 rewritten with a new
11
 
12
  Load any ESM2 models into a FastEsm model to dramatically speed up training and inference without **ANY** cost in performance.
13
 
14
- Outputting attention maps (or the contact prediction head) is not natively possible with SDPA. You can still pass ```output_attentions``` to have attention calculated manually and returned.
15
  Various other optimizations also make the base implementation slightly different than the one in transformers.
16
 
17
  # FastESM2-650
 
11
 
12
  Load any ESM2 models into a FastEsm model to dramatically speed up training and inference without **ANY** cost in performance.
13
 
14
+ The default attention backend is `sdpa`. See the [FastPLMs README](https://github.com/Synthyra/FastPLMs) for a full breakdown of available backends (`sdpa`, `kernels_flash`, `flex`, `auto`) and how to switch between them. Attention maps (`output_attentions=True`) are supported on all backends via a separate naive computation.
15
  Various other optimizations also make the base implementation slightly different than the one in transformers.
16
 
17
  # FastESM2-650
Free AI Image Generator No sign-up. Instant results. Open Now