flash_attention_inference
Models46
Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.
Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.