The goal of this blog post is to explain flash attention in such a way that hopefully anyone who already understands attention will ask themselves:

“Why didn’t I think of this before?” followed by “It’s so easy”.

https://gordicaleksa.medium.com/eli5-flash-attention-5c44017022ad