I don't know much about LLMs but does that mean you have N^2 computation with the context size since every token needs to track how it relates to every other token?
This is great, I've read multiple books and watched videos about the attention mechanism. Now that I understand it, this is the clearest example I've seen on how attention works.
I don't disagree with that. I did add an entire caveat paragraph there.
To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even with a lot of simplification, it can show some interesting patterns.
I mean, there so many headers and layers, it is tricky to make a choice that will resonate with our intuition . Is it some weighted average? Or maybe ablation test?
To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even with a lot of simplification, it can show some interesting patterns.
are you worried later-layer attention gets drowned out by earlier layers just because there are more of them contributing to the sum?
Right now only simple correlations are visible.
I mean, there so many headers and layers, it is tricky to make a choice that will resonate with our intuition . Is it some weighted average? Or maybe ablation test?
When I started, I expected I'd have to experiment a lot to find something comprehensible. But this simple computation can already show some patterns.