The Concept of "Attention" in Deep Learning
Deep learning, in all its glory and jargon, is a bit like a precocious toddler—eager to learn and incredibly sharp, but still occasionally drooling all over a Backyardigans toy. The concept of "attention" in language models isn’t that different from a cavorting kiddo picking out the most exciting toy from a pil…