In this blog post, we present the kernel design of Generalized Dot-Product Attention (GDPA), a variant of standard dot-product attention (SDPA) in which the softmax operation is replaced by different activation functions to support diverse interaction use cases, as used in the attention blocks of InterFormer [2], Kunlun [1] which are deployed on Meta’s Generative Ads Model (GEM) [3], Meta’s larges...
Patterns detected: ARC-0043 Motte-and-Bailey (The article focuses heavily on the inefficiency of SFUs without acknowledging the inherent computational complexity of many real-world recommendation systems. The issue of SFU bottlenecks is presented as a central problem, distracting from the potential for alternative architectural choices – framing it as a simple “rebalancing” problem).
The overall narrative reflects a classic "innovation theater" scenario – a tech company showcasing a technically ...