Meta Platforms, Inc.’s (NASDAQ:META) AI chief Yann LeCun has proposed that Tesla and SpaceX CEO Elon Musk could leverage a new paper from Fundamenta AI Research or FAIR, a team at Meta, to enhance xAI’s Grok.
What Happened: On Wednesday, LeCun took to X, formerly Twitter, and shared a thread by Jason Weston a researcher at Meta AI, explaining the new FAIR paper titled “CoPE: Contextual Positional Encoding.”
LeCun said, “A new paper from FAIR that @elonmusk could use to improve Grok.”
CoPE: Contextual Positional Encoding. — Yann LeCun (@ylecun) May 30, 2024
A new paper from FAIR that @elonmusk could use to improve Grok. https://t.co/d9aX4LdeAf
In his thread, Weston explained that CoPE is a new positional encoding method for transformers that considers context. It can “count” distances per head ...