Since @ylecun introduced his cake analogy at NeurIPS 5 years ago, unsupervised learning and the self-supervised approach in particular, has been a major focus of @facebookai and other AI labs. Progress has been steady and remarkable almost everywhere - NLU, MT, Speech, CV, Video

Not only does this approach often lead to better performance when combined with small amount of supervision, but it offers the promise of a more inclusive and private AI, one that works for all languages and cultures and doesn’t require extensive human labeling. #ResponsibleAI

Here are pointers to some of our latest work relying on self-supervised learning:

Speech https://t.co/4K03h47kIl
CV https://t.co/FcS4QmYQMJ
NLU https://t.co/wCI2q93ybc
MT https://t.co/GIIyZ8qmZ5
Videos https://t.co/o4VWF0r0vK

Overall approach