Let's do a short thread on this incredibly common misunderstanding because why not.
There won\u2019t be a Civil War. These right wing dummies know their guns are no match for the U.S. Army. But by all means, let\u2019s keep lamenting about how the unrealistic worst case scenario is definitely going to happen, so we don\u2019t have to deal with actual problems and challenges.
— Palmer Report (@PalmerReport) October 10, 2021
https://t.co/iQjOSakX2w
More from All
How can we use language supervision to learn better visual representations for robotics?
Introducing Voltron: Language-Driven Representation Learning for Robotics!
Paper: https://t.co/gIsRPtSjKz
Models: https://t.co/NOB3cpATYG
Evaluation: https://t.co/aOzQu95J8z
🧵👇(1 / 12)
Videos of humans performing everyday tasks (Something-Something-v2, Ego4D) offer a rich and diverse resource for learning representations for robotic manipulation.
Yet, an underused part of these datasets are the rich, natural language annotations accompanying each video. (2/12)
The Voltron framework offers a simple way to use language supervision to shape representation learning, building off of prior work in representations for robotics like MVP (https://t.co/Pb0mk9hb4i) and R3M (https://t.co/o2Fkc3fP0e).
The secret is *balance* (3/12)
Starting with a masked autoencoder over frames from these video clips, make a choice:
1) Condition on language and improve our ability to reconstruct the scene.
2) Generate language given the visual representation and improve our ability to describe what's happening. (4/12)
By trading off *conditioning* and *generation* we show that we can learn 1) better representations than prior methods, and 2) explicitly shape the balance of low and high-level features captured.
Why is the ability to shape this balance important? (5/12)
Introducing Voltron: Language-Driven Representation Learning for Robotics!
Paper: https://t.co/gIsRPtSjKz
Models: https://t.co/NOB3cpATYG
Evaluation: https://t.co/aOzQu95J8z
🧵👇(1 / 12)
Videos of humans performing everyday tasks (Something-Something-v2, Ego4D) offer a rich and diverse resource for learning representations for robotic manipulation.
Yet, an underused part of these datasets are the rich, natural language annotations accompanying each video. (2/12)
The Voltron framework offers a simple way to use language supervision to shape representation learning, building off of prior work in representations for robotics like MVP (https://t.co/Pb0mk9hb4i) and R3M (https://t.co/o2Fkc3fP0e).
The secret is *balance* (3/12)
Starting with a masked autoencoder over frames from these video clips, make a choice:
1) Condition on language and improve our ability to reconstruct the scene.
2) Generate language given the visual representation and improve our ability to describe what's happening. (4/12)
By trading off *conditioning* and *generation* we show that we can learn 1) better representations than prior methods, and 2) explicitly shape the balance of low and high-level features captured.
Why is the ability to shape this balance important? (5/12)
You May Also Like
I think a plausible explanation is that whatever Corbyn says or does, his critics will denounce - no matter how much hypocrisy it necessitates.
Corbyn opposes the exploitation of foreign sweatshop-workers - Labour MPs complain he's like Nigel
He speaks up in defence of migrants - Labour MPs whinge that he's not listening to the public's very real concerns about immigration:
He's wrong to prioritise Labour Party members over the public:
He's wrong to prioritise the public over Labour Party
One of the oddest features of the Labour tax row is how raising allowances, which the media allowed the LDs to describe as progressive (in spite of evidence to contrary) through the coalition years, is now seen by everyone as very right wing
— Tom Clark (@prospect_clark) November 2, 2018
Corbyn opposes the exploitation of foreign sweatshop-workers - Labour MPs complain he's like Nigel
He speaks up in defence of migrants - Labour MPs whinge that he's not listening to the public's very real concerns about immigration:
He's wrong to prioritise Labour Party members over the public:
He's wrong to prioritise the public over Labour Party