BC ALL
Saved by @ankitsrihbti

How can we use language supervision to learn better visual representations for robotics?

Introducing Voltron: Language-Driven Representation Learning for Robotics!

Paper: https://t.co/gIsRPtSjKz
Models: https://t.co/NOB3cpATYG
Evaluation: https://t.co/aOzQu95J8z

🧵👇(1 / 12)

 Videos of humans performing everyday tasks (Something-Something-v2, Ego4D) offer a rich and diverse resource for learning representations for robotic manipulation.

Yet, an underused part of these datasets are the rich, natural language annotations accompanying each video. (2/12)
The Voltron framework offers a simple way to use language supervision to shape representation learning, building off of prior work in representations for robotics like MVP (https://t.co/Pb0mk9hb4i) and R3M (https://t.co/o2Fkc3fP0e).

The secret is *balance* (3/12)
Starting with a masked autoencoder over frames from these video clips, make a choice:

1) Condition on language and improve our ability to reconstruct the scene.

2) Generate language given the visual representation and improve our ability to describe what's happening. (4/12)
By trading off *conditioning* and *generation* we show that we can learn 1) better representations than prior methods, and 2) explicitly shape the balance of low and high-level features captured.

Why is the ability to shape this balance important? (5/12)
Because robotics isn't a single thing! While prior work focuses on learning for control, there are so many problems we care about – problems that require different features!

How do we know?

Because we build an evaluation suite of 5 diverse robotics problem domains! (6/12)
Problems like grasp affordance prediction (per-pixel segmentation) tend to require more *low-level* spatial features; edges, object boundaries, textures.

Evaluation: the ARC Grasping dataset (https://t.co/rRI4ya84DL) – CC @andyzengtweets @SongShuran. (7/12)
Learning for control tasks benefit from representations that mix of low and high-level features.

Modeling *multi-frame* contexts (easy with Voltron) is also high-impact!

Evaluation: Franka Kitchen & Adroit Manipulation domains from R3M – CC @aravindr93 @Vikashplus. (8/12)
Really cool is how we can use the generative language model zero-shot, with no extra data.

Given a video & language intent, we can score – in real time – how well the behavior in the video captures the intent.

Transfers to *robot data* – no robots during pretraining! (9/12)
But don't take our word for it – try out our representations yourself... or evaluate your own!

Models & Pretraining: https://t.co/NOB3cpATYG
Evaluation Suite: https://t.co/aOzQu95J8z

Use our models: `pip install voltron-robotics` (10/12)
This project was a huge endeavor; one that would not have been possible without amazing collaborators and mentors – @SurajNair_1 @_anniechen_ @tkollar @chelseabfinn @DorsaSadigh and @percyliang.

Further thanks to @ToyotaResearch, @stanfordnlp, and the @StanfordAILab ! (11/12)
I'm really excited to see the impact of language on shaping representations for robotics... but this isn't the end. The hard parts of robotics remain hard.

Voltron is a building block – a tool. I can't wait to see how y'all use it. Thanks folks – and stay tuned 🤖🚀! (12/12)

More from All

Jaya_Upadhyaya
Jaya_Upadhyaya
@Jayalko1
WEAPONS OF MAHADEV

Sarvaripati Shiva is one of the most fearsome manifestations of Mahadev. Assigned with destroying all of the universes at the end of time, Shiva is one of the most fearsome warriors. He is the Mrityunjaya.
Here are some weapons associated with him:


TRISHULA: the 3 pointed blades denote creation, maintenance, and destruction or the 3 Kalas – Past, Present and Future. They also represent 3 Gunas – Sattva, Rajas and Tamas).
Shiva’s trishul can only be stopped by Shiva himself.
@onlyonenetra @DetheEsha


Trishul is also said to represent Ida, Pingala, and Sushumna. These are the three basic nadis – the left, the right and the central – in the pranamaya kosha, or the energy body of the human system.
@iamkamyabuch @unever_followme


PARASHU: axe with 4 cutting edges which Shiva gave to Parashurama. In his anger, Parashurama used the parashu to get rid of the all the Earth’s tyrannical kshatriya caste twenty-one times over.
@harshasherni


CHANDRAHASA SWORD: The divine sword Chandrahasa literally means ‘the laughter of the moon’. Ravana gave this sword to Indrajit on different occasions especially to fight against Devas.
GIRISH: A special sword of Shiva with unique characteristics.
ALL
Anu Satheesh
Anu Satheesh
@AnuSatheesh5
Thrikkangode Sree Randu Moorthy Temple
Ottappalam, Kerala. 🚩
#keralatemples

This is very unique temple as Shiva and Vishnu both are in the “linga” form in the same Garbhagriham. The Srikovil is in a circular pattern. Temple's Prathishta was done by Parasurama.


Mahadeva and Maha Vishnu in the same Srikovil is very rare Prathishta, but poojas are done separately Dhara is done to Maha Vishnu here. People suffering from “KAIVISHAM” and mental illness are fully cured on drinking a medicinal herb supplied by a nearby family here.


The root of the banyan tree around the Ayyappa Sannidhanam is said to have great powers for skin ailments.

Very ancient temple but was covered with forest for many years.
Later two persons Chakki and Chaathan, while cutting trees their axe hit on the nearby
@SriramKannan77


Vigrahams and it suddenly started bleeding. Later Devaprashnam was held and thus found out that these were murthy's of Shiva and Vishnu which was worshipped long back. And thus temple is constructed here.
Om Namah Sivaye
Om Namo Narayanaya
🕉🙏🕉
ALL
Advertisement
sriram
sriram
@SriramKannan77
'Treasure Gods' - Sangha Nidhi' and 'Padma Nidhi', Tirumala, Tirupati.

------------------------------------

These are the two guards guarding the Wealth and Treasure of Lord Sri Venkateswara.

They are representatives of two treasures of Kubera, the God of Wealth.


One guard who is holding shangam (conch) on both the hands is known as Shanka Nidhi ('nidhi' means wealth).

On the other side, the guard who is holding Lotus flowers on both the hands is known as Padma nidhi.

Shangu is from the ocean which includes all types of wealth like pearls and corals.

Why shagam in his hand? Because shangam only gives the sound of 'omkaram'.

'Padmam' is the symbol of water sources in land like river, lakes, tanks etc. which is very important for all living beings.

As per the tradition, these guards are installed at the third entrance of the temple.

One enters the holy shrine after saluting the first protection threshold - Sankha Nidhi and Padma Nidhi.
ALL
Stanis\u0142aw \u017baryn
Stanisław Żaryn...
@StZaryn
In the last couple of days Russian propaganda has published a lot of content alleging that Poland staged provocations against the Nord Stream 2 pipeline. The content in question has employed narratives in use by the Kremlin in its influence activities against Poland. 1/6

Pattern no. 1
Articles posted to Russian websites pick up false accusations against the Poles. These include bogus claims that Poland sabotaged the construction site of the Russia-to-Germany gas link and carried out a direct attack on the pipeline. 2/6


Pattern no. 2
Some of the articles warn that the Kremlin may take retaliatory actions against Poland. Based on false accusations thrown by Russia, the Kremlin’s propaganda amplifies the calls by some journalists and contributors to embark on aggressive activities against PL. 3/6


Pattern no. 3
The Russian media outlets publish a lot of materials that mock the Polish Navy. The narratives that denigrate this component of the Polish Armed Forces are based on opinions and quotes by single Polish Internet users. 4/6


These information warfare efforts aim to:

- solidify false accusations made against Poland

- threaten the Polish society with a potential response by Russia

- weaken the morale of the Polish soldiers and undermine the image of the army 5/6
ALL
Angela Jamison
Angela Jamison
@AnJamison
@TomthunkitsMind @buzz\_chronicles @buzz\\\_chronicles @buzz_chronicles save as category @pdfmakerapp grab @pdfmakerapp grab this @readwiseio save @readwiseio save thread @threader compile summarize @Threadreaderapp unroll please @ttttreads unroll @unrollthread.com @SaveToBookmarks
ALL

You May Also Like

Nikita Poojary
Nikita Poojary
@niki_poojary
One of the most successful stock trader with special focus on cash stocks and who has a very creative mind to look out for opportunities in dark times

Covering one of the most unique set ups: Extended moves & Reversal plays

Time for a 🧵 to learn the above from @iManasArora

What qualifies for an extended move?

30-40% move in just 5-6 days is one example of extended move

How Manas used this info to book


Post that the plight of the


Example 2: Booking profits when the stock is extended from 10WMA

10WMA =


Another hack to identify extended move in a stock:

Too many green days!

Read
SCREENERS , A SCREENER
Noah Smith
Noah Smith
@Noahpinion
1/Politics thread time.

To me, the most important aspect of the 2018 midterms wasn't even about partisan control, but about democracy and voting rights. That's the real battle.

2/The good news: It's now an issue that everyone's talking about, and that everyone cares about.

3/More good news: Florida's proposition to give felons voting rights won. But it didn't just win - it won with substantial support from Republican voters.

That suggests there is still SOME grassroots support for democracy that transcends

4/Yet more good news: Michigan made it easier to vote. Again, by plebiscite, showing broad support for voting rights as an

5/OK, now the bad news.

We seem to have accepted electoral dysfunction in Florida as a permanent thing. The 2000 election has never really
POLITICS
Advertisement
\u0936\u094d\u0930\u0940\u092e\u093e\u0932\u0940 \u0939\u093f\u0924\u0947\u0936 \u0905\u0935\u0938\u094d\u0925\u0940
श्रीमाली हितेश अवस्थी...
@HiteshAwasthi89
Sri Maha Bhairavar Rudhra alayam, Chengalpet, Tamilnadu

This temple is being constructed merely on the instructions from Bhairava by his devotee Shri Bhairava Siddhantham Swamy. The Temple is fully enshrinedin a transparent glass enclosure..1/n


with Rudrakshas on both the sides of the entrance This temple has a circular Sanctum Sanctorum which has twelve stepseach representing a sign of zodiac and after that twelve steps we find the supreme deity of this temple Sri Kshetra Paata Bhairava. 2/n


Beneath the Bhairava, in the lower level are installed the Ashta Bhairavas, SapthaRishis in the form of stones, and Saptha Kannis in the form of brass lamps. Thereare a number of little blue and white lights in the ceiling that is shaped like a dome.. 3/n


which give an ethereal atmosphere when switched on. The steps representthe twelve signs of the zodiac and when one steps over them to reach thesanctum sanctorum all the doshas pertaining to the horoscope are cleared onceand for all..4/n


The Sthala Vriksham ”Peepal Tree” which is grown on a pedestal, a trident which is considered as Bhairavi, Pancha Mukha Anjaneya, Prathyankara Devi, Subramanya, Rudra Vinayaga, Vaishnavi Devi, Naga Rudra, Eswara, Vana Durga etc. 5/5

#IncredibleBharat
#DekhoApnaDesh
ALL
\u0936\u094d\u0930\u0940\u092e\u093e\u0932\u0940 \u0939\u093f\u0924\u0947\u0936 \u0905\u0935\u0938\u094d\u0925\u0940
श्रीमाली हितेश अवस्थी...
@HiteshAwasthi89
Sri Maha Bhairavar Rudhra alayam, Chengalpet, Tamilnadu

This temple is being constructed merely on the instructions from Bhairava by his devotee Shri Bhairava Siddhantham Swamy. The Temple is fully enshrinedin a transparent glass enclosure..1/n


with Rudrakshas on both the sides of the entrance This temple has a circular Sanctum Sanctorum which has twelve stepseach representing a sign of zodiac and after that twelve steps we find the supreme deity of this temple Sri Kshetra Paata Bhairava. 2/n


Beneath the Bhairava, in the lower level are installed the Ashta Bhairavas, SapthaRishis in the form of stones, and Saptha Kannis in the form of brass lamps. Thereare a number of little blue and white lights in the ceiling that is shaped like a dome.. 3/n


which give an ethereal atmosphere when switched on. The steps representthe twelve signs of the zodiac and when one steps over them to reach thesanctum sanctorum all the doshas pertaining to the horoscope are cleared onceand for all..4/n


The Sthala Vriksham ”Peepal Tree” which is grown on a pedestal, a trident which is considered as Bhairavi, Pancha Mukha Anjaneya, Prathyankara Devi, Subramanya, Rudra Vinayaga, Vaishnavi Devi, Naga Rudra, Eswara, Vana Durga etc. 5/5

#IncredibleBharat
#DekhoApnaDesh
ALL
Yoruba History - by NNP
Yoruba History - by NN...
@YorubaHistory
The Ife Primary Education Research Project (1970-1983).

aka

The Ife 6-Year Primary Project.


In 1970, as the Director of the Institute of Education of the then University of Ife (now Obafemi Awolowo University, Ile-Ife), later Dean Faculty of Education, Prof. Babs Fafunwa & his team embarked on one of the most groundbreaking research projects in education for Yorubaland.


The aim of the project was to test the hypothesis that the best instructional medium for good concept formation at a very tender age is the mother tongue.

This was bolstered by the observation that “of all the continents and peoples of the world, it is only in Africa…

…that formal education is offered in a language that is foreign to the child.”

So, Yoruba was to be used & tested as an exclusive medium of passing information or instructions to a set of pupils.

The project was based on the premise that:

(a). the child will benefit culturally, socially, linguistically & cognitively.

(b). the child’s command of English will be improved if he is taught English as an entirely separate subject by a specialist teacher through the 6 years.
ALL