"A Data-Based Perspective on Transfer Learning"

Different classes in a pretraining dataset can have different effects on downstream accuracy. And you can use this to your advantage. [1/9]

They assess these effects using a simple algorithm that trains different models on different subsets of the data and looks at both the class counts and the predictions for each model on each downstream sample. [2/9]
Using their scoring function, you can intelligently remove subsets of classes from the pretraining dataset in order to significantly raise downstream accuracy. [3/9]
Another use of their method is identifying more granular subpopulations than what a downstream task has annotated. E.g., you can find which CIFAR-10 images look most like ostriches even though CIFAR-10 only has the label “bird”. [4/9]
You can also use a similar idea to understand model failure modes or identify data leakage. [5/9]
And last but not least, you can use it to understand helpful/harmful samples in your pretraining dataset. [6/9]
Overall their algorithm seems like a great tool to have in the toolbox. [7/9]
Paper: https://t.co/CKg0nxmSxE

If you like this paper, consider RTing this (or another!) thread to publicize the authors' work, or following the authors: @saachi_jain_ @hadisalmanX @Alaa_Khaddaj… [8/9]
@saachi_jain_ @hadisalmanX @Alaa_Khaddaj …@RICEric22 @ssung_mminn @aleks_madry

For more paper summaries, you might like following @mosaicml, me, or my newsletter: https://t.co/5BMBC84xY8

As always, comments and corrections welcome! [9/9] https://t.co/8VRLAGmrfQ

More from All

You May Also Like

॥ॐ॥
अस्य श्री गायत्री ध्यान श्लोक:
(gAyatri dhyAna shlOka)
• This shloka to meditate personified form of वेदमाता गायत्री was given by Bhagwaan Brahma to Sage yAgnavalkya (याज्ञवल्क्य).

• 14th shloka of गायत्री कवचम् which is taken from वशिष्ठ संहिता, goes as follows..


• मुक्ता-विद्रुम-हेम-नील धवलच्छायैर्मुखस्त्रीक्षणै:।
muktA vidruma hEma nIla dhavalachhAyaiH mukhaistrlkShaNaiH.

• युक्तामिन्दुकला-निबद्धमुकुटां तत्वार्थवर्णात्मिकाम्॥
yuktAmindukalA nibaddha makutAm tatvArtha varNAtmikam.

• गायत्रीं वरदाभयाङ्कुश कशां शुभ्रं कपालं गदाम्।
gAyatrIm vardAbhayANkusha kashAm shubhram kapAlam gadAm.

• शंखं चक्रमथारविन्दयुगलं हस्तैर्वहन्ती भजै॥
shankham chakramathArvinda yugalam hastairvahantIm bhajE.

This shloka describes the form of वेदमाता गायत्री.

• It says, "She has five faces which shine with the colours of a Pearl 'मुक्ता', Coral 'विद्रुम', Gold 'हेम्', Sapphire 'नील्', & a Diamond 'धवलम्'.

• These five faces are symbolic of the five primordial elements called पञ्चमहाभूत:' which makes up the entire existence.

• These are the elements of SPACE, FIRE, WIND, EARTH & WATER.

• All these five faces shine with three eyes 'त्रिक्षणै:'.