BC DATA SCIENCE
Saved by @zmbnski

An amazing new project from @bearpelican was just released: https://t.co/DBov6sZTVS . A beautiful design; you can auto-generate a melody from chords, chords from a melody, and more.

It's technically brilliant, combining BERT, seq2seq, and Transformer XL
https://t.co/jF3mO5aXiu

 It's also a wonderful example of leveraging and customizing the fastai framework in a deep & thoughtful way.
Here's the full set of blog posts diving in to this project:

https://t.co/jF3mO5aXiu

https://t.co/DrCJxxJRAy

https://t.co/GgatWxa9nM

https://t.co/U9Yp5IZpxt

More from Data science

Ryan J. Gallagher
Ryan J. Gallagher
@ryanjgallag
Tired of word clouds? Want to do better sentiment analysis? Not sure how to look at the words underneath your measures?

Our long overdue paper on generalized word shift graphs is finally here!
https://t.co/lIBXvbMJWX
https://t.co/vSL1REYT8V

So what are they?

1/n


If we have two texts, there are many ways we can compare them. Weighted averages are a particularly useful measure because they're flexible and interpretable

Proportions, Shannon entropy, the KLD, the JSD, and dictionary methods can all be written as weighted averages

2/n

But weighted avgs are also slippery. When we try to compress complex phenomena like happiness, surprise, divergence, or diversity into a single number, it can be unclear what we're measuring

If the measure goes up, what does that mean? Why did it do that? Can we trust it?

3/n

Very often, that's the end of the line and we're left with an uneasy feeling in the pit of our stomach that our weighted avg is actually picking up a data artifact or some other unintended peculiarity

Word shift graphs help us address those concerns

4/n

First, word shifts look under the hood of weighted averages to see what's going on

All weighted averages are a sum of contributions from individual words. We can pull out those words, and rank which ones contribute the most to the difference between two texts

5/n
DATA SCIENCE
Emil Wallner
Emil Wallner
@EmilWallner
Tips for AI writers:

1. Spend 30% of your effort on skimming all student ML papers (e.g. Stanford NLP CS224n) the past 3 years and prototype your favorites

The idea is everything. Pick an area you are interested in and ideally something that has a visual aspect to it

Most of my 'on the top of my mind' ideas were bad in retrospect. Skimming 100s of student papers will give you an overview of what's interesting.

Student papers are overlooked, easy to understand, and have good compute constraints.

2. Spend 30% on your effort on coding

Create an edge to the project. Apply it to something new and use FastAI or Keras to improve the accuracy with 5-30%.

3. Spend 30% writing an in-depth article

Have a north star article in terms of structure and quality. Find something that stretches you to your utmost capability. I used @copingbear’s Style transfer article:

4. Spend 10% marketing your project

Invest a week in studying the strategies to rank on sites like HN and Reddit, then use them. If you have an interesting result and a great article, you've done the hard work.
DATA SCIENCE
Advertisement
Sophie Hill
Sophie Hill
@sophie_e_hill
This is a more wonky thread about how I made this visualization in #Rstats using the awesome visNetwork

First step is to create the underlying network data. We need one file of "nodes" - i.e. the people and organizations. And one file of "edges" - i.e. the connections between them.

I created these by hand, based on excellent investigate journalism:


Now we can pull these together to create a network visualization!

You'll notice that I included a column for "type" in the nodes file. This allows me to use different icons for people vs firms vs political organizations.


All the icons are taken from @fontawesome. I *think* the visNetwork 📦 currently only works with fontawesome version 4.7, which is a bit limited – e.g. I decided to use a book icon to represent the fringe Evangelical Christian sect "Exclusive Brethren"! 😂


I very much enjoyed getting to use the "incognito" icon to represent all the unknown donors that have funded Tory MP Owen Paterson's overseas jaunts!
DATA SCIENCE
Sunrit Jana \U0001f680
Sunrit Jana 🚀
@JanaSunrise
Pandas is an amazing data analysis and manipulation library for python, Really popular when working with ML, Data science or more.

It has a robust data structure, Dataframe for manipulation and analyzing data.

Here's some tips to help you work better with pandas. Let's go! ↓

If you're not aware about what a Dataframe is, It's an optimized data structure for loading data, analysing it, manipulating data in it, and Mostly gathering insights.

It uses Cython backend which transpiles into C for optimized code.

Here's how a dataframe looks like,


Before we start, You need to ensure, you have pandas installed. If you don't, Do that before moving ahead!

Here are the tips, Let's go!


1/ Convert PD series to Dataframe

We all have struggled, when we deal with pandas series. It's always easier to work with Dataframes, rather than series. Here is how you can convert series to dataframe easily.


2/ How to create dummy Dataframe for testing

We always need dataframes for testing and analysing normally, if we do not have data ready. Here is how you can use Pandas API to generate different types of data.
CODING RESOURCE , DATA SCIENCE
Pat Schloss
Pat Schloss
@PatSchloss
Wellll... A few weeks back I started working on a tutorial for our lab's Code Club on how to make shitty graphs. It was too dispiriting and I balked. A twitter workshop with figures and code:


Here's the code to generate the data frame. You can get the "raw" data from https://t.co/jcTE5t0uBT


Obligatory stacked bar chart that hides any sense of variation in the data


Obligatory stacked bar chart that shows all the things and yet shows absolutely nothing at the same time


STACKED Donut plot. Who doesn't want a donut? Who wouldn't want a stack of them!?! This took forever to render and looked worse than it should because coord_polar doesn't do scales="free_x".
DATA SCIENCE

You May Also Like

Reshu Nath
Reshu Nath
@ReshuNath
'Tera mera rishta kya, hindu, hindu,hindutva' is a clear take on 'tera mera rishta kya,la ilaha illallah'. As much as Hindutva hates the monotheistic religions especially Islam, there's a strange wistfulness and subliminal need at display now, to BE that they hate. 1


Charlie Hebdo doesn’t come up as a cautionary tale now but rather 'why cant we be like that'?
Growing up, Hanuman was an adorable deity for us, who swallowed the sun, or who couldn’t figure out what was Sanjeevani booti so he picked up a whole mountain and came back with it. 2

Or he was loyal, admirable and when he cleaved his chest, you saw both Ram and Sita. Now we have a lot of “Angry Hanuman” stickers. Why is a cute childlike God angry? We don’t know. It's a convenient blank space where to paint all the angst of Hindu Khatre mein hain narrative. 3


The other strange thing that has started happening is pictures of Ram appearing sans Sita. “Jai Siya Ram” has become “Jai Shri Ram”. Some people disingenuously explain it as being the same thing - because Shri is a name of Lakshmi. Not really. Siya is an apbhransh of Sita. 4


Shri is a prefix of respect in this case. Hinduism is being militarised. A woman/Sita has no place in it. Even the militaristic Goddess that is Durga has no place in this conception of neo-Hindu divinity, as evidenced by Dilip Mandal wondering where Durga came from. 5
RELIGION
\u0936\u094d\u0930\u0940\u092e\u093e\u0932\u0940 \u0939\u093f\u0924\u0947\u0936 \u0905\u0935\u0938\u094d\u0925\u0940
श्रीमाली हितेश अवस्थी...
@HiteshAwasthi89
Sri Devi Karumariamman mandir, Chennai, Tamilnadu

Sri Devi Karumariamman mandir is in Thiruverkadu, a suburb of Chennai. It is said that God Muruga came here and obtained the 'Vel' (spear) from His mother, Goddess Karumari (an avatar of Parvathi) 1/n


to fight the demon Suran and hence this place was called as 'Vel kadu' and became as Verkadu. As per legend Devi Karumariamman once disguised as a soothsayer visited Bhagwan Surya, the Sun God, to predict His future. 2/n


Not realising that the soothsayer is none other than Sakthi Matha, Surya ignored Her. Angered by his indifference, Devi Karumariamman immediately retreated. Bhagwan Surya lost His glory and the earth plunged into darkness. 3/n


Bhagwan Surya then realised His folly and pleaded forgiveness from Devi Karumariamman. In order to pacify mother Karumariamman, Suryadev said that His rays would fall on Her directly twice in a year and touch her feet. (in the month of Panguni (March- April) & during (Sep - Oct)


Mother Karumari Amman is worshipped here in two forms.A swayambu (self manifested) with only head visible above the ground and another one in a graceful sitting posture with all Parasakthi features,with four arms holding skull, sword, trishul, udukkai (damru). 5/n
ALL
Advertisement
Kanekoa
Kanekoa
@KanekoaTheGreat
*The Evidence Collection*

There is overwhelming evidence of election fraud. Therefore it is the duty of the Executive, Legislative, & Judicial branches to throw out the fraudulent votes & to honor their oath to protect & defend the Constitution of the USA.🇺🇸🦅

A list below.🔻


1/ Navarro Report: The Immaculate Deception - Six Key Dimensions of Election Irregularities -

2/ Analysis of 3,000 U.S. counties shows Joe Biden received 5.6 % more votes in counties using Dominion Voting Systems (Video) -

3/ Antrim Michigan Dominion Voting Systems Forensics Report -

4/ Evidence Grows: ’20 Election Was Rigged -
POLITICS , ALL
Patrick Chovanec
Patrick Chovanec
@prchovanec
I'm going to do two history threads on Ethiopia, one on its ancient history, one on its modern story (1800 to today). 🇪🇹

I'll begin with the ancient history ... and it goes way back. Because modern humans - and before that, the ancestors of humans - almost certainly originated in Ethiopia. 🇪🇹 (sub-thread):


The first likely historical reference to Ethiopia is ancient Egyptian records of trade expeditions to the "Land of Punt" in search of gold, ebony, ivory, incense, and wild animals, starting in c 2500 BC 🇪🇹


Ethiopians themselves believe that the Queen of Sheba, who visited Israel's King Solomon in the Bible (c 950 BC), came from Ethiopia (not Yemen, as others believe). Here she is meeting Solomon in a stain-glassed window in Addis Ababa's Holy Trinity Church. 🇪🇹


References to the Queen of Sheba are everywhere in Ethiopia. The national airline's frequent flier miles are even called "ShebaMiles". 🇪🇹
CULTURE
Rudy Havenstein, listening to Nas all day.
Rudy Havenstein, liste...
@RudyHavenstein
I hate when I learn something new (to me) & stunning about the Jeff Epstein network (h/t MoodyKnowsNada.)

Where to begin?

So our new Secretary of State Anthony Blinken's stepfather, Samuel Pisar, was "longtime lawyer and confidant of...Robert Maxwell," Ghislaine Maxwell's Dad.


"Pisar was one of the last people to speak to Maxwell, by phone, probably an hour before the chairman of Mirror Group Newspapers fell off his luxury yacht the Lady Ghislaine on 5 November, 1991." https://t.co/DAEgchNyTP


OK, so that's just a coincidence. Moving on, Anthony Blinken "attended the prestigious Dalton School in New York City"...wait, what? https://t.co/DnE6AvHmJg

Dalton School...Dalton School...rings a

Oh that's right.

The dad of the U.S. Attorney General under both George W. Bush & Donald Trump, William Barr, was headmaster of the Dalton School.

Donald Barr was also quite a


I'm not going to even mention that Blinken's stepdad Sam Pisar's name was in Epstein's "black book."

Lots of names in that book. I mean, for example, Cuomo, Trump, Clinton, Prince Andrew, Bill Cosby, Woody Allen - all in that book, and their reputations are spotless.
EPSTEIN NETWORK