BC DATA SCIENCE

Saved by @Jacobtldr

Pat Schloss
@PatSchloss 5 years, 8 months ago 1038 views

Save to PDF Share See On Twitter

Wellll... A few weeks back I started working on a tutorial for our lab's Code Club on how to make shitty graphs. It was too dispiriting and I balked. A twitter workshop with figures and code:

When are you doing pie charts?
— #BlackLivesMatter (@surt_lab) October 13, 2020

Here's the code to generate the data frame. You can get the "raw" data from https://t.co/jcTE5t0uBT

Obligatory stacked bar chart that hides any sense of variation in the data

Obligatory stacked bar chart that shows all the things and yet shows absolutely nothing at the same time

STACKED Donut plot. Who doesn't want a donut? Who wouldn't want a stack of them!?! This took forever to render and looked worse than it should because coord_polar doesn't do scales="free_x".

More donuts. Let's get rid of all that messy variation in the data

One pie for @surt_lab, one for @watermicrobe, and one waiting to explode for @a2binny

Fine. Here's a pie for those of you that are still watching... This also took forever to render. The numbers are subject IDs

In all seriousness, here's the type of plot that I encourage for showing relative abundance by taxonomic data. Not fully polished, but you get the idea. Here each diagnosis has about 160 samples. With fewer samples, I'd use geom_jitter rather than geom_histogram

I prefer the boxplot/jitter plot because it allows the viewer to directly compare what I think is important. It also shows the variation in the data. Here's more polished version.

You can see how to do this for other taxonomic levels, incorporate statistical analysis to pick levels to show, and how to add a log scale on y-axis at https://t.co/U30ehefQPE. Thanks for attending my twitter workshop.

More from Data science

Ryan J. Gallagher
@ryanjgallag

Tired of word clouds? Want to do better sentiment analysis? Not sure how to look at the words underneath your measures?

Our long overdue paper on generalized word shift graphs is finally here!
https://t.co/lIBXvbMJWX
https://t.co/vSL1REYT8V

So what are they?

1/n

If we have two texts, there are many ways we can compare them. Weighted averages are a particularly useful measure because they're flexible and interpretable

Proportions, Shannon entropy, the KLD, the JSD, and dictionary methods can all be written as weighted averages

2/n

But weighted avgs are also slippery. When we try to compress complex phenomena like happiness, surprise, divergence, or diversity into a single number, it can be unclear what we're measuring

If the measure goes up, what does that mean? Why did it do that? Can we trust it?

3/n

Very often, that's the end of the line and we're left with an uneasy feeling in the pit of our stomach that our weighted avg is actually picking up a data artifact or some other unintended peculiarity

Word shift graphs help us address those concerns

4/n

First, word shifts look under the hood of weighted averages to see what's going on

All weighted averages are a sum of contributions from individual words. We can pull out those words, and rank which ones contribute the most to the difference between two texts

5/n

Jeremy Howard
@jeremyphoward

An amazing new project from @bearpelican was just released: https://t.co/DBov6sZTVS . A beautiful design; you can auto-generate a melody from chords, chords from a melody, and more.

It's technically brilliant, combining BERT, seq2seq, and Transformer XL
https://t.co/jF3mO5aXiu

It's also a wonderful example of leveraging and customizing the fastai framework in a deep & thoughtful way.

Here's the full set of blog posts diving in to this

Advertisement

Emil Wallner
@EmilWallner

Tips for AI writers:

1. Spend 30% of your effort on skimming all student ML papers (e.g. Stanford NLP CS224n) the past 3 years and prototype your favorites

The idea is everything. Pick an area you are interested in and ideally something that has a visual aspect to it

Most of my 'on the top of my mind' ideas were bad in retrospect. Skimming 100s of student papers will give you an overview of what's interesting.

Student papers are overlooked, easy to understand, and have good compute constraints.

2. Spend 30% on your effort on coding

Create an edge to the project. Apply it to something new and use FastAI or Keras to improve the accuracy with 5-30%.

3. Spend 30% writing an in-depth article

Have a north star article in terms of structure and quality. Find something that stretches you to your utmost capability. I used @copingbear’s Style transfer article:

4. Spend 10% marketing your project

Invest a week in studying the strategies to rank on sites like HN and Reddit, then use them. If you have an interesting result and a great article, you've done the hard work.

Research Engineering a...
@turinghut23

✨✨ BIG NEWS: We are hiring!! ✨✨
Amazing Research Software Engineer / Research Data Scientist positions within the @turinghut23 group at the @turinginst, at Standard (permanent) and Junior levels 🤩

👇 Here below a thread on who we are and what we

We are a highly diverse and interdisciplinary group of around 30 research software engineers and data scientists 😎💻 👉 https://t.co/KcSVMb89yx #RSEng

We value expertise across many domains - members of our group have backgrounds in psychology, mathematics, digital humanities, biology, astrophysics and many other areas 🧬📖🧪📈🗺️⚕️🪐
https://t.co/zjoQDGxKHq
/ @DavidBeavan @LivingwMachines

In our everyday job we turn cutting edge research into professionally usable software tools. Check out @evelgab's #LambdaDays 👩‍💻 presentation for some examples:

We create software packages to analyse data in a readable, reliable and reproducible fashion and contribute to the #opensource community, as @drsarahlgibson highlights in her contributions to @mybinderteam and @turingway: https://t.co/pRqXtFpYXq #ResearchSoftwareHour

Sophie Hill
@sophie_e_hill

This is a more wonky thread about how I made this visualization in #Rstats using the awesome visNetwork

First step is to create the underlying network data. We need one file of "nodes" - i.e. the people and organizations. And one file of "edges" - i.e. the connections between them.

I created these by hand, based on excellent investigate journalism:

Now we can pull these together to create a network visualization!

You'll notice that I included a column for "type" in the nodes file. This allows me to use different icons for people vs firms vs political organizations.

All the icons are taken from @fontawesome. I *think* the visNetwork 📦 currently only works with fontawesome version 4.7, which is a bit limited – e.g. I decided to use a book icon to represent the fringe Evangelical Christian sect "Exclusive Brethren"! 😂

I very much enjoyed getting to use the "incognito" icon to represent all the unknown donors that have funded Tory MP Owen Paterson's overseas jaunts!

You May Also Like

Naval
@naval

How to Get Rich (without getting lucky):

Seek wealth, not money or status. Wealth is having assets that earn while you sleep. Money is how we transfer time and wealth. Status is your place in the social hierarchy.

Understand that ethical wealth creation is possible. If you secretly despise wealth, it will elude you.

Ignore people playing status games. They gain status by attacking people playing wealth creation games.

You’re not going to get rich renting out your time. You must own equity - a piece of a business - to gain your financial freedom.

Suhail
@Suhail

1/ The first 18 months of starting a company is often life or death. I must've made 5 different companies that each failed within 9 mo. 😭 Each time the company failed I figured out what I could do better. Eventually startup #6 got to $40K/mo by month 18. Here’s what I learned...

2/ Stay focused! Ignore things that are a waste of time: meetups & conferences, meetings with no clear agenda, fundraising if you're not fundraising, reading lots of tech media articles, etc. Every week should feel like significant progress in the first year.

3/ Your first 5 hires will be the difference between life or death. Choose carefully. Be picky. Many of the things we do at the company still are a result of those early hires' legacy. Have fun as a tight knit team. It will change & evolve as you get bigger so enjoy this moment.

4/ Growth may be flat for the first 9 months. It's gonna be okay. Almost every company has experienced this: Airbnb had to sell cereal in-between, Slack failed as a gaming company first, Tesla sold only 147 cars after 6 years! You probably won't be an overnight success either.

5/ In the beginning, do customer support yourself. You will learn a lot about why your product sucks. I did 5,000+ support tickets when it was the two of us. Delight customers & fix things fast while you learn. It will help you build an amazing intuition about your customers.

Advertisement

STEALTH JEFF
@drawandstrike

Andy McCabe has some explaining to do.

Andrew McCabe was reported to the FBI’s Office of Public Affairs for making an unauthorized leak of classified info to the media about @GenFlynn in early February 2017.

If you’ve wondered exactly who it was who leaked the classified info from that intelligence report on @GenFlynn’s phone calls with then-Russian ambassador Kislyak to the media, you are right now seeing a huge honking clue who it was.

To those responding to this info by sneering "Nothing's gonna happen":

Up.

Your.

Game.

You just might be too stupid to be following me.

The then-Deputy Director of the FBI get's caught targeting @GenFlynn with an illegal leak so he can 'investigated' by Peter Strzok?

Now that we know it was McCabe who set up Flynn for a fake 'perjury' investigation by his pet attack dog, Peter Strzok, the next name that's going to surface is...Joseph Pientka.

Tweeting Historians
@Tweetistorian

@ChenHuailun here. I'm adding a supplementary thread before today's scheduled one to expand on the origins and early history of the Church of the East (aka the Nestorian Church) ~ahc #jingjiao 1/

The Church of the East traces its origins to the christological position of the School of Antioch, which held that the human and divine and human essences (ουσία) of Christ were united in a single prosopon (πρόσωπον). ~ahc #jingjiao 2/

This ran against the position of the School of Alexandria, which held that the two essences of Christ were united in a single hypostasis (ὑπόστασις). In general terms, hypostasis is more inherent than prosopon. ~ahc #jingjiao 3/

One implication of the Antiochene rejection of the hypostatic union is that Mary could not accurately be called the Theotokos, or God-bearer. Instead, she is the Christotokos, the Christ-bearer. ~ahc #jingjiao 4/

These Christological debates were further complicated by Syriac terms such as 'kyana' (ܟܝܢܐ), associated with 'ousia' but later translated to 'hypostasis' by Syrian Orthodox; and 'qnoma ' (ܩܢܘܡܐ), sometimes contentiously translated as 'hypostasis' or 'person.' ~ahc #jingjiao 5/

Anu Satheesh
@AnuSatheesh5

Sri Mridanga Saileswari Temple Muzhakkunnu, Kannur KERALA

The temple is established by Parasurama and it is one among 108 Durga temples.
Devi will answer all the prayers of her devotees who pray to her with utmost devotion lighting the ghee deepam.

It is the family diety of Kerala Varma Pazhassi Raja.
Devi is present in three forms Kali, Saraswati and Lakshmi.

Legend...👇
Once a Mridungam ( musical instrument) shaped rock fell from heaven here.
@SriramKannan77

Devi originated in the form of Mridangam and Parasurama captivated the essence of Devi into a Vigraham. Vigraham is made of Panchaloham.
Kathakali, an art form of Kerala originated here.

Temple Vigraham was stolen 3-4 times, but because of power of Devi all the attempts were failure. Police records says that Whenever the thieves try to steal, they lose their senses, and they will be in a confused state. So they will abandon the idol.
🕉