BC DATA SCIENCE

Saved by @Jacobtldr

Pat Schloss
@PatSchloss 4 years, 8 months ago 723 views

Save to PDF Share See On Twitter

Wellll... A few weeks back I started working on a tutorial for our lab's Code Club on how to make shitty graphs. It was too dispiriting and I balked. A twitter workshop with figures and code:

When are you doing pie charts?
— #BlackLivesMatter (@surt_lab) October 13, 2020

Here's the code to generate the data frame. You can get the "raw" data from https://t.co/jcTE5t0uBT

Obligatory stacked bar chart that hides any sense of variation in the data

Obligatory stacked bar chart that shows all the things and yet shows absolutely nothing at the same time

STACKED Donut plot. Who doesn't want a donut? Who wouldn't want a stack of them!?! This took forever to render and looked worse than it should because coord_polar doesn't do scales="free_x".

More donuts. Let's get rid of all that messy variation in the data

One pie for @surt_lab, one for @watermicrobe, and one waiting to explode for @a2binny

Fine. Here's a pie for those of you that are still watching... This also took forever to render. The numbers are subject IDs

In all seriousness, here's the type of plot that I encourage for showing relative abundance by taxonomic data. Not fully polished, but you get the idea. Here each diagnosis has about 160 samples. With fewer samples, I'd use geom_jitter rather than geom_histogram

I prefer the boxplot/jitter plot because it allows the viewer to directly compare what I think is important. It also shows the variation in the data. Here's more polished version.

You can see how to do this for other taxonomic levels, incorporate statistical analysis to pick levels to show, and how to add a log scale on y-axis at https://t.co/U30ehefQPE. Thanks for attending my twitter workshop.

More from Data science

Maria Khalusova
@mariaKhalusova

To my JVM friends looking to explore Machine Learning techniques - you don’t necessarily have to learn Python to do that. There are libraries you can use from the comfort of your JVM environment. 🧵👇

https://t.co/EwwOzgfDca : Deep Learning framework in Java that supports the whole cycle: from data loading and preprocessing to building and tuning a variety deep learning networks.

https://t.co/J4qMzPAZ6u Framework for defining machine learning models, including feature generation and transformations, as directed acyclic graphs (DAGs).

https://t.co/9IgKkSxPCq a machine learning library in Java that provides multi-class classification, regression, clustering, anomaly detection and multi-label classification.

https://t.co/EAqn2YngIE : TensorFlow Java API (experimental)

Simon DeDeo
@SimonDeDeo

On Bayesianism, the Many Worlds Interpretation, and personal identity.

Some thoughts worked out in a letter to a friend, which is the kind of thing you do when off Twitter for a glorious week. (🧵)

“Chance is ignorance”—the Bayesian story; all probabilities represent states of mind, not states of the world. One *could* put (some) chances “in the world”, but let’s take Occam’s Razor seriously...

That the probability of a fair coin coming up heads is 50% simply means that marginalizing (tracing, as the physicists say) over the hidden facts leaves you, nearly, maximally ignorant of the outcome.

Quantum uncertainty (access below!) poses an apparent challenge to this story. There seems to be nothing to be ignorant about when it comes to (say) electron spin—there is nothing “inside” the

The electron is a simple object, in other words. So where does the uncertainty come from? One could follow David Wallace’s wonderful interpretation in terms of chaotic dynamics and decoherence, but let’s see if we can take another route...

Advertisement

Emil Wallner
@EmilWallner

Tips for AI writers:

1. Spend 30% of your effort on skimming all student ML papers (e.g. Stanford NLP CS224n) the past 3 years and prototype your favorites

The idea is everything. Pick an area you are interested in and ideally something that has a visual aspect to it

Most of my 'on the top of my mind' ideas were bad in retrospect. Skimming 100s of student papers will give you an overview of what's interesting.

Student papers are overlooked, easy to understand, and have good compute constraints.

2. Spend 30% on your effort on coding

Create an edge to the project. Apply it to something new and use FastAI or Keras to improve the accuracy with 5-30%.

3. Spend 30% writing an in-depth article

Have a north star article in terms of structure and quality. Find something that stretches you to your utmost capability. I used @copingbear’s Style transfer article:

4. Spend 10% marketing your project

Invest a week in studying the strategies to rank on sites like HN and Reddit, then use them. If you have an interesting result and a great article, you've done the hard work.

elvis
@omarsar0

I have always emphasized on the importance of mathematics in machine learning.

Here is a compilation of resources (books, videos & papers) to get you going.

(Note: It's not an exhaustive list but I have carefully curated it based on my experience and observations)

📘 Mathematics for Machine Learning

by Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong

https://t.co/zSpp67kJSg

Note: this is probably the place you want to start. Start slowly and work on some examples. Pay close attention to the notation and get comfortable with it.

📘 Pattern Recognition and Machine Learning

by Christopher Bishop

Note: Prior to the book above, this is the book that I used to recommend to get familiar with math-related concepts used in machine learning. A very solid book in my view and it's heavily referenced in academia.

📘 The Elements of Statistical Learning

by Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie

Mote: machine learning deals with data and in turn uncertainty which is what statistics teach. Get comfortable with topics like estimators, statistical significance,...

📘 Probability Theory: The Logic of Science

by E. T. Jaynes

Note: In machine learning, we are interested in building probabilistic models and thus you will come across concepts from probability theory like conditional probability and different probability distributions.

John Myles White
@johnmyleswhite

This post is pretty bizarre, but it manages to hit on so many false beliefs that I've seen hurt junior data scientists that it deserves some explicit

(1) The notion that R is well-suited to "building web applications" seems totally out of left field. I don't feel like most R loyalists think this is a good idea, but it's worth calling out that no normal company will be glad you wrote your entire web app in R.

(2) It is true that Python had some issues historically with the 2-to-3 transition, but it's not such a big deal these days. On the flip side, I have found interesting R code that doesn't run in modern R interpreters because of changes in core operations (e.g. assignment syntax).

(3) "Most of the time we only need a latest, working interpreter with the latest packages to run the code" -- this is where things get real and reveal some things that hurt data scientists. If this sentence is true, it's likely because you don't share code with coworkers.

(3) Really is a broader issue in data science: people only think of what they need to do their work if no one else existed and code was never maintained. Junior data scientists almost always operate on projects they start from scratch and don't have to maintain for long.

You May Also Like

Chuck Wendig
@ChuckWendig

TRUMP: "I've decided to eat babies."

PEOPLE: "He can't eat babies, that's super illegal."

TRUMP, on TV, eating babies, not even cooking them first: "People are saying that I really am the best baby-eater, folks."

NYT: "Trump Vs. Babies: The Rhetoric On Both Sides Must Stop"

Jeff Flake tweets: "It is a sad day in America when this callous man can eat babies live on TV"

Jeff Flake, 10 seconds later, votes to help him install a series of 4,000 new judges that are very, very pro-babyphagia.

Judge Kavanaugh, found drunk in a Capitol Hill closet: "I LIKE BEER. OKAY? I LIKE BEER AND I LIKE BABIES AND THAT'S NOT SO WRONG. THAT'S RIGHT. I LIKE BABIES DIPPED IN BEER. I JUST DIP EM. IN THE BEER. THE BABIES. THE BABIES I EAT. OKAY? THAT'S AMERICA."

Mike Pence, wringing his hands about the baby-eating: "We must take care of our children. Our youngest must be protected on this troubling day."

Mike Pence then invites a Baby Chef to say a prayer.

Twitter Users: "It's a distraction technique! Trump eating babies is trying to DISTRACT YOU from ALL THE OTHER HORRORS, which are themselves distractions from the BABY-EATING. It's DISTRACTIONS ALL THE WAY DOWN."

Trump: *eats tons of babies at a rally*

*literal tons of babies*

Mitesh Patel
@Mitesh_Engr

Again sharing bank nifty intraday weekly straddle. How to play by selling straddle.
Mostly will work on Monday to Wednesday. Sometimes on Friday.
Why to pay your money to learn this if u can get it here.
U need only practice.
Let’s go.

Say bank nifty future is at 32000.
Sell both 32000 call and put.
Considering current volatile market lest say premium on Monday morning is 350 on both sides.
Sold with 700 points combine premium.
If bank nifty will trade in narrow range say +\- 100 hold it.
If moving 200 points

then exit previous sold straddle. Sell new one. Let’s say BN moved +200
Now sell 32200. If bank nifty will move here within range then u will get profit through theta decay. Follow same procedure for next 150-200 pts move.

This is only for single straddle sell position.

Now in second case sell three straddle. 31900/32000/32100
In case of 100 points move no need to change.
If 200 + then future is 32200.
Exit 31900 and sell 31200 straddle.
If moving another 100 then exit 32000 and sell 32300. Continue same with 100 points move.
If no move then

it will consolidate between 32000-32200 or 32100 to 32300 and u will get theta decay.

This is for normal move. Sudden big move may come anytime. So keep SL in system.
If u r selling at 350-350 then keep SL in system 475 for both legs.
Practice it. No need to attend webinar.

Advertisement

$Ethan \U0001f468\u200d\U0001f3a4$

Ethan 👨‍🎤...
@Booligoosh

First update to https://t.co/lDdqjtKTZL since the challenge ended – Medium links!! Go add your Medium profile now 👀📝 (thanks @diannamallen for the suggestion 😁)

Just added Telegram links to https://t.co/lDdqjtKTZL too! Now you can provide a nice easy way for people to message you :)

Less than 1 hour since I started adding stuff to https://t.co/lDdqjtKTZL again, and profile pages are now responsive!!! 🥳 Check it out -> https://t.co/fVkEL4fu0L

Accounts page is now also responsive!! 📱✨

💪 I managed to make the whole site responsive in about an hour. On my roadmap I had it down as 4-5 hours!!! 🤘🤠🤘

Jonathan Ware
@ReassessHistory

So when it first appeared in Normandy, the Sherman Firefly and it's meaty 17 pounder represented an ideal combo of reliable chassis, great gun and devastating firepower right?

Ummm...

Truth is a bit more complex. /1
#WW2 #SWW #History

Deliveries of Firefly to units only commenced in April 1944,
just a few weeks before Overlord & left minimal time for familiarisation.

Armoured Regiments mostly utilised Fireflies on a basis of 1 per troop of 4 tanks. /2

Finally each armoured regiment troop had access to integral, meaty, anti-tank firepower capable of dealing serious damage to any German AFV likely to be encountered. /3

The dual purpose 75mm gun was quite capable of laying down a reasonable HE shell or AP round for dealing with Panzer III or IV.

Firefly's 17 Pounder had.... problems with HE, but excelled at tank killing. So Tigers and Panthers would need to be wary. /4

Firefly's design history dated back to March 1943 when the General Staff published "Policy on Tanks" which effectively stated the majority of tanks should have a good HE weapon and...

[See image below] /5

Homestays & Villas
@thehomestays

Best resort on the Ganga at Rishikesh