BC DATA SCIENCE
Saved by @zmbnski

Tired of word clouds? Want to do better sentiment analysis? Not sure how to look at the words underneath your measures?

Our long overdue paper on generalized word shift graphs is finally here!
https://t.co/lIBXvbMJWX
https://t.co/vSL1REYT8V

So what are they?

1/n

 If we have two texts, there are many ways we can compare them. Weighted averages are a particularly useful measure because they're flexible and interpretable

Proportions, Shannon entropy, the KLD, the JSD, and dictionary methods can all be written as weighted averages

2/n
But weighted avgs are also slippery. When we try to compress complex phenomena like happiness, surprise, divergence, or diversity into a single number, it can be unclear what we're measuring

If the measure goes up, what does that mean? Why did it do that? Can we trust it?

3/n
Very often, that's the end of the line and we're left with an uneasy feeling in the pit of our stomach that our weighted avg is actually picking up a data artifact or some other unintended peculiarity

Word shift graphs help us address those concerns

4/n
First, word shifts look under the hood of weighted averages to see what's going on

All weighted averages are a sum of contributions from individual words. We can pull out those words, and rank which ones contribute the most to the difference between two texts

5/n
But we can go further

Consider dictionary sentiment analysis. We don't just know the scores of words. We also have an understanding of which words are *more* or *less* positive

We know that there's a point on the sentiment scale that distinguishes positive from negative

6/n
The first thing word shifts do is make this *reference value* explicit

For sentiment analysis, this gives us 4 qualitatively different ways a word can contribute:
1. + word is used more
2. - word is used less
3. + word is used less
4. - word is used more

7/n
We can encode the 4 types of word contribution through 4 types of bars

This is how we construct basic word shift graphs. They give us details of both *what* words contribute and *how* they do so

And we've made it easy for you to make your own:
https://t.co/vSL1REYT8V

8/n
But we can go even further! We generalize the word shift framework so that you can use it for more than just single dictionary sentiment analysis

You can word shifts with multiple dictionaries, entropy-based measures, and any metric that can be written as a weighted avg

9/n
Generalized word shifts account for how words can change scores across texts, allowing us to use context-dependent sentiment dictionaries, or measures like entropy

This ends up giving us 8 qualitatively different ways a word can contribute to the diff between two texts

10/n
There's a lot to unpack here! So to give you practice reading word shift graphs and show you how to use them in practice, we present 5 case studies covering presidential speeches, Moby Dick, U.S. urban parks, 280 character tweets, and labor diversity in the Great Recession

11/n
Word shift graphs are an invaluable tool for unpacking weighted averages and looking at how words affect our measures

We hope this paper can be the ultimate field guide for those are interested in using word shift graphs to help validate their own text-as-data analyses

12/n
All of the word shifts that we mention in the paper are implemented in the Shifterator package
https://t.co/vSL1REYT8V

We have new documentation that includes a comprehensive cookbook for using word shift graphs in Python
https://t.co/p9FdEJYlKN

Please reach out w/ Qs

13/n
I originally presented a draft of this paper at the Text as Data conference in Seattle almost 2 years ago

The Shifterator code has come a long way since then, and I've put a lot of time into it. I hope that people find it useful for their own work
https://t.co/vSL1REYT8V

14/n
Finally, this paper wouldn't have been possible without the hard work of many @compstorylab members past and present

I had a great time collaborating with @mrfrank5790 @lewis_math @andyreagan @ChrisDanforth @peterdodds and Aaron Schwartz on this Story Lab piece!

15/n
In summary: always look at the words!

And please reach out if you have any questions or comments about using word shift graphs!

16/16

More from Data science

Research Engineering at the Turing
Research Engineering a...
@turinghut23
✨✨ BIG NEWS: We are hiring!! ✨✨
Amazing Research Software Engineer / Research Data Scientist positions within the @turinghut23 group at the @turinginst, at Standard (permanent) and Junior levels 🤩

👇 Here below a thread on who we are and what we

We are a highly diverse and interdisciplinary group of around 30 research software engineers and data scientists 😎💻 👉 https://t.co/KcSVMb89yx #RSEng

We value expertise across many domains - members of our group have backgrounds in psychology, mathematics, digital humanities, biology, astrophysics and many other areas 🧬📖🧪📈🗺️⚕️🪐
https://t.co/zjoQDGxKHq
/ @DavidBeavan @LivingwMachines

In our everyday job we turn cutting edge research into professionally usable software tools. Check out @evelgab's #LambdaDays 👩‍💻 presentation for some examples:

We create software packages to analyse data in a readable, reliable and reproducible fashion and contribute to the #opensource community, as @drsarahlgibson highlights in her contributions to @mybinderteam and @turingway: https://t.co/pRqXtFpYXq #ResearchSoftwareHour
DATA SCIENCE
Sophie Hill
Sophie Hill
@sophie_e_hill
This is a more wonky thread about how I made this visualization in #Rstats using the awesome visNetwork

First step is to create the underlying network data. We need one file of "nodes" - i.e. the people and organizations. And one file of "edges" - i.e. the connections between them.

I created these by hand, based on excellent investigate journalism:


Now we can pull these together to create a network visualization!

You'll notice that I included a column for "type" in the nodes file. This allows me to use different icons for people vs firms vs political organizations.


All the icons are taken from @fontawesome. I *think* the visNetwork 📦 currently only works with fontawesome version 4.7, which is a bit limited – e.g. I decided to use a book icon to represent the fringe Evangelical Christian sect "Exclusive Brethren"! 😂


I very much enjoyed getting to use the "incognito" icon to represent all the unknown donors that have funded Tory MP Owen Paterson's overseas jaunts!
DATA SCIENCE
Advertisement
Maria Khalusova
Maria Khalusova
@mariaKhalusova
To my JVM friends looking to explore Machine Learning techniques - you don’t necessarily have to learn Python to do that. There are libraries you can use from the comfort of your JVM environment. 🧵👇

https://t.co/EwwOzgfDca : Deep Learning framework in Java that supports the whole cycle: from data loading and preprocessing to building and tuning a variety deep learning networks.

https://t.co/J4qMzPAZ6u Framework for defining machine learning models, including feature generation and transformations, as directed acyclic graphs (DAGs).

https://t.co/9IgKkSxPCq a machine learning library in Java that provides multi-class classification, regression, clustering, anomaly detection and multi-label classification.

https://t.co/EAqn2YngIE : TensorFlow Java API (experimental)
DATA SCIENCE
Sunrit Jana \U0001f680
Sunrit Jana 🚀
@JanaSunrise
Pandas is an amazing data analysis and manipulation library for python, Really popular when working with ML, Data science or more.

It has a robust data structure, Dataframe for manipulation and analyzing data.

Here's some tips to help you work better with pandas. Let's go! ↓

If you're not aware about what a Dataframe is, It's an optimized data structure for loading data, analysing it, manipulating data in it, and Mostly gathering insights.

It uses Cython backend which transpiles into C for optimized code.

Here's how a dataframe looks like,


Before we start, You need to ensure, you have pandas installed. If you don't, Do that before moving ahead!

Here are the tips, Let's go!


1/ Convert PD series to Dataframe

We all have struggled, when we deal with pandas series. It's always easier to work with Dataframes, rather than series. Here is how you can convert series to dataframe easily.


2/ How to create dummy Dataframe for testing

We always need dataframes for testing and analysing normally, if we do not have data ready. Here is how you can use Pandas API to generate different types of data.
CODING RESOURCE , DATA SCIENCE
Goku Mohandas
Goku Mohandas
@GokuMohandas
🔥 Putting ML in Production! We're going to publicly develop @madewithml's first ML service. Here is the broad curriculum:

- 📦 Product
- 🔢 Data
- 🤖 Modeling
- 📝 Scripting
- 🛠 API
- 🚀 Production

More details (lessons, task, etc.) here: https://t.co/xmMm9XGK9j

Thread 👇


Questions that this thread will answer:

- What is it?
- Who is this course for?
- What is the format?
- What makes this course unique?
- Why constrain to open source tools?
- What are my qualifications?
- Why is this free?
- What are the

What is it?

Putting ML in Production: a guide and code-driven case study on MLOps. We will be developing and deploying Made With ML's first ML service, from Product → ML → Production, with open source tools.

This ML service will act as a foundation for all future ML features and subsequent iterations. The first feature is tagifai - multilabel classification of tags for a project. We'll discuss the need and utility of this feature in the first lesson.

Who is this course for?

- ML developers looking to become end-to-end ML developers.
- Software engineers looking to learn how to responsibly deploy and monitor ML systems.
- Product managers who want to have a comprehensive understanding of the different stages of ML dev.
DATA SCIENCE

You May Also Like

\U0001f1ee\U0001f1f3Sonal \U0001f1ee\U0001f1f3
🇮🇳Sonal 🇮🇳...
@Dharma_Vaarta
Bharat: Advancement of earth’s earliest civilization

Glorious Vedic Architecture and Temple

Architecture in India was an extremely rich form of expression and highly developed science, with a deep heritage of spirituality in it. #FreeHinduTemple


The centre of all architectural development was the temple and temple or Mandir was the earthly home of God, and like the launching pad for the devotee’s consciousness to soar toward the Divine and the spiritual world.


The murti or deity of the Divine would be placed in the temple sanctum, around which all temple activity would revolve.

Such buildings were often elaborately decorated with stone, wood, plaster, etc. and carved or cut to depict the stories from the Puranas or other sacred texts


Temples were always the centre of religious, social and educational activities, practically more so in
ancient times than today.

During a span of more than two thousand years, various dynasties were known for constructing glorious temples,


some of which include the Mauryan, Ashoka, Gupta, Satavahana, Chedi, Chandella, Chalukya, Solanki, Pallava, Shola, Hoysala, Vijayanagar, and others.

Many of these temples remain beautiful architectural wonders today.
ALL , HISTORY
Vibhu Vashisth
Vibhu Vashisth
@VIBHU_Tweet
THE MEANING, SIGNIFICANCE AND HISTORY OF SWASTIK

The Swastik is a geometrical figure and an ancient religious icon. Swastik has been Sanatan Dharma’s symbol of auspiciousness – mangalya since time immemorial.


The name swastika comes from Sanskrit (Devanagari: स्वस्तिक, pronounced: swastik) &denotes “conducive to wellbeing or auspicious”.
The word Swastik has a definite etymological origin in Sanskrit. It is derived from the roots su – meaning “well or auspicious” & as meaning “being”.


"सु अस्ति येन तत स्वस्तिकं"
Swastik is de symbol through which everything auspicios occurs

Scholars believe word’s origin in Vedas,known as Swasti mantra;

"🕉स्वस्ति ना इन्द्रो वृधश्रवाहा
स्वस्ति ना पूषा विश्ववेदाहा
स्वस्तिनास्तरक्ष्यो अरिश्तनेमिही
स्वस्तिनो बृहस्पतिर्दधातु"


It translates to," O famed Indra, redeem us. O Pusha, the beholder of all knowledge, redeem us. Redeem us O Garudji, of limitless speed and O Bruhaspati, redeem us".

SWASTIK’s COSMIC ORIGIN

The Swastika represents the living creation in the whole Cosmos.


Hindu astronomers divide the ecliptic circle of cosmos in 27 divisions called https://t.co/sLeuV1R2eQ this manner a cross forms in 4 directions in the celestial sky. At centre of this cross is Dhruva(Polestar). In a line from Dhruva, the stars known as Saptarishi can be observed.
ALL
Advertisement
Anshul Pandey
Anshul Pandey
@Anshulspiritual
BRAHMA VAIVARTA PURAN: IMPORTANCE AND CHARACTERISTICS.

This Puran introduces us to way of Vedas. On the request of Narad Muni Bhagwan Saavarni had preached this Puran. It is a summary of Dharma, Arth, Moksha etc.


Listening to this Puran one gets automatically attached to Vishnu and Shiva and realize that there is no difference between them. Vyasji divided this book into four parts. It has eighteen thousand shlokas. This whole book is in the form of dialogue between Suutji and Maharishi's.

The first part Brahmkhand, consists of Shrishti description. Here Shri is known as Para Brahma and description of Gokul is also mentioned, It also narrates the argument between Brahma and Narad in which both were defeated, and then Narad went to Shiva to attain more gyan.

Shivji advised him to go to Bhagwan Saavarni for this. Narad goes to Saavarni ashram to seek Knowledge and is totally amazed by the sorroundings.

Next is Prakriti khand which consists of dialogue between Narad and Saavarni where they discuss the

Importance of Shri Krishna through various stories. Then there is the discussion of Prakriti and its various hues. Listening to this gets you wealth.
Next is Ganesh Khand which describes the Punyak vrat done by Parvati ji. After this it narrates the birth of Kartikeya and Ganesh.
RELIGION
Emperor\U0001f451
Emperor👑
@EmperorBTC
Technical Analysis Masterclass- Part 2

'Entering the trade'

Entering a trade without ascertaining a certain things is gambling.

In this masterclass we will learn the pre-requisites to enter a trade.

DON'T ENTER A TRADE WITHOUT DETERMINNG THE FOLLOWING.

Please share.

We understand what reward to risk (popularly called risk to reward) is.
It will be denoted by R:R.

We will also try to bust a few myths about R:R and how to avoid losing trades.

Before entering a trade, you need to determine 3 things.

1. Entry trigger
2. Stop loss
3. Target

1. Entry trigger = Reasons for entering a trade. There could be multiple reasons or a single reason for entry.

Generally a set of reasons AKA confluence is a higher probability trade and a generally a safer entry.

Example of an entry trigger.


2. Stop Loss.

The price in the opposite direction of the trade where the trade is exited, at a loss.

At this level, the reason for the entry becomes invalidated according to TA and the price can then move in the opposite direction, probabilistically.


3. Target is the possible price level that the asset might touch based on previous trends or confluence AND where a possible reversal could occur.

Target is the next path of least resistance from where the price might reverse.

We will always ONLY use TA to determine all 3.
TRADING
Trader knight
Trader knight
@Traderknight007
Pyramiding 2.0

A thread-


What is pyramiding?

It is simply adding to your winners.

Why use pyramiding?

1. It makes your winners bigger than your losers.

• Because you are adding positions when your trades are going your way, you are increasing your gains but your losers will be small comparatively.

2. You lose less during Draw-down time ( it saves you from losing big during whipsaws )

• Many times when the market is in sideways mode, Entering a trade with full risk might not be the best idea.

• So, you test the waters with a small position first, it could be ⅓ or ¼ of your total risk.

• So if the Breakout fails or whipsaws come, then you will only lose a small part of your overall risk.

• Then when position starts to go in your favor you can add to it.

3. It helps you to concentrate on a few trades,

They say that don’t put all your eggs in one basket, but don’t put them in too many either.

• See, Everyone can see charts and find breakouts, pullbacks etc.
ALL , SWING TRADING , PYRAMIDING