What are the implications of using hacked data for research?

A short thread inspired by the fact that, before AWs took it down, #Parler was extensively hacked and user data was leaked.

If you're interested in DB internals, stop what you're doing and watch @CMUDB Quarantine Talk from Nico+Cesar about @SQLServer's Cascades query optimizer: https://t.co/FCdsbHHEaD

Many talks this semester were good. This one is the best. My thread provides key takeaways

[6:50] Microsoft hired Goetz Graffe in the 1990s to help them rewrite original Sybase optimizer into Cascades. This framework is now used across all MSFT DB products (@SQLServer, @cosmosdb, @Azure_Synapse).

[14:43] The optimizer checks whether it has stats it will need before cost-based search. If not, it blocks planning until the DBMS generates them. This is different than other approaches we saw this semester where DBMS says "we'll do it live!" with whatever stats are available.

[21:05] Their Cascades' search starts small/simple and then they make decision on the fly whether to expand search based on the expected query runtime and performance benefit from more search.

[26:33] They explicitly have a property for Halloween Problem. Operators specify whether they protect from it and then optimizer ensures property is satisfied. This is mindblowing. I have never thought about using the optimizer for this but it makes sense.

I have been a Substack and Patreon user for a few years now. And though more than one social media giant has, in recent months, expressed the desire to help writers and journalists monetise their audience, there are very good reasons to take these SOPs with a pinch of salt.

Because the reason Patreon and Substack came into existence was not the need for a new business model. It was as a cure for the old business model - an algorithm-driven ad revenue system that powered the attention economy. The attention economy turned audiences into scrollers...

...who were in it for the next viral hit. Quality of information suffered, the nature of discourse suffered, and as a result, democracy itself suffered. Much of this was enabled by the social media giants who are trying to copy the Substack and Patreon model right now in an...

...attempt to "put creators first". But what we must not lose sight of is that the Substack / Patreon model only emerged as a result of the bad practices the social media giants enabled. The algorithm made a toxic internet possible and they were what hit back. Today, multiple...

...podcasting tools video streaming services have donation buttons built in. But it was not always so. I applaud all attempts that anyone makes to help independent media not have to rely on ad money, but I am not going to ever be able to see Facebook's newsletter tool as a...

Here's an overview of key adoption metrics for deep learning frameworks over 2020: downloads, developer surveys, job posts, scientific publications, Colab usage, Kaggle notebooks usage, GitHub data.

TensorFlow/Keras = #1 deep learning solution.

Note that we benchmark adoption vs Facebook's PyTorch because it is the only TF alternative that registers on the scale. Another option would have been sklearn, which has massive adoption, but it isn't really a TF alternative. In the future, I hope we can add JAX.

TensorFlow has seen 115M downloads in 2020, which nearly doubles its lifetime downloads. Note that this does *not* include downloads for all TF-adjacent packages, like tf-nightly, the old tensorflow-gpu, etc.

Also note that most of these downloads aren't from humans, but are automated downloads from CI systems (but none are from Google's systems, as Google doesn't use PyPI).

In a way, this metric reflects usage in production.

There were two worldwide developer surveys in 2020 that measured adoption of various frameworks: the one from StackOverflow, targeting all developers, and the one from Kaggle.

I am seeing a ton of confusion about protecting yourself on the Internet. So as a retired bastard genius kid, I wanted to share my own experiences and knowledge in this.

DOXXING, PERSONAL CYBERSECURITY, ANONYMITY for the average revolutionary. A thread:

If you're a regular user of huge Internet services like Amazon, Google, Twitter, Facebook, etc., as well as over the age of 18, it's wise to assume that your personally identifiable information IS attainable and IS out there. These services all collect and save info --

to varying degrees, and to varying levels of security. And this information is VALUABLE. Data breaches occur frequently, retroactively, and unknowingly, and the data from said breaches are circulated around and cached in all kinds of easy-to-access places online.

Why does this matter? A (simplified) hypothetical: while someone probably can't find your street address from just your Twitter bio, if your handle is the same as your e-mail, and that e-mail is in a publicly available data breach with a street address attached, you're screwed.

Usually when doxxing happens, a much more elaborate version of said case happens. Doxxing really comes down to investigating all of an individual's available info and digital footprints, and narrowing it down to a single identity.

One of my purposes for 2021 is to simplify Docker for beginners.

Let's start with some definitions:

🏠Docker host
⚙️Docker engine
❓ Docker client
📝Dockerfile
📑Docker Image
🐳Docker Container
🐙Docker Compose
🪣 Docker Registry
🐳🐳🐳Docker Swarm

Definitions Below 👇

🏠Docker (Active) Host

A computer with Docker installed and the daemon running.

It can process Docker and non-Docker workloads.

⚙️Docker Engine

A Client-Server application. It has 3 components:

- "daemon process": A server that is a type of long-running program.

- "Rest API": to specify interfaces that programs can use to talk and interact with the daemon.

- "CLI": A command-line interface client

❓Docker Client

It sends commands to dockerd, using the Docker API.
It can communicate with multiple daemons.

📝Dockerfile

It's a text document that contains all the commands a user could call on the command line to assemble an image.