If you're interested in DB internals, stop what you're doing and watch @CMUDB Quarantine Talk from Nico+Cesar about @SQLServer's Cascades query optimizer: https://t.co/FCdsbHHEaD

Many talks this semester were good. This one is the best. My thread provides key takeaways

[6:50] Microsoft hired Goetz Graffe in the 1990s to help them rewrite original Sybase optimizer into Cascades. This framework is now used across all MSFT DB products (@SQLServer, @cosmosdb, @Azure_Synapse).
[14:43] The optimizer checks whether it has stats it will need before cost-based search. If not, it blocks planning until the DBMS generates them. This is different than other approaches we saw this semester where DBMS says "we'll do it live!" with whatever stats are available.
[21:05] Their Cascades' search starts small/simple and then they make decision on the fly whether to expand search based on the expected query runtime and performance benefit from more search.
[26:33] They explicitly have a property for Halloween Problem. Operators specify whether they protect from it and then optimizer ensures property is satisfied. This is mindblowing. I have never thought about using the optimizer for this but it makes sense. https://t.co/hjjoGCwyvl
[33:16] This is the menu of all the stats that they maintain for tables. Again, the latest research shows @SQLServer has the most accurate stats: https://t.co/d1btkxmsYf
[39:05] @SQLServer uses a general-purpose cardinality estimation framework. This allows them to programmatically select the best data structure to use per expression type. They rank choices by "quality of estimation". This needs further research.
[44:16] Question from @Lin_Ma_: Are you using ML for cardinality estimation?
Answer: @cosmosdb is using it. @SQLServer is more conservative and using a minor form of it.
[53:14] They use heuristics to pre-seed Cascades' memoization table with plans that they think will be good. This allows the search to start from a local optimum instead of a random location in search space.
[54:48] Optimizer uses logical timeouts (# of plans considered) instead of physical timeout (wallclock). This ensures that DBMS always produces plans with same quality under high load. Hand-tuned timeouts for different optimization stages.
[1:00:45] They also use pre-seeding to support DBA provided query plan hints! This is another genius idea that seems obvious once somebody shows it to you.
[1:03:32] This example shows the limitations of Cascades' tree-based plan search. For some optimizations, the DBMS must also consider hypergraphs. See Neumann SIGMOD'08: https://t.co/s825mXPMqK

More from Internet

Many conversations happening on #WhatsApp (WA) groups about new #WhatsAppPrivacyPolicy .
This thread has arguments to help ditch WA & move to @signalapp:
https://t.co/En4fe9VxUN
Share, use, copy-paste, modify with understanding as you deem fit on any platform in whole or part
1/n

Note: No affiliations, conflict of interest
Info presented with NO bias, prejudice, malice or indemnity.
Open to corrections: individual tweets may be deleted, tweets added to thread or corrected as replies.
Points that are unclear or uncertain are marked with "(?)".
2/n

CONTENT OF WA MESSAGES SHALL REMAIN ENCRYPTED END TO END.
BUT, there's data: contacts, group affiliations, co-affiliations, locations (live?), frequency of contacts, *tags* generated when we send or forward a message or file to contacts or groups, links, clicks on links, etc.
3/n

It is unclear whether this data is anonymized.
NOTHING in latest policy *prevents* the collection, retention, sharing or sale by FaceBook (FB: owner of WA) of this data in part or whole whether with identifying information or anonymized.
Meme source:
https://t.co/nMDTUlb0rl
4/n


Companies need to make money & generate profits:
To create software, install & maintain infrastructure.
Google, FB, Insta, Amazon etc sell data created from our content & data generated from our interactions (searches, clicks, purchases etc).
This makes many uncomfortable.
5/n

You May Also Like

1/ Some initial thoughts on personal moats:

Like company moats, your personal moat should be a competitive advantage that is not only durable—it should also compound over time.

Characteristics of a personal moat below:


2/ Like a company moat, you want to build career capital while you sleep.

As Andrew Chen noted:


3/ You don’t want to build a competitive advantage that is fleeting or that will get commoditized

Things that might get commoditized over time (some longer than


4/ Before the arrival of recorded music, what used to be scarce was the actual music itself — required an in-person artist.

After recorded music, the music itself became abundant and what became scarce was curation, distribution, and self space.

5/ Similarly, in careers, what used to be (more) scarce were things like ideas, money, and exclusive relationships.

In the internet economy, what has become scarce are things like specific knowledge, rare & valuable skills, and great reputations.