BC TECH
Saved by @Jacobtldr

We just launched a fun little tool called Phantom Analyzer. It’s a 100% serverless tool that scans websites for hidden tracking pixels.

I want to talk about how we built it 👇

 The idea came about early this year after @mijustin gave us an idea about badges/certification, which developed into “what if we could scan websites for Google Analytics?!”. But we left the idea in Basecamp for many months.
Fast forward to Halloween, we’re thinking about fun ideas we could do to entertain people. We discussed “Phantom Analytics” in response to people getting confused with our product name, and had various ideas. And then we landed on the URL analyzer idea.
Once we’d finalized the spec, Paul got to work on our Halloween themed product and then coded up the HTML/CSS for it all. I then took it, put it into a Laravel application. Nice and easy.
Off the bat, I already knew the base stack I was using.

> Laravel Vapor
> ChipperCI for deployment
> SQS for queues
> DynamoDB for the database

We went with DynamoDB as we don’t want to worry about our database scaling!
So with our infrastructure known, we had a few challenges left to solve:

> How will we scan websites for tracking pixels?
> How will we utilize the queue and check the job is done?
> How will we validate the URL?
For scanning websites, the first thing I did was write out a complex, multi-level regex matching, guzzle executing scanner. But the problem was that it didn’t automatically run the javascript, which often includes additional requests, so the results weren’t accurate.
After spending a long time on that, I was searching for website crawlers and came across Puppeteer, which is a headless Chrome Node.js API. I then searched for how to get it running on Laravel Vapor and saw that someone had already solved that challenge!
I then spent 8 hours trying to start from scratch with Puppeteer, copying from @spatie_be’s Browsershot code, but I just couldn’t get it working. I went to bed, woke up the next morning, and decided that I’d start with Browsershot and simply modify it to what I needed.
I woke up the next day, and within 15 minutes I get a screenshot generated on Laravel Vapor, hooray! I then start to modify Browsershot…

Wait a minute...

Yes, out of the box, Browsershot already had what I needed. Are you kidding me?
So I modified my job and had it all working within minutes. Browsershot returns a list of network requests when loading a web page and returns them, bloody perfect. I then simply compared them against a list of around 10,000 known third party trackers that we had.
The next step was working out how we would get permanent storage in DynamoDB without bringing in anything extra. I wanted to keep it simple. So with DynamoDB as the driver, and “resources” as the storage root, I wrote a command that cached the tracking pixels indefinitely.
One of the initial concerns I had was regarding security. We would be passing user input to the command line and that wouldn't be safe. I spoke with @marcelpociot and he gave me some great advice, and I added in some validation. The active_url rule is fantastic.
I also wanted to have a way so that if a user entered a full URL (e.g. https://t.co/66d4eLDaOu) and not just "https://t.co/GA31muKcta", it still redirected them to the correct results page. Especially since our "tidying up" was opinionated. So we ran this code.
We then had to think about how to configure our Vapor app, and I went with the following settings

> 1024MB of RAM
> 2048 of RAM for the queue (could likely reduce!)
> Warm of 500
> CLI Timeout of 180 seconds

Those settings all worked nicely.
For our Vapor layers, we ran it like this. Very cool. My first time using Layers. Incredible work by the Vapor team (@themsaid @taylorotwell @enunomaduro).
For the "is it ready?" check, I debated using a UUID but I decided that we might have multiple users trying a website at the same time, and they should benefit from the same cache entry (we cache results for 5 minutes).
So for the ping, we went super old school. Interval and redirect when done. Very effective. And when it reloaded the page, it would hit the cache, see the entry and display it.
All in all, this was a fun project to build. I love working with Paul. The only design addition I made was the bats & fade, Paul did everything else. Very grateful for that 😂
I am still in awe over how quickly we deployed this with Vapor. I'm not kidding, it was all coded up and we just created it in the UI, deployed it and we were done. Remarkable experience. Infinite scale without any server work 😎

Hope you all enjoy Phantom Analyzer!

More from Tech

Novid Houellebecq (That's Hollaback)
Novid Houellebecq (Tha...
@GarouGothic
I want to explain what happened to tumblr overnight. And why Twitter as reached it's dream (or have come ever so close to it)

Yahoo, who bought Tumblr years ago, used to have a huge adult presence on the early net. They allowed adult groups and what not.

However, people and bots (just like now) misused the service, and Yahoo were forced to make a choice. They made private the groups (and later closed them down and sold some of it to other companies) and then ended their chatrooms on yahoo messenger...

after a incident with one of the chatrooms vid cams. The damage was done, Yahoo Messenger lost a lot of people - and with the closing of the groups - backpage and Craigslist came more important.

Now backpage is no more and Craigslist is slowly passing away. Tumblr had a semi strong community, but once 2014 came around and both porn, and political bots exploded the quality started to go down,
TECH
Ferruccio Balestreri \U0001f308
Ferruccio Balestreri 🌈...
@0xferruccio
🕵️‍♂️ How Google's PageRank algorithm works

The PageRank algorithm gives each page a rating of its
importance, which is a recursively defined measure of importance, based on if important pages link to it.
It's recursive because the importance of a page refers back to the importance of other pages that link to it

Here's how it works in practice:
1⃣ We start with some pages and crawl them for links


2⃣ Each page has 1/N points (where N as the total number of pages)


3⃣ Add points to each page for the amount of links to it, divided by the number
of links emanating from the sources of these links
TECH
Advertisement
Nicolas
Nicolas
@necolas
A brief analysis and comparison of the CSS for Twitter's PWA vs Twitter's legacy desktop website. The difference is dramatic and I'll touch on some reasons why.

Legacy site *downloads* ~630 KB CSS per theme and writing direction.

6,769 rules
9,252 selectors
16.7k declarations
3,370 unique declarations
44 media queries
36 unique colors
50 unique background colors
46 unique font sizes
39 unique z-indices

https://t.co/qyl4Bt1i5x


PWA *incrementally generates* ~30 KB CSS that handles all themes and writing directions.

735 rules
740 selectors
757 declarations
730 unique declarations
0 media queries
11 unique colors
32 unique background colors
15 unique font sizes
7 unique z-indices

https://t.co/w7oNG5KUkJ


The legacy site's CSS is what happens when hundreds of people directly write CSS over many years. Specificity wars, redundancy, a house of cards that can't be fixed. The result is extremely inefficient and error-prone styling that punishes users and developers.

The PWA's CSS is generated on-demand by a JS framework that manages styles and outputs "atomic CSS". The framework can enforce strict constraints and perform optimisations, which is why the CSS is so much smaller and safer. Style conflicts and unbounded CSS growth are avoided.
TECH
Got Got Need
Got Got Need
@Hudlen_
Thought I'd put a thread together of some resources & people I consider really valuable & insightful for anyone considering or just starting out on their @SorareHQ journey. It's by no means comprehensive, this community is super helpful so no offence to anyone I've missed off...

1) Get yourself on the official Sorare Discord group https://t.co/1CWeyglJhu, the forum is always full of interesting debate. Got a question? Put it on the relevant thread & it's usually answered in minutes. This is also a great place to engage directly with the @SorareHQ team.

2) Bury your head in @HGLeitch's @SorareData & get to grips with all the collated information you have to hand FOR FREE! IMO it's vital for price-checking, scouting & S05 team building plus they are hosts to the forward thinking SO11 and SorareData Cups 🏆

3) Get on YouTube 📺, subscribe to @Qu_Tang_Clan's channel https://t.co/1ZxMsQR1kq & engross yourself in hours of Sorare tutorials & videos. There's a good crowd that log in to the live Gameweek shows where you get to see Quinny scratching his head/ beard over team selection.

4) Make sure to follow & give a listen to the @Sorare_Podcast on the streaming service of your choice 🔊, weekly shows are always insightful with great guests. Worth listening to the old episodes too as there's loads of information you'll take from them.
TECH
Brad Kaellner - Stock Compounder
Brad Kaellner - Stock ...
@bkaellner
Incumbents almost always prevail with sustaining innovation

In disruptive circumstances, the entrants are likely to beat the incumbents

H/t Clayton Christensen, The Innovator’s Solution
TECH

You May Also Like

Jaya_Upadhyaya
Jaya_Upadhyaya
@Jayalko1
SRI MOOKAMBIKA DEVI, KOLLUR (KAR)

There was an Asura, Kaumasura, who was doing penance to please Shiva and get a boon. To prevent him from asking for a boon, Devi Saraswati made him dumb (hence called Mookasur)
Later, Adi Parashakti slayed him and came to be called Mookambika.


It is said that Adi Shankaracharya had a vision in which Devi agreed to follow him provided he did not look back. He kept walking and was assured of her presence due to the sound of anklets. When he reached here in Kollur, he turned back as the sound of anklets had stopped.


So Devi stayed here and merged with the lingam. The place came to be called Mookambika Kshetram.
The linga has integrated on it’s left side MahaKali, Maha Lakshmi and Maha Saraswathi and on the right side- Brahma, Vishnu and Shiva.


A swarna rekha (gold line) divides this linga into left and right portion. Adi Shakthi in this Udhbhavlinga form appears only here.
Devi sits in padmasana and has 4 hands.
Adi Shankaracharya installed the Sri Chakra in front of Devi and also composed the Soundarya Lahiri here.
ALL
Murray \U0001f1fa\U0001f1f8
Murray 🇺🇸...
@Rothbard1776
NEW: @SidneyPowell1 radio interview w/ @toddeherman "The new report has a truckload of evidence of foreign interference in the election in it. That's what people seem to want to deny, even though the FBI & CISA ... issued an advisory and alert back on Oct. 30th & updated on

2/ Nov. 3rd, so for them to deny that now is absolutely absurd. Plus, we have an affidavit from an expert cyber-warfare [individual] who even shows the diagram of all the attacks coming in. We know that the VPN ... the Dominion people left those open and unencrypted on the night

3/ of the election, so that anybody could get in. We know that the packets of information went to Serbia [note: over 100 Dominion employees scrubbed there LinkedIn profiles of any affiliation with the company after the election, many of whom were in Serbia], Liechtenstein,

4/ Spain, Venezuela, Hunan, China & Hong Kong. There was traffic all over into our election system that is supposed to be self-contained, not accessible by the internet at all. We have witnesses who document that it was connected to the internet. We have a study out of

5/ [the Dominion voting system in Antrim] MI, they know that it was connected to the internet. They erased their audit files & their adjudication files, where they dumped over 68% of their ballots into a bin that they call "adjudication" [that would've been thousands + thousands
POLITICS , ELECTIONFRAUD
Advertisement
Marc K\xf6hlbrugge
Marc Köhlbrugge
@marckohlbrugge
💬 I often get Q’s like these:

How did you get your first customers for WIP? How did you grow BetaList’s traffic? Etc.

Makers are looking to reverse-engineer success. I see it everywhere.

I don’t think it works that way and the answers to those questions are mostly useless. 💥

I have built dozens of different products over the last couple of years. The vast majority failed. 😭

Surely if I know the answers to these questions, but still fail over and over again, these answers aren’t that useful. 🤷‍♀️

So what’s a better question to ask? 🤔

99.9% of the questions I receive are about the products that did well. In a way that makes sense, because we quickly forget about those that didn’t succeed.

🧠 This is known as survivorship bias.

Focusing on what survived, while ignoring what made it survive in the first place.

The real question, what you really want to know, is this:

What makes @WIP, @BetaList, and to some extent @AllStartupJobs succeed where my countless other attempts failed?

What separates a failed product 👎 from a successful product 👍?

Honestly, I don’t know. I wish I did.

It’s like Steve Jobs said “I’ll know it when I see it.” 👀

Same is true when we make products. We don’t know upfront what will work. But once we see an inkling of a product that does have potential, it’s not that hard to spot.
STARTUPS
Henry George \u0646
Henry George ن
@intothefuture45
Thread: *Post-Liberal reading list*
1/ Religion and the Rise of Capitalism - R.H. Tawney
Unto This Last - John Ruskin
Liberty, Equality, Fraternity - James Fitzjames Stephen
The Great Transformation - Karl Polanyi
The Quest for Community - Robert Nisbet

2/ Family and Civilisation - Carle C. Zimmerman
Science, Politics and Gnosticism - Eric Voegelin
Liberty Before Liberalism - Quentin Skinner
The Great Debate - Yuval Levin
Lincoln and the Politics of Christian Love - Grant Havers
Where We Are - Roger Scruton

3/ The Humane Vision of Wendell Berry - Various
The Unsettling of America - Wendell Berry
Moral Matters - Mark Dooley
Red Tory - Phillip Blond
Blue Labour - Eds. Ian Geary and Adrian Pabst
Together for the Common Good - Eds. Nicholas Sagovsky & Peter McGrail

4/ The Politics of Virtue - John Milbank & Adrian Pabst
Localism in a Mass Age - Eds. Mark T. Mitchell & Jason Peters
The Antimodern Condition - Peter King
The Principle of Duty - David Selbourne
Nostalgia - Anthony Esolen
A Common Human Ground - Claes G. Ryn

5/ The Politics of Gratitude - Mark T. Mitchell
A Time to Build - Yuval Levin
Remaking One Nation - Nick Timothy
Despised - Paul Embery
Defending Identity - Natan Sharansky
Between Kin and Cosmopolis - Nigel Biggar
The Virtue of Nationalism - Yoram Hazony
CULTURE
Vibhu Vashisth
Vibhu Vashisth
@VIBHU_Tweet
RANI GAIDINLIU:THE LESSER KNOWN FREEDOM FIGHTER

Rani Gaidinliu joined the freedom struggle at 13 &was sentenced to life imprisonment at 16."We r free people—the White should not rule over us.”Thus calling out to her Zeliangrong tribesmen was the voice of 13yr-old Rani Gaidinliu.


She joined the freedom struggle in the present-day Northeast India. Today, her struggle is lost in history as most of her countrymen wouldn’t even have heard her name. B'coz our history textbooks tell us that only Mahatma Gandhi &Pandit Nehru got India its Independence.


At 16 yrs of age, this valiant girl was sentenced to life imprisonment for leading a movement to drive out the colonial rulers from Manipur and Nagaland. Nevertheless, she was released in 1947 after India’s independence and was given the title ,"Rani of Nagas".


EARLY LIFE AND HERAKA MOVEMENT

Gaidinliu belonged to the Rongmei clan of the Zeliangrong tribe in the Tamenglong district of western Manipur. Born on January 26, 1915, she was fifth among eight children.


Her native village was Nungkao, which is present day Tamenglong distt. Brought up in a specially backward region,she couldn't receive any formal education. However,this didn't stop her from joining the freedom struggle. At the age of 10 she came under the influence of her cousin.
ALL