BC TECH
Saved by @Jacobtldr

We just launched a fun little tool called Phantom Analyzer. It’s a 100% serverless tool that scans websites for hidden tracking pixels.

I want to talk about how we built it 👇

 The idea came about early this year after @mijustin gave us an idea about badges/certification, which developed into “what if we could scan websites for Google Analytics?!”. But we left the idea in Basecamp for many months.
Fast forward to Halloween, we’re thinking about fun ideas we could do to entertain people. We discussed “Phantom Analytics” in response to people getting confused with our product name, and had various ideas. And then we landed on the URL analyzer idea.
Once we’d finalized the spec, Paul got to work on our Halloween themed product and then coded up the HTML/CSS for it all. I then took it, put it into a Laravel application. Nice and easy.
Off the bat, I already knew the base stack I was using.

> Laravel Vapor
> ChipperCI for deployment
> SQS for queues
> DynamoDB for the database

We went with DynamoDB as we don’t want to worry about our database scaling!
So with our infrastructure known, we had a few challenges left to solve:

> How will we scan websites for tracking pixels?
> How will we utilize the queue and check the job is done?
> How will we validate the URL?
For scanning websites, the first thing I did was write out a complex, multi-level regex matching, guzzle executing scanner. But the problem was that it didn’t automatically run the javascript, which often includes additional requests, so the results weren’t accurate.
After spending a long time on that, I was searching for website crawlers and came across Puppeteer, which is a headless Chrome Node.js API. I then searched for how to get it running on Laravel Vapor and saw that someone had already solved that challenge!
I then spent 8 hours trying to start from scratch with Puppeteer, copying from @spatie_be’s Browsershot code, but I just couldn’t get it working. I went to bed, woke up the next morning, and decided that I’d start with Browsershot and simply modify it to what I needed.
I woke up the next day, and within 15 minutes I get a screenshot generated on Laravel Vapor, hooray! I then start to modify Browsershot…

Wait a minute...

Yes, out of the box, Browsershot already had what I needed. Are you kidding me?
So I modified my job and had it all working within minutes. Browsershot returns a list of network requests when loading a web page and returns them, bloody perfect. I then simply compared them against a list of around 10,000 known third party trackers that we had.
The next step was working out how we would get permanent storage in DynamoDB without bringing in anything extra. I wanted to keep it simple. So with DynamoDB as the driver, and “resources” as the storage root, I wrote a command that cached the tracking pixels indefinitely.
One of the initial concerns I had was regarding security. We would be passing user input to the command line and that wouldn't be safe. I spoke with @marcelpociot and he gave me some great advice, and I added in some validation. The active_url rule is fantastic.
I also wanted to have a way so that if a user entered a full URL (e.g. https://t.co/66d4eLDaOu) and not just "https://t.co/GA31muKcta", it still redirected them to the correct results page. Especially since our "tidying up" was opinionated. So we ran this code.
We then had to think about how to configure our Vapor app, and I went with the following settings

> 1024MB of RAM
> 2048 of RAM for the queue (could likely reduce!)
> Warm of 500
> CLI Timeout of 180 seconds

Those settings all worked nicely.
For our Vapor layers, we ran it like this. Very cool. My first time using Layers. Incredible work by the Vapor team (@themsaid @taylorotwell @enunomaduro).
For the "is it ready?" check, I debated using a UUID but I decided that we might have multiple users trying a website at the same time, and they should benefit from the same cache entry (we cache results for 5 minutes).
So for the ping, we went super old school. Interval and redirect when done. Very effective. And when it reloaded the page, it would hit the cache, see the entry and display it.
All in all, this was a fun project to build. I love working with Paul. The only design addition I made was the bats & fade, Paul did everything else. Very grateful for that 😂
I am still in awe over how quickly we deployed this with Vapor. I'm not kidding, it was all coded up and we just created it in the UI, deployed it and we were done. Remarkable experience. Infinite scale without any server work 😎

Hope you all enjoy Phantom Analyzer!

More from Tech

Ferruccio Balestreri \U0001f308
Ferruccio Balestreri 🌈...
@0xferruccio
🕵️‍♂️ How Google's PageRank algorithm works

The PageRank algorithm gives each page a rating of its
importance, which is a recursively defined measure of importance, based on if important pages link to it.
It's recursive because the importance of a page refers back to the importance of other pages that link to it

Here's how it works in practice:
1⃣ We start with some pages and crawl them for links


2⃣ Each page has 1/N points (where N as the total number of pages)


3⃣ Add points to each page for the amount of links to it, divided by the number
of links emanating from the sources of these links
TECH
Jason Kint
Jason Kint
@jason_kint
I could create an entire twitter feed of things Facebook has tried to cover up since 2015. Where do you want to start, Mark and Sheryl? https://t.co/1trgupQEH9


Ok, here. Just one of the 236 mentions of Facebook in the under read but incredibly important interim report from Parliament. ht @CommonsCMS https://t.co/gfhHCrOLeU


Let’s do another, this one to Senate Intel. Question: “Were you or CEO Mark Zuckerberg aware of the hiring of Joseph Chancellor?"
Answer "Facebook has over 30,000 employees. Senior management does not participate in day-today hiring decisions."


Or to @CommonsCMS: Question: "When did Mark Zuckerberg know about Cambridge Analytica?"
Answer: "He did not become aware of allegations CA may not have deleted data about FB users obtained through Dr. Kogan's app until March of 2018, when
these issues were raised in the media."


If you prefer visuals, watch this short clip after @IanCLucas rightly expresses concern about a Facebook exec failing to disclose info.
TECH
Advertisement
Daniel Johnson
Daniel Johnson
@DJohnson_CPA
Here’s a SPAC thread covering some basics for new followers. I will pin this to my profile if you want to share to anyone interested in SPACs.

SPAC timelines consist of these stages:
IPO – SPAC raises cash and begins trading as a unit. The cash is placed in Trust.
Searching phase – After IPO, the SPAC management team looks for a company to merge with. This is typically 24 months but can be shorter depending on the SPAC

LOI/Rumor – A letter of intent is announced publicly or Bloomberg/Reuters etc. breaks news of a rumor that a SPAC is close to a deal with a target.

The LOI/rumor stage does not always happen, most go straight to a definitive agreement. Rumors should only be considered to have value if from a trusted news source, internet rumblings are pure speculation. During this stage, a deal can still fall apart and not be signed.

Definitive agreement (DA) – deal officially inked. The SPAC begins filing paperwork with the SEC to have the merger approved to take the target company public. This process typically takes 4+ months after the DA is announced.
TECH
Digital Strada - Partner for Digital Marketing
Digital Strada - Partn...
@DigitalStrada
🙂 Hey - have you heard of @RevolutApp Business before?

🌐 Great international transfer and 🏦 foreign #exchange rates, and various tools to manage your #business.

👉 https://t.co/dkuBrYrfMq

#banking #fintech #revolut #growth #startups
1/10


One place to manage all things business
Get more from your business account with powerful tools that give you total control over your finances.

👉 https://t.co/dkuBrYrfMq
2/10


Accept payments
online at great rates
Receive card payments from around the world with low fees and next-day settlement.

👉 https://t.co/dkuBrYrfMq
3/10


Send and receive international payments, with no hidden fees
Multi-currency accounts allow you to hold, exchange, send and receive funds in 28+ currencies - always at the real (interbank) exchange rate...

👉 https://t.co/dkuBrYrfMq
4/10


Optimise spend with smart company cards
Spend in over 150 currencies at the real (interbank) exchange rate
Stay in control – issue physical and virtual cards, track spending in real-time for your entire team...

👉 https://t.co/dkuBrYrfMq
5/10
TECH
Jen \u26a1
Jen ⚡
@leoniedelt
Been reading up a lot about these jabs, esp as I know they are going to be a condition of a return to normality

Was on a zoom call last evening with a testing company that had some fascinating evidence on protection etc.

I have more reading to do.

I do not want to be coerced into something I don't want, BUT i want normal back as soon as possible.

Basically i'm at the point in my isolation and thinking that I'll take whatever just so this bullshit can end.

That is a desperate mindset, but so is 12 months of being cut off from the world.

All i have is what goes round and round in my head.

There's no one else to help balance it.

So i'm in group 6, and my surgery are doing group 6 right now. And i am at the point i want to do it to get it DONE.

And also to study what happens. Because i have that kind of mind.

I remember Pandemrix.

But we live with risk every day.

I risk my life climbing walls with only a harness around my waist to catch me as i lean back and fall off

And rational risk assessment is at the heart of everything going wrong these last 12 months
TECH

You May Also Like

David Rothkopf
David Rothkopf
@djrothkopf
Pretending that voting for Trump or denying climate change or suggesting the solution to every problem is more guns is a legitimate point of view, worthy of respect is not "how America works." It is how it dies.

People in a free society are entitled to any point of view no matter how obviously wrong or outlandish or destructive provided they do not impose their views on others.

But suggesting that something ignorant or divorced from reality warrants the same kind of treatment in public debate as something based in fact or at least credible is absurd and when done on behalf of a society in public media or academic settings it is self-destructive.

We would not grant media coverage or much bandwidth at all to a group that argued that unicorns exist or that there was a pot of gold at the end of the rainbow. Yet views like denying climate change or basing policy decisions on sweeping, obvious lies, are just as unsound.

Somewhere, somehow, judgments must be made. Some behaviors are wrong and must be condemned. Someone must challenge lies and demand facts. Coddling people who embrace idiocies does not help them no matter how loudly they demand to be coddled.
POLITICS
Suhas S. Hardutt
Suhas S. Hardutt
@HarduttSuhas
THE BATTLE OF SARAIGHAT (1671)

Lachit Borphukan's humiliation of Aurangzeb.

The battle of Saraighat was a naval engagement fought on the Brahmaputra river near Saraighat between the M0ghul empire and Ahom kingdom in 1671.

Guts, resolve and glory!

(Elaborate thread)


Koch was a state situated between the dominions of M0ghuls and Ahoms. For Ahoms, it was a buffer state for keeping M0ghul territorial expansions at bay.(1)

Times changed when the ruler of the Koch kingdom, Nara Narayana passed away in 1587 resulting in Koch being divided into 2. Western Koch was ruled by his son Lakshmi Narayan and the eastern Koch by his nephew, Raghudev.(2)

Lakshmi Narayan formed an alliance with M0ghuls after which he attacked Parikshit, son of Raghudev, in 1602. Parikshit Narayan was defeated while his brother Bali Narayan took refuge in the Ahom Kingdom.(3)

The M0ghuls used it as a pretext to invade Ahom kingdom. The first battle between them took place in 1615 leading to a series of engagements in which both sides had their shares of ups and downs resulting in a stalemate.(4)
ALL
Advertisement
\u0932\u0916\u094b\u092c\u093e \u0932\u094b\u0916\u0902\u0921\u0947
लखोबा लोखंडे...
@LakhobaLokhande
Secrets of Ganapati (Author - Keshav Sitaram Thackeray)

This article is taken from Keshav Sitaram Thackeray's book 'Dharmachi Devale and Devalcha Dharma'. In it, Bal Thackeray's father is claiming how the Hindu deity of Ganesha is imaginary.
(1/20)

Buddha is Ashtavinayak and "Ganapati Bappa Morya" is "Chandragupta Morya".

In the era of democracy, the head of state is the President. But in ancient India, there was a Gana culture in the monarchy and the head of that Gana was called Ganapati
(2/20)

In ancient India, a prince named Siddharth Gautam was born in royalty. He later became the king of Shakya Gana in this Gana culture. In time, Siddhartha attained enlightenment.

Let us now understand the difference between a real Ganapati and an imaginary Ganapati.
(3/20)

Some clever Brahmin manuvadis made the real Ganapati the imaginary Ganapati.

Buddha was the king of the Shakya Gana, so people called him Ganaraja. He founded Buddhism and became the head of the Shakyas. As the head of Gana, people used to call Buddha as Ganapati.
(4/20)

But the Brahmins Brahmanized Ganapati and created a false Ganapati. When Buddha used to give the message of Dharma to the people, two words would come out of his message. Chitta and Malla.

Chitta is the body.
Malla means waste
(5/20)
INDIA
XVIII Airborne Corps\U0001f409
XVIII Airborne Corps🐉...
@18airbornecorps
1 of 16: WE ARE ALL JEWS HERE: THE STORY OF RODDIE EDMONDS

One of the most moving and relevant stories of the Battle of the Bulge, or any American Soldier in any war, is that of Master Sergeant Roddie Edmonds, a Knoxville, Tennessee native, who served with the 106th Infantry.


2 of 16:

Roddie was captured early on in the Battle of the Bulge, on December 19th, when Panzer forces plowed through his unit.

He, along with almost his entire regiment, was forced to surrender.


3 of 16:

The men were transported to the Stalag IX-A POW camp in Ziegenhain, Germany.

Roddie was the senior enlisted American Soldier at the site. As such, he was the conduit between all American Soldiers and their German captors.


4 of 16:

In late January, the camp’s commandant, Major Siegmann [pictured here], ordered Roddie, a Christian, to identify all Jewish Soldiers and order them to stand in formation by themselves the next day.

[Jewish Soldiers were a minority within American units]


5 of 16:

Throughout WWII, captured Jewish Soldiers were often tortured and then killed by their German captors.

In fact, Jewish Soldiers had been told to bury their dog tags before capture. The dog tag identified Jews with the letter “H” for Hebrew.
SOCIETY
Kab
Kab
@Kabamur_Taygeta
#NEIOH

1. Friends Of The Great Planet Earth!

Everything You Seek Is Already Within You!  Each Waking Dream Has False Teachings That  Say There Is A Special Place 'Out There' In The Skies Where People Go That Follow A Certain Doctrine.

2. And Those Same Teachers Explain There Is A Place Of Torment For Others That Arrived On A Different Part Of The Planet And Were Taught A Different Doctrine.

There Is ONE Creator That Has Been Named By Many. It Would Be Loving To Honor And Respect Another's Beliefs,

3. As In Truth, Many Have Lived In Multiple Cultures With Many Beliefs That Now Are Judged! Incarnating Over And Over Is The Joy Of The Soul Experiencing Life! The Soul Fully Understands This Is A Human Experience.

4. Wouldn't It Be Much More Relaxing And Fun To Allow Others Their Freedom, Just As You Want Yours?  Yes Indeed!  Relax And Flow!

Since The Very Beginning Of Your Self-Awareness As An Incarnate Being, You Have Probed Your World For Clues To The Meaning Of Your Existence.

5. You Have Looked In Every Conceivable Crevice And Pondered The Vastness Of The Infinite Cosmos. You Have Scrutinized The Spiritual Wisdom That Was Forthcoming From Those Who Believed Themselves To Be Knowledgeable About Such Matters.
LIFE