BC TECH
Saved by @Jacobtldr

We just launched a fun little tool called Phantom Analyzer. It’s a 100% serverless tool that scans websites for hidden tracking pixels.

I want to talk about how we built it 👇

 The idea came about early this year after @mijustin gave us an idea about badges/certification, which developed into “what if we could scan websites for Google Analytics?!”. But we left the idea in Basecamp for many months.
Fast forward to Halloween, we’re thinking about fun ideas we could do to entertain people. We discussed “Phantom Analytics” in response to people getting confused with our product name, and had various ideas. And then we landed on the URL analyzer idea.
Once we’d finalized the spec, Paul got to work on our Halloween themed product and then coded up the HTML/CSS for it all. I then took it, put it into a Laravel application. Nice and easy.
Off the bat, I already knew the base stack I was using.

> Laravel Vapor
> ChipperCI for deployment
> SQS for queues
> DynamoDB for the database

We went with DynamoDB as we don’t want to worry about our database scaling!
So with our infrastructure known, we had a few challenges left to solve:

> How will we scan websites for tracking pixels?
> How will we utilize the queue and check the job is done?
> How will we validate the URL?
For scanning websites, the first thing I did was write out a complex, multi-level regex matching, guzzle executing scanner. But the problem was that it didn’t automatically run the javascript, which often includes additional requests, so the results weren’t accurate.
After spending a long time on that, I was searching for website crawlers and came across Puppeteer, which is a headless Chrome Node.js API. I then searched for how to get it running on Laravel Vapor and saw that someone had already solved that challenge!
I then spent 8 hours trying to start from scratch with Puppeteer, copying from @spatie_be’s Browsershot code, but I just couldn’t get it working. I went to bed, woke up the next morning, and decided that I’d start with Browsershot and simply modify it to what I needed.
I woke up the next day, and within 15 minutes I get a screenshot generated on Laravel Vapor, hooray! I then start to modify Browsershot…

Wait a minute...

Yes, out of the box, Browsershot already had what I needed. Are you kidding me?
So I modified my job and had it all working within minutes. Browsershot returns a list of network requests when loading a web page and returns them, bloody perfect. I then simply compared them against a list of around 10,000 known third party trackers that we had.
The next step was working out how we would get permanent storage in DynamoDB without bringing in anything extra. I wanted to keep it simple. So with DynamoDB as the driver, and “resources” as the storage root, I wrote a command that cached the tracking pixels indefinitely.
One of the initial concerns I had was regarding security. We would be passing user input to the command line and that wouldn't be safe. I spoke with @marcelpociot and he gave me some great advice, and I added in some validation. The active_url rule is fantastic.
I also wanted to have a way so that if a user entered a full URL (e.g. https://t.co/66d4eLDaOu) and not just "https://t.co/GA31muKcta", it still redirected them to the correct results page. Especially since our "tidying up" was opinionated. So we ran this code.
We then had to think about how to configure our Vapor app, and I went with the following settings

> 1024MB of RAM
> 2048 of RAM for the queue (could likely reduce!)
> Warm of 500
> CLI Timeout of 180 seconds

Those settings all worked nicely.
For our Vapor layers, we ran it like this. Very cool. My first time using Layers. Incredible work by the Vapor team (@themsaid @taylorotwell @enunomaduro).
For the "is it ready?" check, I debated using a UUID but I decided that we might have multiple users trying a website at the same time, and they should benefit from the same cache entry (we cache results for 5 minutes).
So for the ping, we went super old school. Interval and redirect when done. Very effective. And when it reloaded the page, it would hit the cache, see the entry and display it.
All in all, this was a fun project to build. I love working with Paul. The only design addition I made was the bats & fade, Paul did everything else. Very grateful for that 😂
I am still in awe over how quickly we deployed this with Vapor. I'm not kidding, it was all coded up and we just created it in the UI, deployed it and we were done. Remarkable experience. Infinite scale without any server work 😎

Hope you all enjoy Phantom Analyzer!

More from Tech

foone
foone
@Foone
you know how Atari called the Jaguar as a "64bit system", because it had two 32bit CPUs? (or because it had a 64bit bus, but let's not worry about that)
we should apply that logic to the current generation of consoles.
So the Switch is a 256bit console. It's the Nintendo 256.

The Xbox Series X? terrible name.
Better call it the "Xbox 512 Blu-ray".
like the Jaguar CD, the Sega CD... the good old days when consoles had their media type in the name.

Boringly the PS5 is also a Playstation 512 Blu-ray, because both of them are using the same AMD Zen 2 CPU.

The Switch one is weeeird. It's built on an Nvidia Tegra X1, which is an 8 core 64-bit CPU, so that should be 512, right?

Well... no. Four of them can't be used. And later documentation doesn't even mention them.

it's some weird system where the system-on-a-chip has four Cortex-A57 cores and four Cortex-A53 cores but only the A57s are usable?
TECH
Jijo Sunny
Jijo Sunny
@JijoSunny
1/ 👋 Excited to share what we’ve been building at https://t.co/GOQJ7LjQ2t + we are going to tweetstorm our progress every week!

Week 1 highlights: getting shortlisted for YC W2019🤞, acquiring a premium domain💰, meeting Substack's @hamishmckenzie and Stripe CEO @patrickc 🤩

2/ So what is Brew?

brew / bru : / to make (beer, coffee etc.) / verb: begin to develop 🌱

A place for you to enjoy premium content while supporting your favorite creators. Sort of like a ‘Consumer-facing Patreon’ cc @jackconte

(we’re still working on the pitch)

3/ So, why be so transparent? Two words: launch strategy.

jk 😅 a) I loooove doing something consistently for a long period of time b) limited downside and infinite upside (feedback, accountability, reach).
cc @altimor, @pmarca


4/ https://t.co/GOQJ7LjQ2t domain 🍻

It started with a cold email. Guess what? He was using BuyMeACoffee on his blog, and was excited to hear about what we're building next. Within 2w, we signed the deal at @Escrowcom's SF office. You’re a pleasure to work with @MichaelCyger!

5/ @ycombinator's invite for the in-person interview arrived that evening. Quite a day!

Thanks @patio11 for the thoughtful feedback on our YC application, and @gabhubert for your directions on positioning the product — set the tone for our pitch!
TECH
Advertisement
Kevin Roose
Kevin Roose
@kevinroose
On press call, Zuckerberg says FB users "naturally engage more with sensational content" that comes close to violating its rules. Compares it to cable TV and tabloids, and says, "This seems to be true regardless of where we set our policy lines."

Zuckerberg says FB is in the process of setting up a "new independent body" that users will be able to appeal content takedowns to. Sort of like the "Facebook Supreme Court" idea he previewed earlier this year.

Zuckerberg: "One of my biggest lessons from this year is that when you connect more than 2 billion people, you’re going to see the good and bad of humanity."

This is how Facebook says it's trying to change the engagement pattern on its services. https://t.co/3p0PGc912o


.@RebeccaJarvis asks Zuckerberg if anyone is going to lose their job over the revelations in the NYT story. He dodges, says that personnel issues aren't a public matter, and that employee performance is evaluated all the time.
TECH
Devdutta
Devdutta
@Devdutta96
Please please please take your cyber security seriously even if you don’t consider yourself a high risk person. Here are a few easy things to start with. 1/n

Switch from WhatsApp to Signal and enable disappearing messages. Most of us cannot stop using WhatsApp entirely but at least turn off back ups to Google Drive or iCloud. 2/n

Do not have private conversations using the messaging service on any social media platform like Facebook, Instagram or Twitter. 3/n

Switch from Gmail to ProtonMail or any other secure email service provider. 4/n


Switch from Google Docs to CryptPad or RiseUp Pad.
CYBER SECURITY , IT PRIVACY , CYBER SECURITY STEPS
Mateusz Czeladka | Plan \u20b3
Mateusz Czeladka | Pla...
@matiwinnetou
(1) Since some people laugh at Cardano project and it's peer review aspect, let me share a story from work for all of you
(2) There is company called Databricks and they open sourced recently a technology called: https://t.co/gpZyKKiMVD . This technology is super important

(3) In BigData delta allows one to perform DELETE and MERGE operations contrary to Hadoop + Hive where this is not possible, seeking through many partitions finding customer data to remove and then rewriting it is not only expensive as an operation but also very dangerous

(4) This promising technology is something that many companies, which have big data turn into as they need to be GDPR compliant.
(5) I was always reading and accepting all details from their website as given and the "truth"

(6) There is however an amazing Principal Developer at our company that found some of the claims from their website as dubious, e.g. ACID claims on such a distributed file system
(7) It turns out this is not the first time this company advertises something that is simply not true

(8) on another occasion this company made untrue claims about Cassandra database and this then became source of disputes e.g by dr. @martinkl - my fave expert in distributed systems
(9) All professional developers also know of course famous Jepsen tests ->
TECH

You May Also Like

Anu Satheesh \U0001f1ee\U0001f1f3
Anu Satheesh 🇮🇳...
@AnuSatheesh5
Thuravoor Narasimha Swamy Temple
Alappuzha, Kerala
#KeralaTemples 🚩

Thuravoor Narasimha Swamy or Thuravoor Mahakshetram is situated in Alappuzha with Narasimha Swamy and Sudharshana Moorthy as main Prathishta.
Temple has separate Sannidhi for Vadakkanappan ( Narasimha) and 👇


Tekkanappan ( Sudarshana).

Sthala puranam says that Vigraham of Narasimha was brought from Varanasi. Swami Padmapadar, shishya of Adi Sankaracharya, had worshipped the same Vigraham at Kashi.
A Namboodiri, on his way to Kashi saw a powerful light which disappeared here 👇


Namboodiri dug the place and he found a beautiful beautiful Vigraham made of Anjanasila. It is said that the vigraham occupied a site, which was the place of Durga Devi
Sudarshana Moorthy is about 6 Ft, Chaturbahu. Deepavali is famous festival here.
Om Narasimha Swamine Namah
🕉
ALL
Melissa (she/her)
Melissa (she/her)
@LiteraryMouse
In regards to the #CapitolCoup, I’m in a unique position. I monitored forums ahead of time and then was present for the coup itself.

These are my observations based on what I saw and the evidence that’s been collected so far. I reserve the right to adjust my analysis as needed.

Let’s talk about the organization of the coup first. Seeing some people insist that the coup must have been highly planned and coordinated.

Yes and no. This was a #StochasticCoup. We’ll see little evidence of direct communication between Trump and the insurrectionists.

Within the insurrectionists, there was not one group that executed the storming of the Capitol.

Rather, there were a number of smaller groups working independently towards the same goal. I expect you’ll see little evidence that these various groups communicated with each other.

What you will see is detailed planning and coordination within these small groups.

Then there were the joyriders, those Trump supporters who weren’t directly involved in any plotting but who were more than happy to go along for the ride once the coup was in motion.

After four years, the insurrectionists were carefully attuned to Trump’s messaging.

Every tweet and statement of his was treated as marching orders and carefully dissected for meaning.

It’s why shutting down Trump’s Twitter account was so important.
POLITICS , INSURRECTIONISTS
Advertisement
Gareth Harney
Gareth Harney
@OptimoPrincipi
1) The Temple of the Divine Claudius was a vast temple complex on Rome's Caelian Hill, overlooking the valley that would later be occupied by the Colosseum. Though almost nothing now remains of the structure, its scale and grandeur once rivalled anything in the city.. #LostRome


2) The 'Claudium' was intended as a sacred complex to revere Claudius and the imperial cult. Construction began soon after the death and deification of the emperor in 54 AD, initiated by his widow Agrippina who was likely responsible for his supposed death by poisoning. #LostRome


3) The sprawling temple complex sat on a gigantic podium that spread across the Caelian Hill, measuring 180 x 200 metres; with towering retaining walls of travertine that raised the structure 20 metres above the valley below. #LostRome


4) The elevated sacred area was likely entered by steps facing the temple on the west side. The temple itself had a projecting hexastyle porch and was surrounded by gardens and a grand portico, probably the 'porticus Claudia' mentioned in Martial's 'On the Spectacles'. #LostRome


5) Statues of both Agrippina and Claudius’ ill-fated young son Britannicus, sculpted in sumptuous dark Egyptian greywacke, were discovered among the ruins on the Caelian Hill in 1651 – and had almost certainly been displayed in the temple precinct. #LostRome
HISTORY
Mo Rajabifard
Mo Rajabifard
@morajabi
👨‍💻 Last resume I sent to a startup one year ago, sharing with you to get ideas:

- Forget what you don't have, make your strength bold
- Pick one work experience and explain what you did in detail w/ bullet points
- Write it towards the role you apply
- Give social proof

/thread


"But I got no work experience..."

Make a open source lib, make a small side project for yourself, do freelance work, ask friends to work with them, no friends? Find friends on Github, and Twitter.

Bonus points:

- Show you care about the company: I used the company's brand font and gradient for in the resume for my name and "Thank You" note.
- Don't list 15 things and libraries you worked with, pick the most related ones to the role you're applying.
-🙅‍♂️"copy cover letter"

"I got no firends, no work"

One practical way is to reach out to conferences and offer to make their website for free. But make sure to do it good. You'll get:

- a project for portfolio
- new friends
- work experience
- learnt new stuff
- new thing for Twitter bio

If you don't even have the skills yet, why not try your chance for @LambdaSchool? No? @freeCodeCamp. Still not? Pick something from here and learn https://t.co/7NPS1zbLTi
You'll feel very overwhelmed, no escape, just acknowledge it and keep pushing.
LIFE
Enigma
Enigma
@ainittomai
The Eye of Horus. 1/*


I believe that @ripple_crippler and @looP_rM311_7211 are the same person. I know, nobody believes that. 2/*

Today I want to prove that Mr Pool smile faces mean XRP and price increase. In Ripple_Crippler, previous to Mr Pool existence, smile faces were frequent. They were very similar to the ones Mr Pool posts. The eyes also were usually a couple of "x", in fact, XRP logo. 3/*


The smile XRP-eyed face also appears related to the Moon. XRP going to the Moon. 4/*


And smile XRP-eyed faces also appear related to Egypt. In particular, to the Eye of Horus. https://t.co/i4rRzuQ0gZ 5/*
CULTURE