BC TECH
Saved by @Jacobtldr

We just launched a fun little tool called Phantom Analyzer. It’s a 100% serverless tool that scans websites for hidden tracking pixels.

I want to talk about how we built it 👇

 The idea came about early this year after @mijustin gave us an idea about badges/certification, which developed into “what if we could scan websites for Google Analytics?!”. But we left the idea in Basecamp for many months.
Fast forward to Halloween, we’re thinking about fun ideas we could do to entertain people. We discussed “Phantom Analytics” in response to people getting confused with our product name, and had various ideas. And then we landed on the URL analyzer idea.
Once we’d finalized the spec, Paul got to work on our Halloween themed product and then coded up the HTML/CSS for it all. I then took it, put it into a Laravel application. Nice and easy.
Off the bat, I already knew the base stack I was using.

> Laravel Vapor
> ChipperCI for deployment
> SQS for queues
> DynamoDB for the database

We went with DynamoDB as we don’t want to worry about our database scaling!
So with our infrastructure known, we had a few challenges left to solve:

> How will we scan websites for tracking pixels?
> How will we utilize the queue and check the job is done?
> How will we validate the URL?
For scanning websites, the first thing I did was write out a complex, multi-level regex matching, guzzle executing scanner. But the problem was that it didn’t automatically run the javascript, which often includes additional requests, so the results weren’t accurate.
After spending a long time on that, I was searching for website crawlers and came across Puppeteer, which is a headless Chrome Node.js API. I then searched for how to get it running on Laravel Vapor and saw that someone had already solved that challenge!
I then spent 8 hours trying to start from scratch with Puppeteer, copying from @spatie_be’s Browsershot code, but I just couldn’t get it working. I went to bed, woke up the next morning, and decided that I’d start with Browsershot and simply modify it to what I needed.
I woke up the next day, and within 15 minutes I get a screenshot generated on Laravel Vapor, hooray! I then start to modify Browsershot…

Wait a minute...

Yes, out of the box, Browsershot already had what I needed. Are you kidding me?
So I modified my job and had it all working within minutes. Browsershot returns a list of network requests when loading a web page and returns them, bloody perfect. I then simply compared them against a list of around 10,000 known third party trackers that we had.
The next step was working out how we would get permanent storage in DynamoDB without bringing in anything extra. I wanted to keep it simple. So with DynamoDB as the driver, and “resources” as the storage root, I wrote a command that cached the tracking pixels indefinitely.
One of the initial concerns I had was regarding security. We would be passing user input to the command line and that wouldn't be safe. I spoke with @marcelpociot and he gave me some great advice, and I added in some validation. The active_url rule is fantastic.
I also wanted to have a way so that if a user entered a full URL (e.g. https://t.co/66d4eLDaOu) and not just "https://t.co/GA31muKcta", it still redirected them to the correct results page. Especially since our "tidying up" was opinionated. So we ran this code.
We then had to think about how to configure our Vapor app, and I went with the following settings

> 1024MB of RAM
> 2048 of RAM for the queue (could likely reduce!)
> Warm of 500
> CLI Timeout of 180 seconds

Those settings all worked nicely.
For our Vapor layers, we ran it like this. Very cool. My first time using Layers. Incredible work by the Vapor team (@themsaid @taylorotwell @enunomaduro).
For the "is it ready?" check, I debated using a UUID but I decided that we might have multiple users trying a website at the same time, and they should benefit from the same cache entry (we cache results for 5 minutes).
So for the ping, we went super old school. Interval and redirect when done. Very effective. And when it reloaded the page, it would hit the cache, see the entry and display it.
All in all, this was a fun project to build. I love working with Paul. The only design addition I made was the bats & fade, Paul did everything else. Very grateful for that 😂
I am still in awe over how quickly we deployed this with Vapor. I'm not kidding, it was all coded up and we just created it in the UI, deployed it and we were done. Remarkable experience. Infinite scale without any server work 😎

Hope you all enjoy Phantom Analyzer!

More from Tech

Shaun Russell
Shaun Russell
@_ShaunRussell
"I really want to break into Product Management"

make products.

"If only someone would tell me how I can get a startup to notice me."

Make Products.

"I guess it's impossible and I'll never break into the industry."

MAKE PRODUCTS.

Courtesy of @edbrisson's wonderful thread on breaking into comics – https://t.co/TgNblNSCBj – here is why the same applies to Product Management, too.


There is no better way of learning the craft of product, or proving your potential to employers, than just doing it.

You do not need anybody's permission. We don't have diplomas, nor doctorates. We can barely agree on a single standard of what a Product Manager is supposed to do.

But – there is at least one blindingly obvious industry consensus – a Product Manager makes Products.

And they don't need to be kept at the exact right temperature, given endless resource, or carefully protected in order to do this.

They find their own way.
TECH
DHH
DHH
@dhh
Facebook may well be exceptionally nasty and it’s leadership particularly deceitful, but the whole enterprise is the natural conclusion to an ad-powered social network. Ever greater surveillance, hyper targeting, and engagement baiting is what the basic incentives demand.

Facebook cannot change course. The best they can do is keep dishing out empty apologies, commit to inconsequential adjustments to its algorithms, and spend more money on positive PR and negative projection. The rot is at the core of its very being.

This is why we, consumers and citizens, must make the change for Facebook. Antitrust forcing a breakup of the business, and consumer revolt driving #DeleteFacebook. Zuckerberg will never conclude that what he built has become such a net negative for the world on his own.

That’s why its so tragic that WhatsApp sold out to Facebook in particular. Finally a challenger that had a different model and different ethics. But few would say no to $19B, which is why we need antitrust enforcers to do it for them.
TECH
Advertisement
foone
foone
@Foone
I got a Toshiba T1200, which has a bad screen, and a dead CMOS battery, and I thought the hard drive was broken too...


but I rebooted it and heard a KTHUNK and a really loud spin-up noise and holy shit, the hard drive works!


and it still has files on it. the newest date is January 1994.


It's a 21mb hard drive, with ~37k of bad sectors.

This fucker is never going to spin up again, so I better go find some serial cables and copy shit off now.


The backlight is fucked so I'm shining a flashlight on it.
without it, it looks like this.
TECH
Lea Kissner
Lea Kissner
@LeaKissner
We're kicking off the Privacy Tech session at #enigma2021 with Mitch Negus speaking about "NO DATA, NO PROBLEM—GIVING NUCLEAR INSPECTORS BETTER TOOLS WITHOUT REVEALING STATE

A nuclear catastrophe hasn't occurred... yet. So we need to stay vigilant. Nuclear inspectors go in according to treaties to check what's going on and check compliance with treaty rules.

But as sophisticated analytics become more common, states will only want to share the minimum amount of information necessary under the treaty.

But perhaps we can use MPC -- secure multi-party computation

Yao's garbled circuits were first demonstrated on the millionaire's problem -- figuring out who's richer without revealing actual amounts.

Used for other things like cryptocurrency these days.

Can we use this for nuclear inspection?

MPC can be used to compute anything computed by a computer [but it's expensive!]
TECH
Zed A. Shaw, Writer
Zed A. Shaw, Writer
@lzsthw
When you complain about an open source project's management you'll get the reply, "Use something else if you don't like it!" That'd be true except for the tiny problem that *many* projects also strive for complete monopolistic dominance.

Let me tell you about Java in early 2k:

I was programming in Java since it's release, and all through the late 90s into the early 2000s. At first Java was very determined to unseat C++ and it used various propaganda tactics to turn C++ and malloc into a weird enemy for all of us to hate. This worked great!

Everything was fine though because there were plenty of jobs in tech because of the dot-com boom, and then suddenly there were not jobs because of the dot-com bomb.

Overnight it seemed as if the *only* way to get a job was at giant companies, and they *loved* Enterprise Java.

Now, this is the golden ticket for many open source projects and corporations. They all want to be the top dog #1 in the industry for their particular use, and once they gain that position they become abusive by ignoring user needs and extracting wealth from them.

With Java the dominance was so insane that *every* language had to copy Java's weirdness to survive. If you wonder why a language has some odd CamelCase things it's because early Python, Ruby, and others had to copy Java's semantics and style to even be considered "serious".
TECH

You May Also Like

SK
SK
@sk_diary1
Prasanna Venkatesa Perumal #temple -Gunaseelam,Near Trichy,TN #Thread

Gunaseela Mahatmyam,a chapter of Bhavishyottara Purana describes greatness of this place.

Venkatachlapathi of Tirumala Tirupati appeared b4 Gunaseela Rishi here,so Gunaseelam is known as Ten(South)Tirupati


Prasanna Venkatesa Perumal is in a standing posture with 4 arms holding sankha & chakra in the upper hands.The lower right hand is in abhaya pose,blessing devotees &left hand is in kati hasta,resting on waist.

A sceptre(senkkol)rests on right shoulder,with which this murti is believed to cure people affected by mental illness.These patient stay here for a period-48 days& participate in poojas&most of them get relieved of their problems.There is a temple run Mental Rehabilitation center

#storytimethread -As per Sthala Puranam,existence of this temple is due to efforts of Gunaseela Rishi, a disciple of Thalpiya Maharishi.
Thalpiya Maharishi along with his disciple Gunaseela Rishi once went to Himalayas.On his way back, Gunaseela Rishi stayed back at Tirupathi.


He was so wanted to have darshan of Venkatachalapathy in Gunaseelam near his asharam & bless the devotees.

He came back, bathed in the Cauvery,took blessing from his guru &undertook severe penance here in his ashram in Gunaseelam.
RELIGION
Loren Brichter
Loren Brichter
@lorenb
So it turns out that Google Chrome was making everything on my computer slow *even when it wasn’t running*, because it installs something called Keystone which is basically malware.

I made a website because this shouldn’t

Wired first reported on how bad Keystone was 11 years ago when they put it into Google Earth (they seem to put it in all their popular

The fact that Keystone hides itself in Activity Monitor is bizarre. (The only sign of it was excessive CPU usage of WindowServer which is a system process).

I don’t know if Google was doing something nefarious with Keystone, or a third party figured out how to (which Wired warned about). But either way, I’m not inclined to give Google-the-organization the benefit of the doubt (despite the many good people who work on Chrome)...

...since it's been a decade+ and this still hasn't been "fixed".

There is no reason for auto-update software to need to do what Chrome/Keystone was doing. It also has a long history of crashing Macs.
TECH
Advertisement
Vibhu Vashisth
Vibhu Vashisth
@VIBHU_Tweet
शमशान में जब महर्षि दधीचि के मांसपिंड का दाह संस्कार हो रहा था तो उनकी पत्नी अपने पति का वियोग सहन नहीं कर पायी और पास में ही स्थित विशाल पीपल वृक्ष के कोटर में अपने तीन वर्ष के बालक को रख के स्वयं चिता पे बैठ कर सती हो गयी ।इस प्रकार ऋषी दधीचि और उनकी पत्नी की मुक्ति हो गयी।


परन्तु पीपल के कोटर में रखा बालक भूख प्यास से तड़पने लगा। जब कुछ नहीं मिला तो वो कोटर में पड़े पीपल के गोदों (फल) को खाकर बड़ा होने लगा। कालान्तर में पीपल के फलों और पत्तों को खाकर बालक का जीवन किसी प्रकार सुरक्षित रहा।

एक दिन देवर्षि नारद वहां से गुजर रहे थे ।नारद ने पीपल के कोटर में बालक को देख कर उसका परिचय मांगा -
नारद बोले - बालक तुम कौन हो?
बालक - यही तो मैं भी जानना चहता हूँ ।
नारद - तुम्हारे जनक कौन हैं?
बालक - यही तो मैं भी जानना चाहता हूँ ।

तब नारद ने आँखें बन्द कर ध्यान लगाया ।


तत्पश्चात आश्चर्यचकित हो कर बालक को बताया कि 'हे बालक! तुम महान दानी महर्षि दधीचि के पुत्र हो । तुम्हारे पिता की अस्थियों का वज्रास्त्र बनाकर ही देवताओं ने असुरों पर विजय पायी थी।तुम्हारे पिता की मृत्यु मात्र 31 वर्ष की वय में ही हो गयी थी'।

बालक - मेरे पिता की अकाल मृत्यु का क्या कारण था?
नारद - तुम्हारे पिता पर शनिदेव की महादशा थी।
बालक - मेरे उपर आयी विपत्ति का कारण क्या था?
नारद - शनिदेव की महादशा।
इतना बताकर देवर्षि नारद ने पीपल के पत्तों और गोदों को खाकर बड़े हुए उस बालक का नाम पिप्पलाद रखा और उसे दीक्षित किया।
ALL
Noah Smith
Noah Smith
@Noahpinion
1/Politics thread time.

To me, the most important aspect of the 2018 midterms wasn't even about partisan control, but about democracy and voting rights. That's the real battle.

2/The good news: It's now an issue that everyone's talking about, and that everyone cares about.

3/More good news: Florida's proposition to give felons voting rights won. But it didn't just win - it won with substantial support from Republican voters.

That suggests there is still SOME grassroots support for democracy that transcends

4/Yet more good news: Michigan made it easier to vote. Again, by plebiscite, showing broad support for voting rights as an

5/OK, now the bad news.

We seem to have accepted electoral dysfunction in Florida as a permanent thing. The 2000 election has never really
POLITICS
Paras Chopra
Paras Chopra
@paraschopra
1/ Here are my notes from @robertwrighter's book - Nonzero: the logic of human destiny. I enjoyed the book very much.

The basic premise of the book is that history has a direction which favors co-operation and non-zero sum games, and that causes an increase in complexity.


2/ Starting from the first replicating molecule which co-operated with an outer layer to form first proto-cell, evolutionary and cultural history is full of examples where two entities come together to survive and progress a lot more than they would have done individually.

3/ This co-operative entity fares much better than two individual entities because of specialization. If two entities are in the same boat - that they win together or lose together - then trust is implicit.

4/ In a non-zero sum game, trust causes entities to focus on what they do best.

For example, eukaryotic cells - the ones animals and plants have - emerged when two proto-cells merged, and one took the role of energy generator (mitochondria), the other specialized in protection.

5/ Some scientists believe that even nucleus in a cell is a result of an early cell merging with another cell.

So M&A is not a recent phenomena, entities have been merging because of common interests ever since life started.
CULTURE