Okay, here we go.
Neural Volume Rendering for Dynamic Scenes
NeRF has shown incredible view synthesis results, but it requires multi-view captures for STATIC scenes.
How can we achieve view synthesis for DYNAMIC scenes from a single video? Here is what I learned from several recent efforts.
Okay, here we go.
NeRF represents the scene as a 5D continuous volumetric scene function that maps the spatial position and viewing direction to color and density. It then projects the colors/densities to form an image with volume rendering.
Volumetric + Implicit -> Awesome!
Building on NeRF, one can extend it for handling dynamic scenes with two types of approaches.
A) 4D (or 6D with views) function.
One direct approach is to include TIME as an additional input to learn a DYNAMIC radiance field.
e.g., Video-NeRF, NSFF, NeRFlow
Inspired by non-rigid reconstruction methods, this type of approach learns a radiance field in a canonical frame (template) and predicts deformation for each frame to account for dynamics over time.
e.g., Nerfie, NR-NeRF, D-NeRF
All the methods use an MLP to encode the deformation field. But, how do they differ?
A) INPUT: How to encode the additional time dimension as input?
B) OUTPUT: How to parametrize the deformation field?
One can choose to use EXPLICIT conditioning by treating the frame index t as input.
Alternatively, one can use a learnable LATENT vector for each frame.
We can either use the MLP to predict
- dense 3D translation vectors (aka scene flow) or
- dense rigid motion field
More from Tech
The entire discussion around Facebook’s disclosures of what happened in 2016 is very frustrating. No exec stopped any investigations, but there were a lot of heated discussions about what to publish and when.
In the spring and summer of 2016, as reported by the Times, activity we traced to GRU was reported to the FBI. This was the standard model of interaction companies used for nation-state attacks against likely US targeted.
In the Spring of 2017, after a deep dive into the Fake News phenomena, the security team wanted to publish an update that covered what we had learned. At this point, we didn’t have any advertising content or the big IRA cluster, but we did know about the GRU model.
This report when through dozens of edits as different equities were represented. I did not have any meetings with Sheryl on the paper, but I can’t speak to whether she was in the loop with my higher-ups.
In the end, the difficult question of attribution was settled by us pointing to the DNI report instead of saying Russia or GRU directly. In my pre-briefs with members of Congress, I made it clear that we believed this action was GRU.
The story doesn\u2019t say you were told not to... it says you did so without approval and they tried to obfuscate what you found. Is that true?
— Sarah Frier (@sarahfrier) November 15, 2018
In the spring and summer of 2016, as reported by the Times, activity we traced to GRU was reported to the FBI. This was the standard model of interaction companies used for nation-state attacks against likely US targeted.
In the Spring of 2017, after a deep dive into the Fake News phenomena, the security team wanted to publish an update that covered what we had learned. At this point, we didn’t have any advertising content or the big IRA cluster, but we did know about the GRU model.
This report when through dozens of edits as different equities were represented. I did not have any meetings with Sheryl on the paper, but I can’t speak to whether she was in the loop with my higher-ups.
In the end, the difficult question of attribution was settled by us pointing to the DNI report instead of saying Russia or GRU directly. In my pre-briefs with members of Congress, I made it clear that we believed this action was GRU.
Ok, I’ve told this story a few times, but maybe never here. Here we go. 🧵👇
I was about 6. I was in the car with my mother. We were driving a few hours from home to go to Orlando. My parents were letting me audition for a tv show. It would end up being my first job. I was very excited. But, in the meantime we drove and listened to Rush’s show.
There was some sort of trivia question they posed to the audience. I don’t remember what the riddle was, but I remember I knew the answer right away. It was phrased in this way that was somehow just simpler to see from a kid’s perspective. The answer was CAROUSEL. I was elated.
My mother was THRILLED. She insisted that we call Into the show using her “for emergencies only” giant cell phone. It was this phone:
I called in. The phone rang for a while, but someone answered. It was an impatient-sounding dude. The screener. I said I had the trivia answer. He wasn’t charmed, I could hear him rolling his eyes. He asked me what it was. I told him. “Please hold.”
Wish I had the audio of Rush Limbaugh telling me off on the phone on his show when I was six. In the meantime, RIP.
— Shannon Woodward (@shannonwoodward) February 17, 2021
I was about 6. I was in the car with my mother. We were driving a few hours from home to go to Orlando. My parents were letting me audition for a tv show. It would end up being my first job. I was very excited. But, in the meantime we drove and listened to Rush’s show.
There was some sort of trivia question they posed to the audience. I don’t remember what the riddle was, but I remember I knew the answer right away. It was phrased in this way that was somehow just simpler to see from a kid’s perspective. The answer was CAROUSEL. I was elated.
My mother was THRILLED. She insisted that we call Into the show using her “for emergencies only” giant cell phone. It was this phone:
I called in. The phone rang for a while, but someone answered. It was an impatient-sounding dude. The screener. I said I had the trivia answer. He wasn’t charmed, I could hear him rolling his eyes. He asked me what it was. I told him. “Please hold.”
What an amazing presentation! Loved how @ravidharamshi77 brilliantly started off with global macros & capital markets, and then gradually migrated to Indian equities, summing up his thesis for a bull market case!
@MadhusudanKela @VQIndia @sameervq
My key learnings: ⬇️⬇️⬇️
First, the BEAR case:
1. Bitcoin has surpassed all the bubbles of the last 45 years in extent that includes Gold, Nikkei, dotcom bubble.
2. Cyclically adjusted PE ratio for S&P 500 almost at 1929 (The Great Depression) peaks, at highest levels except the dotcom crisis in 2000.
3. World market cap to GDP ratio presently at 124% vs last 5 years average of 92% & last 10 years average of 85%.
US market cap to GDP nearing 200%.
4. Bitcoin (as an asset class) has moved to the 3rd place in terms of price gains in preceding 3 years before peak (900%); 1st was Tulip bubble in 17th century (rising 2200%).
@MadhusudanKela @VQIndia @sameervq
My key learnings: ⬇️⬇️⬇️
Bubble or Bull Market? Join us for a short presentation and candid one on one on 27th Jan, 4pm with Shri \u2066@MadhusudanKela\u2069. \u2066@VQIndia\u2069 \u2066@sameervq\u2069 #bubbleorbullmarket pic.twitter.com/LBvlBrz6mS
— Ravi Dharamshi (@ravidharamshi77) January 24, 2021
First, the BEAR case:
1. Bitcoin has surpassed all the bubbles of the last 45 years in extent that includes Gold, Nikkei, dotcom bubble.
2. Cyclically adjusted PE ratio for S&P 500 almost at 1929 (The Great Depression) peaks, at highest levels except the dotcom crisis in 2000.
3. World market cap to GDP ratio presently at 124% vs last 5 years average of 92% & last 10 years average of 85%.
US market cap to GDP nearing 200%.
4. Bitcoin (as an asset class) has moved to the 3rd place in terms of price gains in preceding 3 years before peak (900%); 1st was Tulip bubble in 17th century (rising 2200%).
You May Also Like
Recently, the @CNIL issued a decision regarding the GDPR compliance of an unknown French adtech company named "Vectaury". It may seem like small fry, but the decision has potential wide-ranging impacts for Google, the IAB framework, and today's adtech. It's thread time! 👇
It's all in French, but if you're up for it you can read:
• Their blog post (lacks the most interesting details): https://t.co/PHkDcOT1hy
• Their high-level legal decision: https://t.co/hwpiEvjodt
• The full notification: https://t.co/QQB7rfynha
I've read it so you needn't!
Vectaury was collecting geolocation data in order to create profiles (eg. people who often go to this or that type of shop) so as to power ad targeting. They operate through embedded SDKs and ad bidding, making them invisible to users.
The @CNIL notes that profiling based off of geolocation presents particular risks since it reveals people's movements and habits. As risky, the processing requires consent — this will be the heart of their assessment.
Interesting point: they justify the decision in part because of how many people COULD be targeted in this way (rather than how many have — though they note that too). Because it's on a phone, and many have phones, it is considered large-scale processing no matter what.
It's all in French, but if you're up for it you can read:
• Their blog post (lacks the most interesting details): https://t.co/PHkDcOT1hy
• Their high-level legal decision: https://t.co/hwpiEvjodt
• The full notification: https://t.co/QQB7rfynha
I've read it so you needn't!
Vectaury was collecting geolocation data in order to create profiles (eg. people who often go to this or that type of shop) so as to power ad targeting. They operate through embedded SDKs and ad bidding, making them invisible to users.
The @CNIL notes that profiling based off of geolocation presents particular risks since it reveals people's movements and habits. As risky, the processing requires consent — this will be the heart of their assessment.
Interesting point: they justify the decision in part because of how many people COULD be targeted in this way (rather than how many have — though they note that too). Because it's on a phone, and many have phones, it is considered large-scale processing no matter what.
Krugman is, of course, right about this. BUT, note that universities can do a lot to revitalize declining and rural regions.
See this thing that @lymanstoneky wrote:
And see this thing that I wrote:
And see this book that @JamesFallows wrote:
And see this other thing that I wrote:
One thing I've been noticing about responses to today's column is that many people still don't get how strong the forces behind regional divergence are, and how hard to reverse 1/ https://t.co/Ft2aH1NcQt
— Paul Krugman (@paulkrugman) November 20, 2018
See this thing that @lymanstoneky wrote:
And see this thing that I wrote:
And see this book that @JamesFallows wrote:
And see this other thing that I wrote:
