Early 20th century videos reprocessed into HFR

Discussion about 120fps HFR as well as future Ultra HFR (240fps, 480fps and 1000fps) playing back in real time on high refresh rate displays. See Ultra HFR HOWTO for bleeding edge experimentation.
Post Reply
User avatar
Chief Blur Buster
Site Admin
Posts: 7868
Joined: 05 Dec 2013, 15:44
Location: Toronto, Ontario, Canada
Contact:

Early 20th century videos reprocessed into HFR

Post by Chief Blur Buster » 01 Sep 2020, 01:07

Although this is only 60fps, this is literally HFR reprocessing done on what are over 100 year old videos -- most of which were originally only about 18 frames per second.

phpBB [video]


phpBB [video]


These are generated from footage more than 100 years old --

Artificial intelligence was used to interpolate and colorize very old black and white footage from year 1902 and 1906. They're not glitch-free, but incredible progress.

In the next five to ten years, it would likely be possible for AI will be able to use its own "imagination" to create a "Holodeck" environment out of old footage, for exploring with a VR headset. There is a wright brothers equivalent here (photogrammetried video). Although some guesswork is involved in adding detail to blurry portions, I would surmise that AI could fill-in / autocomplete missing details (to retina resolution levels) based on a large library of world knowledge, as well as having learned from other historical videos, etc.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

       To support Blur Busters:
       • Official List of Best Gaming Monitors
       • List of G-SYNC Monitors
       • List of FreeSync Monitors
       • List of Ultrawide Monitors

1000WATT
Posts: 276
Joined: 22 Jul 2018, 05:44

Re: Early 20th century videos reprocessed into HFR

Post by 1000WATT » 01 Sep 2020, 11:14

Chief Blur Buster wrote:
01 Sep 2020, 01:07
Although some guesswork is involved in adding detail to blurry portions, I would surmise that AI could fill-in / autocomplete missing details (to retina resolution levels) based on a large library of world knowledge, as well as having learned from other historical videos, etc.
how can a sharpness filter be called an upscale?
Fading out a lot of detail for the upscale effect. And what is the logic in this?
123.jpg
123.jpg (148.62 KiB) Viewed 510 times
I often do not clearly state my thoughts. google translate is far from perfect. And in addition to the translator, I myself am mistaken. Do not take me seriously.

User avatar
Chief Blur Buster
Site Admin
Posts: 7868
Joined: 05 Dec 2013, 15:44
Location: Toronto, Ontario, Canada
Contact:

Re: Early 20th century videos reprocessed into HFR

Post by Chief Blur Buster » 01 Sep 2020, 15:04

It certainly trades certain details for other details, to produce an improved ("you're actually there") immersion effect. We can go yuck on medicore filters --

But this wasn't the point of the post.

Instead of sharpen filters, AI would be like a human graphics artists that repaint full-detail information (redrawing the whole scenery) using the original frames as hint. Even photogrammetry and superresolutioning and sharpening would only be for improving hinting for the AI artist that repaints the whole frames at full 3D detail. Then these artifacts wouldn't show.

The point of this post is that there are many ways to play back the same scenery (without destroying the original material), with various compromise tradeoffs. Let's surmise, that all these filter artifacts eventually disappear with future AI "do-over" algorithms. Smarter AI (of ten years from now or so) would, for example, know what the hell a brick texture looks like, and then draw real bricks in place of blurry black and white bricks, and then so on. Smarter AI would be able to detect for photoshopping artifacts or filter artifacts and keep redoing the image until it passes what it thinks human thinks looks realistic, etc. Rinse and repeat for all frames (and add new frames as necessary).
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

       To support Blur Busters:
       • Official List of Best Gaming Monitors
       • List of G-SYNC Monitors
       • List of FreeSync Monitors
       • List of Ultrawide Monitors

1000WATT
Posts: 276
Joined: 22 Jul 2018, 05:44

Re: Early 20th century videos reprocessed into HFR

Post by 1000WATT » 01 Sep 2020, 16:29

Chief Blur Buster wrote:
01 Sep 2020, 15:04
Smarter AI (of ten years from now or so) would, for example, know what the hell a brick texture looks like, and then draw real bricks in place of blurry black and white bricks, and then so on.
Perhaps this is off topic again. If I distract or mislead readers, please delete my posts.

I see the benefit in such developments for the easy transfer of video material to 3D models, use in games and virtual reality.

But not an improvement in the video itself. Yes, the AI will remove some of the film defects and this will be a plus. But if we talk about bricks, then even a person will not be able to make out exactly which artifacts on a brick wall: drips of rain, dirt, cobwebs, etc. If you start correcting these artifacts, the soul of this video will die.
I often do not clearly state my thoughts. google translate is far from perfect. And in addition to the translator, I myself am mistaken. Do not take me seriously.

User avatar
Chief Blur Buster
Site Admin
Posts: 7868
Joined: 05 Dec 2013, 15:44
Location: Toronto, Ontario, Canada
Contact:

Re: Early 20th century videos reprocessed into HFR

Post by Chief Blur Buster » 02 Sep 2020, 00:48

1000WATT wrote:
01 Sep 2020, 16:29
I see the benefit in such developments for the easy transfer of video material to 3D models, use in games and virtual reality.

But not an improvement in the video itself.
I agree that the original video should never be replaced with these revisualizations.

Remasters of the video / 3D models / games / VR are quite valid, as long as the original video is kept unmodified/preserved. For example, video as a virtual reality education when a user doesn't have a headset.

For the sake of argument, it's basically an independent studio-refilming of the original scene (Aka brand new video) within the imagination of a future supersmart AI rather than through artifacting filters. Not as a classical "video remaster" but an IMAXifivation/virtually refilmed version for the purpose of being able to better immerse oneself in the classical world without a VR headset. Like a remake. But a virtually-generated remake. Perhaps that kind of think-outside-the-box.

I'm still a fan of Hollywood Movie Maker Mode, with 24fps originally, too.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

       To support Blur Busters:
       • Official List of Best Gaming Monitors
       • List of G-SYNC Monitors
       • List of FreeSync Monitors
       • List of Ultrawide Monitors

thatoneguy
Posts: 49
Joined: 06 Aug 2015, 17:16

Re: Early 20th century videos reprocessed into HFR

Post by thatoneguy » 03 Sep 2020, 00:31

First video looks fake as fuck. It looks like video game graphics.

Second video doesn't look as fake but I can't stand how it looks. The various film defects(especially the inconstant luminance shifts) look much more annoying and uncomfortable to the eye in color and at 60fps.
I don't get the point of colorizing film. Remember that Monochrome has its advantages over Color.
I don't get why you would upscale film either considering most film reels can be scanned to at least Full HD. If it is an 8mm reel then at least 480p could be doable which imo is acceptable to look at even on a 40+ inch display nowadays...but maybe I'm just not as picky.

Overall I just don't get the point of AI when it comes to imagery tbh. Even with old 240p video game graphics I would much rather use a CRT Shader then use AI Upscaling. And by the time AI becomes advanced enough to upscale 4k to super duper 32K for Virtual Reality or whatever we'll likely have the hardware to do it natively at that point.

User avatar
Chief Blur Buster
Site Admin
Posts: 7868
Joined: 05 Dec 2013, 15:44
Location: Toronto, Ontario, Canada
Contact:

Re: Early 20th century videos reprocessed into HFR

Post by Chief Blur Buster » 18 Sep 2020, 12:39

It won't always look fake.

It's like how 1980s TRON looks fake as bleep, but today's CGI looks photorealistic.

As AI gets better, it becomes like a human brain imagination, painting photorealistic fakery that doesn't look fake to human eyes. And even recognizing when an image looks fake, and redoing the image until it stops looking fake.

It certainly won't be using the same algorithms; we should always keep originals -- it should not replace a master, but we should be able to ask "Computer, please load this old film into a photorealistic Holodeck environment" eventually. Basically spawn off best-effort remixed photorealistic reality (that looks perfect) off whatever material we wish. It doesn't replace the material.

Yes, I know it is a double edged sword for humanity.

AI that can do that will need to be hundreds or thousands of times more advanced, but work is already in progress to do that type of stuff. Even too, it would decouple the camera, allowing you choice of movement, like walking off the train, or sitting down/standing up inside the train, or looking at the train controls, etc. (That the computer would non-fake-look reproduce for you on a best-effort basis based on world history knowledge).

Real human artists would then use the AI tools to further enhance games, like add grafitti or post-apocalypticfeel or fiction elements to existing game art AI-generated from real life, etc.) Even though game artists will stop doing photogrammetry eventually because of such smart AI being able to recreate worlds from films/etc, they will move on to different enhancement tasks (using the AI as an assistant) to expand imagination and whatever.

But the same thing can be a great history learning tool. Imagine asking a computer, "Please create a reproduction of New York's World Fair 1939" so that you can explore the world in VR. AI would analyze all photos & footage, and generate a world based on that, in a complete-repaint way (rather than film-reprocess way).

Basically instead of planar-to-planar (like these YouTube links), it'd be planar-to-jump-into-a-holodeck (like VR headset) or FPS movements around a photorealistic environment. Instead of it being simply a "rewatch a remastered experience".

There are major flaws with using simplified AI algorithms as seen in these videos, but if you maximize the window, you will see that the humans walk much more realistically than you see in black and white films (which are much jerkier + wrong framerate, e.g. 18-sped-to-24), so even though there are ugly artifacts, other parts of immersion is improved when viewing the video in fullscreen mode and trying your hardest to ignore the flaws and focussing on things like hats & walking motion, as well as the horses, etc -- it's correctly real-life speed now (and everybody criscrosses because there was no fast vehicles back in those olden days).

The "convert film to a Holodeck" is what I viewing this through that "lens" (perspective)

Instead of applying AI algorithms to the image, the future AI would instead be a instant realtime game artist instead, a painter, recreating the whole scenery from scratch. This allows retina resolution + retina framerate + retina photorealism.

The stuff I post here is just very Wright Brothers earlybird glimpses...
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

       To support Blur Busters:
       • Official List of Best Gaming Monitors
       • List of G-SYNC Monitors
       • List of FreeSync Monitors
       • List of Ultrawide Monitors

Post Reply