Anyone ever upscaled FullHd to 4K with machine learning?

frd · February 11, 2020, 12:10am

Hi folks,

I am rendering a short animation and someone suggested to render in FHD and then upscale through AI. Has anyone ever done this on Linux? Would you recommend any program, tutorial, or wiki? This is all very new to me so I need some guidance.

Thanks

paperdigits · February 11, 2020, 12:30am

AI like Artificial Intelligence?

frd · February 11, 2020, 12:38am

yeah

ggbutcher · February 11, 2020, 1:48am

I just know of a couple of AI-based resizers for stills, AI Gigapixel comes to mind. If you’re producing a stack of stills as your renders, you could probably use one of these, frame-by-frame…

Edit: Oh, Linux… did find a reddit thread about running Topaz AI Gigapixel in wine, but it required patching…

hanatos · February 11, 2020, 7:45am

you mean like dlss?

my (old man’s biased) understanding was that it doesn’t work:

CarVac · February 11, 2020, 1:08pm

If it’s lineart, waifu2x does pretty well.

I’ve only ever used it on web services because I think it needs an Nvidia GPU.

PhotoPhysicsGuy · February 11, 2020, 1:47pm

madVR (not open source) also has an upscaler which is neural network/machine learning based, and I think it’s been trained to work with a lot of real world stuff as well. So, it can be implemented to work nicely.

afre · February 11, 2020, 7:16pm

What expectations do you or your friend have of the AI? Could you also give us an example?

frd · February 13, 2020, 12:42pm

Well, it seems some CG studios are doing that. But it seems to be either inhouse tools or some proprietary solutions so I was wondering if there was anything like this in th FLOSS world.

paperdigits · February 13, 2020, 3:38pm

github.com

alexjc/neural-enhance/blob/master/README.rst

Neural Enhance
==============

.. image:: docs/OldStation_example.gif

**Example #1** — Old Station: `view comparison <http://enhance.nucl.ai/w/0f5177f4-9ce6-11e6-992c-c86000be451f/view>`_ in 24-bit HD, `original photo <https://flic.kr/p/oYhbBv>`_ CC-BY-SA @siv-athens.

----

`As seen on TV! <https://www.youtube.com/watch?v=LhF_56SxrGk>`_ What if you could increase the resolution of your photos using technology from CSI laboratories? Thanks to deep learning and ``#NeuralEnhance``, it's now possible to train a neural network to zoom in to your images at 2x or even 4x.  You'll get even better results by increasing the number of neurons or training with a dataset similar to your low resolution image.

The catch? The neural network is hallucinating details based on its training from example images. It's not reconstructing your photo exactly as it would have been if it was HD. That's only possible in Hollywood — but using deep learning as "Creative AI" works and it is just as cool!  Here's how you can get started...

1. `Examples & Usage <#1-examples--usage>`_
2. `Installation <#2-installation--setup>`_
3. `Background & Research <#3-background--research>`_
4. `Troubleshooting <#4-troubleshooting-problems>`_
5. `Frequent Questions <#5-frequent-questions>`_

|Python Version| |License Type| |Project Stars|

This file has been truncated. show original

frd · February 13, 2020, 6:24pm

Thanks, will check it out.

afre · February 13, 2020, 7:58pm

I ask because it would depend on your CG. They use in-house not only because they have the resources but they also need to cater to their specific content. I guess at this point you are going by trial and error.

frd · February 13, 2020, 8:15pm

Yeah, also discovered that in some cases they use this: https://topazlabs.com/gigapixel-ai/

KristijanZic · February 14, 2020, 3:27am

Is there anything for this purpose that uses OpenCL?

I have AMD Vega 64 card in my machine so anything with CUDA is not for me.

vdmokstati · February 21, 2020, 3:30pm

this is what you need