While adding a debug filter for a reported issue, I thought I might as well add a new one as well: Aspect Adjustment. I think everyone is familiar with the concept by now, but apparently missing as a gui filter in g’mic…
2 Likes
One I’ve been sitting on for a while: a simple patch-based 2x upscale.
Quite similar to the recursive2x filter, but with several changes that finally means it can work without various post-scale fixups:
- Match patches from upscaled → original, rather than original → downscaled
- Match on low frequency, then add the high frequency part to upscaled
- Tweak the image saturation for matching
#@cli gcd_upscale_patch2x
#@cli : Double image size using randomized self-similar matching.
#@cli : $ image.jpg +gcd_upscale_patch2x
gcd_upscale_patch2x :
foreach {
m:=[im,iM]
(0.0625,0.125,0.0625;0.125,0.25,0.125;0.0625,0.125,0.0625)
+r[0] 200%,200%,100%,100%,1 convolve. .. rm..
[-1] +b[0] 0.67 sub[0] .
mix_channels[-2,-1] (3,-1,-1;-1,3,-1;-1,-1,3)
matchpatch.. .,3,3,1,12,13 rm.
warp[0] .,0 add[0,1] k[0] c $m
}
This has been pushed to filter updates, should be available soon. It is randomised, so results will not be 100% the same each time (it’s usually quite consistent though and PSNR is good).
1 Like
Added gcd_seams, to find optimal vertical paths (usually in a gradient image). This is derived from the algorithm behind seamcarve, but with a few cleanups/speedups and works multi-channel. Mainly I just like the cleverness of the method and wanted to review it to add better comments, but somebody else might find other uses.
#@cli gcd_seams
#@cli : Find vertical paths with optimal variance for a given gradient map.
#@cli : $ image.jpg gradient_norm gcd_seams
gcd_seams :
foreach {
# Calculate low matrix (backwards propagation).
+f[0] "<i+min(j(-1,1),j(0,1),j(1,1))"
# Initialise seams, top matrix.
100%,100%,100%,100% +rows[1] 0
=> grad,low,seam,top
repeat h#0-1 {
# get next row from low matrix
nr:=$>+1 +z[low] 0,$nr,100%,$nr => cmp
# Find optimum matches between two 1D matrices.
+*[top,cmp] # vertical weights
+shift[top] 1 *. [cmp] # diagonal backward weights
+shift[cmp] 1 *. [top] # diagonal forward weights
+[-2,-1] # summed diagonal weights
j[cmp] [top]
f[cmp] ">max(j(-1)+i#-2,j(-2)+i#-1)" # forward pass: f(x)
shift[cmp] 1 +[-2] [cmp] # calculate all f(x-1) + vertical
shift[cmp] 1 +[cmp,-1] # calculate all f(x-2) + diagonal
>[-2,-1] f. "<j(1)<0?1:-i" => offs # backward pass: offsets
# Add matched row to seams.
j[seam] [offs],0,$>
# Distribute matched energy within top matrix.
f[top] "j(i#-1)" rm[offs]
# Add next energy row to top matrix.
+z[grad] 0,$nr,100%,$nr +[top,-1]
}
# Calculate seam energy.
j[grad] [top],0,100% rm[low,top]
f[grad] "<j(i#-1,1,0,0,0,1)" rm[seam]
}
1 Like
I guess this can be used to improve seamcarving?
Well there are some speedups there, so yes I suppose so. I was considering moving all of it to math parser rather than a loop over rows, but it’s quite a lot of effort for possibly no gain (other than learning from it)!
Well, I find that the math parser allows me to do things that wouldn’t be easily feasible otherwise. If you look at my code, you will see that I use math parser all the time. Once you get used to it, you don’t go back easily.
Indeed, one thing I’ve been thinking about is complex convolution, which will probably need math parser: convolution of a two-channel image taken as (r,i) and convolved with a similar kernel. If it’s fast enough, there may be some advantages by using matrix factorisation on the kernels, but I haven’t thought it through properly yet.
Added new GUI filter Upscale [Patch2x]
- This adds a gui for the
gcd_upscale_patch2xcommand, but also for a new cli commandgcd_upscale_tile2x. - The
gcd_upscale_tile2xcombines a solver method and a self-similar match method, in tiled windows usingimg2patches. That’s about as sharp as I’m likely to get without using CNN methods, so I’m done with upscalers for now
3 Likes
Welcome back! Any plans for your next projects? Some one did wanted focus stacking on G’MIC. I wonder if you got a idea for that.
No particular plans, but that’s because the list of ideas is enormous - usually when a filter arrives these days it’s because it’s been sitting around doing nothing for a while!
Only briefly looked at focus stacking while working on pyramid methods, suppose it could use a re-read. It didn’t look tough to make something functional.
1 Like
This could be super interesting to try focus stacking using neural networks
More generally, that could be super interesting that G’MIC filter developers (other than me) start playing with the nn_lib. I’m not sure how much this library will evolve, but I could at least do a little presentation of how it works, if you are interested (maybe with a video chat?).
Well, funny you mention the nn_lib stuff, I’ve been thinking exactly that - if it can be boiled down to a series of steps on how to train (for a non-expert), I’d certainly give it a go as a basis for filters.
I only know the absolute basics about the design of neural nets, so I’m afraid that would be the starting point for me!
Edit: hah, I like the comparison there - tile2x is approaching CNN quality in some cases, which usually means some of the techniques could be applied to the CNN method to improve it too.
This suggests I’d first have to write a kind of tutorial that explains how to use the lib, train a simple network and use it for some task.
Maybe image upscaling…
1 Like
If you can find the time to write such a thing, it would be a service to us all - whether imagined or actual, the bar seems too high to get into it!
To add: I’m not expecting “nn_lib for dummies” either, if it takes major effort to learn sometimes that’s just how it is.
I haven’t gotten into nn_lib thing either. Seems even more abstract than 3D things, and I only have a brief idea on that.
Uggghhhhh pleasssseeeee do this.
Do you think there is a training dataset somewhere that I could use to train a neural network for focus stacking ?
@andabata might be able to provide you with some images
I do have a couple of stacks from mainly mushrooms (they tend not to run away while shooting a stack). How many do you need? and it what format ? You can poke me in #pixls.us
A problem i experience with the OSS stackers is that they don’t have an ability to retouch the images. Even the mushrooms don’t move, there is quite often an insect moving around on them, and creating wierdness.