Smart cropping with afre

exercise
#1

Yet another exercise. I decided to make a new thread for it this time. I love cropping using the resize and crop commands and derivatives. Cropping also has other applications. I am figuring out a few things currently.

1. Given x0,y0,x1,y1, crop but so that it fits a certain aspect ratio. That presents two cases: the actual crop has points within (1a; smaller image) the original 4 or outside (1b; bigger image).

2. Idea 1b presents us with a case where the crop could overtake the original image’s bounds, which I wouldn’t want. In this case, the crop would be smart enough to stretch in the opposite direction (2a). The final problem that I could think of ATM is if both edges along one axis is reached, then no more growing is possible (2b); adjust accordingly.

Sounds like a mouthful already. I have more thoughts but lets start with this for now. Any suggestions? Any takers?

(G'MIC staff) #2

Isn’t this already what the command resize_ratio2d does?

#3

Yes, but I want it to apply to a crop. I will study resize_ratio2d.

PS Let me elaborate on what I have in mind. In a GUI app, you can easily crop by aspect ratio. You start dragging and an appropriately constrained crop rectangle forms either from a corner or the centre.

CLI doesn’t have that option, the rectangle you make is what you crop and it doesn’t adhere to a specified aspect ratio unless you do the mental math and have the dexterity to operate the mouse well. Or you could determine the coordinates. Nah, how about write a script to do all of that, with the considerations I highlighted in the OP.

Except I have bigger ambitions. I want to mark a ROI and expand or shrink it to a specified size and / or aspect ratio. E.g., the aspect ratio for the cropped tiger (the ROI) below is 1.699669 (515x303). What if I wanted an aspect of 1.5 and the rectangle to be 2x, or a minimum width of 700px?

focused_tiger

PPS I am aware of @snibgo’s (consider yourself pinged :stuck_out_tongue:) crop2det, and that much of what I am asking about is a common problem. My thought process is: how do I crop a PlayRaw image, for instance, based on its ROI? It an exercise that I want to work through with you all.

(Alan Gibson) #4

A script could be written that took an image, and x0,y0,x1,y1 coordinates and an aspect ratio, and returned an image, as in @afre’s OP.

As afre says, the actual returned image might be within the specified coordinates, or outside them. Or a combination of both.

From just that information, there is no unique solution. The script could return one of a large number of possibilities, and they would all be valid. How does the script decide which one is best?

My Crop to detail uses a fairly primitive measure: sharp parts of the image are more important (more salient) than blurred areas. This works fairly well for my own photos because I use a “full frame” camera, with a sensor about 24x36mm. It wouldn’t work well with phone cameras and their tiny sensors.

A more intelligent system might:

  • identify faces, and decide they are more salient than non-faces;

  • identify human-content versus nature (eg buildings versus bushes) and decide which is more salient;

  • identify other aesthetic features, eg colour, tone, line, patterns, texture etc, and base saliency on those. I fear this would most easily be implemented by a machine-learning system.

#5

@snibgo I don’t intend to discuss how one would approach identifying the ROI. That is a deeper discussion to be had another time. A long time ago, I used to come up with a lot of interesting stuff but I don’t have the energy, smarts, memory and determination to do it anymore. :cry:

What I have been thinking about is how to approach the cropping about 1 chosen continuous ROI. Could be expanded to multiple ROI in the future but again I want to keep it simple for now.

#6

I just found a issue with autocrop if this is related.

To clarify:

image

The above is a small palette with transparency info. If you apply s y,{h}, and apply autocrop, note that the transparency info is still retained. If I type in mirror x, the transparency info is no longer retained. Ooh, and I noticed one pixel get cropped out as well.

(G'MIC staff) #7

I’m not sure I understand what you expect and what you get. Could you please give us the command you are using for your experiment ?

#8

s y,{h} autocrop

The result in this picture -

Evidently it crops one pixel to the right

The expect result is that all fully transparent pixel info has been removed, and all opaque pixels are retained. Neither are those two are met.

(G'MIC staff) #9

No, that is expected for me.
If you use autocrop without arguments, then G’MIC considers that the color to remove is the first encountered in the image, and in your case, that’s color 0,0,0,255.
So it removes the first pixel.

If you want another color, then specify it, e.g autocrop 0,0,0,0.
Note also that transparent pixels is quite a vague notion, because transparent pixels can be colored. If you specify autocrop 0,0,0,0, only black transparent pixels will be removed
(assuming your 4-channel image is RGBA, which is something G’MIC does not care at all for autocrop :stuck_out_tongue: ).

1 Like
#10

This would remove all fully transparent pixels at ends.

[palette] split_opacity autocrop_seq 0 append c
#11

What do you do when rr2d doesn’t give you what you want?

in_a=22
in_b=10
in_c={$in_a*$in_b}
v={$in_a+$in_b}
iw={w}
ih={h}
vw={$in_c==0?floor(w>h?$v:w/h*$v):$in_a}
vh={$in_c==0?floor(h>w?$v:h/w*$v):$in_b}
mini_width={ceil(w/$vw)}
mini_height={ceil(h/$vh)}
full_width={$mini_width*$vw}
full_height={$mini_height*$vh}

+rep_lbchstatfunc_to_pal[^0] ia
rr2d[^0,{$!-1}] {$vw},{$vh},1,4,5

rr2d gives me different result than what’s expected which is vw, and vh values for result width and height sometimes. So, I am stumped at what to do. I want to minimize distortion, so scaling the entire image isn’t gonna give me what I’d like. Right now, I’m using shift, and rescale without interpolation after the rr2d as a workaround. I think rr2d does work better with bigger images, but smaller ones, probably not. Shrink_x, and shrink_y didn’t help. Sometimes shrink actually increases/decreases from the value from what I want.

In other words, I’m looking to fit and scale at the same time.

#12

I don’t understand what your problem is. Show not tell.


Ha ha, I totally forgot about this thread. I will elaborate on my OP later…

#13

Here’s the problem right now. I’m trying to automatically crop to specified ratio from the center.

The ratio of the end image is 20:10 which is 2:1

This is needed for the mosaic project to work with multiple images with minimal distortion from scaling.

#14

Oh, I have my own filter for that. I really should update my community file sometime with the utility filters that I have accumulated; I am too shy and uncertain, and never used GitHub to make pull requests or changes. :blush::stuck_out_tongue:

Tip: don’t use resize commands; use crop. I am sure you can write it. :slight_smile::+1:

#15

@afre - Here’s my solution. I’ll upload this to github after my mosaic cli gets updated with support for custom mini-images size rather than automatically based on the [0] image.

#@cli rep_aspect_crop: width_ratio>=1,height_ratio>=1,-100>=shift_factor<=100
rep_aspect_crop:
skip ${3=0}
old_ratio={w/h}
new_ratio={$1/$2}
shift_factor=$3

if {abs($shift_factor)>100} v + error "Shift factor can only be in ranges [-100,100] inclusive!" fi
if {$old_ratio!=$new_ratio}
if {$new_ratio>$old_ratio} {w},{w/$new_ratio},{d},{s} shift.. 0,{(h#1-h#0)/2+((-$shift_factor/200)*(h#0-h#1))} f. "i#0" rm.. else {h*$new_ratio},{h},{d},{s} shift.. {(w#1-w#0)/2+((-$shift_factor/200)*(w#0-w#1))},0 f. "i#0" rm.. fi fi
(G'MIC staff) #16

That’s why resize accept percentage as arguments.
Find the right percentage (between 0% and 100%), and use the 0.5 centering, and you are done:

$ gmic sp lena resize 50%,50%,1,100%,0,0,0.5,0.5
#17

Ah, so that’s what the last two stand for! I haven’t really thought of that nor understood it. I guess the aspect ratio command isn’t actually needed at all though it can be useful for a example or for easier access. This is what I came up with.

skip ${3=0}
shift_factor=$3
if {($1/$2)<(w/h)} r 100%,{(min($1,$2)/max($1,$2))*(w/h)*100}%,1,100%,0,0,0.5,{.5+$shift_factor/2} else r {(min($1,$2)/max($1,$2))*(w/h)*100}%,100%,1,100%,0,0,{.5+$shift_factor/2},0.5 fi
#19

Plus interpolation=0. I always forget that resize can do crops. In my non-programmer brain, separating resize and crop makes more sense. So, I stubbornly do the same thing but with a command using crop, which I know is less efficient.

#20

Actually, now I believe there is actually a working solution to my current problem. Combining your suggestion and rr2d would give me a way to actually make the mosaic filter work i.e you don’t see alpha gaps.

This is my test script. This forces image to stay within a ratio though yes I know it does more than changing ratio, but it’s pretty much the working solution for my mosaic filter.

variable width and height seem to use the last image info rather than its being independent, not sure how to get around that though that’s not really needed for mosaic filter.

10,10,1,3

150,150,1,3

rr2d 220,100,1,5

r 220,100,1,100%,0,0,0,.5,.5
#21

Stats are always retrieved from the last image unless you specify otherwise. I find that even % fails sometimes, so I need to explicitly mention the stat; e.g., {3,w}. The confusing thing is that there is more than one way to do this.