Your choice of input actually set that up. ‘Contrast’ is about the rate at which tone changes, which can be considered with a graph of a line with a certain slope. Steeper the slope, higher the contrast, and vice versa. A simple contrast tool’s graph will ‘pivot’ about the center as contrast is changed, so a value of 128 on a 0-255 scale will sit on that pivot, and not change as the slope is changed.
You can demonstrate such with a tone curve. To increase contrast with a curve, take the lower-left control point and drag it right, and then drag the upper-right point left by an equal amount. Should look like this:
Note the midpoint of the line still intersects the center of the graph. X=128, Y=128. Now, a decrease in contrast looks like this:
Still, the midpoint of the line intersects the graph center. X=128, Y=128…