Numbers are there because pixels are represented by some value. So we can’t escape doing maths on pixels. However, whatever value you read is code for something. Math is a language, numbers are representations of some quantity. Bothering about numbers is like bothering about words in a text. You don’t care about individual words, you care about their meaning.
We are tackling fundamental epistemology here.
Image processing uses models, like science does. Models represent something real in an abstract way that allows general cognition. Using models is nice to get away from empiricism and particular cases.
Your display is modeled as an electron to photon converter. Using that model, we can derivate an (ICC) profile that links input (RGB code values turned into voltage) to output (light interprated as color by the human). That model is based on some assumptions, for example 100% luminance means media white, and RGB values are bounded in [0-100]%. But your actual display can be standard sRGB in dim surround or a custom HDR stuff with a gamma 2.6 in bright surround, it doesn’t matter, as long as we have a standard display against which the non-standard display is profiled. As long as we know how to go from standard RGB space to non-standard display space (which is what the ICC profile is for), we don’t care.
So, whenever you stumble upon some “standard” value for stuff (like middle grey), you need to assert in which standard it belongs, and what the assumptions/expectations of that standard are. Because there are many standards for many uses.
For example, it took me years of diving into color management to finally learn (thanks Troy) that sRGB is a color space meant for dim surround lighting. So, whenever you save or display a JPEG sRGB file, some brightness compensation should be applied to simulate dim viewing conditions if you are not in a dim surround. Well, nobody does that. But using sRGB specs to display whatever in a bright surround will fail. Yet nobody cares.
So, 18% middle grey is part of a standard. Your camera RGB space is not standard, and your display RGB space is probably not standard either. A standard is only a connection space to go from one input/output medium to another, or work images in a reproductable way (for example, if you need to apply styles/presets/batch processing, you need to remove image peculiarities first).
Going from/to standard to/from non-standard only needs a transform. ICC profiles define a kind of transform. Unfortunately, no ICC profile will be able to say where the actual middle-grey is in your sensor raw image, because it depends on your hardware exposure. So you need to define that transform yourself, using digital exposure compensation, to shift the raw mid-tones until they match your display mid-tones.
The only question you need to ask yourself when editing is “how bright is bright enough to see midtones”. The actual value of middle-grey is none of your business, since every color space conversion takes care of that.
In the past I did better and didn’t care about numbers, but the more I red the more I feel like image processing is a complex thing and this makes me feel like numbers could help to provide some kind of anchor.