XYZ on its own is not sufficient to describe the appearance of a colour. You also need to know the conditions under which it is viewed (surrounding colour, illumination etc.).
In a typical colour management workflow, you have to deal with many different white points and viewing conditions. Rather than having specific conversions between each pair of devices in a system (which results in a LOT of possible transforms), ICC colour management uses a common Profile Connection Space (PCS) as the target for ICC profiles. That way, each ICC profile converts from a device colourspace to the PCS (or the other way around). You can build colour transforms by plugging profiles together using the PCS as a common intermediate space. The ICC PCS just happens to be D50 based.
Conversion between device white points and the PCS whitepoint is handled in the profiles. There is some extra complexity in putting the profile together to account for viewing conditions, but using the profiles is simplified.
The full explanation of the reasons for a D50 whitepoint in Display profiles is on the ICC website here:
Why is the media point of a display profile always D50?