When you scroll up or down through slices, that should be interpreted in world space, i.e. scrolling up can mean right, anterior and superior, while scrolling down can mean left, posterior and inferior, respectively to the orientation of the render window.
There can be other conventions, see T8802.
The main thing is that it should be consistent, and direction should not depend on how the slices have been stacked up to make the geometry of the renderers.
This was consistent until the renderers could only be initialised by global re-init and (non-global) re-init worked only with 'upright' images. However, if the axes can be flipped, the stepper direction has to be inverted accordingly, when the slice navigation controller is initialised.