Maybe the format doesn’t match the data? The colors are 32-bit float in VL, so you have to set the format to that. If you use a 8 bit per channel texture use the ToInteger node for the color to sample them down to 8-bit per channel.
And for grayscale you can extract one channel of the color and use a single channel texture format. Like R32_Float or so.