Asset loading not working with high dpi monitor

[mnordine]

I'm using a 4k monitor with a dpi of 120, with a UI scaling setting of 1.25. on Windows 10 The algorithm used in _TextureLoaderFile loads @1x assets, even though I have a high dpi monitor.

[mnordine]

I was messing around with basing this on dpi, and I came up with this:

class _TextureAtlasLoaderFile extends TextureAtlasLoader {

  String _sourceUrl = "";
  bool _webpAvailable = false;
  bool _corsEnabled = false;
  num _pixelRatio = 1.0;

  static bool _highDpi;

  _TextureAtlasLoaderFile(String sourceUrl, BitmapDataLoadOptions options) {

    if (options == null) options = BitmapData.defaultLoadOptions;

    var pixelRatio = 1.0;
    var pixelRatioRegexp = new RegExp(r"@(\d)x");
    var pixelRatioMatch = pixelRatioRegexp.firstMatch(sourceUrl);

    if (pixelRatioMatch != null) {
      var match = pixelRatioMatch;
      var maxPixelRatio = options.maxPixelRatio;
      var originPixelRatio = int.parse(match.group(1));
      var devicePixelRatio = env.devicePixelRatio;
      var loaderPixelRatio = minNum(devicePixelRatio, maxPixelRatio).round();

      if (_isHighDpi()) loaderPixelRatio = minNum(maxPixelRatio, 2);

      pixelRatio = loaderPixelRatio / originPixelRatio;

      sourceUrl = sourceUrl.replaceRange(match.start, match.end, "@${loaderPixelRatio}x");
    }

    _sourceUrl = sourceUrl;
    _webpAvailable = options.webp;
    _corsEnabled = options.corsEnabled;
    _pixelRatio = pixelRatio;
  }

  bool _isHighDpi()
  {
    if (_highDpi != null) return _highDpi;

    final diff = 300 - 56;
    final xs = new Iterable.generate(diff, (i) => 56 + i);
    final match = xs.firstWhere((x) => window.matchMedia('(max-resolution: ${x}dpi)').matches, orElse: () => 0);
    if (match == null) return false;

    _highDpi =  match > 100;
    return _highDpi;
  }
}

Seems to work for me, I'm working to make it more flexible

[bp74]

Hi, yes this is an issue. At the time we were implementing the HiDpi support we were mainly thinking about mobile devices where most of them use a 2x or 3x ratio. Obviously you are right that on the desktop and higher dpi monitors a factor or 1.25 oder 1.50 is quite common. I have to look at a few other libraries how they are handling this. As far as i know most of them are using the same approach as StageXL does, which is adding the @2x and @3x suffix to the filename. We probably need to add a new configuration to the BitmapDataLoadOptions where the user can configure the available images sizes in a better way.

The current approach with the "maxPixelRatio" field is not enough. Maybe we should introduce a List of available image pixel ratios like [1x, 1.25x, 1.5x, 2x, 3x, etc.]. This way the user has the option to create images with arbitrary image pixel ratios and the library will choose the one that best fits the display pixel ratio.

Will think about it a little bit more. Feedback is welcome!

[bp74]

Okay i have pushed a commit with better support for different HiDpi images.

before (still possible, but deprecated)

StageXL.bitmapDataLoadOptions.maxPixelRatio = 3;

/// 1x resolution files are named "{imageName}@1x.png"
/// 2x resolution files are named "{imageName}@2x.png"
/// 3x resolution files are named "{imageName}@3x.png" 

var resourceManager = new ResourceManager();
resourceManager.addBitmapData("background", "images/background@1x.jpg");
resourceManager.addTextureAtlas("atlas", "images/atlas@1x.json");
await resourceManager.load();

now

StageXL.bitmapDataLoadOptions.pixelRatios = <double>[1.00, 1.25, 1.50, 2.00, 3.00];

/// 1.00x resolution files are named "{imageName}@1.00x.png"
/// 1.25x resolution files are named "{imageName}@1.25x.png"
/// 1.50x resolution files are named "{imageName}@1.50x.png"
/// 2.00x resolution files are named "{imageName}@2.00x.png"
/// 3.00x resolution files are named "{imageName}@3.00x.png"

var resourceManager = new ResourceManager();
resourceManager.addBitmapData("background", "images/background@1.00x.jpg");
resourceManager.addTextureAtlas("atlas", "images/atlas@1.00x.json");
await resourceManager.load();

Please give it a try and tell me how it works.

[mnordine]

Unfortunately, I think it's a bit more complex than even that. In my original scenario, user has a 4k monitor (3840x2160), and is likely to play the game full screen. In that scenario, I'd want to serve 2x assets, not 1.25x assets. I was thinking of having something like BitmapDataLoadOptions.useDpi. I'm using the above dpi algo in my app, and it seems to be working fine, at least for my needs with 1x, 2x assets. Unfortunately, Safari doesn't support max-resolution media queries, so we fall back to previous behaviour for it and mobile.

[bp74]

Okay maybe it's like this: The coordinate system of your stage is e.g. 500x300 but when you make it full screen, this coordinate system get's scaled to 1920x1080. Even if the devicePixelRatio is 1.0 your BitmapDatas would look pixelated because the stage is scaled by a factor of >3. In this situation it would be better to use the HiDpi images, even if the screen isn't HiDpi.

[bp74]

If you want to use HiDpi images for up-scaled Stages, you could do it like this:

var stageScale = math.min(stage.stageHeight / stage.sourceHeight, stage.stageWidth / stage.sourceWidth);
var ratioScale = StageXL.environment.devicePixelRatio * stageScale;
var pixelRatios = <double>[1.0, 1.25, 1.50, 2.0, 3.0];
pixelRatios.removeWhere((r) => r < ratioScale && r != 3.0);
StageXL.bitmapDataLoadOptions.pixelRatios = pixelRatios;

For example the available pixelRatios are [1.0, 1.25, 1.50, 2.0, 3.0]. If your Stage is scaled by a factor of 2.3333, this would result in BitmapDataLoadOptions.pixelRatios = [3.0]. So only HiDpi images with 3.0 should be available when loading Bitmaps.

You can try it out here - just open the URL and look which images are loaded from the server. Change the browser window and reload the page to see the difference:

http://www.stagexl.org/temp/hidpiscale/index.html
http://www.stagexl.org/temp/hidpiscale/index.dart

[mnordine]

That scenario doesn't help my initial use case, unfortunately. I'm not really concerned about the initial stage size either. If they initially load the app in a tiny window, I still want them to have 2x assets on a high dpi monitor, since they are likely to go full screen. I don't think the new algorithm allows for that. For instance

I'm using a 4k monitor with a dpi of 120, with a UI scaling setting of 1.25

So, given StageXL.bitmapDataLoadOptions.pixelRatios = [1.0, 2.0];
1.0 would be returned from the new algorithm.

[bp74]

Why not start with this? This will always load 2x assets.

StageXL.bitmapDataLoadOptions.pixelRatios = [2.0];

[mnordine]

Because I still want 1x assets loaded for low dpi monitors

[bp74]

if (StageXL.environment.devicePixelRatio > 1.0) {
  StageXL.bitmapDataLoadOptions.pixelRatios = [2.0];
}

[mnordine]

Yeah, that issue still has its problems. If a user has a low resolution screen and increases the ui scaling to 1.25, it'll load 2x assets even though they have a low resolution. I was thinking the dpi would be a better solution thinking it'd be independent of any ui scaling. However, on further testing, it seems the dpi returned by the browser is dependent on ui scaling. I might just have to live with this and go with your solution...

[nilsdoehring]

FYI, in general, web devs moved away from using devicePixelRatio to detect HiDPI. But I think the solution you're looking for is in this article: https://www.kirupa.com/html5/detecting_retina_high_dpi.htm

Just 'port' the JS example to Dart and you should be good to go. You would set the available pixelRatios only if the condition is matched. By the way, your 120dpi monitor is not considered HiDPI, this starts at 192dpi – but the solution can be adjusted for whatever dpi you have in mind.