Retina images generated using ipython display functions do not have their dimensions set

[has2k1]

Bug description

Images generated via jupyter/ipython and tagged as retina should be displayed in half the true pixel dimensions. For html format, the image tag should contain the height and width that are half the true size.

Generally, it is not possible to control the display size (and therefore the perceived resolution) of the output image using ipython display functions.

Steps to reproduce

---
title: "IPython Retina Image"
format: html
---

```{python}
from IPython.display import Image, display_png

url = "https://quarto.org/docs/gallery/websites/websites-quarto.png"
img = Image(url=url, metadata={"height": 663, "width": 900}, retina=True, embed=True)
display_png(img)
```

Expected behavior

A a figure with image tag that has height and width. i.e.

<img src="..." height="331" width="450" ...>

Actual behavior

<img src="..." class="img-fluid figure-img">

Quarto check output

$ quarto check
Quarto 1.5.31
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.13: OK
      Dart Sass version 1.70.0: OK
      Deno version 1.41.0: OK
      Typst version 0.11.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.5.31
      Path: /Applications/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: v2022.11
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: TinyTex
      Path: /Users/me/Library/TinyTeX/bin/universal-darwin
      Version: 2022

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.12.1
      Path: /Users/me/.uvenv/plotnine/bin/python3
      Jupyter: 5.7.2
      Kernels: gnuplot, python3

[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........OK
      Version: 4.2.3
      Path: /Library/Frameworks/R.framework/Resources
      LibPaths:
        - /Library/Frameworks/R.framework/Versions/4.2/Resources/library
      knitr: 1.45
      rmarkdown: 2.25

[✓] Checking Knitr engine render......OK

[mcanouil]

The size are properly set for the image.

I used a slight variation of your code:

---
format: html
---

```{python}
from IPython.display import Image, display

url = "https://quarto.org/docs/gallery/websites/websites-quarto.png"
display(Image(url=url, height=663, width=900, retina=True, embed=True))
```

Note that the issue seems to be when you use embed option.
I suggest to use embed via Quarto/Pandoc which actually gives the proper and desired output:

---
format: html
embed-resources: true
---

```{python}
from IPython.display import Image, display

url = "https://quarto.org/docs/gallery/websites/websites-quarto.png"
display(Image(url=url, height=663, width=900, retina=True))
```

I am not entirely sure there is a bug here.

[has2k1]

Note that the issue seems to be when you use embed option.

embed was just to get the downloaded image to show up using a single copy-pastable example. You get the same "non-retina" image if you use a download images.

First Download the image.

wget "https://quarto.org/docs/gallery/websites/websites-quarto.png"

---
title: "IPython Retina Image"
format: html
---

```{python}
from IPython.display import Image, display
display(Image("websites-quarto.png", retina=True))
```

which gives

The same snippet in jupyter notebook gives

[has2k1]

This is what we get in jupyter frontend environments (jupyter-lab or notebook).

This is what we get in quarto

As a result when quarto renders ipynb notebooks, the images are not displayed with the same apparent resolution.

[has2k1]

Looking at the issue using matplotlib

The python code-block in here should display a 600 x 400 image, but the true pixel size should be 1200 x 800.

---
title: "IPython Retina Image"
---

```{python}
# Do not crop the empty margins so that final image size is predictable
%config InlineBackend.print_figure_kwargs = {"bbox_inches": None}

import matplotlib.pyplot as plt
import numpy as np

# Set the output to Retina
# i.e. display_width = pixel_width / 2
#      display_height = pixel_height / 2
from matplotlib_inline.backend_inline import set_matplotlib_formats
set_matplotlib_formats("retina")

fig, ax = plt.subplots()

# Create a 600 x 400 display image
# Because it is retina, the pixel size should be 1200 x 800
fig.set_dpi(100)
fig.set_size_inches(6, 4)
fig.set_layout_engine("none")

x = np.linspace(0, 4*np.pi, 100)
y1 = np.sin(x)
y2 = np.cos(x)

ax.set_title("Sine and Cosine")
ax.plot(x, y1)
ax.plot(x, y2)
ax.legend(['Sine', 'Cosine'])
```

As with using the core ipython display methods, the height and width attributes are missing.

Yet if you change the meta information and set the fig-dpi to the same DPI as fig.set_dpi or even omit fig.set_dpi

---
title: "IPython Retina Image"
format:
   html:
      fig-dpi: 100
---

you get the desired result!

[cderv]

@has2k1 we are doing special handling for retina considering the dpi

quarto-cli/src/core/jupyter/jupyter.ts

Lines 1916 to 1933 in 163a059

	// base64 decode if it's not svg
	const outputFile = join(options.assets.base_dir, imageFile);
	if (mimeType !== kImageSvg) {
	const imageData = base64decode(imageText);
	// if we are in retina mode, then derive width and height from the image
	if (
	mimeType === kImagePng && options.figFormat === "retina" && options.figDpi
	) {
	const png = new PngImage(imageData);
	if (
	png.dpiX === (options.figDpi * 2) && png.dpiY === (options.figDpi * 2)
	) {
	width = Math.round(png.width / 2);
	height = Math.round(png.height / 2);
	}
	}
	Deno.writeFileSync(outputFile, imageData);

Though as you see we do it when the dpi of the encoded png is coherent with the the fig-dpi option from Quarto. In that case we change width and height accordingly, otherwise we leave it untouched.

This is because we set this option to be the one to use by matplotlib

quarto-cli/src/resources/jupyter/lang/python/setup.py

Lines 19 to 28 in 163a059

	# matplotlib defaults / format
	try:
	import matplotlib.pyplot as plt
	plt.rcParams['figure.figsize'] = (fig_width, fig_height)
	plt.rcParams['figure.dpi'] = fig_dpi
	plt.rcParams['savefig.dpi'] = fig_dpi
	from IPython.display import set_matplotlib_formats
	set_matplotlib_formats(fig_format)
	except Exception:
	pass

So when you set fig-dpi the same as the option you modify yourself in your example it works, and when you remove call fig.set_dpi also.

So I do believe it works as expected as long as no manual override is done.

If some config is modified at the cell level, then this automated sizing won't apply.

For the downloaded image, it is the same. If the DPI of the image does not match the fig-dpi option (by default 96 for HTML format), then no size will be computed and set. The downloaded image in your example as a dpi of 72 I believe.

Currently fig options are only at document level for Jupyrter engine, but hopefully we can adapt to make them work better at cell level

• fig-width has no effect with jupyter engine #3155

In addition to all this, as no specific width and height are specify on the image and because responsive image is opt-in by default for HTML, the rendered size gets tweaked. If you set fig-responsive: false you will get the width and height defined in your metadata passed to Image() in your first example.

I tried to play with Retina = False or True in Image() but it does not seem the image is marked somehow. From the doc it seems Retina = True, means size will be computed based on this from read sizes. But I can't make it work on my end.

Anyhow.

Does it help understanding the behavior here ?

Not the best probably, but I don't think there is a bug per-se.

What would you have expected from Quarto to do exactly ? Always curious of ideas to improve!

[has2k1]

@cderv, the link to the typescript code was very helpful at locating the problem. I have a PR #9538, let's see what has broken.

[cderv]

You're welcome ! Happy to help any time. Feel free to ping me directly here or on slack if any question with Quarto