Problem parsing raw LaTeX table environment with option (like H position)

[bhattmaulik]

Bug description

I am using Quarto to prepare a document. I want to render it as PDF/Latex document. In this document, I need some tables, which I am formatting using kableExtra package. I want to HOLD the position of the table, so that they don't appear on the next page. But when I add "HOLD_position" option in the kable_styling, it adds an extra [H] above the table. Interestingly, even if I have set tbl-cap-position below the table, this [H] remains above the table instead of moving with the caption.

A minimal example is given below.

I created an issue with the repository of kableExtra, thinking that it is an issue with the package. But from the comment in that issue, I realized that this issue doesn't happen if I use R Markdown document instead of Quarto. Therefore, I am creating a bug report here.

Steps to reproduce

---
title: "Exp_KableExtra"
author: "Maulik"
date: "`r Sys.Date()`"
output: pdf_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```


```{r}
library(tidyverse) #optional 
library(knitr)
library(kableExtra)
```

Now I am adding some data.

```{r}
#| echo: false  
#| label: tbl-cars
#| tbl-cap: "Some cars"  
head(mtcars) |> kableExtra::kbl(booktabs = TRUE) |> kable_styling(latex_options = c("striped", "bordered","HOLD_position"),full_width = FALSE) %>%   column_spec(2, width = "30em") 

Expected behavior

There should not be [H] in the table caption.

Actual behavior

There is an [H] written above the table.

Your environment

RSudio Version: 2024.04.0 Build 725, OS: Windows 11

Quarto check output

$ quarto
$ quarto check
Quarto 1.5.31
[>] Checking versions of quarto bin
ary dependencies...
Pandoc version 3.1.13: OK
Dart Sass version 1.70.0: OK
Deno version 1.41.0: OK
Typst version 0.11.0: OK
[>] Checking versions of quarto dep
endencies......OK
[>] Checking Quarto installation...
...OK
Version: 1.5.31
Path: C:\Users\Hp\AppData\Loc
al\Programs\Quarto\bin
CodePage: 1252

[>] Checking tools.................
...OK
TinyTeX: (external install)
Chromium: (not installed)

(|) Checking LaTeX.................
...
(/) Checking LaTeX.................
...
(-) Checking LaTeX.................
...
() Checking LaTeX.................
...
(|) Checking LaTeX.................
...
(/) Checking LaTeX.................
...
(-) Checking LaTeX.................
...
() Checking LaTeX.................
...
(|) Checking LaTeX.................
...
(/) Checking LaTeX.................
...
(-) Checking LaTeX.................
...
() Checking LaTeX.................
...
(|) Checking LaTeX.................
...
(/) Checking LaTeX.................
...
(-) Checking LaTeX.................
...
() Checking LaTeX.................
...
(|) Checking LaTeX.................
...
(/) Checking LaTeX.................
...
(-) Checking LaTeX.................
...
() Checking LaTeX.................
...
(|) Checking LaTeX.................
...
(/) Checking LaTeX.................
...
(-) Checking LaTeX.................
...
() Checking LaTeX.................
...
(|) Checking LaTeX.................
...
(/) Checking LaTeX.................
...
[>] Checking LaTeX.................
...OK
Using: TinyTex
Path: C:\Users\Hp\AppData\Roa
ming\TinyTeX\bin\windows
Version: 2024

(|) Checking basic markdown render.
...
(/) Checking basic markdown render.
...
(-) Checking basic markdown render.
...
() Checking basic markdown render.
...
(|) Checking basic markdown render.
...
(/) Checking basic markdown render.
...
(-) Checking basic markdown render.
...
() Checking basic markdown render.
...
(|) Checking basic markdown render.
[>] Checking basic markdown render.
...OK

[>] Checking Python 3 installation.
...(None)

  Unable to locate an installed

version of Python 3.
Install Python 3 from https:/
/www.python.org/downloads/

(|) Checking R installation........
...
(/) Checking R installation........
...
(-) Checking R installation........
...
[>] Checking R installation........
...OK
Version: 4.4.0
Path: C:/PROGRA1/R/R-441.0
LibPaths:
- C:/Users/Hp/AppData/Local
/R/win-library/4.4
- C:/Program Files/R/R-4.4.
0/library
knitr: 1.46
rmarkdown: 2.26

(|) Checking Knitr engine render...
...
(/) Checking Knitr engine render...
...
(-) Checking Knitr engine render...
...
() Checking Knitr engine render...
...
(|) Checking Knitr engine render...
...
(/) Checking Knitr engine render...
...
(-) Checking Knitr engine render...
...
() Checking Knitr engine render...
...
(|) Checking Knitr engine render...
...
(/) Checking Knitr engine render...
...
(-) Checking Knitr engine render...
...
() Checking Knitr engine render...
...
(|) Checking Knitr engine render...
...
(/) Checking Knitr engine render...
[>] Checking Knitr engine render...
...OK

[cderv]

Thanks for the report:

This is seems to be an issue with table processing when the table is using cross-reference. If you remove the tbl- prefix in the label, the table will be the same as what kableExtra is producing

\begin{table}[H]
\centering
\begin{tabular}[t]{l>{\raggedleft\arraybackslash}p{30em}rrrrrrrrrr}
\toprule
  & mpg & cyl & disp & hp & drat & wt & qsec & vs & am & gear & carb\\
\midrule
\cellcolor{gray!10}{Mazda RX4} & \cellcolor{gray!10}{21.0} & \cellcolor{gray!10}{6} & \cellcolor{gray!10}{160} & \cellcolor{gray!10}{110} & \cellcolor{gray!10}{3.90} & \cellcolor{gray!10}{2.620} & \cellcolor{gray!10}{16.46} & \cellcolor{gray!10}{0} & \cellcolor{gray!10}{1} & \cellcolor{gray!10}{4} & \cellcolor{gray!10}{4}\\
Mazda RX4 Wag & 21.0 & 6 & 160 & 110 & 3.90 & 2.875 & 17.02 & 0 & 1 & 4 & 4\\
\cellcolor{gray!10}{Datsun 710} & \cellcolor{gray!10}{22.8} & \cellcolor{gray!10}{4} & \cellcolor{gray!10}{108} & \cellcolor{gray!10}{93} & \cellcolor{gray!10}{3.85} & \cellcolor{gray!10}{2.320} & \cellcolor{gray!10}{18.61} & \cellcolor{gray!10}{1} & \cellcolor{gray!10}{1} & \cellcolor{gray!10}{4} & \cellcolor{gray!10}{1}\\
Hornet 4 Drive & 21.4 & 6 & 258 & 110 & 3.08 & 3.215 & 19.44 & 1 & 0 & 3 & 1\\
\cellcolor{gray!10}{Hornet Sportabout} & \cellcolor{gray!10}{18.7} & \cellcolor{gray!10}{8} & \cellcolor{gray!10}{360} & \cellcolor{gray!10}{175} & \cellcolor{gray!10}{3.15} & \cellcolor{gray!10}{3.440} & \cellcolor{gray!10}{17.02} & \cellcolor{gray!10}{0} & \cellcolor{gray!10}{0} & \cellcolor{gray!10}{3} & \cellcolor{gray!10}{2}\\
\addlinespace
Valiant & 18.1 & 6 & 225 & 105 & 2.76 & 3.460 & 20.22 & 1 & 0 & 3 & 1\\
\bottomrule
\end{tabular}
\end{table}

[H] is added by kableExtra in the above.

When the tbl- prefix label is added, this activates cross referencable table in Quarto, which leads to a processing of the produce table. And this processing is currently broken as we now get in the tex output from the previous intermediate tex table the following

\begin{table}

\caption{\label{tbl-cars}Some cars}

\centering{

[H]
\centering
\begin{tabular}[t]{l>{\raggedleft\arraybackslash}p{30em}rrrrrrrrrr}
\toprule
  & mpg & cyl & disp & hp & drat & wt & qsec & vs & am & gear & carb\\
\midrule
\cellcolor{gray!10}{Mazda RX4} & \cellcolor{gray!10}{21.0} & \cellcolor{gray!10}{6} & \cellcolor{gray!10}{160} & \cellcolor{gray!10}{110} & \cellcolor{gray!10}{3.90} & \cellcolor{gray!10}{2.620} & \cellcolor{gray!10}{16.46} & \cellcolor{gray!10}{0} & \cellcolor{gray!10}{1} & \cellcolor{gray!10}{4} & \cellcolor{gray!10}{4}\\
Mazda RX4 Wag & 21.0 & 6 & 160 & 110 & 3.90 & 2.875 & 17.02 & 0 & 1 & 4 & 4\\
\cellcolor{gray!10}{Datsun 710} & \cellcolor{gray!10}{22.8} & \cellcolor{gray!10}{4} & \cellcolor{gray!10}{108} & \cellcolor{gray!10}{93} & \cellcolor{gray!10}{3.85} & \cellcolor{gray!10}{2.320} & \cellcolor{gray!10}{18.61} & \cellcolor{gray!10}{1} & \cellcolor{gray!10}{1} & \cellcolor{gray!10}{4} & \cellcolor{gray!10}{1}\\
Hornet 4 Drive & 21.4 & 6 & 258 & 110 & 3.08 & 3.215 & 19.44 & 1 & 0 & 3 & 1\\
\cellcolor{gray!10}{Hornet Sportabout} & \cellcolor{gray!10}{18.7} & \cellcolor{gray!10}{8} & \cellcolor{gray!10}{360} & \cellcolor{gray!10}{175} & \cellcolor{gray!10}{3.15} & \cellcolor{gray!10}{3.440} & \cellcolor{gray!10}{17.02} & \cellcolor{gray!10}{0} & \cellcolor{gray!10}{0} & \cellcolor{gray!10}{3} & \cellcolor{gray!10}{2}\\
\addlinespace
Valiant & 18.1 & 6 & 225 & 105 & 2.76 & 3.460 & 20.22 & 1 & 0 & 3 & 1\\
\bottomrule
\end{tabular}

}

\end{table}%

So our caption insertion is somehow broken here.

I though we fixed that previously, but it may have been something else...

[cderv]

@cscheid I believe this is another occurences of

• Raw LaTeX tables with cross-references no longer render #7236
  fixed by
• latex: fix 1.4 regression on parsing raw table syntax #7240

Our current regex used for floatref element

quarto-cli/src/resources/filters/modules/patterns.lua

Lines 38 to 41 in 163a059

	-- note the capture pattern here is different because we need to capture the
	-- body of the float environment as well as the environment itself
	local latex_table = "(\\begin{table})(.*)(\\end{table})"
	local latex_table_star = "(\\begin{table%*})(.*)(\\end{table%*})"

covers less cases than our others one

quarto-cli/src/resources/pandoc/datadir/init.lua

Lines 1711 to 1712 in 163a059

	local latexTableWithOptionsPattern = "(\\begin{table}%[[^%]]+%])(.*)(\\end{table})"
	local latexTablePattern = "(\\begin{table})(.*)(\\end{table})"

Especially we don't cover options like \begin{table}[H] so the match we do for the environment part before doing the caption is not correct IMO

quarto-cli/src/resources/filters/quarto-pre/parsefiguredivs.lua

Line 91 in 163a059

local b, e, begin_table, table_body, end_table = raw.text:find(patterns.latex_table)

For the non floatref case we were doing more test I believe

quarto-cli/src/resources/filters/quarto-pre/table-captions.lua

Lines 170 to 175 in 163a059

	elseif hasRawLatexTable(raw) then
	for i,pattern in ipairs(_quarto.patterns.latexTablePatterns) do
	if raw.text:match(pattern) then
	raw.text = applyLatexTableCaption(raw.text, tblCaptions[idx], tblLabels[idx], pattern)
	break
	end

quarto-cli/src/resources/filters/quarto-pre/table-captions.lua

Lines 198 to 215 in 163a059

	function applyLatexTableCaption(latex, tblCaption, tblLabel, tablePattern)
	local latexCaptionPattern = _quarto.patterns.latexCaptionPattern
	-- insert caption if there is none
	local beginCaption, caption = latex:match(latexCaptionPattern)
	if not beginCaption then
	latex = latex:gsub(tablePattern, "%1" .. "\n\\caption{ }\\tabularnewline\n" .. "%2%3", 1)
	end
	-- apply table caption and label
	local beginCaption, captionText, endCaption = latex:match(latexCaptionPattern)
	if #tblCaption > 0 then
	captionText = stringEscape(tblCaption, "latex")
	end
	if #tblLabel > 0 then
	captionText = captionText .. " {#" .. tblLabel .. "}"
	end
	latex = latex:gsub(latexCaptionPattern, "%1" .. captionText:gsub("%%", "%%%%") .. "%3", 1)
	return latex

I believe the two patterns (one without options and one with options) was needed because default Lua patterns does not support a ? operator for "maybe this".

Maybe another library would help here, but I believe we need to match \begin{table} and \begin{table}[H] and maybe more variation like patterns in init.lua are defining.

Reprex

---
title: "Table"
format: pdf
keep-md: true
keep-tex: true
---

::: {#tbl-test}

```{=latex}
\begin{table}[H]
\centering
\begin{tabular}[t]{l|r|r|r|r|r|r|r|r|r|r|r}
\hline
  & mpg & cyl & disp & hp & drat & wt & qsec & vs & am & gear & carb\\
\hline
Mazda RX4 & 21.0 & 6 & 160 & 110 & 3.90 & 2.620 & 16.46 & 0 & 1 & 4 & 4\\
\hline
Mazda RX4 Wag & 21.0 & 6 & 160 & 110 & 3.90 & 2.875 & 17.02 & 0 & 1 & 4 & 4\\
\hline
Datsun 710 & 22.8 & 4 & 108 & 93 & 3.85 & 2.320 & 18.61 & 1 & 1 & 4 & 1\\
\hline
Hornet 4 Drive & 21.4 & 6 & 258 & 110 & 3.08 & 3.215 & 19.44 & 1 & 0 & 3 & 1\\
\hline
Hornet Sportabout & 18.7 & 8 & 360 & 175 & 3.15 & 3.440 & 17.02 & 0 & 0 & 3 & 2\\
\hline
Valiant & 18.1 & 6 & 225 & 105 & 2.76 & 3.460 & 20.22 & 1 & 0 & 3 & 1\\
\hline
\end{tabular}
\end{table}
```

Caption

:::

[cscheid]

I think we should do something more drastic starting in Quarto 1.6, which is that if a raw html block contains table (a crossref environment), we should emit a warning, and say that this will stop working in the future.

[cscheid]

I believe the two patterns (one without options and one with options) was needed because default Lua patterns does not support a ? operator for "maybe this".

That's correct.

[cscheid]

Note, also, that we use patterns.latex_table in two places. The first is the one you mentioned:

quarto-cli/src/resources/filters/quarto-pre/parsefiguredivs.lua

Line 91 in 163a059

local b, e, begin_table, table_body, end_table = raw.text:find(patterns.latex_table)

But we also use it here:

quarto-cli/src/resources/filters/quarto-pre/parsefiguredivs.lua

Line 679 in 163a059

local b, e, begin_table, table_body, end_table = raw.text:find(patterns.latex_table)

[cscheid]

The more I look at this, the less I want to fix it.

I think we should just encourage users to not use HOLD_position and instead add tbl-pos: H.

@bhattmaulik, the following will work for you:

```{r}
#| echo: false  
#| label: tbl-cars
#| tbl-cap: "Some cars"  
#| tbl-pos: H
head(mtcars) |> kableExtra::kbl(booktabs = TRUE) |> kable_styling(latex_options = c("striped", "bordered"),full_width = FALSE) %>%   column_spec(2, width = "30em") 
```

[bhattmaulik]

Thank you @cscheid sir for this suggestion. I didn't know that tbl-pos: H is also a valid YAML field in quarto. For my case, I solved it by adding the following lines in the preamble of the LaTeX document:

\AtBeginDocument{
\floatplacement{table}{H}
}

But when other users also had the same issue, the issue was re-opened at the kableExtra repository. After that, I decided to open a separate issue here.
And thank you @cderv sir for your explanation of how quarto works under the hood. Quarto + LaTeX has been my use case as I teach statistics and R programming (especially with scrbook and other koma-script document classes). Your explanation helps me refine my own document template that I use regularly.

[mine-cetinkaya-rundel]

I also ran into this. It would also be good to add tbl-pos to the documentation at https://quarto.org/docs/reference/formats/pdf.html.

[cscheid]

I also ran into this. It would also be good to add tbl-pos to the documentation at https://quarto.org/docs/reference/formats/pdf.html.

This is an instance of the discussion we had last week about limitations of our autocompletion/validation. The problem is that we support more than just fig-pos, tbl-pos, lst-pos, etc. The prefix is whatever the prefix used by the crossreferencing environment, including custom crossrefs. So the YAML intelligence doesn't know ahead of time which are the ones that are valid.

In other words: yes, it's a bug. But fixing it is harder than it seems at first glance.