Code does not show up in pdf since v5.1.0 #10719
Comments
|
Thanks for reporting. It would have been easier with a smaller sample project but here is the cause of the problem: the produced \usepackage[draft]{minted}
\fvset{breaklines=true, breakanywhere=true}I guess this comes from Jupyter Book template. If you want to fix this at your locale try to update your post-processing of tex file to remove those two consecutive lines, I expect this will cure the problem, can you please confirm later? The
However note the following: all those things are pinning Sphinx to The above came from I had to modifiy the latex font config because Inconsolate is not at my locale. Finally, |
jfbu
changed the title
|
I notice file The is a way compatible with HTML output (via MathJaX, untested) and PDF via LaTeX (tested see screenshot). With it PDF will look like Else, the rendering in pdf is: and the console output of latex run complains: For bold math, maybe this tex.sx answer is useful especially this comment although it dates somewhat (2017). Alternative might be for LaTeX (untested here) using EDIT: regarding |
jfbu
changed the title
|
Wait! After some digging where I grepped throughout the executablebooks works and was not finding minted I finally found out the lines with \usepackage[draft]{minted}
\fvset{breaklines=true, breakanywhere=true}Remove the lines. Now if you want linebreaks at every character in code cells you should modify line You may also consider adding OTHER thing : I must investigate why with However I will now close this issue as not due to a Sphinx bug. |
|
Thanks to your reporting I became aware of a bug of 5.1.0 #10723 which will be fixed in next release (code-blocks always with frame border even with |
@astonzhang Update. I checked the documentation of https://github.com/gpoore/fvextra (which is used for the So, after dropping
You can, if you so desire... (if you work in genomics with long DNA strains then you will want to do that... ;-) ). |
|
To sum up the original report is explained by an incompatibility showing up at Sphinx 5.1.0 with a preamble edit https://github.com/gpoore/minted added a first version of Please take note that:
For this reason I will mark this incompatiblity with extra LaTeX preamble material as "wont-fix". |
|
@humitos I notice your thumbs up so I am pinging you to read this again because my initial comment was written at a stage when I had misunderstood the structure of the project and thought it was using jupyter-book as its sole document builder. As I explained in other comments I was right that the |
|
@jfbu Many thanks for diving deep into this issue! I just removed and lifted the sphinx version constraint in d2l-ai/d2l-book@cbfbdf4 Now the code shows up in our latest preview PDF using http://preview.d2l.ai.s3-website-us-west-2.amazonaws.com/d2l-en/master/d2l-en.pdf e.g. (note the red arrow for wrapping up),
|
Thanks! I agree that boldsymbol should be used for greek symbols. Would you like to send a PR to D2L: https://github.com/d2l-ai/d2l-en/ ? We acknowledge every community fix: https://d2l.ai/chapter_preface/index.html#acknowledgments :) |
|
Besides, a major pain point for our generated PDF is that it's hard to distinguish code input and output: Actually in earlier versions of Sphinx (in 2018), it's easier to distinguish input and output. See an example below with "
Since you mentioned configurations of |
I will give a look to your d2l-book processing. You can always add LaTeX code locally to redefine VerbatimColor either via an injected Maybe the following is relevant:
|
Good. This is the expected rendering when Sphinx wraps long code lines. Since Sphinx 5.1.0 in case that happens near bottom of page, the very long wrapped code lines can continue at top of next page. Prior, it did not (see #8686). For customization of the little red arrow check https://www.sphinx-doc.org/en/master/latex.html#the-sphinxsetup-configuration-setting, particularly |
|
@astonzhang The fact that you see a border around the code-block with Sphinx 5.1.0 despite |
|
@astonzhang Since Sphinx 4.1.0 (released Jul 12, 2021) the rST i.e. using rst mark-up, an input such as: will render in LaTeX as Now you only need to add to preamble a suitable So in your case if you can create classes say should achieve your goal. In place of Now, it you can't do that but have some means to insert code in latex output before input or output cell, all you need to do is inject at right places local usage of For example you only need to find a way to inject such things. But the "container class" idea is probably simpler as it can be done without worrying if for HTML or LaTeX output. BTW, I get the feeling HTML input cells already have some such CSS class, but I am too little equipped and could not check how output cells look because of I also have various problem with PyTorch at my locale... |
|
Happy Saturday! Our Sphinx-built book is being used by many professors and students who prefer reading on PDF (eye-friendly). Since our code input block and code output block use the same style ( Thanks for looking into this problem. Let me unblock you to evaluate our notebooks for code output generation on your locale (cd ../d2l-en):
Then you will get a very small pdf that only consists of the Preliminaries chapter with both code input and code output. |
|
For more context on our d2lbook, it starts from input .md files with code and here is how these source files on our GitHub repo are transformed into output HTML and PDF (see our doc: https://book.d2l.ai/develop/pipeline.html): Thus, when you run The specific build pipeline for pdf generation is:
|
|
@jfbu Before discussing the solution, let's see what rst and tex files look like using sphinx 5.1.1. For example, in
Unfortunately, in
So, even if we can edit the .tex file (actually I did post-edit the above d2l-en.tex file with this script for other ad-hoc hacking) by following your suggestion below
it's still very hard to tell which Therefore, is it possible to show some special tag like Besides, my tex-processing script is too hacky and strongly depends on sphinx output/syntax. For example, if sphinx changes any latex-specific syntax in the future, my script will break... Any suggestions are welcome. Thanks! |
|
@astonzhang and will now get some sleep... I encountered PyTorch problems on my macos x such as pytorch/pytorch#36941, pytorch/pytorch#20030, https://stackoverflow.com/questions/61525299/cannot-import-torch-image-not-found and will address those also tomorrow. (after making the mistake to start some brew install stuff; I will rather go the macports way and possibly use I will examine build pipeline tomorrow too. The https://github.com/spatialaudio/nbsphinx processing is able to define all needed environment wrappers to customize locally the sphinxVerbatim. as a last resort I have some crazy idea about hijacking Sphinx LaTeX writer handling of captions for code-blocks. We can provide a title only to signal for example that this is an output cell. Then modify the Sphinx latex macro to not at all prepare a title but rather change VerbatimColor. but this is no good, because probably if we can do that while keep same source for both HTML and PDF outputs, which clearly must be a requirement, then we have a way to inject surely some LaTeX code at the right place. Update, now at 35:45 build time As you can see, I don't have access to a mainframe or even a decently fast computer... |
Quick reply: Can you simply remove that linear-algebra.md file and the corresponding linear-algebra entry from https://github.com/d2l-ai/d2l-en/blob/master/chapter_preliminaries/index.md? Will that just make your computer feel happier? :) Good night! Update: @jfbu To allow you to play with d2lbook without relying on pytorch/tensorflow/mxnet, I just created a new sphinx-latex branch containing the minimal code to test any crazy idea. Could you replace all your d2l-en files with those from the sphinx-latex branch on your locale? Then it takes < 1 min to execute
The corresponding
and the
Actually, if you run
All in all, now we'll need to make the code output |
|
Thanks for the trimmed-down set-up which was indeed much helpful. The build was stalled yesterday due to PyTorch not finding libomp, an issue I will address perhaps later at my locale. edit: PyTorch officially does not support my too old mac os x; besides I don't have a GPU and don't have access to cloud services from my non-financed location. So that's pretty much all to it until I decide to buy some new hardware as noone will do it for me... Finally I understand PyTorch originates in FaceBook AI team, which I do not intend to share anything with for reasons of ethics. Try this: diff --git a/d2lbook/rst.py b/d2lbook/rst.py
index 43819f4..faec73a 100644
--- a/d2lbook/rst.py
+++ b/d2lbook/rst.py
@@ -110,7 +110,12 @@ def _process_rst(body):
break
j += 1
i = j
+ elif line.startswith('.. code::'):
+ # reset LaTeX code-block rendering parameters
+ lines[i] = '.. raw:: latex\n\n \\diilbookstyleinputcell\n\n' + lines[i]
elif line.startswith('.. parsed-literal::'):
+ # reset LaTeX code-block rendering parameters
+ lines[i] = '.. raw:: latex\n\n \\diilbookstyleoutputcell\n\n' + lines[i]
# add a output class so we can add customized css
lines[i] += '\n :class: output'
i += 1
diff --git a/d2lbook/sphinx_template.py b/d2lbook/sphinx_template.py
index f662bdb..8f16a5d 100644
--- a/d2lbook/sphinx_template.py
+++ b/d2lbook/sphinx_template.py
@@ -100,9 +100,46 @@ MONO_FONT
\fancyhead[LE,RO]{{\py@HeaderFamily }}
}
\makeatother
+
+% Defines macros for code-blocks styling
+\definecolor{d2lbookOutputCellBackgroundColor}{RGB}{239,254,255}
+\definecolor{d2lbookOutputCellBorderColor}{RGB}{204,204,204}
+\def\diilbookstyleoutputcell
+ {\sphinxcolorlet{VerbatimColor}{d2lbookOutputCellBackgroundColor}%
+ \sphinxcolorlet{VerbatimBorderColor}{d2lbookOutputCellBorderColor}%
+ \sphinxsetup{verbatimwithframe,verbatimborder=3pt}%
+ }%
+%
+\definecolor{d2lbookInputCellBackgroundColor}{rgb}{.95,.95,.95}
+\def\diilbookstyleinputcell
+ {\sphinxcolorlet{VerbatimColor}{d2lbookInputCellBackgroundColor}%
+ \sphinxsetup{verbatimwithframe=false,verbatimborder=0pt}%
+ }%
+% memo: as this mark-up uses macros not environments we have to reset all changed
+% settings at each input cell to not inherit those or previous output cell
+% memo: Sphinx 5.1.0, 5.1.1 ignore verbatimwithframe Boolean, so for this
+% reason we added an extra verbatimborder=0pt above.
''',
-'sphinxsetup': 'verbatimwithframe=false, verbatimsep=2mm, VerbatimColor={rgb}{.95,.95,.95}'
+
+'sphinxsetup': '''verbatimsep=2mm,
+ VerbatimColor={rgb}{.95,.95,.95},
+ VerbatimBorderColor={rgb}{.95,.95,.95},
+ pre_border-radius=3pt,
+ ''',
}
+# memo: Sphinx 5.1.0+ has a "feature" that if we don't set VerbatimColor to
+# some value via the sphinxsetup key or via \sphinxsetup raw macro, it
+# considers no colouring of background is required. Above we by-passed usage
+# of \sphinxsetup, because \sphinxcolorlet was more convenient. So we set
+# VerbatimColor in 'sphinxsetup' global key to work around that "feature".
+# The exact same applies with VerbatimBorderColor: it has to be set at least
+# once via 'sphinxsetup' or via \sphinxsetup raw macro else frame is black.
+#
+# memo: the Sphinx 5.1.0+ added pre_border-radius must be used in 'sphinxsetup'
+# (it can be modified later via extra raw \sphinxsetup)
+# because at end of preamble Sphinx decides whether or not to load extra package
+# for rendering boxes with rounded corners. N.B.: pre_border-radius is
+# unknown in Sphinx < 5.1.0 and will cause breakage.
SPHINX_CONFIGS
At my locale with Sphinx 5.1.0 and with current Sphinx 5.x it will produce
I used a 5.1.0 added feature of rounded corners, simply remove the (I don't have Inconsolata at my locale so I used FreeMono which is font in code cells in above screenshot) Indeed your setup adds an "output" class, but one would have to modify sphinx/sphinx/writers/latex.py Lines 1751 to 1800 in c2f1f89 to let the LaTeX writer detect the class in the node and react to it. The above patch is not good style. I considered adding a usage of About
Indeed it is no good to rely on Sphinx output syntax... I don't have specific suggestion as I have not looked in details at what your post-processing script does. I see now you e.g. add a |
|
@jfbu Thanks a ton!! |
|
@astonzhang you are welcome ;-)
.. parsed-literal::
def foo():
return None
.. parsed-literal::
def foo():
return *None*and here is its rendering to HTML (using Notice that the first instance was handled exactly as if ...it is very peculiar that one can not be sure fully in advance of what happens. Here is what sphinx/sphinx/writers/html5.py Lines 403 to 408 in e47ef71 Now, in your use case it seems always the Notice how some keywords are highlighted, but this highlighting is a bit strange in this context. My suggestion is that perhaps you may consider replacing the lines with lines. The current code incorporating #10719 (comment) looks elif line.startswith('.. parsed-literal::'):
# reset LaTeX code-block rendering parameters
lines[i] = '.. raw:: latex\n\n \\diilbookstyleoutputcell\n\n' + lines[i]
# add a output class so we can add customized css
lines[i] += '\n :class: output'
i += 1and may thus be modified to do the replacement. This will remove any highlighting but keep the architecture for background colouring, addition of a frame, both in HTML and PDF. However it will inihibit rendering of actual reST mark-up for emphasis or boldness or for embedded links. I thus don't know what is best for d2l-book. |
|
In future we may incorporate #10736 suggestion. If this was already in place your issue with distinguishing code-blocks in PDF would have been much easier to handle: you already add an "output" class for HTML, and with #10736 suggestion the LaTeX is aware of that class. it is only a matter to define in |
I am not sure this is related to Sphinx. The prompts are not in the At my locale in the |
I have identified an instance of this phenomenon at https://book.d2l.ai/user/create.html#project-from-scratch Consider the following screenshot from this html web page rendering:
This has no highlighting. In the PDF output we can see it has no background colouring and no frame. It is an instance where the Sphinx mechanism decided that .. [1]: for reducing Sphinx LaTeX console output refer our docs The rst source looks like this So this is indeed an example where <pre class="output literal-block">[d2lbook:config.py:L12] INFO Load configure from config.ini
[d2lbook:build.py:L143] INFO 0 notebooks are outdated
[d2lbook:build.py:L149] INFO Evaluating notebooks in parallel with 4 CPU workers and 4 GPU workers
[d2lbook:build.py:L53] INFO === Finished "d2lbook build eval" in 00:00:02
[d2lbook:build.py:L289] INFO 0 rst files are outdated
[d2lbook:build.py:L53] INFO === Finished "d2lbook build rst" in 00:00:02
Running Sphinx v5.1.1which we can contrast with how the previous output cell is rendered <div class="output highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">d2lbook</span><span class="p">:</span><span class="n">config</span><span class="o">.</span><span class="n">py</span><span class="p">:</span><span class="n">L12</span><span class="p">]</span> <span class="n">INFO</span> <span class="n">Load</span> <span class="n">configure</span> <span class="kn">from</span> <span class="nn">config.ini</span>If Trying out the rst in a Sphinx dummy project I receive a bunch of warnings (7 of them) where the I identified that the trigger is the presence of the backtick So these lines trigger warnings about bad mark-up !!! But there are other things. Lines such as these ones also contribute to Sphinx using the parsed literal branch and not the code-block branch: Finally I identified after some search that presence of this in the contents by itself causes that Sphinx HTML and LaTeX writers do not style as highlighted block! The cause is the backslash in ...this is good illustration of what I have always considered a strange handling by Sphinx of edit: I mean it is surprising that Ideally the conversion tool from |
Great suggestion. I'll take a look at the whole book and decide if to keep |
Agreed. Please keep me posted of this change. Once it's available in a later version of Sphinx we may make our d2lbook implementation cleaner. |
Ideally it would be Globally it looks like a rather complex topic... |
Describe the bug
Given the following latex content produced by Sphinx from jupyter notebooks:
sphinx==5.0.2 produces pdf nicely:
However, sphinx==5.1.0 produces pdf without code:
How to reproduce this bug
Modify https://github.com/d2l-ai/d2l-en/blob/master/config.ini#L32 as
so that code is not evaluated for generating output (thus no need to install more libraries to run code & save time in reproducing the error).
Then run
Finally, access the output pdf under
_build/pdf/d2l-en.pdfExpected behavior
No response
Your project
https://github.com/d2l-ai/d2l-en/
https://github.com/d2l-ai/d2l-book
Screenshots
No response
OS
Linux
Python version
3.8
Sphinx version
5.1.0
Sphinx extensions
No response
Extra tools
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: