Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SCDoc improvements: 3. Some HTML tags are enabled. #6265

Open
wants to merge 10 commits into
base: develop
Choose a base branch
from

Conversation

prko
Copy link
Contributor

@prko prko commented Apr 22, 2024

This pull request enables you to incorporate HTML tags in order to enhance the appearance of the rendered HTML in both the web browser and SC Help Browser. Additionally, "\n" has been added to certain tags to make the rendered HTML more readable when viewed in a text editor.

Purpose and Motivation

While working on writing educational material using SCdoc, I encountered the need to use HTML tags in order to improve the readability of the printed SCDoc. I used these tags to add empty lines and control line height, and so on.

Some opinions have already been expressed in PRs #6260 and #6263. I am aware that some believe I may be doing things incorrectly. However, I am doing this in order to complete the task of dividing PR #6254.

This is a revised PR that addresses the relevant part of PR #6254. I plan to keep this PR open for several weeks to gather more feedback on my work.

Types of changes

  • Documentation
  • New feature

To-do list

  • Code is tested
  • All tests are passing
  • Updated documentation
  • This PR is ready for review

…adability of rendered HTML

Some HTML tags are enabled. "\n" is added to some tags to improve the readability of rendered HTML.
typo correction & removing unnecessary part
to place image with resize attribute
@JordanHendersonMusic JordanHendersonMusic added the comp: SCDoc scdoc syntax, parser, and renderer. for changes to schelp files, use "comp: help" label Apr 24, 2024
Internal and External links for iframe, audio, video and image now work
more examples and typo correction
@prko
Copy link
Contributor Author

prko commented Apr 24, 2024

updated again (deleted previous ones)

Converted PDF from the corresponding reference in SCDoc included in this PR:

@capital-G
Copy link
Contributor

I really appreciate the effort here and I see similarities to your proposed #6304 , but using replace to replace generated SCDoc HTML rendered code seems a hacky way of doing it as in this case we work against the output of SCDoc instead of telling it to "ignore" it.

For example, <p class="foo"> and < p class="foo"> are both valid HTML p tags, but would not be the same in the current output, and parsing HTML even with the power of regex is not possible, although a limited set is possible.

Maybe there should be a focus on how to extend the SCDoc syntax to include a raw or html tag which simply (by-) passes any HTML to the generated file.

So, instead of working against the SCdoc generator, we can just work with it :)

The definition of the SCdoc syntax is stated here
Lexical definition: https://github.com/supercollider/supercollider/blob/develop/SCDoc/SCDoc.l
Grammar definition: https://github.com/supercollider/supercollider/blob/develop/SCDoc/SCDoc.y

which are used by Bison to generate a parser, see https://www.oreilly.com/library/view/flex-bison/9780596805418/ch01.html

I think it should be feasible to include a new tag here, if you are interested in this we can take a look at this together, but currently the build_parser.sh is broken (at least on my machine), maybe we should add this to the CI in general?

@smoge
Copy link
Contributor

smoge commented May 10, 2024

Maybe there should be a focus on how to extend the SCDoc syntax to include a raw or html tag which simply (by-) passes any HTML to the generated file.

I thought about it today about an HTML:: tag that bypasses the renderer. It's an idea.

@prko
Copy link
Contributor Author

prko commented May 10, 2024

@smoge

I thought about it today about an HTML:: tag that bypasses the renderer. It's an idea.

Wow, I think it can simplify a lot of things! I really like it!


@capital-G

The definition of the SCdoc syntax is stated here
Lexical definition: https://github.com/supercollider/supercollider/blob/develop/SCDoc/SCDoc.l
Grammar definition: https://github.com/supercollider/supercollider/blob/develop/SCDoc/SCDoc.y
I think it should be feasible to include a new tag here, if you are interested in this we can take a look at this together

I have modified the files you refer to several times over the past year to implement subsubsection, but to no avail.

Recently @smoge also recommended learning Bison and Flex...
I replied that I can not learn them because I do not know C++. Now @capital-G is recommending the same thing.... When two people on the forum recommend the same thing in the same week... Hard to say no again...

In between

  • @smoge uploaded a draft on modal tags.
  • I found the story of how to remove the math:: tag at ......

OK! Modifying these files without exact knowledge is the reason for non-success. After I finish some urgent personal stuff, I will have a look at it.

currently the build_parser.sh is broken (at least on my machine), maybe we should add this to the CI in general?

I do not know what build_parser.sh and what CI is... However, if you do I will try to learn it!


@smoge @capital-G
Thank you for helping me and giving me a great idea!

@prko
Copy link
Contributor Author

prko commented May 13, 2024

@capital-G
First, I want to know if I understand how to use build_parser.sh.
Could you check if I have fixed the problem correctly?

  1. install flex and bison
    done in Ubuntu 22.04.4 LTS
  2. Edit SCDoc.l and SCDoc.y in https://github.com/supercollider/supercollider/blob/develop/SCDoc
  3. run the following code in a terminal window: ./build_parser.sh in the folder on the local machine of the same folder above.
  4. I got the following error:

    (base) parallels@ubuntu-linux-22-04-desktop:~/Downloads/supercollider/SCDoc$ ./build_parser.sh
    SCDoc.y:54.1-14: warning: deprecated directive: ‘%error-verbose’, use ‘%define parse.error verbose’ [-Wdeprecated]
    54 | %error-verbose
    | ^~~~~~~~~~~~~~
    | %define parse.error verbose
    SCDoc.y: warning: fix-its can be applied. Rerun with option '--update'. [-Wother]

  5. I did the following: bison --update SCDoc.y, and got the following error:

    (base) parallels@ubuntu-linux-22-04-desktop:~ /Downloads/supercollider/SCDoc$ bison --update SCDoc.y
    SCDoc.y:54.1-14: warning: deprecated directive: ‘%error-verbose’, use ‘%define parse.error verbose’ [-Wdeprecated]
    54 | %error-verbose
    | ^~~~~~~~~~~~~~
    | %define parse.error verbose
    bison: file ‘SCDoc.y’ was updated (backup: ‘SCDoc.y~’)

  6. I replaced the text in line 54 with %define parse.error verbose
  7. re-run: ./build_parser.sh

@smoge
Copy link
Contributor

smoge commented May 13, 2024

It was a minor deprecation and easily fixable with bison SCDoc.y --update

Also, the commented line that compiles with gcc, works with a minor change.

I had heard about this "breakage" before. But it does not seem to be the case. It's all right.

@prko
Copy link
Contributor Author

prko commented May 13, 2024

@smoge @capital-G
I have success in implementing html:: modal tag in the same way of teletype::.

Here I have a new question. The followings for example seems not so fun to typing:

  • html::<br>::
  • html::<br><br>
  • html::<hr>::
  • html::<p>::

etc...

I think it would be better to make a poll for the new modal tags.
For example

  • html::<br>:: -> lin::1::
  • html::<br><br>:: -> lin::2::
  • html::&nbsp;:: -> sp::1::

I also would like to have a tag for table with <thead> and <th>.

I think @capital-G would wish some modal tags related to this....
How do you think about it?

@prko
Copy link
Contributor Author

prko commented May 14, 2024

  • html::<br>:: -> lin::1::
  • html::<br><br>:: -> lin::2::

The followings seem to be better:

  • html::<br>:: -> br::l1::
  • html::<br><br>:: -> br::l2::

Also:

  • html::<p style='page-break-before: always;'>:: -> br::pgb::

@capital-G
Copy link
Contributor

capital-G commented May 14, 2024

I think we should limit this more towards a <div> or context-based approach here instead of introducing all possible features of HTML through it. SCDoc should be independent of HTML features as otherwise we limit SCDoc for all time to come to be viewed only within a browser environment.

We could nonetheless include something like admonitions in myst or rst with an either limited or open set of possible classes.

Something like

context:: math
    \int_0^\inf = \sin(x)~dx
::

would then be converted to

<div class="scdoc-context math">
    \int_0^\inf = \sin(x)~dx
</div>

This makes it possible and very easy to edit such divs via JS or CSS and adjust/modify their style or content.

Edit: To include arbitrary HTML content (this should be limited to HTML exclusive features such as embedding a video via iframes etc!) we could also add

html::
<iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/dQw4w9WgXcQ?si=WEomThZlWcYABnhC" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
::

which would be simply forwarded/passed as

<iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/dQw4w9WgXcQ?si=WEomThZlWcYABnhC" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

@smoge
Copy link
Contributor

smoge commented May 14, 2024

Yes, let's follow the KISS principle. Let's add one or two tokens to the grammar, allowing more expressivity on the SCLang side. It will be better for future ideas and adaptations. Bison works just fine; I don't think it's 'archaic. ' It's a good tool. But grammars are a hard part of programming. Changes will always be more complicated.

@prko
Copy link
Contributor Author

prko commented May 15, 2024

We could nonetheless include something like admonitions in myst or rst with an either limited or open set of possible classes.

context:: math
\int_0^\inf = \sin(x)~dx
::

html::

<iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/dQw4w9WgXcQ?si=WEomThZlWcYABnhC" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> ::

Yes, these are good!

I already use HTML in SCDoc using the SCHTMLRenderer in this PR.

There is a problem: by using HTML I should type in too much code to use <style> (often also <span>). You can see examples in the attached PDF at the end of this post. For the user's coding experience, row HTML is flexible but not so amusing. To solve this, we should add more CSS definitions in SCDoc.css, linked to the modal tags. Of course, users can add custom.css if they want to customise some CSS definitions.
Moreover, I think now this PR, #6263 and #6266 should be merged by adding the following modal tags.

Thus, we need the following modal tags:

The following PDF is a guide to all these features.
It is written with the wrong implementation because I did not know about adding new modal tags.
But it helps to understand what I mean.

Advanced SCDoc features _ SuperCollider 3.14.0-dev Help.pdf

Is there anything I have missed?
Do we need to make a poll on the scsynth forum about this?

@capital-G
Copy link
Contributor

capital-G commented May 15, 2024

As mentioned above, I personally would strongly avoid any HTML beyond an iframe - though there are always exceptions to the rule. (Relying too heavily on HTML will loose the ability for export schelp to latex/md and other formats).

By implementing a context (represented by wrapping the content inside a div that has a class as an attribute), it is possible to implement most of what you ask for.
Additional functionality or styles can then be implemented via js and css for that context, and can be reviewed/discussed on an atomic basis.
If these features don't get accepted, but you feel the need to share them with a group of people, you could simply distribute a frontend.js and custom.css file for the group.

In general, I don't think that development efforts within SuperCollider are best spent on making SCDoc equal to other documentation or typesetting tools. If the community agrees that there is an urgency to include such features, then we should replace SCDoc with a tool dedicated to documentation like doxygen.

@prko
Copy link
Contributor Author

prko commented May 15, 2024

In general, I don't think that development efforts within SuperCollider are best spent on making SCDoc equal to other documentation or typesetting tools. If the community agrees that there is an urgency to include such features, then we should replace SCDoc with a tool dedicated to documentation like oxygen.

I would like to mention that the community here are all experts in using SC and many of them are professional level programmers. They are not so dependent on the SuperCollider help document as they often understand just by looking at the implementation of the methods.

However, my recent PRs are for my students who are outside the community. They are studying various subjects, are not musicians (I used to teach musicians, mainly composers, but not anymore) and are learning Audacity, Reaper and programming with SC for the first time in their lives.

I do not use PDF or other tools to present code because some characters can be converted, so errors can occur when evaluating pasted code. I mainly use SCD files and my own tutorials in Korean written in SCDoc. SCD files are good for code, but not good for writing explanations as comments. SCDoc is good for linking the help documents and other help documents. The advantage of using SCDoc is that you can embed images. However, it is not very eye-friendly and its readability needs to be improved by adjusting the lead (spacing between lines), table width, line width between items, etc. The current fonts are also not very good for Korean.

The use of the online documentation system is also very limited. There are computers with wired internet in the lecture room, but many students use their own laptops with Wi-Fi, but the Wi-Fi connection in the room is very unstable.

I do not think there is an urgent need to implement such features for the community.
I think implementing the features I recently suggested will be much faster than replacing Doxygen (<- Oxygen was a typo).

However, I think it would not be a bad idea to start a poll to replace SCDoc with Oxygen. What do you think? If I make a poll, it will be ridiculous because I am not competent to replace SCDoc with Oxygeon.

Or would it be better to close all my recent PRs related to this?

(added:)
I updated #6266. The subsubsection seems to be correctly implemented now.
I will do what @capital-G recommends. Should I add only div and context in the same way of teletype?

@prko
Copy link
Contributor Author

prko commented May 16, 2024

I have updated #6266. The subsubsection now seems to be implemented correctly.
Regarding this PR, I will do as @capital-G recommends. Could you give a specific list of what you think should be added? If you could be generous enough to include some of the above, I would appreciate it. But if not, it is not a problem as I understand what you have written.

@capital-G
Copy link
Contributor

@prko

However, I think it would not be a bad idea to start a poll to replace SCDoc with Oxygen. What do you think? If I make a poll, it will be ridiculous because I am not competent to replace SCDoc with Oxygeon.

I suppose you mean Doxygen?
I don't think we need to actually replace SCDoc as it still has it charms and history but also it limitations, but therefore also some perks that would most likely get lost on a transition.

I think the recent PRs showed that there is still interest in SCDoc and we will also hopefully find a way to have create a way to improve it over time, but please be aware that SCDoc will never become a perfect typesetting tool as this is out of scope. (BTW, doxygen is also not a typesetting tool^^)

Maybe a bit OT but if you are interested in writing course material I can maybe recommend a tool I wrote https://github.com/capital-G/sc_kernel which allows you to write Jupyter Notebooks with sclang.
Combined with jupyter book you can combine these notebooks into a book which can be exported to a HTML webpage or a PDF via tex.

An example of this can be found
https://capital-g.github.io/musikinformatik-sose2021/00_basics/sc_dimensions.html
where the source code is located at
https://github.com/capital-G/musikinformatik-sose2021

The Problem in general

I think the DOM structure of the SCDocs HTML could be improved, as it is currently almost impossible to get hold of specific sections of a document via JS and CSS and adjust their style, because the DOM is essentially flat (and not deterministic) rather than nested within divs which create namespaces.

image

Normally every kind of section and element (e.g. a paragraph) should be nested with a div.
But introducing this would also break the existing CSS.
I think this could be justified as this could lay a foundation for really new looks on the documentation which is currently really limited due to this lack.

But instead of going for such a big step I think it is best to implement something less drastic and would be useful even even if we have a proper nested DOM by allowing to create a context wrapper explicitly.

Proposition

Context

I think #6265 (comment) already states how admonitions work within MyST/RST and this could be mimiced to create a context syntax within SCDoc.

Something like

context::{foo bar}
this is a context
::

should then result in

<div class="foo bar">
  this is a context
</div>

Via scdoc.css we could then address this contexts via

.foo {
  background-color: red;
}

.bar {
  font-weight: bold;
}

The set of global classes could then be discussed in an atomic basis.
It is basically a generalization of the NOTE and WARNING boxes which could be replaced via

context::{warning}
this is a warning
::

If you could extend the SCDoc parser to include the parsing of the context including classes like above it would be really great!!

Maybe this could also allow to enter raw HTML for elements such as iframe

context::{raw-html}
<iframe>...</iframe>
::

via js

    // pseudo-code
    for (let element of document.getElementsByClassName("raw-html")) {
        const text = element.textContent;
        element.innerHTML = element.textContent;
    }

Maybe there could be a syntax to create an inline-context via span classes, but I think this should be something for a 2nd PR.

@prko
Copy link
Contributor Author

prko commented May 17, 2024

@capital-G
Thanks, I got it!

Do I need to add the modal tag context in the same way as teletype? Or would you like to do it, since it is your idea? I do not care who does it. What you suggest is much better than what I did in this PR!


Why have I tried to improve SCDoc?

  • Copying and pasting code from external sources can change the characters in the code, including whitespace.
  • Using an external application with SC is a bit clumsy.
  • The SCD file cannot contain images, tables, etc. Before Qt, this was fine as it has rtfd format!
  • SCDoc can contain images, tables, lists and many other things, but there are some missing features.
  • Allowing students to customise SCDoc using my /SCClassLibrary/SCDoc/SCDocRenderer.sc file and /HelpSources/custom.css takes too long.

Adding the modal tag context in SCDoc seems to resolve most of these things!
In any case, #6263 seems to be closed...


I suppose you mean Doxygen?
(BTW, doxygen is also not a typesetting tool^^)

Yes, it was a typo. Sorry!

From my experience,

  • Using non-standard characters, as suggested in SCDoc: add various punctuation marks for non-code text and comments in code #6263, is not convenient in typesetting tools either.
  • Adjusting the layout page by page is a lot of work, with slight changes in lead, font width, paragraph spacing, and spacing between the list (including images, tables, etc.) and the paragraphs above and below.
  • For me, there is not much

Maybe a bit OT but if you are interested in writing course material I can maybe recommend a tool I wrote https://github.com/capital-G/sc_kernel which allows you to write Jupyter Notebooks with sclang.
Combined with jupyter book you can combine these notebooks into a book which can be exported to a HTML webpage or a PDF via tex.

I am already familiar with your great tool and I like to use it. However, more than 90% of my students use Windows and very few of them use MacOS. In 10 years of teaching experience, I only meet 1 or 2 students who use Linux. So I cannot use your wonderful tool with my students. I think that a teacher and his students should use the same tool to avoid many technical problems.

@prko prko mentioned this pull request May 19, 2024
4 tasks
writing style change of the method added in this PR
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp: SCDoc scdoc syntax, parser, and renderer. for changes to schelp files, use "comp: help"
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants