SCDoc: add various punctuation marks for non-code text and comments in code

[prko]

This PR allows you to insert various quotation marks, ellipses and brackets using predefined character combinations.

Purpose and motivation

When writing educational material using SCdoc, I needed to use

• various quotation marks for non-code, text other than ' and ",
• an apostrophe for non-code text other than ',
• various brackets for non-code text other than () and []: brackets to indicate the year, brackets for additional explanation, and brackets to indicate the origin of terms (most Korean computer music terms are translations of European terms, and often an English expression is translated into a different expression when used for a different situation).

Inserting such punctuation from the character map is somewhat time-consuming and reduces productivity.
To solve these problems and improve productivity, I created this PR.

Types of changes

• Documentation
• New feature

To-do list

• Code is tested
• All tests are passing
• Updated documentation
• This PR is ready for review

[prko]

Converted PDF from the corresponding reference in SCDoc included in this PR:

• Various punctuation marks for non-code text _ SuperCollider 3.14.0-dev Help.pdf

• Various punctuation marks for non-code text _ SuperCollider 3.14.0-dev Help with custom css.pdf

[JordanHendersonMusic]

A part of #6260

[prko]

@JordanHendersonMusic
The bugs described in #6278 can be avoided with this PR.

The complete list of the same bug in built-in core schelp files:
#6278 (comment)

[prko]

In this PR, _ is used to enable various punctuation marks, but a word starting with _ is related to primitives by its colour. This can be a bit confusing for developers or other users who know these relationships, it may be better to use / instead of _.

[prko]

Converted PDF from the corresponding reference in SCDoc included in this PR:
Non-code punctuation marks and symbols _ SuperCollider 3.14.0-dev Help.pdf

[telephon]

why not instead of SCDocHTMLRenderer.renderNonCodePunctuation(this.escapeSpecialChars(string))

this.renderNonCodePunctuation(this.escapeSpecialChars(string))?

But also I find it hard to unsderstand what kind of transfromations are applied to what kind of section. Why has \CODE no renderNonCodePunctuation?

It would be good if you could try and give intention revealing names so that all in all we improve the clarity and not make it even harder.

[prko]

why not instead of SCDocHTMLRenderer.renderNonCodePunctuation(this.escapeSpecialChars(string))

this.renderNonCodePunctuation(this.escapeSpecialChars(string))?

I followed your instructions and updated all the relevant parts. When I started creating the PRs, I used the format this.newMethodInMyPR. However, I didn't use this. when I was calling more than two methods. I cannot remember the reason exactly, but the form you recommended is better! Thanks!

But also I find it hard to unsderstand what kind of transfromations are applied to what kind of section. Why has \CODE no renderNonCodePunctuation?

I did not add renderNonCodePunctuation to \code, \codeblock etc. because I thought that special characters could be used in comments, but that they were not so useful. I have now added renderNonCodePunctuation to \code, \codeblock etc.

It would be good if you could try and give intention revealing names so that all in all we improve the clarity and not make it even harder.

I will do it! Thanks for pointing it out.

[telephon]

Thanks! Yes, if you think about the intent of each of the string manipulations (also those that you haven't touched), maybe you can find a clear reasoning. Something like:

prepareStringForCode
prepareStringForNonCode

these could then include the escapeSpecialChars and renderNonCodePunctuation.

By the way "render" implies that something is done to convert to graphics. This is not really the case here, so the above would be better maybe.

But:

I am now a little wary about all the special syntax this introduces in the helfile. Already now it is really hard if you don't know it, and making it more complicated won't help much in people being able to read the source of help docs and learning from them how to write them.

Maybe the new LaTeX support might make a clearer and more powerful way of doing this?

[prko]

Thanks! Yes, if you think about the intent of each of the string manipulations (also those that you haven't touched), maybe you can find a clear reasoning. Something like:

prepareStringForCode
prepareStringForNonCode

these could then include the escapeSpecialChars and >renderNonCodePunctuation.

I changed the name of the method from .renderNonCodePunctuation to .prepareStringForNonCode (I need time to think about how to combine it with escapeSpecialChars.):

Instead of using .renderNonCodePunctuation, I think .prepareStringForNonCode seems appropriate. Um, and the current method, escapeSpecialChars, only replaces three characters, &, < and >, which prevent corruption when converting SCDoc contents to HTML... So you can add them to prepareStringForNonCode.

By the way "render" implies that something is done to convert to graphics. This is not really the case here, so the above would be better maybe.

In other related PRs I have used the method names pareHTML and parseMathML. In this PR I have used the term .renderNonCodePunctuation because the font size changes when some brackets are used:
https://github.com/prko/supercollider/blob/014a425e28343822cf43460be9800256a417fa63/SCClassLibrary/SCDoc/SCDocRenderer.sc#L57-L62

		.replace(/*"%E2%9D%AA"*/"/f(/", "<span style='font-size: 0.85em'>❪") // MEDIUM FLATTENED
		.replace(/*"%E2%9D%AB"*/"/f)/", "❫</span>")
		.replace(/*"%E2%9D%A8"*/"/m(/", "<span style='font-size: 0.85em'>❨") // MEDIUM
		.replace(/*"%E2%9D%A9"*/"/m)/", "❩</span>")
		.replace(/*"%E2%9D%B2"*/"/s(/", "<span style='font-size: 0.71em'>❲") // TORTOISE SHELL
		.replace(/*"%E2%9D%B3"*/"/s)/", "❳</span>")

However, I was not sure if the method name was appropriate. Thanks for pointing this out!

---

Maybe the new LaTeX support might make a clearer and more powerful way of doing this?

I am intending to include the following characters:

		.replace("&hsp;",    "ㅤ")
		.replace(" ",   " "  ) // space.
		.replace(" ",    " "   ) // space.
		.replace(" ", " ") // hair space.  
		.replace(" ",  " " ) // narrow no-break space.
		.replace(" ",   " "  ) // en space.  
		.replace(" ",   " "  ) // em space.  
		.replace("&emsp16;", " " ) // 6-per-em space.
		.replace(" ", " ") // 4-per-em space.  
		.replace(" ", " ") // 3-per-em space.  
		.replace(" ", " ") // thin space.  
		.replace(" ",   " "  ) // non breaking space.  
		.replace(" ", " ") // punctuation space.  
		.replace(" ",  " " ) // figure space.  
		.replace("ㅤ", "ㅤ") // Hangul Filler.

The various spaces above and some of the characters included in the PR whether they are linked with the size of characters or not, can not be used via KaTes if I correctly understand KaTeX....

I am now a little wary about all the special syntax this introduces in the helfile. Already now it is really hard if you don't know it, and making it more complicated won't help much in people being able to read the source of help docs and learning from them how to write them.

Even though I made this PR, I am not sure which way is better:

• Copy special characters from the character map
  (since SC-IDE does not have its own character map, the system's built-in character map should be used, but the workflow is not as different as using typesetting tools)
• Place the symbols using the method described in this PR.

What I can say is,

• In LaTeX and Mathematica, the special symbols are entered via code, not by selecting characters from the character map.
• I switched from Finale to Dorico in 2018. I was a long-time Finale user. I find Dorico's workflow much more intuitive and faster. One reason for this is Dorico's popover, which allows users to place all musical symbols by typing in words, similar to what I did in this PR.

If someone wanted to write the following paragraph in SCD right now

Sie können jeden einzeiligen Ausdruck ausführen, indem Sie auf eine beliebige Stelle in der Zeile klicken und die „Enter“-Taste drücken. Bitte beachten Sie, dass die „Enter“-Taste nicht mit der „Return“-Taste identisch ist.

" and " should be copied from the character map or the text should be written using e.g. Microsoft Word.

However, if this PR is approved, " and " can easily be expressed in SCDoc.
So I think it would be a good idea to discuss making it easier to abbreviate each symbol. However, I cannot push this idea if none of the main developers agree. I also understand what you are thinking, as I was also very hesitant before making this PR.