Yarn file spec: add file header object for storing file metadata, which encompasses all nodes #183

blurymind · 2020-05-18T19:51:54Z

@desplesda @hylyh @daviddq This has been bothering me for a while, but I was wondering if we can add an extra object to yarn files where we can give the file a bit more information about its identity and capabilities:

tagStyle: "html" | "bbcode" (used to automatically detect the tag style of the document)
textColourPalette: Array<hexString> (used to store the colour picker's accumulated coloursin the palette) - it will allow us to keep its palette history in the file
userVariables: Array<string> (used to store all variables that are used in the file, we can use it for autocompletion and better debugging when playtesting)
userCommands - same as userVariables but for commands
other metadata - such as author, language, last edited user etc

Right now a yarn json file looks like this:

[
	{
		"title": "Start",
		"tags": "err erg",
		"body": "A: Hey, I'm a character in a script!\n<<test>>\nB: And I am too! You are talking to me!\nB: What would you prefer to do next?\n\n[[Leave|Leave]]\n[[Learn more|LearnMore]]",
		"position": {
			"x": 283,
			"y": 207
		},
		"colorID": 0
	},
	{
		"title": "Leave",
		"tags": "ad ber",
		"body": "A: Oh, goodbye!\nB: You'll be back soon!",
		"position": {
			"x": 190,
			"y": 539
		},
		"colorID": 0
	},
]

a Yarn text file:

title: Start
tags: 
colorID: 0
position: 108,232
---
<b>Bold</b> text
<u>Underlined</u> text
Text in <i>italics</i> 
Text in <color=#ff0000>red color</color> and <color=#0000ff>blue color</color>
<color=#00ff00><b>Green and bold</b></color> text
<b><i>bold and italics</i></b> 
<b><u>bold and underlined</u></b> 
<i><u>italics and underlined</u></i>
<i><u><color=#e900ff>italics, underlined and pink</color></u></i> 
<img>image</img>
<<command>>
[[This link brings you to a new node:|link]]
===
title: link
tags: 
colorID: 0
position: 526,227
---
Empty Text
===

My proposal is to add a header to the file, maybe something like this:

[
	{
		"title": "YARN_HEADER",
		"tagStyle": "html",
                "textColourPalette": ["#E9967A", "#F08080"],
                "userVariables" : ["talkedToBird", "foundClock", "apples", "etc"],
                "userCommands": ["avatar", "speed",  "setTime"],
                "fileInfo": {
                       "language": "english",
                          ....
                  }
	},
	{
		"title": "Start",
		"tags": "err erg",
		"body": "A: Hey, I'm a character in a script!\n<<test>>\nB: And I am too! You are talking to me!\nB: What would you prefer to do next?\n\n[[Leave|Leave]]\n[[Learn more|LearnMore]]",
		"position": {
			"x": 283,
			"y": 207
		},
		"colorID": 0
	},
	{
		"title": "Leave",
		"tags": "ad ber",
		"body": "A: Oh, goodbye!\nB: You'll be back soon!",
		"position": {
			"x": 190,
			"y": 539
		},
		"colorID": 0
	},
]

a Yarn text file (not sure where to put the metadata yet):

title: YARN_HEADER
tagStyle: html,
textColourPalette: ["#E9967A", "#F08080"]
userVariables : "talkedToBird", "foundClock", "apples", "etc"
userCommands: "avatar", "speed",  "setTime"
language: english
 ...
===
title: Start
tags: 
colorID: 0
position: 108,232
---
<b>Bold</b> text
<u>Underlined</u> text
Text in <i>italics</i> 
Text in <color=#ff0000>red color</color> and <color=#0000ff>blue color</color>
<color=#00ff00><b>Green and bold</b></color> text
<b><i>bold and italics</i></b> 
<b><u>bold and underlined</u></b> 
<i><u>italics and underlined</u></i>
<i><u><color=#e900ff>italics, underlined and pink</color></u></i> 
<img>image</img>
<<command>>
[[This link brings you to a new node:|link]]
===
title: link
tags: 
colorID: 0
position: 526,227
---
Empty Text
===

The text was updated successfully, but these errors were encountered:

blurymind · 2020-05-18T20:02:09Z

of course the header data can be top level and all yarn nodes can be somehow nested under nodes or something, but a change such as this would require changing the parsers- ideally with keeping backwards compatibility

desplesda · 2020-05-18T22:19:41Z

Given that we removed JSON support from the language in Yarn Spinner v1.0, should this project file be in JSON?

blurymind · 2020-05-18T23:52:42Z

Bondagejs uses json files and so does gdevelop and ctjs because of that. We need to keep it for those and other html5 engines

…

On Mon, May 18, 2020, 11:19 PM Jon Manning ***@***.***> wrote: Given that we removed JSON support from the language in Yarn Spinner v1.0, should this project file be in JSON? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://github.com/YarnSpinnerTool/YarnEditor/issues/183#issuecomment-630465791>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABRRWVNYJ5UWRF2Z4NXRDWLRSGYAVANCNFSM4NEL5PGA> .

daviddq · 2020-05-19T05:56:36Z

Regarding the JSON file: adding the header as a node with different structure is dangerous. Loaders that are not prepared will try to read it as a normal node and they will explode trying to access members that don't exist. We could use a meta-node, identified by the "title" and put all metadata inside the body. That way loaders that don't know about the metadata will load an additional "messy" unconnected (and unreachable) node, but won't explode loading the file. On the editor we can recognize the meta-node, read and hide it.

Regarding the YARN file: safely modifying this format is a challenge. If this file already supports comments we could put all the meta as comments or put the meta inside the body.

Long term proposal:

Start versioning the files:

YARN

version: 2
===
tagStyle: html,
textColourPalette: ["#E9967A", "#F08080"]
userVariables : "talkedToBird", "foundClock", "apples", "etc"
userCommands: "avatar", "speed",  "setTime"
language: english
 ...
===
title: Start
tags: 
colorID: 0
position: 108,232
---
<b>Bold</b> text
<u>Underlined</u> text
...

JSON

{
  version: '2',
  meta: {
    "tagStyle": "html",
    "textColourPalette": ["#E9967A", "#F08080"],
    "userVariables" : ["talkedToBird", "foundClock", "apples", "etc"],
    "userCommands": ["avatar", "speed",  "setTime"],
    ...
  },
  nodes: [
	{
		"title": "Start",
		"tags": "err erg",
		"body": "A: Hey, I'm a character in a script!\n<<test>>\nB: And I am too! You are talking to me!\nB: What would you prefer to do next?\n\n[[Leave|Leave]]\n[[Learn more|LearnMore]]",
		"position": {
			"x": 283,
			"y": 207
		},
		"colorID": 0
	},,
...
}

Be able to import/export all versions from the editor. That way, authors can keep using old versions for their old productions and switch to the new version whenever they want. Note that they will be able to load a "version 1" file and save it as "version 2".

blurymind · 2020-05-19T14:12:51Z

Given that we removed JSON support from the language in Yarn Spinner v1.0, should this project file be in JSON?

Json is the parsing standard for html5/javascript game engines. Why did you decide to drop it :) That could be a problem for swapping out bondagejs for yarnSpinner on gdevelop if yarnspinner can one day run in a js app

desplesda · 2020-05-19T14:19:01Z

We’re approaching a point where we need to have a conversation about the structure of the Yarn project structure, and how this editor relates to the other Yarn Spinner project structures. @blurymind and @daviddq, we should probably have a chat to make sure that we don’t accidentally misalign the projects soon - what’s the best place to talk, Twitter, Slack, etc?

daviddq · 2020-05-19T14:52:24Z

Slack, Discord, Hangouts, IRC, ... 🤷

desplesda · 2020-05-19T14:58:10Z

Probably the best way would be for you both to join the Yarn Spinner slack? (It’s currently 1AM for me so right now‘s not the best time, but it’s a great place to start the conversation from!) https://lab.to/narrativegamedev

daviddq · 2020-05-19T15:28:33Z

I joined. Any preferences:

blurymind · 2020-05-20T16:32:51Z

I'm in the UK and can join after work (18:00 uk time)

daviddq · 2020-05-21T13:45:52Z

This has been bothering me for a while, but I was wondering if we can add an extra object to yarn files

Additionally to the solution proposals I provided before, we can have additional files that hold the meta data (user/workspace/project related metadata)

desplesda · 2020-05-21T13:49:54Z

A simple text file format that encodes key-value pairs, and doesn’t introduce any package dependencies (either in Yarn Editor, bondage, or Yarn Spinner) would be ideal. Something easy to parse like ini files, maybe.

I’d suggest that this be opened as an issue in the Yarn Spinner repo, as a proposal to enhance the format.

blurymind · 2021-06-08T18:49:07Z

I added a v2 yarn JSON file type, which looks as follows:

old type (note how the root is an array - huge problem for expand ability):

[
	{
		"title": "Start",
		"tags": "err erg",
		"body": "A: Hey, I'm a character in a script!\n<<test>>\nB: And I am too! You are talking to me!\nB: What would you prefer to do next?\n\n[[Leave|Leave]]\n[[Learn more|LearnMore]]",
		"position": {
			"x": 283,
			"y": 207
		},
		"colorID": 0
	},
	{
		"title": "LearnMore",
		"tags": "rawText",
		"body": "A: HAHAHA\nBlah blah more..",
		"position": {
			"x": 534,
			"y": 534
		},
		"colorID": 0
	}
]

new type (root is an object - suddenly a much more flexible file):

{
	"header": {
		"lastSavedUnix": "2021-06-08T19:13:17.277Z",
		"language": "en-GB",
		"markupLanguage": "bbcode",
		"filetypeVersion": "2"
	},
	"nodes": [
		{
			"title": "Start",
			"tags": "err erg",
			"body": "A: Hey, I'm a character in a script!\n<<test>>\nB: And I am too! You are talking to me!\nB: What would you prefer to do next?\n\n[[Leave|Leave]]\n[[Learn more|LearnMore]]",
			"position": {
				"x": 312,
				"y": 123
			},
			"colorID": 0
		},
		{
			"title": "LearnMore",
			"tags": "rawText",
			"body": "A: HAHAHA\nBlah blah more..",
			"position": {
				"x": 534,
				"y": 534
			},
			"colorID": 0
		}
	]
}

type is detected by the root being an array (OLD) or an object (NEW)
You can also select your preferred version number in settings

I am not sure how to do this for ordinary yarn text file, since that has no nesting like json and xml do - the stucture we save in it will have to be flat, but if we can agree on one - I would be eager to implement it.

blurymind changed the title ~~Yarn file spec: global settings object~~ Yarn file spec: global settings object for file metadata May 18, 2020

blurymind changed the title ~~Yarn file spec: global settings object for file metadata~~ Yarn file spec: add file header object for storing file metadata, which encompasses all nodes May 18, 2020

JujuAdams mentioned this issue Jan 21, 2021

[Discussion] Process to update Yarn Editor to the Yarn 2.0 spec? #236

Closed

FaultyFunctions added enhancement New feature or request question Further information is requested and removed question Further information is requested labels Jan 22, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Yarn file spec: add file header object for storing file metadata, which encompasses all nodes #183

Yarn file spec: add file header object for storing file metadata, which encompasses all nodes #183

blurymind commented May 18, 2020 •

edited

blurymind commented May 18, 2020

desplesda commented May 18, 2020 via email

blurymind commented May 18, 2020 via email

daviddq commented May 19, 2020 •

edited

blurymind commented May 19, 2020 •

edited

desplesda commented May 19, 2020

daviddq commented May 19, 2020

desplesda commented May 19, 2020

daviddq commented May 19, 2020

blurymind commented May 20, 2020

daviddq commented May 21, 2020

desplesda commented May 21, 2020

blurymind commented Jun 8, 2021 •

edited

Yarn file spec: add file header object for storing file metadata, which encompasses all nodes #183

Yarn file spec: add file header object for storing file metadata, which encompasses all nodes #183

Comments

blurymind commented May 18, 2020 • edited

blurymind commented May 18, 2020

desplesda commented May 18, 2020 via email

blurymind commented May 18, 2020 via email

daviddq commented May 19, 2020 • edited

blurymind commented May 19, 2020 • edited

desplesda commented May 19, 2020

daviddq commented May 19, 2020

desplesda commented May 19, 2020

daviddq commented May 19, 2020

blurymind commented May 20, 2020

daviddq commented May 21, 2020

desplesda commented May 21, 2020

blurymind commented Jun 8, 2021 • edited

blurymind commented May 18, 2020 •

edited

daviddq commented May 19, 2020 •

edited

blurymind commented May 19, 2020 •

edited

blurymind commented Jun 8, 2021 •

edited