How to group parts?

[cecilios]

I was looking at the new section for Parts to the MNX-Common by Example page and it raised the question of how to group parts (the MusicXML part-group elements). I've been looking in the specification and unless I've missed something, I have not found any reference to this.

I would appreciate some comments on this or what ideas are you considering for representing groups. Thank you.

In my oppinion grouping parts is a very important organizational element and this could affect current high level MNX structure, and should be addressed sooner than details of the specification.

[notator]

Yes. I also think this needs fixing.
§4.1 says that a Part can have one or more staves that "relate(s) to a single performer or group of performers". That can't be right.
A Part defines a group of staves, connected by a single barline. Voices can move from one of these staves to another. If, for example, the Part was a group of woodwind instruments, should a bassoon voice on a bassoon staff really be allowed to move to a clarinet staff?
Note also that in §6.11 (and §6.1.9 Example 10) a Part can have "part-name" and "part-abbreviation" attributes, but there is no provision for naming individual staves. So there's currently no way to represent a group of woodwinds connected by a single barline.

[clnoel]

What about choral pieces that sometimes have the SATB each on a separate staff, and sometimes have the female voices on one staff and the male voices on one staff? You have to be able to move a "voice" from one staff to another to correctly represent the original.

-- Christina

[notator]

Sorry, I'm still trying to understand this myself.
Probably I was wrong, and §4.1 is correct:

Each part is a grouping of related musical material that relates to a single performer or set of performers.

and all that's missing is the ability to name particular staves and/or voices. (§6.11 looks a bit unfinished...)

§4.5 says (my emphasis)

[...] A voice is a set of sequences in different measures, but within the same part. The sequences within this set can be thought of as constituting a single musical voice throughout the score. Thus they are an organizational construct, rather than a notational one.
A given voice need not be expressed in every single measure of a part. It may be present in some measures, and absent in others.

According to §6.2.2, sequences can have both a cross-measure "voice" attribute (so four sequences of "sequence" elements in the same Part could be labelled as belonging to voices S, A, T and B) and a "staff" attribute denoting the staff index (from top to bottom in the Part). So @clnoel 's example should be no problem.
However, as far as I can see, we are not told explicitly how to order the voices on each staff.
Presumably, as with staves, voices should ALWAYS occur in the XML in the same order as they are to appear in the score (top to bottom).

Apropos: I had a problem with MNX-Common by Example, Multiple Voices. In that example, the lower voice is presented first, so it should really have its stems pointing upwards. I had to go through a few hoops to make the example look like the given graphic. I think that voices MUST be presented in top to bottom order, otherwise things get unnecessarily complicated, -- and that the example does not actually match the graphic.

Summary of my current understanding (is this right?):

1. Sequences belong to the same voice if they have the same voice identifier.
2. A particular voice does not have to have a sequence in every measure in the score, but a particular voice identifier means the same voice whenever it occurs in the score.
3. Voice identifiers in "sequence" elements (hence voices) are simply a technique for defining layout. They have no necessary relation to the voice's content (Klangfarbe etc.)
4. We need to be able to give a printed name to each voice and/or staff. How else could it be made clear where the voices move when they move from two staves (S+A, T+B) to four staves (S,A,T,B) and back?

Is the distinction between (non-printed) voice identifier and (printed) voice name the distinction needed to solve Issue #183? Is the (internal MNX-Common) voice identifier "layout", and the voice name (printed in the score) "analytical"?

[clnoel]

However, as far as I can see, we are not told explicitly how to order the voices on each staff.
Presumably, as with staves, voices should ALWAYS occur in the XML in the same order as they are to appear in the score (top to bottom).

If we alter this to not knowing how to order the staves in the system, then I agree here. Voices should appear on staves wherever their associated notes should appear. But the ordering of the staves is a layout issue that needs to continue to be addressed.

Whenever the system layout changes we will need labels. This occurs at the beginning of the piece, anywhere the arrangement of voices on staves changes, anywhere the number of visible staves changes, and probably some other places that an editor chooses to put one in for clarity.

[notator]

@clnoel I'm not sure I understand you there.
According to §4.1:

Staves in CWMN are identified within a part by a unique staff index. The topmost staff in a part has a staff index of 1; staves below the topmost staff are identified with successively increasing indices.

I think things should be kept as simple as possible: voices in staves, staves in parts, parts in systems and systems in the score should all be consistently organised: The order in which the elements occur in the XML should be the top to bottom order in the score. Any of these orderings (except systems in the score) can change at any point in the score/XML.
But for that to work we need staff/voice names (in the score)...

[clnoel]

Oh... arg... I skipped the "part" in the hierarchy. I meant staves ordered within the part not within the system. Parts would be ordered within the system. My point, though, is that voices don't order in the staff. Once a voice is in a staff, it can't really have a vertical order relative to other voices, because the note sequences might cross each other musically.

[notator]

I don't see any problem with voices crossing musically on a staff. An upper voice (the one with stems up) can have pitches below those of the lower voice (that has stems down).
I'd like to suggest that the top-bottom order of voices in a part should remain the same during the whole score, regardless of which staff (inside the part) they are written on.
To extend your example of (S+A, T+B) changing to (S, A, T, B): If S and T were tacet for a few measures, A and B could be notated on a single staff, but their top-bottom order still would not change (i.e. A would be notated above B on the staff).
I think its common practice for the notated top-bottom order of instruments per system not to change during a score, even if instruments sometimes get left out or are moved to different staves.

[notator]

This issue is collecting TODO items for the voice to system hierarchy, and may end up as a Pull Request, so here's a summary of the items so far (comments and corrections welcome, of course):

• It must be possible to give each voice in a score a printed name (§6.2.2, §6.11). This is in addition to the "voice identifier" used to clarify voice leading when the voice is anonymous (e.g. when a voice changes staves on a piano's grand staff).
• The top to bottom order of voices (in each system of a score) should never change, either in the XML or in the printed score. (Note that not all voices have to be present all the time, so the "top to bottom order" can always have gaps). This means that:
  • the top to bottom order of Parts (in XML and score) should be fixed (in §6.1.7)
  • voices should always be presented in top to bottom order inside each Part, regardless of the staff (inside the Part) to which they are assigned (§6.11).
  • sequence elements must be presented in top to bottom order inside Measures (§6.2.1). (Voices are defined as sequences of Sequence elements across different Measures. Voice identifiers are used for disambiguation when the number of voices changes.)

I'd like to discuss whether there should be a maximum number of voices per staff.
(Collision checking gets exponentially more difficult as the number of voices per staff increases, so it might encourage more software developers to read/write standard MNX-Common if the requirement to support unlimited numbers of voices per staff were removed. Unlimited voices per staff could be allowed in some other profile.)
My own feeling is that standard MNX-Common should have a maximum of two voices per staff. Each voice is rhythmically independent of the others so, in the above SATB example, the two voices could then only be named "S+A" and "T+B" if each just consisted of simple two-part chords.

[bhamblok]

@notator I disagree on limiting the number of voices per staff. There are tons of examples in literature where there are more than 2 voices on a staff (to name most of Bach's preludes & fugues as an example). In choral music, there are also lot's of examples where all voices are again split up in multiple voices like "SSAATTBB", and still visualised on 2 staves.

[notator]

@bhamblok you're right, of course, about the Bach examples, but I'm still not sure that this belongs in the standard profile. As I said, I think this needs discussing.
Can you give an example of SSAATTBB visualized on 2 staves with more than 2 (rhythmically independent) voices per staff?

[adrianholovaty]

@notator This GitHub issue has gotten quite a ways off-topic...! Regarding multiple voices, I don't see a good reason to limit the number of voices in an MNX-Common file.

To the original issue topic (how to group parts), this still needs to be designed. My gut take on this is that we'd have a separate section, somewhere in the document, which would let us specify the groups by listing IDs. (For example, "this group consists of parts P1 and P2, and it uses a bracket.")

[cecilios]

@adrianholovaty Thanks for your thoughts. They are in line with current MusicXML approach. Days after opening this issue I found #33 that seem to address the same concerns but, probably, with a very different implementation. I still think that this issue should receive more priority because depending on the final decision it could affect significantly to current MNX skeleton.

[notator]

@adrianholovaty Okay, granted: the question of how many voices per staff there should be in the standard profile is off-topic. The idea really belongs in the debate about profiles (#68, #175), so I'll save it for when we get back to those issues.

But the rest of the above thread, containing a review of the container hierarchy, is very much on-topic. (@cecilios asked for more discussion around the subject of grouping parts, because the result might affect the basic structure of MNX-Common documents. The container hierarchy is the basic structure of an MNX document.) The current state of this discussion is that

• Parts should not be grouped by putting them in another container, and
• Parts should be ordered (in the XML) in their top to bottom order in the score. (I have yet to hear a convincing argument as to why the Parts should be an unordered collection.)

This affects the way brackets can be defined. You said:

My gut take on this is that we'd have a separate section, somewhere in the document, which would let us specify the groups by listing IDs. (For example, "this group consists of parts P1 and P2, and it uses a bracket.")

Here are some thoughts to follow that up:

• The information could go in global measure directions element(s) (where we know how many Parts currently exist).
• It must be possible to define "nested" brackets and their type (e.g. a curly piano bracket for a single piano Part, left of a square bracket that encompasses the piano part and other Parts. Or a square bracket, grouping two trumpet staves, left of a larger square bracket encompassing the whole brass section.) Maybe that can be done by implication: Brackets that overlap are simply printed with the small one to the left of the longer one.
• Do we need more than two bracket types ("curly", "square") ?
• If Parts are always in a fixed top to bottom order, then bracket definitions only need to include the top and bottom part IDs. If the Parts are in random order, then you need to define the bracket using an unordered set of part IDs (which would be error prone, and make the tracking of bracket heights more difficult when Parts disappear and reappear).
• This means that Parts MUST have an ID. It could either be a mandatory ID attribute in the Part definition (§6.1.9, §6.11), or there could be an automatic allocation of part IDs (maybe the string "part" followed by an integer that indicates the top to bottom position of the parts in the score). N.B.: The Part ID is not the same as the part's name, because there could be more than one part with the same name (e.g. "Violin").

BTW: I think #33 could now be closed.

[notator]

Sorry but, looking closely at the problem of extracting Parts, I've come to the conclusion that the above review of the container hierarchy must be wrong.

I now think that the Spec is only talking about anonymous voices that can be played by one performer (so the voices don't need names). A Part should be an entity that can be extracted from a score to be played by one performer.
As I pointed out above, §4.1 is wrong after all, when it says:

Each part is a grouping of related musical material that relates to a single performer or set of performers.

The spec needs to distinguish between different staff types that occur at different levels in the container hierarchy:

• A Part (for a single performer) can contain more than one part-staff. A part-staff can contain more than one (anonymous) voice. For example, a piano grand staff consists of two part-staves.
• A system-staff can contain more than one (named) Part (e.g. 8 Horns). Each system-staff can contain more than one voice, each of which can be associated with (contain?) more than one (named) Part.

If this is correct, a "part-group" can be a set of system-staves connected by a single barline.

So far, so good, but a problem arises, because MNX-Common is designed for flexible layout.
Given an orchestra score containing eight horns (horn Parts) for example:
There is no information in the XML as to where the system breaks happen, so there can be no information as to how many system-staves to use for the eight horns on each system. Also, there can be no information in MNX-Common as to how to name the system-staves in the score to say which Parts they apply to.
These things can neither be imported into, nor exported from, instantiating applications. They can only exist in the instatiations themselves. The instantiating applications (or their users) have to provide this information themselves.

Nevertheless, it should still be possible to declare the fixed, top to bottom order of the (single performer) Parts in the global measure directions, and to define (nested) brackets (independently of the group-barlines) that enclose them...

Still working on this.

[clnoel]

Okay, if we're still looking at making definitions, I want to talk about the following visual groupings we see in a piece of music.

1. Staff - a set of horizontal lines, where musical notation can be arranged for display
2. Grand Staff - two staves vertically linked by a curly brace, and sharing measure barlines, used to facilitate a part that has enough range to be confusing when put on a single staff. (e.g. piano or organ)
3. Grouped Staves - a set of staves or grand staves connected by a curly or square brace, and not sharing measure barlines, used to indicate that the parts involved are closely linked. (e.g. SATB voices or 8 Horns).
4. System - a set of staves, grand staves, and/or grouped staves connected by a vertical system barline, representing every part in the current layout.

I believe that what I am calling "Grouped Staves" above is the source of the original question (Part-Groups), especially in regards to how this might affect the MNX-Common hierarchy.

If that understanding is correct, then in my opinion, "Grouped Staves" are entirely a layout question, as is the ordering of the staves in the system, and as such are completely separate from the musical/notational definition of the notes in the parts, just as the number of measures per system are separate (because they depend on page size). And we should not be worrying about it right now, because we are trying to separate out such layout concerns into their own section rather than mixing them in. This way, the same MNX document can be easily used to either faithfully reproduce the original by using that layout information, or to easily order/reorder or selectively include whichever parts are wanted by disregarding it.

Which doesn't mean that the parts don't need a name to be used for display. Probably they need both a full name and an "abbreviated" name, such as "Soprano" and "S". The layout specification can then choose when/how to display these names.

If we are ready to start talking about how to map parts onto staves to create layouts, then I'm more than willing to help dig into it, but I think we're not there yet, considering that we are still defining how repeats and second endings work when we only have one part.

--Christina

[cecilios]

Thanks @clnoel , a very clear explanation.

I believe that what I am calling "Grouped Staves" above is the source of the original question (Part-Groups),

Yes, that were my concerns.

in my opinion, "Grouped Staves" are entirely a layout question

I see three approaches to this:

• The semantic approach suggested by @joeberkovitz at Support part groups #33
• Your approach, a layout question to be solved later
• The MusicXML approach as suggested by @adrianholovaty at How to group parts? #185 (comment)

I'm not voting or suggesting any approach. My concerns are that it is necessary to define which approach will be choosen so that we all are aware of possible implications, instead of proceeding with details and later, when finally this be studied, having to go back or having to implement 'hacks' to avoid having to go back.

[clnoel]

Actually, I just realized that #57 (page and system flow) is in active review right now, so we do need to be worrying about this now!

My point about this being a layout issues is that in a complex orchestral piece, you might have the 8 horns part-group as a grouped stave on the conductor's score, be printed out as the entire layout for the horns themselves, or ignored entirely in order to produce a score that is just one of the 8 horns. And all of those are valid options for which it would be nice to not have to have separate MNX files for. Instead, we specify each of the 8 horn parts separately, and define score-layout elements for the composer, for the horn group, and for each horn separately.

Each of these different score-layouts would treat the group in different ways. So, if we have something like:

<score-layout name="Full Score">
    <system-layout name="Grouped Parts">
		<part-layout part="Violin 1"/>
		<group-parts type="bracket">
			<part-layout part="Horn 1"/>
			...
			<part-layout part="Horn 8"/>
		</group-parts>
		...
		<part-layout part="Piano" type="Grand"/>
	</system-layout>
	<page-layout page-number="1">
		<system system-layout-id="Grouped Parts" start-measure-id=1 end-measure-id=4 show-name="Full"/>
	</page-layout>
	<page-layout page-number="2">
		<system system-layout-id="Grouped Parts" start-measure-id=5 end-measure-id=10 show-name="Abbreviated"/>
	</page-layout>
	...
</score-layout>
<score-layout name="Horn Section">
	<system-layout name="Horn Solo">
		<part-layout part="Horn 1"/>
	</system-layout>
	<system-layout name="All Horns">
		<part-layout part="Horn 1"/>
		...
		<part-layout part="Horn 8"/>
	</system-layout>
	<page-layout page-number="1">
		<system system-layout-id="Horn Solo" start-measure-id=1 end-measure-id=4 show-name="Full"/>
		<system system-layout-id="Horn Solo" start-measure-id=5 end-measure-id=8 />
		<system system-layout-id="All Horns" start-measure-id=9 end-measure-id=12 show-name="Full"/>
	</page-layout>	
	...
</score-layout>
<score-layout name="Horn 1">
	<system-layout name="Horn 1">
		<part-layout part="Horn 1" combine-full-measure-rests=true/>
	</system-layout>
	<page-layout page-number="1" show-name="Full">
		<system system-layout-id="Horn 1" start-measure-id=1 end-measure-id=4/>
		<system system-layout-id="Horn 1" start-measure-id=5 end-measure-id=8/>
		<system system-layout-id="Horn 1" start-measure-id=9 end-measure-id=12/>
		...
	</page-layout>
	...
</score-layout>
<score-layout name="Horn 2">
	<system-layout name="Horn 2">
		<part-layout part="Horn 2" combine-full-measure-rests=true/>
	</system-layout>
	<page-layout page-number="1" show-name="Full">
		<system system-layout-id="Horn 2" start-measure-id=1 end-measure-id=12/>		
		...
	</page-layout>
	...
</score-layout>

There are a lot of things in this proposal that are out of scope for this discussion (and in fact, I should probably post it over at #57 ) but one thing it definitely does is show how we can group parts. Also, I'm not attached to the naming conventions for any of this, just the general idea.

[cecilios]

@clnoel Nice approach! I like it

[clnoel]

@notator I'd especially like your input. I've posted an expanded version at #57 (comment) .

[notator]

@clnoel Many, many thanks for

• pointing out that Provide a way to encode system breaks and page breaks #57 exists and is in active review.
• posting some real code that we can discuss! :-)

I've got a lot to say about the code, but its rather off topic here so I'll continue that discussion in #57, as you suggest.

I think we can finish with this issue (#185) when we're really sure what we mean by a "Part" and a "group of Parts".

Further to my last posting, I now think that a "Part" should be defined in terms of booklets rather than performers: A "Part" is the smallest sub-unit of an MNX-Common file that can be printed in a separate booklet. (All the Parts defined in the file having the same number of measures as the Full Score.) Booklets containing just one, or an arbitrary combination of Parts can be defined and printed.
Here are some examples, to illustrate what I mean:

• When one hires the score and parts for a classical symphony, one gets maybe six copies of the Viola Part. There's only one Viola Part defined in the MNX-Common file, but it can be printed several times, and played by more than one player. Even if there is only one Part defined for the Violas, some of its systems may contain more than one concurrent (part-)staff (divisi Violas).
• Not all the Parts defined in an MNX-Common file need to be printed in the Full Score. It would be perfectly possible to include a Part containing a piano reduction of the Full Score in the file. Such a Part would not be included in the Full Score itself but, for example, could be included with the choir Part(s) to create a score that can be used in rehearsals.
• Cues: It should be possible to define cues (Parts written on small staves) that can be included in booklets containing instrumental Parts. The same cue could very well be useful in different instrumental Part-booklets.

@cecilios, @clnoel: I think we are very close to agreement on the answer to this issue's question: "How to group Parts?".
The simple answer is to draw a continuous barline through all the (system-)staves that contain the Parts, and define appropriate brackets to the left of the system.
But this still needs refining a bit: I'm not sure what you think, but I suspect that part-staves should not be joined by the same barline. Groups of Parts containing Parts that have part-staves should only be indicated using the brackets on the left of the system. Barlines that join part-staves are important for pointing out single players to a conductor, so should not be overwritten.
For example: If a piano Part (on a grand staff) is part of a percussion group, should there be a single barline that goes through all the percussion Parts and the piano's grand staff? My inclination is to say no, and that the group should only be indicated by the bracket(s) to the left of the system.
What do you think?

Now for #57! :-)

[cecilios]

@notator:

For example: If a piano Part (on a grand staff) is part of a percussion group, should there be a single barline that goes through all the percussion Parts and the piano's grand staff?

In my opinion this should be like in MusicXML: an option in the group definition.

[notator]

@cecilios Yes. Agreed. Good idea!

[clnoel]

@notator Agreed on the "booklet" idea. That would also resolve the way I was struggling to define the soprano "part" of an SATB piece when there is a point at which the SA line has three notes. Especially if there is only one such place throughout the piece (like the final note). It didn't feel right to have to define a second-soprano part for the whole piece because of that one divisi, but we had restricted part to one performer. I like the idea of thinking in "booklets" although I'm not sure I'm attached to the name itself.

(off-topic: I have a choral background, not a composer/symphonic one, so I tend to think in terms of those pieces of music!)

@cecilios Yes. Attaching a barline attribute to the group-parts tag should do that. single-part or cross-part should be sufficient options.

[cecilios]

@cnoel

single-part or cross-part should be sufficient options.

And what about Mensurstrich barlines? Probably the set of options should be that of MusicXML.

[clnoel]

@cecilios 🤦
Oops. Despite having been very involved in a Mensurstrich conversation not that long ago, it completely slipped my mind. You're right. The MusicXML group-barline enumeration covers all three options. Does anyone ever connect parts with dashed/dotted barlines even when the part's barlines are solid?

@notator What about percussion parts? In a full score they are frequently done on an instrument-by-instrument basis, but there are usually far fewer performers than the number of potential instruments. I can't imagine anyone ever putting out a "booklet" for a triangle part even though it is a stave in the conductor score. Although I guess you could.

[notator]

Thanks @clnoel for the link to the group-barline enumeration. Those options should, of course, also be supported by MNX-Common.
They don't, of course, answer the question as to whether the system-staff barlines should override the part-staff barlines. That's an option which, as @cecilios said in #185 (comment), should go in the group definition. I've made a note to do that in my next posting to #57. Has anyone got a link to a corresponding option in MusicXML?

I can't imagine anyone ever putting out a "booklet" for a triangle part even though it is a stave in the conductor score.

Particular Parts don't have to appear in every system (that's something I'm looking at for #57)... :-)

Its up to the author of the file to decide how to organise percussion parts. Printed booklets can be defined to contain any combination of Parts, and the number of staves in each Part can vary from system to system (possibly from measure to measure). Presumably, its possible to define the number of stafflines in a staff (I haven't checked).
Percussion booklets should especially benefit from the flexibility of this scheme: The same physical triangle(s) can be defined to be a Part that exists in different booklets at different times. You could make a booklet for just the triangle(s) if they were intended for a particular player, or if they were placed a long way from the other instruments and the players needed to walk a long way to get to them without carrying booklets about...

[clnoel]

@notator I was responding to what you said:

A "Part" is the smallest sub-unit of an MNX-Common file that can be printed in a separate booklet.

Although I just think I was being nit-picky. Because, as you point out, you could print just the triangle part if you really wanted to.

I agree about the flexibility of having these score-layouts for percussion. The conductor can get his one-line staves with the percussion parts on it, and the percussionists themselves can get whatever other notation they want, like having the triangle part appear on a staff with another part, except with a triangle-shaped notehead.

(Off-topic: I've seen some weird percussion pieces come through Musicnotes recently. There's no standard at all that I can see.)

[notator]

@clnoel Its very good, even important, to be nit-picky here. All, even rather vague, reservations need to be raised and investigated so that we can be sure we're talking the same language, and that there's nothing we've overlooked. Having to think a bit more about percussion parts was, I think, very useful.

@adrianholovaty I don't think we can move on to #57 before the results of this issue (#185) have been incorporated into the Draft Spec.
It turns out that @cecilios concerns were completely justified. Its currently impossible to group Parts (as in group barline) because the Spec has no "system" container that can contain concurrent "system-staves".
We can't discuss how to define systems or system breaks if such things don't exist in the Spec.
(A "system-staff" is a staff element (contained by a "system") that contains voices that are, or contain, lists of one or more PartIDs).

The Spec also needs to provide clear definitions of what it currently calls "staff" and "part".

Staff:
My suggestion would be to rename the current "staff" as "part-staff". A "part-staff" is a staff element (contained by a Part) that contains anonymous voices.

Part:
§4.1 says

A score consists of multiple parts. Each part is a grouping of related musical material that relates to a single performer or set of performers. It has the same temporal extent as the score overall, but presents a slice of content that is relevant to a single instrument or a group of related instruments.

As you can see from the above discussion, defining a Part in terms of performers or sets of performers (or even instruments or groups of related instruments) leads to endless misunderstandings, and is much less powerful than defining it in terms of printability.
§6.1.9 says

The part element represents a set of measures which describe a single part within the score.

which doesn't help much either.

There's no hurry for this, but I think this issue should have even higher priority than #57, which is currently under "Active Review". Maybe you'd like to wait for the next co-chair meeting?

(Off topic: I'm nearly ready with MNX-Common by Example, Ties, and will then be starting on the Slur examples.)

[clnoel]

@notator Are you suggesting we put a section in the "Notational concepts" area to define systems before we define what their representational structure is? I guess I can see that, but I feel like #57 is the place where we are creating that definition.

I agree that redefining the part definition might be good, especially as there is, in fact, confusion around how to distinguish between two parts that are printed on the same staff due to spacing/layout concerns. Defining it as applicable to a single performer does not allow for a divisi, and therefore is not really adequate.

However, I don't understand the problem that the part-staff and system-staff thing is solving. It doesn't solve a grouping problem and it doesn't solve a problem in the part description that is covered by this topic (it has nothing to do with defining which sequence of notes belongs in each part). It is actually pretty clear what a staff is from the current description. Maybe we need a "staff-type" enumeration to facilitate default staff definitions inside the part and overriding staff definitions in the layout structure, instead?

[joeberkovitz]

@clnoel I've been thinking about the current state of play here over the last few days and I'm uncomfortable about just one aspect of it: reusable layout definitions. Let's refer to your recent example which was super helpful in clarifying what's being proposed (see the hornsSplit staff layout definition for instance).

The following things are bothering me — I am not taking a hard stand on these points yet, but I want to float these thoughts to see how others feel.

1. In terms of the rendered result, reusable definitions add nothing to this proposal. There is nothing you can encode with a reusable definition that cannot be encoded with repeated "inline" definitions. So we could get everything we want without reuse, and leave this question to a separate issue.
2. Let's ask what reusable definitions actually do bring to the table. I can only think of one thing: they allow an application that already reuses named layout definitions to encode definitions as first-class concepts. I don't see them being used by manual encoders, or by applications that are not pro-level notation editors. But... how likely is it that different notation editors are going to have reusable layout definitions which are organized exactly like the MNX schema? Seems like a bad bet.
3. Reusable definitions add non-trivial complexity to both the schema and the document topology, as the layout elements now form a directed acyclic graph instead of a tree. If we ever allow groups-within-groups or any other recursive construct, the potential for cycles would exist which is tricky to validate. And having a -definition variant of each reusable element (which would be important for validation) doubles the number of elements.
4. The interaction between definitions and style properties (or, actually, any inheritable layout properties at all) is going to be quite complex to get right. A reusable definition will inherit totally different properties from its ancestors for each different place it is used. These properties may interact with what's in the definition so that it does not really behave like a copy of itself in each instance where it is referenced. It can all be carefully speced out, but why take this on here and now?

Based on the above my feeling is that while reusable definitions are nice to have, they are not necessary to put this issue to bed, and they carry some risk of overreach. Better to put them off.

(By the way let's not discuss how a parser can handle reusable defs mechanically, which is pretty obvious. This is not a debate about feasibility. It's just about keeping things as simple as possible when we don't know all the ramifications and are trying to get things done.)

[joeberkovitz]

One clarification - I’m only questioning the need for reuse of definitions below the level of <system-layout>. The reuse of system layouts themselves within and across <score> elements is essential.

[clnoel]

New issue for that is now created, @joeberkovitz . Just to be clear, I still advocate for it, but I'm willing to discuss it later.

[joeberkovitz]

With the closure of #266 and #270 and the breakout of #269 (thank you @clnoel) I have no more discussion points to offer here. This feels to me like a really significant chunk of progress: I believe the layout concept is going to turn out to be one of the most useful innovations in MNX. Once we merge a PR, I expect to use the layout structure proposed here to support styles in #263, as well as other remaining gaps in MNX 1.0.

[clnoel]

I feel like it's been a while since anyone had anything new to this issue, so I'm going to re-re-summarize. @adrianholovaty I think this represents our consensus for needed changes. I'd appreciate up votes from the various people, including lurkers, if they are in agreement. (And, of course, comments if you are in disagreement!)

• <layouts>: New element, optional child of <mnx>, to contain <system-layout> elements (and possibly other -layout elements if we implement them, and possibly style elements when we figure those out).
• <system-layout: Should become a zero or more child of <layouts> instead of <mnx>
• <group-layout>: The Grouping Symbol enum needs better descriptions for the options.
• <staff-layout>: It needs an optional part-staff attribute (default =1).
• <voice-layout>: New element, a "zero or more" child of <staff-layout>. Should have attributes label, labelref (like <staff-layout>), and stem. Can have one or more <part-layout> elements as children.
  • stem attribute is a Stem Direction enum, where the options are 'default', 'up', 'down' and 'float'. In <voice-layout>, it defaults to 'default', which uses stem directions (if any) defined in the part, and is equivalent to 'float' if they are not defined in the part. 'float' means that the stem directions are determined algorithmically based on staff location of the notehead(s).
• <part-layout> needs stem (like voice-layout), for when it is not inside a voice-layout. Using a part-layout with a stem inside a voice-layout is discouraged. It also needs a part-voice optional attribute to indicate that it refers only to certain sequences with the matching voice labels.
• <system-layout-change>: New element, a child of <system> (in the score hierarchy), with a location (Measure Location) and a layout attribute, used to indicate when the layout changes mid-system, usually (but not necessarily) due to stemming changes.

Side issues (out-of-scope) include:
#247: Measurement units and styling
#266: Eliminating hierarchy layers, especially a score with no pages, but perhaps also part-layout at any level of the hierarchy, including directly under score
#246: Directions that appear differently in different layouts, including clefs (may include staff labels)
#121: Multi-measure-rests
#250: Transposing instruments
#272: minimize-noteheads, possibly as a style option instead of a direct attribute
#269: Reusable layouts
#274: mode as a qualifier for part-voice layouts

Note that, since other people don't seem to want to discuss it, I've tabled my suggested mode attribute to another issue rather than try to keep pushing it.

[cecilios]

@clnoel As you request comments: I have no experience with orchestral scores and on how to prepare the different score parts for the performers. Thus, it is difficult for me to contribute and to spot problems or missing cases. I agree with current proposal. My overall impression is that current proposal for the structure seems to cover normal use cases so it is good to me. You, the active participants, have done a good job developing this proposal. Thank you.

IMO this can be closed and move to the side issues, that could add more light.

[notator]

@clnoel:
Many thanks for the above summary. I'd like to suggest making a few, rather subtle, changes before converting it into a PR. If we subsequently decide that this PR needs correcting, we can always issue new PRs, but its obviously best to get things as right as we can now.
Changes are in two main areas:

• The stem direction attributes in <voice-layout> and <part-layout>.
• We should assume as little as possible about <part>'s internal structure before completing the discussions about layout, visual styles and dimensions.

So here's your summary again (edited), with my comments behind, or enclosed in, // marks:

• <layouts>: New element, optional child of <mnx>. // Is a container for <part-layout>, <voice-layout>, <staff-layout>, <group-layout> and <system-layout> definitions. (If it exists, <layouts> must contain at least one definition.)
• <system-layout>: Should become a zero or more child of <layouts> instead of <mnx>.
• <group-layout>: The Grouping symbol enum needs better option descriptions. // These descriptions could be added later. They need not be part of the PR.
• <staff-layout>: Needs an optional part-staff attribute (default = 1). // Can contain one or more <voice-layout> elements OR one or more <part-layout> elements
• <voice-layout>: New element, optional child of <staff-layout>. Attributes: label, labelref (like <staff-layout>, and stem. // Must contain one or more <part-layout> elements. (Note: I think <voice-layout> can only contain homophonic <part>s, not polyphonic, i.e. grand staff, <part>s. That's something we need to clean up later when we have completed all the discussions about layout, visual styles and dimensions etc., and can turn our attention to the internal structure of <part>)
  • stem // this optional attribute has values that are a VoiceStemDirection enum having values "part|up|down|float". Default is "part", which uses stem directions defined by the first contained <part-layout>. 'float' means that the stem directions are determined algorithmically based on staff location of the notehead(s). If stem="up" or stem="down" or stem="float", stem directions defined by contained <part-layout> elements are ignored.
• <part-layout> // Parent elements are <voice-layout> and <staff-layout>. Needs an optional stem attribute and an optional part-voice attribute (default = 1). (Note: I imagine organising (homophonic) part-voice similarly to (polyphonic) part-staff, but that discussion belongs in the discussion of <part>'s internal structure, so is out-of-bounds here.)
  • stem // this optional attribute has values that are a PartStemDirection enum, having values "up|down|float". Default (when the stem attribute is missing) is float, meaning that the stem directions are determined algorithmically based on staff location of the notehead(s) in the referenced <part>.
• <system-layout-change>: New element, a child of <system> (in the score hierarchy), with a location (Measure Location) and a layout attribute, used to indicate when the layout changes mid-system, usually (but not necessarily) due to stemming changes.

I think the above summary could now be converted into a Pull Request.

Edit: The code for the example at https://w3c.github.io/mnx/docs/mnx-reference/examples/system-layouts/ needs to be updated to conform to this summary. Also, when the PR has been merged, we should add more example diagrams and code, especially to illustrate the use of "missing levels". This can be done using further PRs.

I'd like to take this opportunity to thank you @clnoel for all your work in this epic thread. Something like this can only be achieved as a cooperation. Disagreements are inevitable from time to time, of course, but I think we've come through unscathed. :-)

[clnoel]

@notator:
Agreement notes:

• I'm willing to table stem directions as a possible subset of the style hierarchy debate. Assuming that the parts/sequences don't define their own stem directions, your definitions make sense, even though I'm cringing at have two stem-direction enums. We should consider revisiting as part of the other debate, though. Support style properties with a uniform mechanism #263
• Your point about homophonic/polyphonic is exactly what I was trying to reach for about when it is appropriate to group parts into a voice-layout. However, I'm not sure if that's verifiable. Can you make an issue for it, since you bring it up?
• Added the system layout example to the list of needed changes

Disagreements:

• We've agreed to table the idea of -layout definitions other than system-layout into Reusable layouts below system #269, so the layouts element should not reference those.
• I don't want to make a new PR for the grouping symbol enum. This seems like the optimal time to do this. I'm not suggesting new values, just a better defintion, possibly a picture, of the existing values (bracket|brace|line|none|square)
• You have set part-voice to a default of 1, which means that you missed the point of my definition. I've clarified (see the already existing definition of the matching attribute in sequence).

So here's the newly combined summary, with changes or reversions labelled with // marks. I've also pulled out the internal "notes".

• <layouts>: New element, optional child of <mnx>. //Is a container for one or more <system-layout> definitions.
• <system-layout>: Should become a zero or more child of <layouts> instead of <mnx>.
• <group-layout>: The Grouping symbol enum needs better option descriptions, //possibly including pictures (or an example that shows each one!)
• <staff-layout>: Needs an optional part-staff attribute (default = 1). Can contain one or more <voice-layout> elements OR one or more <part-layout> elements
• <voice-layout>: New element, optional child of <staff-layout>. Attributes: label, labelref (like <staff-layout>), and stem. Must contain one or more <part-layout> elements.
  • stem: this optional attribute has values that are a VoiceStemDirection enum having values "part|up|down|float". Default is "part", which uses stem directions defined by the first contained <part-layout>. 'float' means that the stem directions are determined algorithmically based on staff location of the notehead(s). If stem="up" or stem="down" or stem="float", stem directions defined by contained <part-layout> elements are ignored.
• <part-layout>: Parent elements are <voice-layout> and <staff-layout>. Needs an optional stem attribute and an optional part-voice attribute.
  • part-voice: // this optional Voice ID attribute, when it appears, associates this part-layout with a sequence or set of sequences in the part with the matching voice attribute. If it is empty or missing, the part-layout is associated with the entire part.
  • stem: this optional attribute has values that are a PartStemDirection enum, having values "up|down|float". Default (when the stem attribute is missing) is float, meaning that the stem directions are determined algorithmically based on staff location of the notehead(s) in the referenced <part>.
• <system-layout-change>: New element, a child of <system> (in the score hierarchy), with a location (Measure Location) and a layout attribute, used to indicate when the layout changes mid-system, usually (but not necessarily) due to stemming changes.
• //An updated System Layout example.

P.S.
This issue has been extremely important to me as correctly getting MusicXML layouts into Musicnotes' internal format is extremely harrowing. We actually have an internal structure that represents scores by the number of systems on each page and the number of measures in each system, and gives each system what we call (for legacy reasons) a "voice file", representing which parts appear in which order and the system grouping symbols and staff labels. Getting that info out of the MusicXML measure attributes and print directives has been one of our major sources of conversion error, especially since they might be defined on any of the parts in the file. Having it in one defined place is going to help us out a lot. I hope I haven't gone too far in trying to push our structures on everybody else, or maybe it's just that everybody else uses something like our structures (or wishes they were!). In any case, I have greatly enjoyed the collaboration on this issue.

[notator]

@clnoel: Okay, we can go ahead and make a PR from your current version! :-)

Here's what I'm thinking in detail:
Agreements:

I'm willing to table stem directions as a possible subset of the style hierarchy debate. Assuming that the parts/sequences don't define their own stem directions, your definitions make sense, even though I'm cringing at have two stem-direction enums. We should consider revisiting as part of the other debate, though: #263

Yes. This will need to be re-visited after the PR has been merged. As I said, we need some graphic examples and code illustrating "missing levels". These should. I hope, show why I think the stem directions need to be done like that.
Both homophonic, and polyphonic <part> elements generally assume that stem directions "are determined algorithmically based on the staff location of their noteheads". But the stem directions of homophonic <part>s actually depend on the way they are combined on real staves, and that can't be foreseen inside the <part>. Note that <event> currently has an orient attribute ("up|down"), which would be overridden by a <voice-layout>'s stem attribute. Maybe <event> orient is a mistake in homophonic <part>s? The only way to be sure that we've really got it right is to write, and discuss, demo examples. That can be done later.

Your point about homophonic/polyphonic is exactly what I was trying to reach for about when it is appropriate to group parts into a voice-layout. However, I'm not sure if that's verifiable. Can you make an issue for it, since you bring it up?

I've added this to the list of issues that I want to bring up once the PR is merged. Adding a grand staff to a <voice-layout> must be made (verifiably) impossible.

Disagreements:

We've agreed to table the idea of -layout definitions other than system-layout into Reusable layouts below system #269, so the layouts element should not reference those.

Okay. I've made a note that I should create a PR for reusable layout definitions inside <layouts>, when #269 has been resolved.

I don't want to make a new PR for the grouping symbol enum. This seems like the optimal time to do this. I'm not suggesting new values, just a better defintion, possibly a picture, of the existing values (bracket|brace|line|none|square)

Its precisely because we are not wanting to change the enum, but only add the descriptions that I think this could wait for a new PR. Practically speaking, the creation of diagrams and definitions may mean a little work and discussion that need not hold this PR up. I don't really mind if this is done before or after the PR is merged.

You have set part-voice to a default of 1, which means that you missed the point of my definition. I've clarified (see the already existing definition of the matching attribute in sequence).

I think you missed my point. :-) But I'm quite happy with your current version. We will have to revisit this once we start discussing <part>'s internal structure. Conveniently, part-voice and part-staff are the only interfaces between the -layout elements and <part>, so the solution should not be too difficult. This is on my <part>-discussion TODO list. (To reiterate: I think we should complete all the layout, visual style and dimensions discussions before starting to talk about <part>'s internal structure.)

All the best,
James

[adrianholovaty]

@clnoel Thanks for shepherding this issue through, and thanks for the executive summary! I can put a pull request together as a next step.

The Grouping symbol enum needs better option descriptions, //possibly including pictures (or an example that shows each one!)

In the meantime, could you put together these pictures — and (if we're lucky) maybe even example documents for each one?

[clnoel]

brace: a curved symbol similar to { that sits to the left of the system barline without touching it.

bracket: a straight line with curved ends that intersects the system barline just above and below the staves it includes.

line: a straight line that sits to the left of the system barline without touching it.

none: no symbol is displayed

square: a straight line with straight ends that connect to the top and bottom staff lines of the staves it includes.

(These images were generate in MuseScore)

[notator]

The way the final barlines are drawn in these examples has nothing to do with the Grouping symbol enum, so I suggest leaving them out. It would be best to leave the time-signatures and rests out as well.

brace: a curved symbol similar to { that sits to the left of the system barline without touching it.

bracket: a straight line with curved ends that intersects the system barline just above and below the staves it includes.

line: a straight line that sits to the left of the system barline without touching it.

none: no symbol is displayed

square: a straight line with straight ends that connect to the top and bottom staff lines of the staves it includes.

[joeberkovitz]

In all the discussion of part groups, reductions, and other relatively fancy stuff, I don't know that we've talked about a relatively simple case: an ensemble piece in which there is a conductors' score with all parts (using one layout) and a set of part scores (which all use an identical layout except for the choice of part).

It seems to me that this is a pretty common use case. But the current proposal doesn't appear to have a way to express the fact that all the part scores share the same layout, because it is the layout itself which designates which part is to be shown. Thus, each part has both a <system-layout> with a contained <staff-layout> pointing at the part of interest, and a corresponding <score> which (alone) makes use of that system layout.

I'm not going to stop the train for this but it seemed worth pointing out to see if it's a concern shared by anyone else. I feel as though the current structure may be optimized more from a rendering point of view rather than an authoring point of view.

[notator]

I'm working on, and am soon going to post, a proposal for describing the labels, brackets etc. in a system's left margin. These things are purely a matter of layout, and cannot be related to name and short-name attributes inside <part> ( <part> cannot know how it is going to be combined on a staff with other <part>s.) In other words, the left-margin labels are going to be described inside <system-layout>, separately from the staff content definitions. This proposal takes #276 into account, and should also solve #269 (because <group-layout> no longer has to be recursive).

This means that part scores cannot share the same layout. Each part score needs its own <system-layout> definition and therefore <score>. So the current way of doing things is correct.

Update: The left-margin labels proposal, mentioned above, has been posted in #277

[adrianholovaty]

Hello all contributors to issue #185! There's now a pull request that brings this issue's agreed-upon changes into the MNX spec. Here is the link: #281

Please have a look and leave comments there if you have a chance.

[adrianholovaty]

With the changes from pull request #281 now merged (as of commit 9d19e1b), I'm now marking this as closed. For any follow-up tweaks and additions, let's use separate issues.

Thanks to all who contributed to this — I think we landed on something really elegant and powerful!