Preparing for PRE revival (Piano Roll Editor)

[scorster]

Recently I needed to work in 3.6.2's Piano Roll Editor (aka the PRE). The experience went better than my previous encounters but there's much to improve. So I'm starting a comprehensive list of 3.6.2 issues to aide in a healthy repatriation of the Piano Roll Editor in MuseScore 4, when the time comes. If interest ensues I'll gladly separate various issues into individual 3.6.2 bug reports for the advancement of v4.x.

Here's the MuseScore 3.6.2 PRE issue list:

1. There's no good reason for the hardcoded 2K upper value limit for the Len field (located in the upper right of the PRE panel.) This has been addressed in MuseScore 3.7 (aka MuseScore 3 Evolution) where the value limit is now 60K.

2. We can’t drag the tail of notes to change their playback duration in the PRE—instead we must descend into the PRE’s basement and drag a duration slider vertically to alter the horizontal representation of the playback duration. Recommendation, in short: Allow the user to edge-edit the tail of notes directly in the Piano Roll Editor.

2b) With multiple notes selected lengthening one of the selected notes (by dragging the yellow lines in the PRE's basement) fails to lengthen them all. In constrast, with multiple notes selected, all durations change when I change the value of the LEN field.
Suggestion: A tail drag should mutually affect the tail of all selected note.

3. After clicking on a "note" in the Piano Roll Editor the "basement's" editing handle is not always visible or draggable. For instance, in Duration(NON-multiplier) via the note's yellow duration handle is often invisible because it's buried beneath the green handle of an unselected note. Interestingly, clicking on the display of the green handle selects the yellow handle making it draggable. Recommendation: If a PRE "note" is selected its handle should be visible and draggable in front of the handles of non selected notes.

3b) The PRE's basement view only shows Duration(Multiplier) values up to 1000. However we can drag upward beyond 1000, but then it's impossible to drag back down after mouseUp, so the user must resort to the Duration field. The inconvenient workaround to attain a greater drag range is to switch to Duration "non multiplier" view. Recommendation: Allow the user to zoom or scroll the view.

3c) Sometimes the upper PRE view fails to update until clicked. Ideally it would update whenever the user changes a property.

4. Sometimes clicking a "note" in the PRE view alters its playback duration (usually changing it from above 2000 to back down to 2000. Seems to happen mainly when the note starts at beat 1. I don't have steps to replicate, but it happens regularly.) This may be related to the unneeded ceiling on the Len value, which is 2000 in MuseScore 3.6.

5. On the left we have Duration and Duration (multiplier) fields—unfortunately their full names and tooltip labels are truncated.

6. In either flavor of the Duration field pressing enter doesn't commit the value. Nor does clicking Up or Down in the spinner button, which is yet a stranger oversight. Indeed, the Set button is well positioned and well-advertised … but at least 200 times I've forgotten to take that extra step. One would expect pressing enter would suffice, or merely existing the field. Recommendation: Clicking a spinner button should suffice to enter the value. So should pressing return after editing a value.

7. The Pitch spinner (located to the left of Len) does not operate at all.

8. My understanding is that Len and Duration(Multiplier) both show a value of x‰, so I'd think they would always show the same value. And exception is that Len appears to adhere to 3.6.2's 2K limit, so when Duration(Multiplier) is > 2000 Len only shows 2000.
   Recommendation: Regarding UI, it would make sense to label the field ‰ ... or permille. Or better yet, for normal human consumption, display in as a normal percentage, and if four digit accuracy is required then display as 200.0%

9. When my selection contains notes of varying playback duration the Duration field displays the value of the first clicked note. It should indicate that a mix of values is selected.

10. This is a BIG one. If the PRE is left open, switching to another score leaves the PRE open, which is not an issue except that the PRE fails to update. Instead it displays the previous score's data! Even worse, the PRE scrolls when the current score plays, and that leads one to believe the PRE is displaying the data of the current score! When I've worked on nearly identical scores this has caused great confusion.

11. Also very important: MuseScore 4 plugin developers need the Piano Roll Editor's properties exposed to their plugins or they will be unable to take advantage of the Piano Roll Editor, including properties like Len and Len(multiplier).

I've illustrated some of these behaviors in a screencast as an unlisted youTube video:

https://youtu.be/H2x6cJ28w2U

12. The following would be amazingly useful! Allow the user to superimpose editable play durations on a standard staff and/or tablature staff. This would be the equivalent of "overlaying" the Piano Roll Len data on notational staves, like the view available in Garageband and Logic. Logic enables this feature under Editor>View>Duration Bars. Unfortunately, by default, changing the length of the "duration bar" affects the note's face value. That's NOT the proposed affect in MuseScore, and if I've been clear, that should be obvious. Additionally I see that Garageband now behaves that way too. Garageband's legacy approach was the perfect example of the feature I've described. I'll look to see if there's a setting that allows play duration editing without altering the face value displayed in Garageband's and Logic's notation view. I've raise the question here on Apple's Logic Pro X discussion board.

13. Most apps advance the view by a "screen with" when the user clicks to the right of a horizontal scroll bar's "thumb." So, if four bar of data are showing, clicking to the right of the "thumb" updates the screen to show the next four bar. But presently the PRE barely moves forward on a "right of thumb" click. Additionally clicking the right or left scroll buttons makes a surprisingly minuscule change.

This is a hefty list indeed. Nevertheless, if a anyone following knows of other issues, or has additional requests, please contribute to this conversation.

scorster

[enh3]

Hi scorster,

I'm very glad to hear that this is being discussed! There are many improvements in MuseScore 4, but the lack of a piano roll editor is one of the things keeping me in 3.6.2 (The other is the undoing of this fix: https://musescore.org/en/node/294836). Your list is great, but one thing I would add is better zoom integration. This may be a Mac only issue (I'm on macOS Monterey 12.6.2), but the cmd plus mousewheel and cmd plus mousewheel described on this page https://musescore.org/en/handbook/3/piano-roll-editor are not very responsive (you have to scroll several times for it to have an effect), and as far as I can tell, it's the only way to zoom. What I would propose is that in addition to the original controls :

1. When the PRE is open and it's window is active, the classic cmd + or cmd - zooms the PRE (horizontally?) and cmd shift + or - zooms vertically.
2. When the PRE is open and it's window is active, the zoom controls in the view menu work for the PRE (perhaps both vertically and horizontally? (I wouldn't use this, but it might be useful for people that want a general zoom from the menu.
3. There could also be zoom controls right in the interface, in the form of a + and - sign on the extremities of the horizontal and vertical scroll bars. 

Of course, the most important would be getting the original controls smoother and more responsive, but eventually it would be nice to have a couple options.

Thanks again for initiating this conversation. I'm looking forward to seeing MuseScore 4 progress.

Kjel

[ShibuyaCyana]

It's kinda hard to make a good piano roll...I mean, there're so many DAWs around in the market and very few of them have the fame of "a good piano roll".
I'm agree with many points in OP's video, the PRE in 3.6 are behaving strangely...some more examples:

1. when dividing chord:
   
2. when moving notes around:
   

Some random wild thoughts:

Note editing

1. It's strange to adjust note length or offset by a slider below; better be done visually.
2. when user edits a note, especially length since it'll have overlap problem, there're a few solutions - or user intent interpretations:

• legato edit, i.e. cut the length of the notes being overlapped, tho the prior note may be not a single note but a chord;
  • this is the old default behavior;
  • 
  • the behavior of the notes being cut should be carefully considered, e.g. its length might be remembered and restored if user move the newly edited note away - or maybe not?
  • the newly edited note would be in the same voicing with overlapped notes.
• stack, i.e. explicitly create a let ring style chord like this:
  
  • same voice with overlapped notes, as in legato edit.
• new voice, or even new staff, maybe even auto move notes above/below to an existing staff (auto hand division);
• actual length/position override, this remove the need of a lane of strange position/duration sliders below;
  • this won't change voice group;
• dummy note, which is not displayed in notation staff, maybe used to trigger articulation or other VST commands
  • this should be as free as possible, not belonging to any voice or staff and don't have restrictions of overlapping etc.

So these are five different edit modes, a quick sum up:

• legato edit, user designated voice;
• stack edit, user designated voice;
• new voice/staff, user designated voice, but may change existing note voice group;
• length/position override, only apply to existing notes;
• dummy notes, no voice designation, not shown on staff and free to move anywhere.

So when edit/insert a note in piano roll, in order to decide how to place new notes and existing notes on the staff, the program only needs to keep track of the current edit mode and the voice belonging of every existing notes, no new properties needed. As for dummy notes, they can be simply regarded as MIDI data.
3. or alternatively, maybe just let user write/record everything onto the piano roll, then apply edit modes and voice grouping afterwards. This might be a super fast workflow.

Lanes

I'm referring to the area below the piano roll, now displaying length, duration, velocity etc in 3.6.

1. use them to display velocity, pitch and other MIDI messages, like every other DAW.
2. maybe automation for VST and articulations.
3. support multi line display, such as velocity and pedal (cc64) together.

Zooming and hotkeys

1. we're all having our own zooming habits. better let user define which modifier keys (ctrl, alt, shift) to use, and adjust polarity and speed.
2. grid could be adaptive, changing according to zoom level, and could be locked.
3. if using the input system stated above, we'll have voices, staffs, edit modes and tools for users to choose from, that will be a lot of hotkeys. maybe use modifiers like shift+click=insert note for tools as well.

[scorster]

enh3 and ShibuyaCyana,

Thanks for your extensive feedback on the Piano Roll Editor!

Lot's of great thoughts, examples and descriptions.

scorster

[blackears]

Hi. I'm the guy who wrote the Piano Roll Editor for MuseScore 3 (there was one in MuseScore 2, but after the overhaul, it was pretty much a new editor). I did write an update for MuseScore 4 about two years ago now which both ports the old code and adds some new features including many of the points discussed above. However, from my discussions with the team, they are not interested in using this patch. My sense from what they said is they want to design something new that integrates with their code base better. They are also concerned about unusual timings, such as tuplets that cross bar lines which the old code doesn't really address. I also never got any feedback from the team either before or after the takeover by Ultimate Guitar as to how to best integrate the editor with the note engine, which can account for some of the jankiness. Both the before and after teams seem to regard the piano roll editor more as a way to tweak note playback timings rather than as an editor in its own right.

I had been keeping the patch @scorster linked up to date for a while, but after a discussion with Martin decided to stop since it does not look like it's ever going to be applied. It was also a real pain updating it each time it went out of sync with the main branch. I've gotten some hints that the team may be looking at it as a possible starting point for whatever they are writing, but based on the issues in the bug tracker I've not seen any evidence any kind of a piano roll editor is being worked on. Maybe when MuseScore moves into version 5 they might start adding these extended features.

[cbjeukendrup]

That we are not interested is not the case, but there are two difficulties:

• the PR is very big, so reviewing it is a complicated task and it is difficult to find time for that. Reviewing commit-by-commit is also not really an option because the Git history is not very streamlined.
  It would have been easier if the PR was only a piano roll, and not also an automation system etc.; and ideally, broken down into easily reviewable portions, either as commits or as multiple PRs.
• it was not really coordinated with us. Usually, our preferred way of working is to agree on a design with the design team before working on the implementation. And as far as playback is concerned, all changes would need to be carefully coordinated with the developers team, because of the new playback engine which has been built in src/engraving/playback. (That's another reason that it would be preferable to have just the piano roll changes, and not the automation stuff, which is where I expect most conflicts.)
  So because of the uncertainty around playback, my advice back then was actually to wait a bit. But I can also understand why you didn't wait, because if you just wait, then nothing happens...

So this pull request puts us in a painful situation: we absolutely appreciate how much time and effort you put into this, and we would really want to give it the recognition it deserves by merging it somehow, but right now we can't, because of the reasons mentioned above.

Anyway, when we're finally ready for a piano roll again, I promise we'll make sure to involve you, if you're still happy to work with us. I'll ask the design team if they happen to have piano roll designs ready, and if so, to reach out to you about it.

[blackears]

I can still help, although I can't devote as much time as I did in the past. The automation part is mostly just the user interface. I didn't know how it was supposed to hook into the rest of the program so I just created some generic animation classes to hold key data and designed the UI around accessing them. The animation classes are just holding data right now and don't affect anything else. They could be removed from the core classes without really affecting either the PRE or how the note engine either.

A lot of the PRE was ported from MuseScore 3, so there wasn't really a case-by-case to start with.

Martin did give me a design document back when I started, but I found it very difficult to communicate with anyone. I used to post on the MuseScore discord group, but didn't get much feedback aside just from help about how to compile the project. I could use someone on the developer team providing feedback on the best way to have the PRE interact with the notation engine.

[blackears]

Mostly I need to be able to have someone on the team I can talk to. I've found it almost impossible to communicate with anyone on the core MuseScore team about what is going on and what is needed. A lot of these issues could be cleared up if I could simply talk to someone about what is going on.

I feel that I'm back in the position where I could help with this, but I'm going to get no communication with anyone, so any work I do is just going to sit in the PR for another year and the whole issue will just be ignored and forgotten.

[scorster]

@blackears

Thanks for your reply!!

I believe I saw a youtube of yours (from about three years ago) regarding v3 PRE development ... and from two years ago regarding v4 development.

Please keep your ear to the rail and let us know what you hear!

The disappearance of the PRE is a huge v4 regression. And sadly it blocks porting of the new TAB Ring plugin to MuseScore 4.

Additionally we must act cautiously when opening MuseScore 3.6 or 3.7 score in MuseScore 4. MuseScore 4 discards any Len duration adjustments made MuseScore 3's Roll Editor PRE. Those values are simply ignored, thus play durations revert to face value durations.

So it follows, after opening a MuseScore 3.6 or 3.7 score processed with the TAB Ring plugin we find all the pertinent play durations reverted to face value—and that erases precisely what TAB Ring alters to bring realistic Laissez Vibrer to MuseScore! So the question is "Why does MS 4 ignore Len values? Perhaps there's a legacy upper limit on the Len field value that disallows importing Len values from MS3.6 scores into MS 4.2 scores.. The impact is also discussed in the last section of this forum post.

I wonder if delays to reinstating the PRE hinge on conceptualizing/implementing the integration of note velocities along with dynamics and dynamic velocities.

@blackears, I'm quite grateful for the enormous amount of work you've devoted to the 3.x PRE, plus your efforts on moving forward toward a 4.x version.

Thank youfor taking the time to comment here and for illuminating the PRE's history and current state of affairs!

scorster

[enh3]

Hi all,

Thanks @blackears for your incredible work and for your insight here. Discovering the PRE was one of my favourite moments of using MuseScore. I felt like I knew the program pretty well at that point, and then it was like finding a new chapter in a book you've read several times, and it's a really good chapter. I really hope that we can get your contributions integrated into MuseScore4 where they belong. I don't have too much development experience, especially not in collaborative contexts or on huge codebases, but let me know if there's any way I can help. I do have some limited C++ background and would love to chip in in a small way if I can. I can also write/revise documentation if that would help.

Kjel

[blackears]

Thanks. The main problem is integration. As @cbjeukendrup said, there's a lot of code in the PR and I can understand that it would take some effort to go through it. That said, I've had no contact from any developer at MuseScore to resolve any of these issues, or even just to talk about what's going on. I've taken the code as far as I can by myself. If this is to ever make it into MuseScore, I need to work with someone on the team who can tell me what needs to be done and can answer some questions I have about integrating with the note engine.

[enh3]

That makes sense. I'm sure it's no easy task, and they are working on a lot of different priorities as they try to bridge between many improvements of MuseScore4 and the wider functionality of MuseScore3. I do hope the team finds time to reach out to you to organize an implementation. In any case, you've got a bump from me my friend!

[blackears]

So I decided to backport the changes I made for the 4.0 branch to 3.7. @Jojo-Schmitz just merged it, so it should be available in the Musescore Evolution as soon as it finishes compiling https://sites.google.com/view/musescore-3-evolution/

It doesn't address everything above, but is a significant improvement over it was before. There is also some jankiness when you move notes around - sometimes notes that shouldn't be deleted will be. I don't know how to fix this because I wasn't able to figure out how the note engine was handling the commands I was sending it. It also doesn't refresh the screen properly if you undo adjusting note play time adjustments. If someone else wants to look into this, that would be welcome.

[scorster]

blackears wrote > I decided to backport the changes I made for the 4.0 branch to 3.7. @Jojo-Schmitz just merged it, so it should be available in the Musescore Evolution.

Excellent news!! Do you have a change log so we can be mindful of the improvements?

A couple of questions:

• I lightly followed some of the v4 PRE pull request. Do you know if the work is now in a state where where it could be merged into v4 when development is ready.

• Did you look into v4 and the Len property? My understanding is that MuseScore 4 reads Len properties but ignores them, so they are not heard in playback. Hopefully it won't be very complicated to get MuseScore 4's note engine to play ball with Len.

Thanks so much for moving this long dormant project forward and backporting it to Evolution.

[blackears]

You can check the PRs themselves to see what changed on a code level. I put together this youtube video explaining the changes for users: https://youtu.be/CemYFBE_vS4

This cannot be merged into the 4 branch directly. 4 uses a completely graphics system for rendering - but most of the non-API specific stuff should be the same for both. I also did some bug fixing as well as just porting, so at this point it would probably best to report the new 3.7 version to the 4.x series.

That said, the main problem is the almost complete lack of communication with the MuseScore team. I can't port this if no one will tell me what issues they're having with it. And I don't want to submit anther PR and have it wait for more than a year. Despite what Casper said, I have no evidence to think that MuseScore even wants it at this point.

I did not look into how 4.x treats the len property. I was just working with the old 3 codebase.

[scorster]

@blackears Thanks for your dedication and recent work on the Piano Roll Editor! Glad to see it now merged into MuseScore Evolution (aka MuseScore 3.7)

Your youTube video is quite helpful toward understanding the recent PRE updates.

I hope the MuseScore 4 team eventually sees the value of reviving the Piano Roll Editor in MS4.

[KJinx101]

@blackears
I was looking at the potential of JACK
https://www.youtube.com/watch?v=vVOdveT6WnU
And am wondering if there's a way that this can be utilized to fake a PRE...
With the idea being something that reads from/modifies a 'dummy' score (which people normally use to make mock-ups in MS4)
through the interface of a piano roll. So the instruments on the main score would be muted or something, but the dummy score would match the main score by default (and have its audio played whenever the main score presses play), but changes would be made possible through the 'Piano Roll' which really edits the separate dummy score. Or something like that.

It might perhaps be a clunky pseudo method for having a PR, but if it is easily implemented as either a plugin, or a pull where users who want a PR can easily download it, it should be doable, right?
You have experience making the Piano Roll so I wanted to run the idea/ workaround by you. Thanks. Love the work you're still putting into MS Evolution!

[MarcSabatella]

FWIW, I don't think anyone on the MU4 team has expressed anything but complete support for bringing a PRe back. The holdup isn't resistance to the idea in general, but rather a desire to architecture the internals of the score representation to make a much more fully functional version of a PRE available - one not constrained by measure or tuplet boundaries, etc. Which is why it is repeatedly mentioned as something being looked for MU5, when the internal rearchitecting is complete.

[dompaon]

HI ALL and first thanks to developpers and everyone around MUSESCORE, its been a significant part of my life for several years and I couldnt imagine to miss it.
Currently i face a little problem I'd like to solve and I guess people around are clever enough with whats inside musescore to help me.
I use the PRE for analysing drum patterns I want to migrate to my BeatBuddy pedal (thousands of patterns) .
To do the mappings, I have to check which notes are kick, snare, hat etc...... Its very convenient for me to define a .DRM drum set and see on the left side keyboard the note number and octave. I have created and saved it.
I do audition the pattern with a wonderful midi/audio browser called Resoniq (a diamond in its category) and i drag it to musescore.
This creates a new sheet in MS and this sheet receives the defaut drum map which is less detailed than mine.
I have tried all I can to understand where MS stores this default pattern but no way to find it. Does anyone know where it is located and how to change it ? Thanks by advance . Dom from France

[MarcSabatella]

The default drumset is defined within the MuseScore source code here on GitHub, so you wouldn't be able to change that except by recompiling. You can however create a template - just a normal score saved to your Templates folder - containing your custom definition.

For further help in using MuseScore, better to ask in the official Support forum on musescore.org

[dompaon]

thanks Marc. I was expecting this thats why i posted here. Can you please refresh my memory, is MS open source ? so do I have a chance to find the map in the code, change it and recompile ?

[MarcSabatella]

Yes, just see the main GitHub page for MuseScore for the source. But - that's going to be a ton of work, and the resulting scores won't be compatible with others, and you'll have to repeat the work for each update, etc. Why not simply create the template? that takes but seconds and you're done.

[dompaon]

you're right, a ton of work and i can make it with the template, thats what i just did and it does the job for me. Ill add to the wishlist for MS5 a default drm file read from user config, like shortcuts. Thanks for ansewring though

[VighneshJadha]

Use the Piano Roll Viewer. It is like a Piano Roll Editor but you cannot edit from the piano roll.

[dompaon]

Hi, what is the piano roll viewer ?

[ParadoxV5]

I would like our piano roll editor (PRE) to double as a piano roll viewer for playback, similar to Synthesia or any of the handful of alternatives. That means, in addition to what’s shown on the GIFs above:

• Display multiple instruments in parallel, differentiated by at least a few color options
  • PRE users could also use this feature; I haven’t researched.
• Choice of vertically scrolling down (good for piano previewing) in addition to horizontally left (good for score composing)
  • Upward-rising notes seems to be the new trend. I care not. I find it more for show and less practical.
  • The vertical orientation can replace the basic piano keyboard dock MuseScore currently have.
• Non-goal: features to help practice a piano
  • MuseScore is a score engraving app, not an instrument practice app. This at most could be an out-of-place plugin.
  • What would be on-topic instead would be MIDI input via a MIDI keyboard. MuseScore 3 have this feature without needing the PRE, haven’t checked for MuseScore 4.
• Non-goal: special effects such as fire
  • I care not. They can be plugins if there’re people who ask too much. (plugin system when tho)

[P.S.] Let me know if I should file this as separate feature request(s) after reviving the PRE as-is.