why my training model dont have pkl file

[lyh512796310]

i want to know that how to get the “dictionary.pkl” file
and how to generate this file

[songhe17]

Same problem here

[remyhuang]

from collections import Counter
import pickle

all_elements= []
for midi_file in you_all_midi_files:
    events = model.extract_events(midi_file)
    for event in events:
        element = '{}_{}'.format(event.name, event.value)
        all_elements.append(element)

counts = Counter(all_elements)
event2word = {c: i for i, c in enumerate(counts.keys())}
word2event = {i: c for i, c in enumerate(counts.keys())}
pickle.dump((event2word, word2event), open('dictionary.pkl', 'wb'))

I’m not sure if you can run it directly, but it should be easy for you to modify.

[dhrumilp15]

You can use glob to grab your midi files! We can pull out extract_events from model.py to make the dictionary.

from collections import Counter
import pickle
import glob
import utils
def extract_events(input_path, chord=False):
    note_items, tempo_items = utils.read_items(input_path)
    note_items = utils.quantize_items(note_items)
    max_time = note_items[-1].end
    if chord:
        chord_items = utils.extract_chords(note_items)
        items = chord_items + tempo_items + note_items
    else:
        items = tempo_items + note_items
    groups = utils.group_items(items, max_time)
    events = utils.item2event(groups)
    return events

all_elements= []
for midi_file in glob.glob("./data/*/*.mid*", recursive=True):
    events = extract_events(midi_file) # If you're analyzing chords, use `extract_events(midi_file, chord=True)`
    for event in events:
        element = '{}_{}'.format(event.name, event.value)
        all_elements.append(element)

counts = Counter(all_elements)
event2word = {c: i for i, c in enumerate(counts.keys())}
word2event = {i: c for i, c in enumerate(counts.keys())}
pickle.dump((event2word, word2event), open('dictionary.pkl', 'wb'))

This grabs all files in the data folder with file extensions that start with .mid (This covers .mid and .midi files).

[ShacoLAY]

Thank you for the amazing work on this project!

However, I have a problem for several days and would like to ask for help, I was able to generate my own music training dictionary according to the above code, but it does not work with the original REMI-tempo-checkpoint (I am directly replacing the previous dictionary with my own) and then the following error occurs:

In addition, can I use my own training data to generate my own checkpoint and evaluation my own music?
Do you have any insights about this problem?

Thanks again for your kind help!

[dedededefo]

感谢您在这个项目上所做的出色工作！

但是我这几天有问题想求助，我可以按照上面的代码生成自己的音乐训练字典，但是用原来的REMI-tempo-checkpoint不行（我直接用我自己的字典替换以前的字典），然后出现以下错误：

另外，我可以使用自己的训练数据来生成自己的检查点并评估自己的音乐吗？ 你对这个问题有什么见解吗？

再次感谢您的帮助！

Hello, have you solved this problem? I have the same problem

[dedededefo]

您可以使用 glob 来获取您的 MIDI 文件！我们可以从中取出来extract_events制作model.py字典。

from collections import Counter
import pickle
import glob
import utils
def extract_events(input_path, chord=False):
    note_items, tempo_items = utils.read_items(input_path)
    note_items = utils.quantize_items(note_items)
    max_time = note_items[-1].end
    if chord:
        chord_items = utils.extract_chords(note_items)
        items = chord_items + tempo_items + note_items
    else:
        items = tempo_items + note_items
    groups = utils.group_items(items, max_time)
    events = utils.item2event(groups)
    return events

all_elements= []
for midi_file in glob.glob("./data/*/*.mid*", recursive=True):
    events = extract_events(midi_file) # If you're analyzing chords, use `extract_events(midi_file, chord=True)`
    for event in events:
        element = '{}_{}'.format(event.name, event.value)
        all_elements.append(element)

counts = Counter(all_elements)
event2word = {c: i for i, c in enumerate(counts.keys())}
word2event = {i: c for i, c in enumerate(counts.keys())}
pickle.dump((event2word, word2event), open('dictionary.pkl', 'wb'))

这将抓取文件data夹中文件扩展名以.mid(This cover .midand .midifiles) 开头的所有文件。

您可以使用 glob 来获取您的 MIDI 文件！我们可以从中取出来extract_events制作model.py字典。

from collections import Counter
import pickle
import glob
import utils
def extract_events(input_path, chord=False):
    note_items, tempo_items = utils.read_items(input_path)
    note_items = utils.quantize_items(note_items)
    max_time = note_items[-1].end
    if chord:
        chord_items = utils.extract_chords(note_items)
        items = chord_items + tempo_items + note_items
    else:
        items = tempo_items + note_items
    groups = utils.group_items(items, max_time)
    events = utils.item2event(groups)
    return events

all_elements= []
for midi_file in glob.glob("./data/*/*.mid*", recursive=True):
    events = extract_events(midi_file) # If you're analyzing chords, use `extract_events(midi_file, chord=True)`
    for event in events:
        element = '{}_{}'.format(event.name, event.value)
        all_elements.append(element)

counts = Counter(all_elements)
event2word = {c: i for i, c in enumerate(counts.keys())}
word2event = {i: c for i, c in enumerate(counts.keys())}
pickle.dump((event2word, word2event), open('dictionary.pkl', 'wb'))

这将抓取文件data夹中文件扩展名以.mid(This cover .midand .midifiles) 开头的所有文件。

Hello, why do I report an error when using my MIDI data to generate a dictionary? In the case of multiple MIDI, it will report an error, but one or two will be fine. What's the use of a dictionary

[dedededefo]

from  collections  import  Counter 
import  pickle

all_elements = []
用于 you_all_midi_files 中的 midi_file：
    事件 = 模型。extract_events ( midi_file )
    用于 事件 中的 事件：
         element  =  '{}_{}'。格式（事件。名称，事件。值）
         all_elements。附加（元素）

counts  =  Counter ( all_elements )
 event2word  = { c : i  for  i , c  in  enumerate ( counts.keys ( ) )}
 word2event = { i : c for i , c in enumerate ( counts.keys  ( ) )
 } pickle。转储（（event2word，word2event），打开（'dictionary.pkl'    , 'wb' ))

我不确定你是否可以直接运行它，但它应该很容易修改。

Must a dictionary of the same size as the pre training checkpoint be generated

[dedededefo]

您可以使用 glob 来获取您的 MIDI 文件！我们可以从中取出来extract_events制作model.py字典。

from collections import Counter
import pickle
import glob
import utils
def extract_events(input_path, chord=False):
    note_items, tempo_items = utils.read_items(input_path)
    note_items = utils.quantize_items(note_items)
    max_time = note_items[-1].end
    if chord:
        chord_items = utils.extract_chords(note_items)
        items = chord_items + tempo_items + note_items
    else:
        items = tempo_items + note_items
    groups = utils.group_items(items, max_time)
    events = utils.item2event(groups)
    return events

all_elements= []
for midi_file in glob.glob("./data/*/*.mid*", recursive=True):
    events = extract_events(midi_file) # If you're analyzing chords, use `extract_events(midi_file, chord=True)`
    for event in events:
        element = '{}_{}'.format(event.name, event.value)
        all_elements.append(element)

counts = Counter(all_elements)
event2word = {c: i for i, c in enumerate(counts.keys())}
word2event = {i: c for i, c in enumerate(counts.keys())}
pickle.dump((event2word, word2event), open('dictionary.pkl', 'wb'))

这将抓取文件data夹中文件扩展名以.mid(This cover .midand .midifiles) 开头的所有文件。

Do you have to generate a dictionary of the same size as the pre training checkpoint to train your own data

[Hikari-Tsai]

Just for the record, this discuss only for train task from scratch. If you use finetune.py to train your model,just need copy the original dictionary.pkl from pretrain document.