Error(s) in loading state_dict for SoftmaxNN

[ceteri]

Thank you for this library OpenNRE which is excellent.

When running the following example code:

import opennre

model_name: str = "wiki80_bert_softmax"
model: opennre.model.softmax_nn.SoftmaxNN = opennre.get_model(model_name)

input_sent: dict = {
    "text": "He was the son of Máel Dúin mac Máele Fithrich, and grandson of the high king Áed Uaridnach (died 612).",
    "h": { "pos": (18, 46), },
    "t": { "pos": (78, 91), },
}

result: tuple = model.infer(input_sent)
print(result)

The following exception gets thrown:

  File "/Users/paco/src/OpenNRE/opennre/pretrain.py", line 161, in get_model
    m.load_state_dict(torch.load(ckpt, map_location='cpu')['state_dict'])
  File "/Users/paco/src/OpenNRE/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 2041, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for SoftmaxNN:
	Unexpected key(s) in state_dict: "sentence_encoder.bert.embeddings.position_ids". 

Looking at the contents of that dictionary at inference time:

 m.load_state_dict(torch.load(ckpt, map_location='cpu')['state_dict'])

tensor contents

('sentence_encoder.bert.embeddings.position_ids', 
 tensor([[
           0,   1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,
          14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,
          28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,
          42,  43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,
          56,  57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,
          70,  71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,
          84,  85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,
          98,  99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,
         112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125,
         126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139,
         140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153,
         154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167,
         168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181,
         182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195,
         196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209,
         210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223,
         224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237,
         238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251,
         252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265,
         266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279,
         280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293,
         294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307,
         308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321,
         322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335,
         336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349,
         350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363,
         364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377,
         378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391,
         392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405,
         406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419,
         420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433,
         434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447,
         448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461,
         462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475,
         476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489,
         490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503,
         504, 505, 506, 507, 508, 509, 510, 511
]]))

Should that dictionary element "sentence_encoder.bert.embeddings.position_ids" be removed?

Or is there another known workaround?

[naoki623]

感谢您的来信，我已成功收取。

[disperaller]

This issue is addressed by https://github.com/thunlp/OpenNRE/issues/312, however when i tried installing transformers==3.4.0, a problem with Rust came up saying no rust compiler found. Thus, after a little search, the following command should install the Rust compiler for you: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs/ | sh
However, another problem came up, if you stuck on the installation of this rust compiler, then you may encounter the same issue as me. The solution is to export 2 environment vars first, then run that command:

1. export RUSTUP_UPDATE_ROOT=https://mirrors.ustc.edu.cn/rust-static/rustup
2. export RUSTUP_DIST_SERVER=https://mirrors.ustc.edu.cn/rust-static
   Then, try pip install the 3.4.0 transformers, it should work.