NA SM: fix NA_NO_BLOCK progress path to be aligned with notification path

[mdorier]

I know busy-spinning is meant for networks like GNI, where blocking is inefficient, but if I try it with na+sm, it simply doesn't work.

I do this on the server:

hg_opt.na_init_info.progress_mode = NA_NO_BLOCK;

then pass this hg_opt to margo_init_opt so it gets passed to Mercury when MErcury is initialized. For now I'm not using busy-spinning on the client side (the client is initialized normally).

When the client looks up the server's address, I get the following error:

# NA -- Error -- /tmp/mdorier/spack-stage/spack-stage-mercury-2.0.0rc1-uca52ro7tw7sm5lm5tgitbcmyl2ti2rs/spack-src/src/na/na_sm.c:2536
 # na_sm_addr_event_send(): sendmsg() failed (No such file or directory)
# NA -- Error -- /tmp/mdorier/spack-stage/spack-stage-mercury-2.0.0rc1-uca52ro7tw7sm5lm5tgitbcmyl2ti2rs/spack-src/src/na/na_sm.c:2279
 # na_sm_addr_lookup_insert_cb(): Could not send addr events
# NA -- Error -- /tmp/mdorier/spack-stage/spack-stage-mercury-2.0.0rc1-uca52ro7tw7sm5lm5tgitbcmyl2ti2rs/spack-src/src/na/na_sm.c:2210
 # na_sm_addr_map_insert(): Could not execute insertion callback
# NA -- Error -- /tmp/mdorier/spack-stage/spack-stage-mercury-2.0.0rc1-uca52ro7tw7sm5lm5tgitbcmyl2ti2rs/spack-src/src/na/na_sm.c:3435
 # na_sm_addr_lookup(): Could not insert new address
# HG -- Error -- /tmp/mdorier/spack-stage/spack-stage-mercury-2.0.0rc1-uca52ro7tw7sm5lm5tgitbcmyl2ti2rs/spack-src/src/mercury_core.c:1220
 # hg_core_addr_lookup(): Could not lookup address na+sm://5729/0 (NA_NOENTRY)
# HG -- Error -- /tmp/mdorier/spack-stage/spack-stage-mercury-2.0.0rc1-uca52ro7tw7sm5lm5tgitbcmyl2ti2rs/spack-src/src/mercury_core.c:3850
 # HG_Core_addr_lookup2(): Could not lookup address
# HG -- Error -- /tmp/mdorier/spack-stage/spack-stage-mercury-2.0.0rc1-uca52ro7tw7sm5lm5tgitbcmyl2ti2rs/spack-src/src/mercury.c:1490
 # HG_Addr_lookup2(): Could not lookup na+sm://5729/0 (HG_NOENTRY)

[mdorier]

Note: I tested this with Mercury 2.0.0rc1 and 2.0.0a1.

[soumagne]

@mdorier yes you currently need to set both the server and the client in that mode. It would not work otherwise currently, this mode is special and bypasses the notification mechanism.