I am trying MQTTNet managed client in of my POC but its very slow.

[andy-mate4tech]

Describe your question

We have an EMQX broker and I am trying the MQTTNet managed client sample application with it.
The behaviour I am seeing is different to the PAHO Python client and want some suggestions on why this is happening and if I start using it in production what kind of challenges will I face. It's happening on the subscription side where it's not sending the PUBREL, and PUBCOMP in a timely fashion.

So my proof of concept is to use the default Managed Client example code to subscribe to a topic called /bigfile where I am publishing some big file in chunks of 10MB each.

This is from EMQX broker log trace using the managed client:

2024-03-14T08:22:16.702388+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: SUBACK(Q0, R0, D0, PacketId=1, ReasonCodes=[2,2]) 2024-03-14T08:22:17.235645+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"1"],", ",["Payload=",{error,[],[<<233,148,221,71,107,221,253,140,87,173,76,112,1,114,170,57,184,79,120,135,125,173,141,125,130,153,45,39,130,140,68,47,193,160,74,212,107,239,78,247,91,244,86,9,37,78,211,78,83,12,233,69,48,69,112,177,85,45,120,156,81,68,13,41,218,4,215,0,180,29,34,28,121,183,132,157,166,143,39,55,85,237,235,64,172,236,244,0,118,164,38,152,172,80,226,12,115,1,18,152>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:18.305082+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"2"],", ",["Payload=",{error,[],[<<234,59,15,237,143,30,53,67,178,92,123,107,120,89,37,237,75,45,42,59,207,63,158,133,127,61,1,242,32,125,89,70,199,230,146,205,113,178,5,190,224,132,5,173,4,5,57,240,102,124,78,238,46,112,68,16,178,181,57,144,253,14,41,121,142,157,216,169,73,61,67,135,75,156,77,11,195,50,224,138,197,92,33,80,40,193,188,177,185,243,225,81,157,29,131,111,37,211,62,254>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:19.242964+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"3"],", ",["Payload=",{error,[],[<<240,237,135,204,233,144,14,50,27,55,58,153,184,199,103,171,108,158,9,77,105,115,210,20,75,245,128,16,85,138,209,15,235,29,211,245,249,170,66,196,9,145,3,158,67,131,96,71,111,121,224,152,37,37,1,244,148,112,203,193,229,143,192,229,161,213,199,253,197,93,106,146,15,185,13,159,79,169,0,158,97,70,153,198,202,175,89,237,171,179,193,68,209,4,123,201,18,182,23,199>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:20.178207+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"4"],", ",["Payload=",{error,[],[<<209,62,12,225,6,206,81,118,198,154,248,76,116,197,226,251,240,237,79,209,234,241,164,30,224,72,162,176,149,175,23,146,202,25,44,43,229,232,207,249,236,90,150,207,61,228,171,177,26,197,14,212,205,44,44,192,63,57,8,199,97,209,130,23,90,27,42,124,186,29,6,180,14,195,134,187,71,156,70,51,27,99,175,206,118,45,252,122,217,16,16,53,135,26,157,59,149,235,121,184>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:21.137457+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"5"],", ",["Payload=",{error,"4hyJ",[<<228,233,222,80,66,159,185,216,145,157,6,171,119,44,41,161,242,202,25,123,129,131,196,80,146,46,159,162,12,49,105,13,188,167,145,233,68,138,135,123,53,152,120,202,29,188,154,58,53,44,65,58,165,222,108,248,167,117,247,186,81,132,222,24,201,96,36,185,233,165,36,251,226,249,11,34,68,70,238,253,32,81,129,189,55,155,59,65,42,41,135,201,227,220,30,23>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:21.321553+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_received, packet: PUBREC(Q0, R0, D0, PacketId=1, ReasonCode=0) 2024-03-14T08:22:21.321733+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: PUBREL(Q1, R0, D0, PacketId=1, ReasonCode=0) 2024-03-14T08:22:24.044901+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"6"],", ",["Payload=",{error,"u",[<<147,71,59,103,221,215,236,87,77,118,61,19,251,150,255,112,157,225,80,53,192,33,156,75,231,107,31,123,113,143,153,84,77,71,70,111,55,251,148,77,70,63,123,54,93,247,114,51,65,114,125,162,222,134,197,35,119,187,48,9,117,98,144,87,171,35,119,145,142,42,70,179,162,119,215,118,249,255,112,81,81,116,100,81,199,130,247,27,53,56,230,110,239,75,183,255,31,189,85>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:24.045169+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"7"],", ",["Payload=",{error,"k",[<<236,223,227,128,93,107,9,121,254,12,84,83,194,10,140,212,125,55,179,175,159,215,55,163,67,86,55,163,195,86,54,163,67,87,180,178,211,252,76,190,131,87,119,190,3,232,246,29,26,50,166,0,24,166,42,160,192,51,112,2,174,19,207,110,190,67,210,154,127,211,79,217,214,114,182,110,132,49,75,135,180,227,242,12,144,157,83,55,144,93,201,81,166,71,204,152,6,65,124>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:24.165872+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"8"],", ",["Payload=",{error,[],[<<177,86,238,251,184,164,187,190,173,237,116,15,205,122,146,92,21,162,14,207,21,23,146,81,225,150,183,145,34,121,180,212,155,81,5,197,67,215,157,80,60,222,137,180,226,119,167,89,251,246,75,12,12,226,119,58,180,226,174,164,83,168,35,131,99,154,23,238,236,118,173,190,214,250,122,101,141,198,170,73,173,79,4,181,183,127,31,38,189,18,222,103,217,124,186,93,236,119,196,188>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:24.710518+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_received, packet: PUBREC(Q0, R0, D0, PacketId=2, ReasonCode=0) 2024-03-14T08:22:24.710667+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: PUBREL(Q1, R0, D0, PacketId=2, ReasonCode=0) 2024-03-14T08:22:34.278360+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"9"],", ",["Payload=",{error,"p",[<<249,10,28,232,218,226,5,70,109,0,89,236,42,111,5,185,129,99,203,183,83,134,189,245,215,0,222,174,123,123,47,101,191,94,35,213,148,133,81,28,33,4,251,212,140,223,174,142,47,215,248,58,81,9,8,65,241,79,110,90,113,34,64,41,5,148,213,111,82,6,40,167,222,164,28,80,158,191,73,1,160,196,120,147,66,64,201,242,38,69,128,82,249,77,138,1,165,227,155>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:34.278587+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"10"],", ",["Payload=",{error,[],[<<252,104,152,204,251,95,38,157,228,62,213,248,137,144,115,20,165,63,217,27,156,94,173,26,98,199,59,190,49,158,146,230,199,146,59,150,215,167,152,241,78,98,41,171,195,172,203,61,109,49,143,182,71,183,22,182,102,225,41,12,2,229,249,146,209,104,191,212,87,155,109,76,20,59,242,241,156,107,87,132,94,95,98,225,156,227,162,136,70,176,74,160,91,171,211,52,75,209,142,160>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:34.278712+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"11"],", ",["Payload=",{error,[19,29,8],[<<234,65,205,216,209,77,100,217,67,186,163,123,118,65,242,109,84,35,245,25,31,136,59,118,54,135,185,188,117,167,22,122,78,241,29,161,221,202,87,189,21,150,22,96,252,235,193,234,212,98,158,51,152,197,146,159,239,117,13,11,91,157,134,163,165,120,216,187,115,210,173,177,82,247,163,119,121,216,39,115,164,11,148,246,126,103,104,78,12,130,76,113,138,211,169,6,121>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:34.278879+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"12"],", ",["Payload=",{error,"*",[<<246,34,131,10,194,0,48,142,243,217,44,186,147,124,118,80,29,233,235,132,224,82,192,140,130,252,13,27,52,197,135,60,6,179,159,34,149,150,87,64,66,2,116,232,214,101,143,159,154,125,60,196,190,87,106,206,151,37,51,78,201,49,230,31,52,81,100,117,122,220,73,79,185,80,72,57,132,190,74,79,42,81,109,199,104,112,34,72,79,59,125,115,166,192,183,0,11,194,48>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:34.278987+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"13"],", ",["Payload=",{error,[],[<<240,207,91,195,57,188,26,143,65,199,33,16,71,56,35,174,222,128,189,162,11,21,224,73,177,150,34,166,237,153,211,5,11,160,53,0,38,74,101,152,254,53,232,14,28,10,172,172,166,150,188,172,26,71,125,198,150,199,103,28,126,234,24,15,31,131,202,196,148,197,167,164,177,94,71,189,115,168,131,169,116,51,222,104,78,6,204,161,198,85,190,254,28,211,75,148,53,166,152,171>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:34.279270+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"14"],", ",["Payload=",{error,"z",[<<248,210,3,201,197,7,146,92,255,60,248,147,194,76,105,6,20,48,214,76,49,214,152,67,15,164,103,25,165,173,105,96,34,12,239,66,195,181,209,3,173,15,161,34,216,194,193,136,198,65,127,132,33,115,81,212,156,176,145,252,176,188,0,186,156,239,228,167,37,64,125,210,136,3,97,7,133,110,129,152,51,240,131,254,91,232,193,33,111,31,49,180,7,133,55,103,193,143,161>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:34.279435+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"15"],", ",["Payload=",{error,[],[<<227,221,116,217,124,225,115,118,128,49,232,173,166,246,146,75,239,83,69,167,39,183,58,28,119,183,18,163,194,202,16,164,246,27,11,168,231,87,143,112,72,184,65,168,125,142,72,84,152,0,81,170,122,228,200,155,192,19,110,189,166,122,205,115,189,253,235,224,243,239,73,198,87,113,40,83,253,180,244,56,147,59,167,74,204,30,243,75,176,96,43,38,196,139,76,232,113,118,109,56>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:34.279583+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"16"],", ",["Payload=",{error,[],[<<224,239,210,237,82,223,226,152,189,119,14,248,237,17,223,18,56,198,112,137,103,147,79,9,131,235,253,176,251,194,116,116,76,240,144,6,37,35,73,164,146,78,36,147,85,33,3,33,3,135,17,37,21,170,175,162,64,159,84,169,218,39,30,141,49,19,110,88,109,64,141,250,126,17,109,70,133,26,196,163,46,130,87,121,74,237,76,22,34,171,38,91,20,154,197,91,112,83,63,171>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:34.279782+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"17"],", ",["Payload=",{error,[],[<<137,255,19,157,240,130,165,91,119,117,185,197,134,77,147,251,56,183,21,214,56,214,193,96,143,187,114,242,130,250,107,122,119,122,17,15,2,60,196,99,184,38,188,163,68,106,39,203,248,148,126,198,81,126,168,215,19,170,83,213,243,137,37,122,245,187,82,60,131,167,229,148,238,155,183,70,58,211,251,145,205,181,9,27,227,221,34,211,171,98,194,43,83,212,253,120,12,186,110,230>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:34.279933+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"18"],", ",["Payload=",{error,[3],[<<188,142,108,198,116,170,53,95,6,190,243,200,155,107,28,45,77,247,115,108,27,46,108,46,121,82,196,8,253,226,30,190,254,69,96,248,46,103,183,144,222,9,32,1,84,46,119,57,18,236,48,29,87,101,101,59,193,126,90,158,17,209,111,172,104,88,177,125,108,231,28,53,24,198,124,235,149,178,144,159,246,164,159,34,177,225,25,73,225,140,86,150,255,235,246,147,26,130,62>>,"... The ","1048476"," bytes of this log are truncated"]}]],")"] 2024-03-14T08:22:34.280099+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_received, packet: PUBREC(Q0, R0, D0, PacketId=3, ReasonCode=0) 2024-03-14T08:22:34.280143+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: PUBREL(Q1, R0, D0, PacketId=3, ReasonCode=0) 2024-03-14T08:22:49.280806+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_received, packet: PUBREC(Q0, R0, D0, PacketId=4, ReasonCode=0) 2024-03-14T08:22:49.280932+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_sent, packet: PUBREL(Q1, R0, D0, PacketId=4, ReasonCode=0) 2024-03-14T08:22:49.281009+00:00 [MQTT] new-ak@10.7.48.142:56444 msg: mqtt_packet_received, packet: PUBCOMP(Q0, R0, D0, PacketId=1, ReasonCode=0)

You can see here the PUBREC was sent to the broker almost after 5 packets is sent to the subscribed client and the PUBCOMP is sent even after so many other transactions.

If I use the Paho client using python this how my broker log trace looks like:

2024-03-04T11:54:27.833987+00:00 [MQTT] sub_py1@10.7.48.80:40120 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"3"],", ",["Payload=",{error,"C",[<<233,204,14,224,32,225,109,214,85,48,36,168,39,181,147,108,200,5,142,135,48,15,48,204,102,250,7,174,41,119,165,34,165,231,118,29,13,153,230,20,43,55,4,13,60,94,56,16,49,68,119,27,190,87,230,162,80,95,203,94,237,48,189,150,60,208,195,120,166,29,90,229,114,163,158,133,116,85,98,35,160,73,198,117,101,37,180,95,128,4,204,23,94,90,229,9,113,74,94>>,"... The ","52428700"," bytes of this log are truncated"]}]],")"] 2024-03-04T11:54:35.514646+00:00 [MQTT] sub_py1@10.7.48.80:40120 msg: mqtt_packet_received, packet: PUBREC(Q0, R0, D0, PacketId=3, ReasonCode=0) 2024-03-04T11:54:35.514877+00:00 [MQTT] sub_py1@10.7.48.80:40120 msg: mqtt_packet_sent, packet: PUBREL(Q1, R0, D0, PacketId=3, ReasonCode=0) 2024-03-04T11:54:35.542517+00:00 [MQTT] sub_py1@10.7.48.80:40120 msg: mqtt_packet_received, packet: PUBCOMP(Q0, R0, D0, PacketId=3, ReasonCode=0) 2024-03-04T11:54:46.324408+00:00 [MQTT] sub_py1@10.7.48.80:40120 msg: mqtt_packet_sent, packet: [[80,85,66,76,73,83,72,40,81,"2",44,32,82,"0",44,32,68,"0"],", ",[[84,111,112,105,99,61,"/bigfile",44,32,80,97,99,107,101,116,73,100,61,"4"],", ",["Payload=",{error,[],[<<231,150,124,42,209,102,38,9,88,214,3,175,15,34,71,103,214,225,206,144,234,114,146,107,53,195,115,124,41,220,66,161,99,84,206,51,72,154,118,224,142,197,205,7,229,206,14,39,86,117,230,84,82,29,77,128,197,203,40,62,111,114,49,137,138,205,206,10,39,14,207,157,74,24,74,62,51,43,229,136,61,75,70,118,85,197,136,181,241,8,40,121,105,151,137,175,166,161,232,148>>,"... The ","52428700"," bytes of this log are truncated"]}]],")"] 2024-03-04T11:54:51.823267+00:00 [MQTT] sub_py1@10.7.48.80:40120 msg: mqtt_packet_received, packet: PUBREC(Q0, R0, D0, PacketId=4, ReasonCode=0) 2024-03-04T11:54:51.823510+00:00 [MQTT] sub_py1@10.7.48.80:40120 msg: mqtt_packet_sent, packet: PUBREL(Q1, R0, D0, PacketId=4, ReasonCode=0) 2024-03-04T11:54:51.850270+00:00 [MQTT] sub_py1@10.7.48.80:40120 msg: mqtt_packet_received, packet: PUBCOMP(Q0, R0, D0, PacketId=4, ReasonCode=0)

Here you can see it's very much following the QOS2 protocol where when the broker is sending the packet on the subscribed topic to the client it immediately follows the PUBREC, PUBREL and PUBCOMP before sending another packet.

Which project is your question related to?

• ManagedClient