Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Networking rampage and accumulated fixes #3060

Merged
merged 17 commits into from Apr 7, 2020
Merged

Networking rampage and accumulated fixes #3060

merged 17 commits into from Apr 7, 2020

Conversation

nwf
Copy link
Member

@nwf nwf commented Apr 2, 2020

  • This PR is for the dev branch rather than for master.
  • This PR is compliant with the other contributing guidelines as well (if not, please describe why).
  • I have thoroughly tested my contribution.
  • The code changes are reflected in the documentation at docs/*.

This represents the work done thus far on a few issues; #3032 and easy aspects of #3004 (xref #3006) as well as refactoring and simplification around the network-facing things I use regularly. I've been carrying these patches locally for a while and would like to push them upstream, at least to dev, for others to beat about a bit and so that other things stop feeling quite so blocked on these. Much of this has been subject to testing via the test harness in my https://github.com/nwf/nodemcu-firmware/tree/dev-active branch (recall #2983), so I feel fairly confident in having not broken anything obvious.

The mqtt and tls "stop using espconn->reverse" refactoring could be done for other network-facing modules (http, coap, and websocket) as well and would probably be similarly beneficial. Then we can remove the ->reverse field from espconn and make a little more progress on #3006. I don't feel confident in changing code in the remaining ones, absent test support.

ETA: removed espconn->reverse from tls, too, and fixed a stack imbalance issue in the new TLS cert/key callback code.

@nwf
Copy link
Member Author

nwf commented Apr 3, 2020

ETA, again: found and fixed a memory leak in the new tls cert/key callback API (just a typo, sorry). I've adjusted my test harness to exercise the new API better and believe that this class of bug is no longer present.

@marcelstoer
Copy link
Member

Cool! I like every PR that removes more code than it adds. Can we land this on dev sooner rather than later?

@nwf
Copy link
Member Author

nwf commented Apr 4, 2020

I'm happy to hit merge and deal with any fallout, but I suspect it would be better if someone else gave things a quick run-through first. ;)

nwf added 12 commits April 5, 2020 03:30
This is the easiest part of nodemcu#3004 .
It removes a bunch of functions that were never called in our tree.
Further work on nodemcu#3004

While here, remove `mqtt`'s charming DNS-retry logic (which is neither
shared with nor duplicated in other modules) and update its :connect()
return value behavior and documentation.
A write-only global!  How about that.
All the TLS stuff moved over there a long time ago, and
net_createUDPSocket should just do what it says on the tin.
We can barely function as a TLS client; being a TLS server seems like a
real stretch.  This code was never called from Lua anyway.
There is nothing "ssl_packet" about this structure.  Get rid of the
terrifying "pbuffer" pointer.

Squash two structure types together and eliminate an unused field.
Split out espconn_mbedtls_parse, which we can use as part of our effort
towards addressing nodemcu#3032
The new feature part of nodemcu#3032
Subsequent work will remove the old mechanism.
@nwf nwf force-pushed the for-upstream branch 2 times, most recently from e38ce43 to 5bf1665 Compare April 5, 2020 22:55
@marcelstoer
Copy link
Member

marcelstoer commented Apr 7, 2020

I think it would be very valuable (and thus welcome by the community) if we merged dev back to master in the coming weeks. It's been almost six months but there are a few things pending: https://github.com/nodemcu/nodemcu-firmware/milestone/14. So, I wouldn't mind landing this on dev soonish as it looks quite mature.

nwf added 3 commits April 7, 2020 12:58
Everywhere we have the mqtt_state_t we also have the lmqtt_userdata.
Instead, just place the espconn structure itself at the top of the user
data.  This enlarges the structure somewhat but removes one more layer
of dynamic heap usage and NULL checks.

While here, simplify the code a bit.
@nwf
Copy link
Member Author

nwf commented Apr 7, 2020

I'm worried about the tls callback changes (1cc2040), which probably merit some more discussion than they've gotten (as part of #3062). I've pulled those from this PR and will open another for the next round of rampaging.

@nwf nwf merged commit f5aa0af into nodemcu:dev Apr 7, 2020
@nwf
Copy link
Member Author

nwf commented Apr 7, 2020

Argh, I didn't really mean to squash all those together. I'm feeling jittery about more of them, now, too, but whatever, I think I can live with some egg on my face and follow-up commits to fix bugs as we find them.

marcelstoer pushed a commit that referenced this pull request Jun 9, 2020
* espconn: remove unused espconn code, take 1

This is the easiest part of #3004 .
It removes a bunch of functions that were never called in our tree.

* espconn: De-orbit espconn_gethostbyname

Further work on #3004

While here, remove `mqtt`'s charming DNS-retry logic (which is neither
shared with nor duplicated in other modules) and update its :connect()
return value behavior and documentation.

* espconn: remove scary global pktinfo

A write-only global!  How about that.

* net: remove deprecated methods

All the TLS stuff moved over there a long time ago, and
net_createUDPSocket should just do what it says on the tin.

* espconn_secure: remove ESPCONN_SERVER support

We can barely function as a TLS client; being a TLS server seems like a
real stretch.  This code was never called from Lua anyway.

* espconn_secure: more code removal

* espconn_secure: simplify ssl options structure

There is nothing "ssl_packet" about this structure.  Get rid of the
terrifying "pbuffer" pointer.

Squash two structure types together and eliminate an unused field.

* espconn_secure: refactor mbedtls_msg_info_load

Split out espconn_mbedtls_parse, which we can use as part of our effort
towards addressing #3032

* espconn_secure: introduce TLS cert/key callbacks

The new feature part of #3032
Subsequent work will remove the old mechanism.

* tls: add deprecation warnings

* luacheck: net.ifinfo is a thing now

* tls: remove use of espconn->reverse

* mqtt: stop using espconn->reverse

Instead, just place the espconn structure itself at the top of the user
data.  This enlarges the structure somewhat but removes one more layer
of dynamic heap usage and NULL checks.

While here, simplify the code a bit.

* mqtt: remove redundant pointer to connect_info

Everywhere we have the mqtt_state_t we also have the lmqtt_userdata.

* mqtt: doc fixes

* mqtt: note bug

* tls: allow :on(...,nil) to unregister a callback
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants