Unable to make HTTP/HTTPS POST request with file

[kuax]

ISSUE TYPE

• Bug Report

DIONAEA VERSION

0.11.0

CONFIGURATION

No custom configuration

OS / ENVIRONMENT

• The official Docker image of dionaea, so I'm guessing Ubuntu 18.04?

SUMMARY

When making an HTTP/HTTPS POST request, dionaea is unable to process it.

STEPS TO REPRODUCE

Spin up a container using the official dionaea image and opening port 80:

sudo docker run --rm -it -p 80:80 dinotools/dionaea:latest

Make a POST request to port 80 also sending a file (I used Postman to test the request)

EXPECTED RESULTS

No error and a correct HTTP response

ACTUAL RESULTS

Dionaea fails due to some type error

[09122020 12:12:55] root (unknown file):0: There was an error in the Python service
Traceback (most recent call last):
  File "binding.pyx", line 778, in dionaea.core.handle_io_in_cb
  File "lib/dionaea/python/dionaea/http.py", line 637, in handle_io_in
    self.handle_POST()
  File "lib/dionaea/python/dionaea/http.py", line 702, in handle_POST
    environ=tmp_environ
  File "/usr/lib/python3.6/cgi.py", line 566, in __init__
    self.read_single()
  File "/usr/lib/python3.6/cgi.py", line 757, in read_single
    self.read_binary()
  File "/usr/lib/python3.6/cgi.py", line 779, in read_binary
    self.file.write(data)
TypeError: write() argument must be str, not bytes

[welcome]

Thanks for opening your first issue here! Be sure to follow the issue template!

[phibos]

Thanks for reporting the issue. I had a look at the cPython code but was unable to find a solution yet. Looking forward to hopefully fix it in the next few days.

[kuax]

Thanks for the response. I had a quick look when I found out about this issue, but was not really able to determine where the issue lies after the http.py line 702 traceback 🤔 usually this kind of exception is just about forgetting to do a .decode('utf-8') on the data, but I'm not sure in this case

[timeaston]

Hey, the type error is just b'' vs string expected data, however you have the larger issue that TLS/SSL data is not being captured.

I was looking into why I couldn't see https POST data today & found the SSL_read() implementation in connection_tls_io_in_cb function (src/connection_tls.c) looks to be incorrect. If you do a single SSL_read() you only obtain the headers & nothing more on a basic https POST, need to continue to append the SSL_read() data in a loop until you get an error with the expected error being SSL_ERROR_WANT_READ.

Did a quick test with the below & it seems to now work, not sure if there's some events this may break as my testing was rather limited to some straight forward scripted https get/post back-and-forth. Perhaps resetting all event handlers to default at the end shouldn't happen and only do those for specific cases?

---
 src/connection_tls.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/connection_tls.c b/src/connection_tls.c
index 3a60e48..589bcdd 100644
--- a/src/connection_tls.c
+++ b/src/connection_tls.c
@@ -484,7 +484,7 @@ void connection_tls_io_in_cb(EV_P_ struct ev_io *w, int revents)
 
        unsigned char buf[recv_throttle];
        int err=0;
-       if( (err = SSL_read(con->transport.tls.ssl, buf, recv_throttle)) > 0 )
+       while( (err = SSL_read(con->transport.tls.ssl, buf, recv_throttle)) > 0 )
        {
 //             g_debug("SSL_read %i %.*s", err, err, buf);
                g_string_append_len(con->transport.tls.io_in, (gchar *)buf, err);
@@ -553,8 +553,8 @@ void connection_tls_io_in_cb(EV_P_ struct ev_io *w, int revents)
                        connection_tls_disconnect(con);
                        break;
                }
-       } else
-               if( err > 0 )
+       }
+
        {
 
                /* restore io handlers to fit default */
--