supported HTTP/2 #278

chobits · 2023-07-08T05:58:41Z

try to fix: #25

RFC: https://httpwg.org/specs/rfc7540.html#CONNECT

WIP: not ready now

…s to be changed RFC: https://httpwg.org/specs/rfc7540.html#CONNECT The location where the code needs to be changed is as follows: 1. client read before data proxying: check and process HTTP/2 header ":authority" 2. client write before data proxying: Once backend connection is successfully established, the proxy sends a HEADERS frame containing a 2xx series status code to the client, as defined in 3. client read/write during data proxying: read/write DATA frames from/to client

…connect_module into for_pull_request

chobits · 2023-07-08T06:09:09Z

phrase from RFC as following

Client sends HEADERS frame (:method = CONNECT, :authority: <host>[:<port>]) to the proxy server.
The proxy server establishes a connection with backend server, then it sends HEADERS frame with 2xx status.
The proxy server reads data from backend/client, then transmits the received data to client/backend.
- if proxy server reads data from backend, then assemble it into DATA frame and sent it to client.
- if proxy server reads DATA frame from client, then send the payload of DATA frame to backend .

…tream

chobits · 2023-07-16T12:11:20Z

HINT: How upstream reads request body from http/2 DATA frame:

If we dont wanna modify the http/2 logic in the nginx core, we need to use the logic of upstream reading request body.

ngx_http_v2_state_read_data
 -> ngx_http_v2_process_request_body(r, pos) // pos --> h2c head buf
   ngx_memcopy pos -> r->request_body->buf->last
                 data saved in r->request->body->buf->[pos, last]
   -> ngx_http_v2_filter_request_body(r)
         cl->b <= r->request->body->buf
         r->request_body->busy/free <-- ngx_chain_update_chains --- cl->b
   -> rb->post_handler(r);
       -> ngx_http_upstream_init
          -> ngx_http_upstream_init_request

              if (r->request_body) {
                  u->request_bufs = r->request_body->bufs;
              }

            -> ngx_http_upstream_connect
             -> ngx_http_upstream_send_request
              -> ngx_http_upstream_send_request_body(r, u, do_write);
                out <- u->request_bufs
                r->request_body->bufs <- NULL
                for ( ;; ) {
                     if (do_write) {
                         rc = ngx_output_chain(&u->output, out);    // send to upstream fd
                     }
                     ...
                }

…ream via http/2 protocol

…ding 200 OK ngx_http_v2_send_output_queue() will do this for us.

chobits · 2023-07-25T12:42:24Z

source notes for http/2 request body reading logic( DATA frame receiving logic):


------
http/2 connection handler：
    rev->handler = ngx_http_v2_read_handler;
                      -> ngx_http_v2_state_head
                        -> ngx_http_v2_state_data
    c->write->handler = ngx_http_v2_write_handler;

------
ngx_http_v2_state_read_data
 -> ngx_http_v2_process_request_body(r, pos) // pos --> h2c head buf
   -> ngx_memcopy(pos -> r->request_body->buf->last)
      // data saved in r->request->body->buf->[pos, last]
   -> ngx_http_v2_filter_request_body(r)
         cl->b <= r->request->body->buf
         r->request_body->busy/free <-- ngx_chain_update_chains --- cl->b
   -> rb->post_handler(r);              // TODO: is it ngx_http_init_upstream always?
       -> ngx_http_upstream_init
          -> ngx_http_upstream_init_request

              if (r->request_body) {
                  u->request_bufs = r->request_body->bufs;
              }

            -> ngx_http_upstream_connect
             -> ngx_http_upstream_send_request
              -> ngx_http_upstream_send_request_body(r, u, do_write);
                out <- u->request_bufs
                r->request_body->bufs <- NULL
                for ( ;; ) {
                    if (do_write) {
                        rc = ngx_output_chain(&u->output, out);    // send to upstream fd
                    }

                    if (r->reading_body) {
                        /* read client request body */
                        rc = ngx_http_read_unbuffered_request_body(r);
                              -> ngx_http_v2_read_unbuffered_request_body(r);
                                -> ngx_http_v2_process_request_body
                                  -> ngx_http_v2_filter_request_body
                    }
                }

chobits · 2023-07-31T07:19:28Z

If proxy_connect does not connect to backend server and DATA frame from client has already been sent to nginx, the logic backtrace is as following:

header frame + data frame ==> nginx
handle header frame , connecting to backend but waiting establishment of this connection
handle DATA frame ---. following bt log

(gdb)
1190        return ngx_http_v2_state_complete(h2c, pos, end);
(gdb) bt
#0  ngx_http_v2_state_read_data (h2c=h2c@entry=0x562d049be0d0, pos=0x562d049d1920 "hello HTTP/1.1\r\nHost: test.com\r\n\r\n", pos@entry=0x562d049d18f9 "GET /hello HTTP/1.1\r\nHost: test.com\r\n\r\nhello HTTP/1.1\r\nHost: test.com\r\n\r\n", end=end@entry=0x562d049d1920 "hello HTTP/1.1\r\nHost: test.com\r\n\r\n") at src/http/v2/ngx_http_v2.c:1190
#1  0x0000562d038190c7 in ngx_http_v2_state_data (h2c=0x562d049be0d0, pos=0x562d049d18f9 "GET /hello HTTP/1.1\r\nHost: test.com\r\n\r\nhello HTTP/1.1\r\nHost: test.com\r\n\r\n", end=0x562d049d1920 "hello HTTP/1.1\r\nHost: test.com\r\n\r\n") at src/http/v2/ngx_http_v2.c:1087
#2  0x0000562d038175c6 in ngx_http_v2_state_head (h2c=0x562d049be0d0, pos=0x562d049d18f9 "GET /hello HTTP/1.1\r\nHost: test.com\r\n\r\nhello HTTP/1.1\r\nHost: test.com\r\n\r\n", end=0x562d049d1920 "hello HTTP/1.1\r\nHost: test.com\r\n\r\n") at src/http/v2/ngx_http_v2.c:947
#3  0x0000562d038196cb in ngx_http_v2_read_handler (rev=0x7f98954e4250) at src/http/v2/ngx_http_v2.c:435
#4  0x0000562d037c2221 in ngx_epoll_process_events (cycle=0x562d048a4c80, timer=<optimized out>, flags=<optimized out>) at src/event/modules/ngx_epoll_module.c:901
#5  0x0000562d037b654e in ngx_process_events_and_timers (cycle=cycle@entry=0x562d048a4c80) at src/event/ngx_event.c:248
#6  0x0000562d037bfd8f in ngx_worker_process_cycle (cycle=0x562d048a4c80, data=<optimized out>) at src/os/unix/ngx_process_cycle.c:721
#7  0x0000562d037be1c3 in ngx_spawn_process (cycle=cycle@entry=0x562d048a4c80, proc=proc@entry=0x562d037bfc52 <ngx_worker_process_cycle>, data=data@entry=0x0, name=name@entry=0x562d038c96ea "worker process", respawn=respawn@entry=-3) at src/os/unix/ngx_process.c:199
#8  0x0000562d037bf4c1 in ngx_start_worker_processes (cycle=cycle@entry=0x562d048a4c80, n=1, type=type@entry=-3) at src/os/unix/ngx_process_cycle.c:344
#9  0x0000562d037c06b0 in ngx_master_process_cycle (cycle=0x562d048a4c80) at src/os/unix/ngx_process_cycle.c:130
#10 0x0000562d037942ed in main (argc=1, argv=<optimized out>) at src/core/nginx.c:383

debug log

2023/07/31 15:14:22 [debug] 522#0: *16 event timer add: 17: 10000:22052976
2023/07/31 15:14:22 [debug] 522#0: *16 http finalize request: -4, "?" a:1, c:2
2023/07/31 15:14:22 [debug] 522#0: *16 http request count:2 blk:0
2023/07/31 15:14:22 [debug] 522#0: *16 http2 frame complete pos:0000562D049D191B end:0000562D049D191B
2023/07/31 15:14:22 [debug] 522#0: *16 http2 read handler
2023/07/31 15:14:22 [debug] 522#0: *16 SSL_read: 48
2023/07/31 15:14:22 [debug] 522#0: *16 SSL_read: -1
2023/07/31 15:14:22 [debug] 522#0: *16 SSL_get_error: 2
2023/07/31 15:14:22 [debug] 522#0: *16 http2 frame type:0 f:1 l:39 sid:1
2023/07/31 15:14:22 [debug] 522#0: *16 http2 DATA frame
2023/07/31 15:14:22 [debug] 522#0: *16 malloc: 0000562D04A15910:65536
2023/07/31 15:14:22 [debug] 522#0: *16 http2 frame complete pos:0000562D049D1920 end:0000562D049D1920

chobits · 2023-07-31T08:59:22Z

for http2 & upstream:

read upstream and send data to downstream: ngx_http_upstream_process_non_buffered_request
read downstream and send data to upstream: ngx_http_upstream_read_request_handler->ngx_http_upstream_send_request

chobits · 2023-08-02T10:27:26Z

Unfortunately, there is no low-level IO operation API of c->recv()/send() provided in nginx's HTTPv2 implementation. So if we want to support http/2 CONNECT tunnel, we need to use request_body API for reading DATA frames from http/2 request and output_filter for sending DATA frames through the http/2 CONNECT tunnel.

This makes the implementation of supporting HTTP/2 in this module more complex.

doublex · 2023-12-19T13:05:13Z

Great feature!!
Thanks for your efforts!

doublex · 2023-12-22T12:28:29Z

Question:
Is the hard part downstream or upstream?
Would it be easier if only the downstream-connection is http2 (so eavesdroppers think it is http2)?
E.g.:
Client -> http2:connect -> nginx:chobits -> http11 -> website

chobits · 2023-12-25T02:36:25Z

Question: Is the hard part downstream or upstream? Would it be easier if only the downstream-connection is http2 (so eavesdroppers think it is http2)? E.g.: Client -> http2:connect -> nginx:chobits -> http11 -> website

This pr is under development, so some part of the logic is not completed. so it could not support the HTTP2 now.

chobits added 3 commits July 8, 2023 12:01

supported http/2: return 400 while failing to parse connect host

672721a

Merge branch 'for_pull_request' of github.com:chobits/ngx_http_proxy_…

c58c08a

…connect_module into for_pull_request

chobits changed the title ~~supported HTTP/2~~ [WIP] supported HTTP/2 Jul 8, 2023

chobits added 5 commits July 14, 2023 20:55

supported http/2 (part3): process connect host and port from HTTP/2 s…

9a7255d

…tream

supported http/2 (part3.1): added debug log and testing for port

4fc823e

supported http/2 (part4): added template for test case

d6bf313

supported http/2 (part5): sent "200" status for tunnel establishment

b71ebfa

supported http/2 (part5.1): completed logic of sending status 200 OK

2ed77da

chobits added 3 commits July 21, 2023 19:48

supported http/2 (part5.2): completed logic of sending data to downst…

bdae33b

…ream via http/2 protocol

supported http/2 (part5.3): removed logic of handling event after sen…

0ae1f3f

…ding 200 OK ngx_http_v2_send_output_queue() will do this for us.

supported http/2 (part5.4): added request body reading logic

357a2d4

supported http/2 (part5.5): refactor IO operations to buf/chain API

37c42ba

chobits marked this pull request as draft September 4, 2023 03:18

chobits changed the title ~~[WIP] supported HTTP/2~~ supported HTTP/2 Sep 4, 2023

chobits mentioned this pull request Dec 11, 2023

quic protocol #292

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

supported HTTP/2 #278

supported HTTP/2 #278

chobits commented Jul 8, 2023 •

edited

chobits commented Jul 8, 2023 •

edited

chobits commented Jul 16, 2023 •

edited

chobits commented Jul 25, 2023

chobits commented Jul 31, 2023 •

edited

chobits commented Jul 31, 2023 •

edited

chobits commented Aug 2, 2023 •

edited

doublex commented Dec 19, 2023

doublex commented Dec 22, 2023

chobits commented Dec 25, 2023

supported HTTP/2 #278

Are you sure you want to change the base?

supported HTTP/2 #278

Conversation

chobits commented Jul 8, 2023 • edited

chobits commented Jul 8, 2023 • edited

chobits commented Jul 16, 2023 • edited

chobits commented Jul 25, 2023

chobits commented Jul 31, 2023 • edited

chobits commented Jul 31, 2023 • edited

chobits commented Aug 2, 2023 • edited

doublex commented Dec 19, 2023

doublex commented Dec 22, 2023

chobits commented Dec 25, 2023

chobits commented Jul 8, 2023 •

edited

chobits commented Jul 8, 2023 •

edited

chobits commented Jul 16, 2023 •

edited

chobits commented Jul 31, 2023 •

edited

chobits commented Jul 31, 2023 •

edited

chobits commented Aug 2, 2023 •

edited