New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migrate request context on mitmHandler #5
Comments
Same with |
And strangely enough, any headers set in |
You should instantiate mitmHandler as follows:
Because middleware is implemented by As middleware is implemented by |
Ahh. not quite sure how I missed that, thanks @telanflow ! |
@telanflow can you elaborate on which context I should set/get the value to get it to pass from I am using if req.Method == http.MethodConnect {
testID, err = pkg.ExtractTestID(req)
} else {
// if not, we really need a testID to be present in the context to continue
var ok bool
testID, ok = ctx.Context.Value("testid").(uint64)
if !ok { err = errors.Errorf("invalid testid in context: %+v", req.Context().Value("testid")) }
}
if testID <= 0 || err != nil {
logger.WithError(err).Error("could not extract testid")
return req, newHttpResponse(req, http.StatusForbidden, "Forbidden", "invalid credentials provided")
}
if req.Method == http.MethodConnect {
ctx.Context = context.WithValue(ctx.Context, "testid", testID)
return req.WithContext(context.WithValue(req.Context(), "testid", testID)), nil
} |
I tried req = req.WithContext(context.WithValue(req.Context(), "testid", testID))
ctx = ctx.WithRequest(req)
ctx.Context = context.WithValue(ctx.Context, "testid", testID)
return req, nil |
I'm sorry, i went out to play on China National Day. you can do that: // set value
ctx.Context = context.WithValue(ctx.Context, "testid", testID)
// get value
ctx.Context.Value("testId") Or // set value
req.WithContext(context.WithValue(req.Context(), "testid", testID))
// Get value
req.Context().Value("testId") |
@telanflow is it possible it won't propagate to the actual requests because the GET request is inside the CONNECT tunnel, hence a different proxy context perhaps? e.g.
I am doing it as you stipulated:
|
yes, It's best to set it above the request |
@telanflow can you perhaps elaborate? Not sure what you mean; proxyInstance.OnRequest().DoFunc(func(req *http.Request, ctx *mps.Context) (*http.Request, *http.Response) {
logger := log.WithField("ip", req.RemoteAddr)
var testID uint64
var err error
// if this is a CONNECT, scrape the testID from the request TLS metadata
if req.Method == http.MethodConnect {
testID, err = pkg.ExtractTestID(req)
} else {
// if not, we really need a testID to be present in the context to continue
var ok bool
testID, ok = ctx.Request.Context().Value("testid").(uint64)
if !ok { err = errors.Errorf("invalid testid in context: %+v", req.Context().Value("testid")) }
}
if testID <= 0 || err != nil {
logger.WithError(err).Error("could not extract testid")
return req, newHttpResponse(req, http.StatusForbidden, "Forbidden", "invalid credentials provided")
}
// if this is a request to our proxy, set context and pass for the subsequent real requests to work
if req.Method == http.MethodConnect {
req = req.WithContext(context.WithValue(req.Context(), "testid", testID))
ctx = ctx.WithRequest(req)
ctx.Context = context.WithValue(ctx.Context, "testid", testID)
return req, nil
}
....
} |
Each request spawns a new Handler for the execution middleware (MITMHandler, TunnelHandler)
eg. Lines 94 to 108 in 00583b5
Lines 44 to 58 in 00583b5
|
But proxy context should pass on between those handlers, right? |
Take a look at the latest submission so that should work @hazcod |
@telanflow thanks for the quick response, but still returns nil: proxyInstance.OnRequest().DoFunc(func(req *http.Request, ctx *mps.Context) (*http.Request, *http.Response) {
logger := log.WithField("ip", req.RemoteAddr)
var testID uint64
var err error
// if this is a CONNECT, scrape the testID from the request TLS metadata
if req.Method == http.MethodConnect {
testID, err = pkg.ExtractTestID(req)
} else {
// if not, we really need a testID to be present in the context to continue
var ok bool
log.Info(req.Context().Value("testid"))
testID, ok = ctx.Context.Value("testid").(uint64)
if !ok { err = errors.Errorf("invalid testid in context: %+v", req.Context().Value("testid")) }
}
if testID <= 0 || err != nil {
logger.WithError(err).Error("could not extract testid")
return req, newHttpResponse(req, http.StatusForbidden, "Forbidden", "invalid credentials provided")
}
// if this is a request to our proxy, set context and pass for the subsequent real requests to work
if req.Method == http.MethodConnect {
ctx.Context = context.WithValue(ctx.Context, "testid", testID)
return req, nil
}
...
} Returns:
|
Is this row not getting the value? log.Info(req.Context().Value("testid")) Where do they set a testid? one thing to note: |
@telanflow I've split it up to make it more clear; proxyInstance.OnRequest().DoFunc(func(req *http.Request, ctx *mps.Context) (*http.Request, *http.Response) {
logger := log.WithField("ip", req.RemoteAddr).WithField("method", req.Method)
if req.Method != http.MethodConnect {
return req, nil
}
// if this is a CONNECT, scrape the testID from the request TLS metadata
testID, err := pkg.ExtractTestID(req)
if err != nil {
logger.WithError(err).Error("could not extract testid")
return req, newHttpResponse(req, http.StatusForbidden, "Forbidden", "invalid credentials provided")
}
ctx.Context = context.WithValue(ctx.Context, "testid", testID)
return req, nil
})
proxyInstance.OnRequest().DoFunc(func(req *http.Request, ctx *mps.Context) (*http.Request, *http.Response) {
logger := log.WithField("ip", req.RemoteAddr).WithField("method", req.Method)
// if not, we really need a testID to be present in the context to continue
var ok bool
logger.Info(req.Context().Value("testid"), ctx.Context.Value("testid"))
testID, ok := ctx.Context.Value("testid").(uint64)
if !ok { err = errors.Errorf("invalid testid in context: %+v", req.Context().Value("testid")) }
if testID <= 0 || err != nil {
logger.WithError(err).Error("could not extract testid")
return req, newHttpResponse(req, http.StatusForbidden, "Forbidden", "invalid credentials provided")
}
logger = logger.WithField("testid", testID)
// if this is a request to our proxy, set context and pass for the subsequent real requests to work
if req.Method == http.MethodConnect {
return req, nil
}
return req, nil
}) And the logs, showing the testID context being set the first time
|
@telanflow : this made me think, will proxyInstance := mps.NewHttpProxy()
mitmHandler := mps.NewMitmHandlerWithContext(proxyInstance.Ctx)
proxyInstance.HandleConnect = mitmHandler
proxyInstance.HttpHandler = mitmHandler
proxyInstance.OnRequest().DoFunc(...) |
Hmm, same result with: proxyInstance := mps.NewHttpProxy()
proxyInstance.HandleConnect = mps.NewMitmHandlerWithContext(proxyInstance.Ctx)
proxyInstance.UseFunc(func(req *http.Request, ctx *mps.Context) (*http.Response, error) {
...
} |
@telanflow sorry for bugging, but any idea? |
I'm sorry. I've been a little busy lately. The code is as follows: func main() {
proxy := mps.NewHttpProxy()
proxy.HandleConnect = mps.NewMitmHandlerWithContext(proxy.Ctx)
proxy.OnRequest().DoFunc(func(req *http.Request, ctx *mps.Context) (*http.Request, *http.Response) {
// request Context with value
reqCtx := context.WithValue(req.Context(), "testid", "1")
req = req.WithContext(reqCtx)
// mps.Context with value
ctx.Context = context.WithValue(ctx.Context, "testid", "2")
return req, nil
})
proxy.OnRequest().DoFunc(func(req *http.Request, ctx *mps.Context) (*http.Request, *http.Response) {
// get req.Context() value
reqVal := req.Context().Value("testid")
log.Printf("Method: %s URL: %s req:testid: %v", req.Method, req.URL, reqVal)
// get mps.Context.Context value
ctxVal := ctx.Context.Value("testid")
log.Printf("Method: %s URL: %s ctx:testid: %v", req.Method, req.URL, ctxVal)
return req, nil
})
_ = http.ListenAndServe(":8888", proxy)
} logs:
|
@telanflow thanks for the support, but this not seem to propagate with your latest mps release. liTLS := tls.NewListener(li, &tls.Config{
Certificates: []tls.Certificate{*serverCrt, caCert},
ClientAuth: tls.RequireAndVerifyClientCert,
ClientCAs: clientCACertPool,
PreferServerCipherSuites: true,
MinVersion: tls.VersionTLS12,
MaxVersion: tls.VersionTLS13,
Renegotiation: tls.RenegotiateNever,
SessionTicketsDisabled: true,
NextProtos: []string{"http/1.1", "http/1.0"},
}) Because I get EDIT: in your case, you are -always- setting testID because your are not skipping the first handler if is -not- CONNECT. try with: func main() {
proxy := mps.NewHttpProxy()
proxy.HandleConnect = mps.NewMitmHandlerWithContext(proxy.Ctx)
proxy.OnRequest().DoFunc(func(req *http.Request, ctx *mps.Context) (*http.Request, *http.Response) {
if req.Method != http.MethodConnect { return req, nil }
// request Context with value
reqCtx := context.WithValue(req.Context(), "testid", "1")
req = req.WithContext(reqCtx)
// mps.Context with value
ctx.Context = context.WithValue(ctx.Context, "testid", "2")
return req, nil
})
proxy.OnRequest().DoFunc(func(req *http.Request, ctx *mps.Context) (*http.Request, *http.Response) {
if req.Method == http.MethodConnect { return req, nil }
// get req.Context() value
reqVal := req.Context().Value("testid")
log.Printf("Method: %s URL: %s req:testid: %v", req.Method, req.URL, reqVal)
// get mps.Context.Context value
ctxVal := ctx.Context.Value("testid")
log.Printf("Method: %s URL: %s ctx:testid: %v", req.Method, req.URL, ctxVal)
return req, nil
})
_ = http.ListenAndServe(":8888", proxy)
} |
you should know that an HTTP request is executed sequentially to the middleware, and if middleware 1 skips it, middleware 2 will not get the value.
You can set MPS Proxy as a property of another object to uniformly set your own values. package main
import (
"context"
"errors"
"github.com/telanflow/mps"
"log"
"net/http"
"os"
"os/signal"
"syscall"
)
type Srv struct {
server *http.Server
proxyHandler *mps.HttpProxy
ctx context.Context
signChan chan os.Signal
}
func NewSrv(addr string) *Srv {
proxyHandler := mps.NewHttpProxy()
proxyHandler.HandleConnect = mps.NewMitmHandlerWithContext(proxyHandler.Ctx)
return &Srv{
signChan: make(chan os.Signal),
proxyHandler: proxyHandler,
ctx: context.Background(),
server: &http.Server{
Addr: addr,
Handler: proxyHandler,
},
}
}
func (s *Srv) Run() {
// proxy on request
s.proxyHandler.OnRequest().DoFunc(s.handleRequest1)
s.proxyHandler.OnRequest().DoFunc(s.handleRequest2)
// listen quit notify
signal.Notify(s.signChan, syscall.SIGINT, syscall.SIGKILL, syscall.SIGTERM, syscall.SIGQUIT)
// start server
go func() {
log.Println("proxy server started")
err := s.server.ListenAndServe()
if errors.Is(err, http.ErrServerClosed) {
return
}
if err != nil {
s.signChan <- syscall.SIGKILL
log.Fatalf("Http Fail: %v", err)
}
}()
// stop
<-s.signChan
s.server.Shutdown(context.Background())
log.Fatal("proxy server exit.")
}
// Handle Request 1
func (s *Srv) handleRequest1(req *http.Request, ctx *mps.Context) (*http.Request, *http.Response) {
if req.Method != http.MethodConnect { return req, nil }
// mps.Context with value
s.ctx = context.WithValue(s.ctx, "testid", "3")
return req, nil
}
// Handle Request 2
func (s *Srv) handleRequest2(req *http.Request, ctx *mps.Context) (*http.Request, *http.Response) {
if req.Method == http.MethodConnect { return req, nil }
// get mps.Context.Context value
ctxVal := s.ctx.Value("testid")
log.Printf("Method: %s URL: %s s.ctx:testid: %v", req.Method, req.URL, ctxVal)
return req, nil
}
func main() {
// start proxy server
srv := NewSrv("127.0.0.1:8888")
srv.Run()
} logs:
|
Sorry, I'm not good at English, haha |
@telanflow but wouldn't that create a race condition on Also, the first middleware will only need to run on CONNECT while a second can only verify a non-CONNECT. |
I'm sorry, I don't really understand what you want to do. Can you describe it? can you give a complete example of a failed requirement? |
@telanflow yeah sorry for not specifying myself clearly enough.
Issue a test request: % ALL_PROXY="http://localhost:9999" curl -k -L https://google.be/
out of scope request% And you'll see the issue: INFO[0001] request method=CONNECT url="//google.be:443"
INFO[0001] set testid ip="127.0.0.1:58652" method=CONNECT testid=foo
INFO[0001] request method=GET url="https://google.be:443/"
WARN[0001] testid not found in context ip="127.0.0.1:58652" method=GET proxy_ctx=context.Background req_ctx=context.Background testid=0 |
but if you would use your |
you should be aware of the HTTPS proxy process:
the lifetime of the Context and the request binding, you can't get the Context from the last request in the next request.
you can avoid concurrency problems by lock |
Hmm, but if the |
Hi, so I have a TLS listener with my mps proxy.
During handshake, I generate a variable and need to pass this to my
.OnRequest(...)
function.However, anything I set on req Context with the following function is not propagated?
The text was updated successfully, but these errors were encountered: