Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

parse ipaddress from output http traffic under http proxy #160

Open
djerryz opened this issue Aug 25, 2022 · 5 comments
Open

parse ipaddress from output http traffic under http proxy #160

djerryz opened this issue Aug 25, 2022 · 5 comments
Labels
Type: Enhancement Most issues will probably ask for additions or changes.

Comments

@djerryz
Copy link

djerryz commented Aug 25, 2022

When use socks proxy and access "https://www.google.com" , logs output like "ipaddress_anythinghere.txt".
And i can parse info ipaddress( in file name ), Host (in file content).

But use http proxy , logs output like "domainname_anthinghere.txt", try to pare info from it , just can get Host ,usually the Host equal to domainname so i did not know the traffic send to which ipaddress (which one i want to port scan).
@djerryz djerryz added the Type: Enhancement Most issues will probably ask for additions or changes. label Aug 25, 2022
@djerryz djerryz changed the title parse ipaddress from output http traffic parse ipaddress from output http traffic under http proxy Aug 25, 2022
@djerryz
Copy link
Author

djerryz commented Aug 25, 2022 •
edited

real word scene, after i test host bind to difference ip address, i analysis traffic log and try to store in db , but i do not know which traffic should related to which ip address~

@djerryz
Copy link
Author

djerryz commented Aug 25, 2022 •
edited

look through the source code , seems necessary to create a variable to maps between domain names and IP addresses when the domain first time come to httpproxy.OnRequest function.
for sure , should do some dns resolve work

@ehsandeep
Copy link
Member

@djerryz Please include example information/logs for the specific issue mentioned in this case.

@djerryz
Copy link
Author

djerryz commented Sep 1, 2022

output with http proxy - log name: www.baidu.com:443-cc3heitgu9resig1q300.txt

GET /arcade/intro HTTP/1.1
Host: www.baidu.com
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Sec-Ch-Ua: "Chromium";v="104", " Not A;Brand";v="99", "Google Chrome";v="104"
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua-Platform: "Linux"
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.101 Safari/537.36

HTTP/1.1 404 Not Found
Content-Length: 210
Content-Type: text/html; charset=iso-8859-1
Date: Thu, 25 Aug 2022 06:31:40 GMT
Server: Apache

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>

output with socks proxy - log name: 45.113.192.101:443-cc3hetdgu9resig1q3lg.txt

GET https://45.113.192.101:443/arcade/intro HTTP/1.1
Host: www.baidu.com
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Sec-Ch-Ua: "Chromium";v="104", " Not A;Brand";v="99", "Google Chrome";v="104"
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua-Platform: "Linux"
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.101 Safari/537.36

HTTP/1.1 404 Not Found
Content-Length: 210
Content-Type: text/html; charset=iso-8859-1
Date: Thu, 25 Aug 2022 06:31:40 GMT
Server: Apache

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>

In sock model log , i can get info : scheme: https , domain: www.baidu.com , ipaddress: 45.113.192.101, port: 443.
But in http model log , i just can get info : domain: www.baidu.com , port: 443.

in my case, i handle traffic log in async way , so when i get one http model log , i don't know which ipaddress and scheme when this traffic sending .
@ehsandeep

@djerryz
Copy link
Author

djerryz commented Sep 1, 2022

need Equivalent information as standard output

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Type: Enhancement Most issues will probably ask for additions or changes.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ehsandeep @djerryz