dirbuster scans often never finish

[allendemoura]

i've adjusted my config file to use 50 threads and only the common.txt wordlist which is a lot shorter. my scans still run for hours and often never finish at all, thus never generating the nice markdown reports. the only option i see is to disable dirbusting in AutoRecon (how?) and do it manually.

potential compounding issue, when i press enter, i get the feroxbuster menu and the output of autorecon seems messed up. never shows the feroxbuster scan menu though i wonder if it is being triggered under the hood and potentially mucking things up?

[Tib3rius]

Could you post the following information to help me try and replicate:

1. The OS you're using, plus whether you are using bash or zsh.
2. A link to the wordlist (I assume common.txt is from SecLists but want to be sure)
3. An example box (HTB, THM, etc.) you're encountering these issues with.

[allendemoura]

1. Kali (VMware image from offsec), zsh (default)
2. youre correct
3. offsec PG practice Algernon

i've also looked thru the commands.log and the noticed that feroxbuster runs the -e flag, but i cant find it in the ferox documentation. what is that flag doing?

[allendemoura]

i started another control group experiment scan against algernon this morning with the above options, its currently about to cross the 1 hour mark as we speak. i have avoided triggering the ferox menu described above, just trying to get a full scan to complete for benchmark purposes. unfortunately something i've only been able to accomplish once (on localhost) in my week of messing around with your (awesome!) tool.

my VM has 4 cores and 4GB of RAM allocated. CPU usage showing very low during the dirbuster scans.

[Tib3rius]

Interesting. On my installation, -e is defined:

-e, --extract-links              Extract links from response body (html, javascript, etc...); make new requests based on findings

They must have changed it. I should probably replace single character options with full ones either way.

Out of interest could you find out what version of feroxbuster is running on your system? That is likely causing some kind of issue (as I can't reproduce the issue with my current setup).

[allendemoura]

v 2.10.1, which i think is the latest version in the kali repos

it was installed using the supplied script in your readme, i used gobuster before. though FWIW i tried running autorecon with gobuster also, and ran into some similar issues, though i think it mightve been due to the API (oops i was confusing it with a different box. its a smartermail portal) on port 9998 of algernon returning 200 on a LOT. i would have to rerun it to be sure if i'm remembering correctly.

[allendemoura]

also! this just in, the control group scan i mentioned above just finished, 1h49m. would it help if i provided the generated report?

[Tib3rius]

Yes please! If it's not too much trouble to zip up the output directory?

[allendemoura]

np, thx for your responsiveness on this

algernon.zip

[allendemoura]

after some digging i think i've uncovered something, i was probably editing the wrong config file.... i run autorecon as root with sudo, but edited my ~/user config 🤦‍♂️

that being said. the scans being run are your defaults, which per my tests (running busters outside of autorecon) seem to just be extraordinarily long running tests, at least on my hardware. perhaps it just takes 2 hours to run a gobuster/ferox scan on a medium wordlist with six file extensions. how long are these tests taking for you?

[Tib3rius]

Ah yeah that might be your issue. If you use -c or --config you can specify the file (I usually have one in my home dir).

Also I feel like I should point out that AutoRecon was originally designed for, and still has a main use-case for the OSCP exam, which traditionally had 4-5 machines you were supposed to attack at once. So the defaults were chosen with that in mind. If you're scanning a single box then it makes sense to up the number of threads certain tools use.

One of the main reasons I wrote AutoRecon v2 was to make it more customizable, so you could control the number of threads that certain tools use via a config file or even on the command line, without needing to edit the actual command being run.

My own config file uses the following settings and I run it against single boxes quite often. If you add more extensions to the dirbuster scan I would probably up the number of threads.

# Configure regular AutoRecon options at the top of this file.

nmap-append = '-T4 --min-rate=1000 --min-parallelism=1000'
verbose = 2
exclude-tags = 'wkhtmltoimage'

# Configure global options here.
# [global]
# username-wordlist = '/usr/share/seclists/Usernames/cirt-default-usernames.txt'

# Configure plugin options here.
[dirbuster]
threads = 20
ext = 'txt,html,php'

[allendemoura]

excellent, thanks. looks like when i'm actually running the common.txt wordlist, it brings the scan from 2h down to 4minutes.

i'm a current OSCP student preparing for the exam so that is definitely my use case.

what i would love to be able to do, is configure AutoRecon to run with dirbuster settings that were a bit less thorough than your defaults, so they could finish around the same time as the other scans, and generate a report, and then perhaps run the deeper dirb scans as an auxiliary. my option right now is to just run those deeper scans manually, but is there capability in AutoRecon for something like this?

edit: i see that i can supply multiple wordlists in the config file. but this will hold up the report generation, right?

[Tib3rius]

Yeah, unfortunately report generation only happens once all plugins have finished for a particular target, so adding multiple wordlists will just delay this.

I expect I'll have to change that behavior at some point.

[allendemoura]

ok. thanks for your time help with learning the tool!

[GingerRouxt]

is there a way to add dirsearch to autorecon?

[Tib3rius]

@GingerRouxt it's already in. Use --dirbuster.tool=dirsearch

[GingerRouxt]

@GingerRouxt it's already in. Use --dirbuster.tool=dirsearch

i just logged back in to delete my question. LOL. I found where to use it. Thank you my friend!. I love the tool