New, license violation-free version of WPScan #16336

paralax · 2018-01-02T14:46:54Z

built a list of WP plugins myself, no external sources, included
built an updater to keep it updated
use the new file in the updated crawl/wpscan plugin

no more license violations, however no vulnerability information

andresriancho · 2018-01-03T15:44:10Z

w3af/plugins/crawl/wpscan/update.py

+        for line in html.splitlines():
+            if line.strip().startswith("<li><a href="):
+                f.write(re.sub("<[^<]+>", "", line).strip().rstrip('/') + '\n')
+    print('done')


Could you add some kind of safety check here? If the HTML failed to download, or the file format changed, the user will still get "done" in the output and no indication that the update failed.

Maybe count the number of plugins extracted and write something like this to the console?

Found and processed 5498 plugins. Done!

andresriancho · 2018-01-03T15:44:57Z

w3af/plugins/crawl/wpscan/update.py

+
+
+URL = 'http://plugins.svn.wordpress.org/'
+PLUGIN_FILE = 'w3af/plugins/crawl/wpscan/plugins.txt'


Please write a unittest that makes sure that the plugins file has been updated in the last month.

andresriancho · 2018-01-03T16:16:23Z

Thanks for the PR!

I'm very happy that you found a way to be nice to the wpscan guys and have the same features we had before 👍

An important thing to change is the name ;-) We don't want to use wpscan since it might confuse users, the wpscan team might have registered the name, etc.

Also, if you could write a unittest using the mock responses like in https://github.com/andresriancho/w3af/blob/fe6de41c5bd539a6597fc571a0d83852b8c7defd/w3af/plugins/tests/crawl/test_find_backdoors.py that would be amazing.

You can use nosetests w3af/plugins/tests/crawl/test_wpscan.py to run the test suite.

andresriancho

I would really love to see this plugin in production!

@paralax what can I do to make this happen?

andresriancho · 2018-01-03T15:54:20Z

w3af/plugins/crawl/wpscan.py

+        if not self._exec:
+            raise RunOnce()
+        else:
+            domain_path = fuzzable_request.get_url().get_domain_path()


The plugin tries to find wordpress plugins in all site paths at least one time.

The plugin list has ~70k entries. This means that if the site has 10 paths, this plugin will generate 700k HTTP requests, which is something we can't do (at least not as default).

What do you think about implementing something like

w3af/w3af/plugins/crawl/wordpress_enumerate_users.py

Lines 54 to 61 in a53d59b

# Check if there is a wordpress installation in this directory

domain_path = fuzzable_request.get_url().get_domain_path()

wp_unique_url = domain_path.url_join('wp-login.php')

response = self._uri_opener.GET(wp_unique_url, cache=True)

# If wp_unique_url is not 404, wordpress = true

if not is_404(response):

self._enum_users(fuzzable_request)

? What this does is check if the wp-login.php file exists in the path, if it does it will perform all the checks it wants, otherwise it just ignores the path.

If something like this is implemented, I would still keep the self._already_tested in order to prevent checking if wp-login.php exists in the path more than once.

Also, if we do it like this, we could remove this code:

if not self._exec: raise RunOnce()

And everything related with it. We want to check if there site has more than one wordpress installation (in different paths), so raising RunOnce doesn't make sense.

andresriancho · 2018-01-03T15:56:34Z

w3af/plugins/crawl/wpscan.py

+        :param fuzzable_request: A fuzzable_request instance that contains
+                               (among other things) the URL to test.
+        """
+        self._plugin_list = open(os.path.join(self.BASE_PATH, 'plugins.txt'), 'r').readlines()


Several problems here:

The plugin list is kept in memory even when not used

The plugin list is read every time crawl is called.

Move this to the place where you use it (_dir_name_generator) and use xreadlines() to prevent the whole file from being in-memory

andresriancho · 2018-01-03T15:57:28Z

w3af/plugins/crawl/wpscan.py

+            try:
+                dir_url = base_path.url_join(directory_name + '/')
+            except ValueError, ve:
+                msg = 'The "%s" line at "%s" generated an ' \


This happens with the chinese / russian names in plugins.txt?

andresriancho · 2018-01-03T16:03:06Z

w3af/plugins/crawl/wpscan.py

+        :return: None, data is stored in self.output_queue
+        """
+        try:
+            http_response = self._uri_opener.GET(dir_url, cache=False)


This does an HTTP GET to target.com/wp-content/plugins/{plugin-name}/, correct?

What if the plugin is installed, but "directory indexing" is off? What is shown in the output when a request like this one is sent? Maybe the plugins.txt file should contain not only the plugin name but also a file in the plugin? A readme.txt or something?

andresriancho · 2018-01-03T16:05:10Z

w3af/plugins/crawl/wpscan.py

+            dir_url = base_path.url_join(directory_name + rand_alnum(5) + '/')
+            invalid_http_response = self._uri_opener.GET(dir_url,
+                                                         cache=False)
+            if is_404(invalid_http_response):


Just a stupid comment, but after coding a lot in python I got used to:

if not is_404(invalid_http_response): return

That way, the rest of the code that goes below is indented at the same level of the if statement and you don't have "problems" with the 80 column 'restriction' imposed by PEP-8.

andresriancho · 2018-01-03T16:11:24Z

w3af/plugins/crawl/wpscan.py

+
+        return """
+        This plugin finds WordPress plugins.
+        While it is not possible to fingerprint the plugin version automatically,


Challenge: Since all wordpress plugins seem to be in http://plugins.svn.wordpress.org/ and we have the different releases (tags: http://plugins.svn.wordpress.org/1000eb/tags/) it should be possible to not only identify that a plugin is present but also identify its version?

For example:

In the initial commit of plugin http://plugins.svn.wordpress.org/1000eb/trunk/, the file screenshot-1.png had md5sum X

In release 2.0 the same file has md5sum Y

In release 3.0 the file was removed and file screenshot-2.png was added

Using that information it should be possible to fingerprint the version, right?

I'm not saying that this should be implemented in order for the PR to be approved, but it would be a nice thing to have. After that we just need a list of vulnerable wordpress plugins and we can cross-reference them to give the user very valuable results.

paralax · 2019-03-03T00:12:53Z

andres sorry to say but i no longer work on w3af, i left the job i where i was using w3af last fall. as such i am no longer in a position to develop and maintain this plugin. - jose

…

On Sat, Mar 2, 2019 at 6:18 PM Andres Riancho ***@***.***> wrote: ***@***.**** commented on this pull request. I would really love to see this plugin in production! @paralax <https://github.com/paralax> what can I do to make this happen? ------------------------------ In w3af/plugins/crawl/wpscan.py <#16336 (comment)>: > + # Internal variables + self._exec = True + self._already_tested = DiskSet(table_prefix='wpscan') + + def crawl(self, fuzzable_request): + """ + Get the file and parse it. + + :param fuzzable_request: A fuzzable_request instance that contains + (among other things) the URL to test. + """ + self._plugin_list = open(os.path.join(self.BASE_PATH, 'plugins.txt'), 'r').readlines() + if not self._exec: + raise RunOnce() + else: + domain_path = fuzzable_request.get_url().get_domain_path() The plugin tries to find wordpress plugins in all site paths at least one time. The plugin list has ~70k entries. This means that if the site has 10 paths, this plugin will generate 700k HTTP requests, which is something we can't do (at least not as default). What do you think about implementing something like https://github.com/andresriancho/w3af/blob/a53d59b73c493013a90fccf468cf443e47f869d0/w3af/plugins/crawl/wordpress_enumerate_users.py#L54-L61 ? What this does is check if the wp-login.php file exists in the path, if it does it will perform all the checks it wants, otherwise it just ignores the path. If something like this is implemented, I would still keep the self._already_tested in order to prevent checking if wp-login.php exists in the path more than once. Also, if we do it like this, we could remove this code: if not self._exec: raise RunOnce() And everything related with it. We want to check if there site has more than one wordpress installation (in different paths), so raising RunOnce doesn't make sense. ------------------------------ In w3af/plugins/crawl/wpscan.py <#16336 (comment)>: > + def __init__(self): + CrawlPlugin.__init__(self) + self._update_plugins = False + self._plugin_list = [] + # Internal variables + self._exec = True + self._already_tested = DiskSet(table_prefix='wpscan') + + def crawl(self, fuzzable_request): + """ + Get the file and parse it. + + :param fuzzable_request: A fuzzable_request instance that contains + (among other things) the URL to test. + """ + self._plugin_list = open(os.path.join(self.BASE_PATH, 'plugins.txt'), 'r').readlines() Several problems here: - The plugin list is kept in memory even when not used - The plugin list is read every time crawl is called. Move this to the place where you use it (_dir_name_generator) and use xreadlines() to prevent the whole file from being in-memory ------------------------------ In w3af/plugins/crawl/wpscan.py <#16336 (comment)>: > + self._already_tested.add(domain_path) + self._bruteforce_plugins(domain_path) + + def _dir_name_generator(self, base_path): + """ + Simple generator that returns the names of the plugins to test. + + @yields: (A string with the directory, + a URL object with the dir name) + """ + for directory_name in self._plugin_list: + directory_name = "wp-content/plugins/" + directory_name.strip() + try: + dir_url = base_path.url_join(directory_name + '/') + except ValueError, ve: + msg = 'The "%s" line at "%s" generated an ' \ This happens with the chinese / russian names in plugins.txt? ------------------------------ In w3af/plugins/crawl/wpscan.py <#16336 (comment)>: > + msg = 'The "%s" line at "%s" generated an ' \ + 'invalid URL: %s' + om.out.debug(msg % (directory_name, + os.path.join(self.BASE_PATH, 'plugins.txt'), + ve)) + else: + yield directory_name, dir_url + + def _send_and_check(self, base_path, (directory_name, dir_url)): + """ + Performs a GET and verifies that the response is a 200. + + :return: None, data is stored in self.output_queue + """ + try: + http_response = self._uri_opener.GET(dir_url, cache=False) This does an HTTP GET to target.com/wp-content/plugins/{plugin-name}/ <http://target.com/wp-content/plugins/%7Bplugin-name%7D/>, correct? What if the plugin is installed, but "directory indexing" is off? What is shown in the output when a request like this one is sent? Maybe the plugins.txt file should contain not only the plugin name but also a file in the plugin? A readme.txt or something? ------------------------------ In w3af/plugins/crawl/wpscan.py <#16336 (comment)>: > + """ + try: + http_response = self._uri_opener.GET(dir_url, cache=False) + except: + pass + else: + if not http_response.get_code() == 200: + return + # + # Looking good, but lets see if this is a false positive + # or not... + # + dir_url = base_path.url_join(directory_name + rand_alnum(5) + '/') + invalid_http_response = self._uri_opener.GET(dir_url, + cache=False) + if is_404(invalid_http_response): Just a stupid comment, but after coding a lot in python I got used to: if not is_404(invalid_http_response): return That way, the rest of the code that goes below is indented at the same level of the if statement and you don't have "problems" with the 80 column 'restriction' imposed by PEP-8. ------------------------------ In w3af/plugins/crawl/wpscan.py <#16336 (comment)>: > + generated by the framework using the result of get_options(). + + :param OptionList: A dictionary with the options for the plugin. + + :return: No value is returned. + """ + pass + + def get_long_desc(self): + """ + :return: A DETAILED description of the plugin functions and features. + """ + + return """ + This plugin finds WordPress plugins. + While it is not possible to fingerprint the plugin version automatically, Challenge: Since all wordpress plugins seem to be in http://plugins.svn.wordpress.org/ and we have the different releases (tags: http://plugins.svn.wordpress.org/1000eb/tags/) it should be possible to not only identify that a plugin is present but also identify its version? For example: - In the initial commit of plugin http://plugins.svn.wordpress.org/1000eb/trunk/, the file screenshot-1.png had md5sum X - In release 2.0 the same file has md5sum Y - In release 3.0 the file was removed and file screenshot-2.png was added Using that information it should be possible to fingerprint the version, right? I'm not saying that this should be implemented in order for the PR to be approved, but it would be a nice thing to have. After that we just need a list of vulnerable wordpress plugins and we can cross-reference them to give the user very valuable results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16336 (review)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFW90YsGIEgFJt-j5ZoZATuRVQPAvsBIks5vSwasgaJpZM4RQr45> .

-- jose nazario For Privacy Act and Paperwork Reduction Act Notice, see page 2

andresriancho · 2019-03-03T14:27:51Z

No problem, completely understandable 👍

Will add this to my TODO list

paralax added 3 commits January 2, 2018 09:43

list of wordpress plugins scraped from the web

cc97eb6

wp plugin list updater

3b0d6de

Create wpscan.py

cdb3088

andresriancho reviewed Jan 3, 2018

View reviewed changes

andresriancho reviewed Mar 2, 2019

View reviewed changes

andresriancho self-assigned this Mar 3, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New, license violation-free version of WPScan #16336

New, license violation-free version of WPScan #16336

paralax commented Jan 2, 2018 •

edited

andresriancho Jan 3, 2018

andresriancho Jan 3, 2018

andresriancho commented Jan 3, 2018

andresriancho left a comment

andresriancho Jan 3, 2018

andresriancho Jan 3, 2018

andresriancho Jan 3, 2018

andresriancho Jan 3, 2018

andresriancho Jan 3, 2018

andresriancho Jan 3, 2018

paralax commented Mar 3, 2019 via email

andresriancho commented Mar 3, 2019



		URL = 'http://plugins.svn.wordpress.org/'
		PLUGIN_FILE = 'w3af/plugins/crawl/wpscan/plugins.txt'

	# Check if there is a wordpress installation in this directory
	domain_path = fuzzable_request.get_url().get_domain_path()
	wp_unique_url = domain_path.url_join('wp-login.php')
	response = self._uri_opener.GET(wp_unique_url, cache=True)

	# If wp_unique_url is not 404, wordpress = true
	if not is_404(response):
	self._enum_users(fuzzable_request)

New, license violation-free version of WPScan #16336

Are you sure you want to change the base?

New, license violation-free version of WPScan #16336

Conversation

paralax commented Jan 2, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andresriancho commented Jan 3, 2018

andresriancho left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paralax commented Mar 3, 2019 via email

andresriancho commented Mar 3, 2019

paralax commented Jan 2, 2018 •

edited