diff --git a/README.md b/README.md index 89740ff..a0b176f 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@ ## Framework for Testing WAFs (FTW) [![Build Status](https://travis-ci.org/fastly/ftw.svg?branch=master)](https://travis-ci.org/fastly/ftw) -##### Purpose -This project was created by researchers from ModSecurity and Fastly to help provide rigorous tests for WAF rules. It uses the OWASP Core Ruleset V3 as a baseline to test rules on a WAF. Each rule from the ruleset is loaded into a YAML file that issues HTTP requests that will trigger these rules. +##### Purpose +This project was created by researchers from ModSecurity and Fastly to help provide rigorous tests for WAF rules. It uses the OWASP Core Ruleset V3 as a baseline to test rules on a WAF. Each rule from the ruleset is loaded into a YAML file that issues HTTP requests that will trigger these rules. Goals / Use cases include: @@ -29,7 +29,8 @@ If you require an environment for testing WAF rules, there has been one created ## Running Tests while overriding destination address in the yaml files to custom domain * *start your test web server* -* `py.test test/test_default.py --ruledir=test/yaml --destaddr=domain.com` +* `py.test test/test_default.py --ruledir=test/yaml --destaddr=domain.com +--port 443 --protocol https` ## Run integration test, local webserver, may have to use sudo * `py.test test/integration/test_logcontains.py -s --ruledir=test/integration/` @@ -43,7 +44,7 @@ If you require an environment for testing WAF rules, there has been one created 3. Get the logs, store them in an array of strings and return it from `get_logs()` 4. Make use of `py.test fixtures`. Use a function decorator `@pytest.fixture`, return your new `LogChecker` object. Whenever you use a function argument in your tests that matches the name of that `@pytest.fixture`, it will instantiate your object and make it easier to run tests. An example of this is in the python file from step 1. 5. Write a testing configuration in the `*.yaml` format as seen in `test/integration/LOGCONTAINSFIXTURE.yaml`, the `log_contains` line requires a string that is a regex. FTW will compile the `log_contains` string from each stage in the YAML file into a regex. This regex will then be used alongside the lines of logs passed in from `get_logs()` to look for a match. The `log_contains` string, then, should be a unique rule-id as FTW is greedy and will pass on the first match. False positives are mitigated from the start/end time passed to the `LogChecker` object, but it is best to stay safe and use unique regexes. -6. For each stage, the `get_logs()` function is called, so be sure to account for API calls if thats how you retrieve your logs. +6. For each stage, the `get_logs()` function is called, so be sure to account for API calls if thats how you retrieve your logs. ## Making HTTP requests programmatically Although it is preferred to make requests using the YAML format, often automated tests require making many dynamic requests. In such a case it is recommended to make use of the py.test framework in order to produce test cases that can be run as part of the whole. @@ -53,4 +54,3 @@ Generally making an HTTP request is simple: 3. provide the instance of the input class to `HttpUA.send_request()` *For some examples see the http integration tests* - diff --git a/ftw/pytest_plugin.py b/ftw/pytest_plugin.py index 9349f52..df94d8e 100644 --- a/ftw/pytest_plugin.py +++ b/ftw/pytest_plugin.py @@ -6,7 +6,7 @@ def get_testdata(rulesets): """ In order to do test-level parametrization (is this a word?), we have to - bundle the test data from rulesets into tuples so py.test can understand + bundle the test data from rulesets into tuples so py.test can understand how to run tests across the whole suite of rulesets """ testdata = [] @@ -37,6 +37,22 @@ def destaddr(request): """ return request.config.getoption('--destaddr') +@pytest.fixture +def port(request): + """ + Destination port override for tests + """ + + return request.config.getoption('--port') + +@pytest.fixture +def protocol(request): + """ + Destination protocol override for tests + """ + + return request.config.getoption('--protocol') + @pytest.fixture def http_serv_obj(): """ @@ -69,11 +85,16 @@ def pytest_addoption(parser): parser.addoption('--rule', action='store', default=None, help='fully qualified path to one rule') parser.addoption('--ruledir_recurse', action='store', default=None, - help='walk the directory structure finding YAML files') + help='walk the directory structure finding YAML files') parser.addoption('--with-journal', action='store', default=None, help='pass in a journal database file to test') parser.addoption('--tablename', action='store', default=None, help='pass in a tablename to parse journal results') + parser.addoption('--port', action='store', default=None, + help='destination port to direct tests towards', choices=range(1,65536), + type=int) + parser.addoption('--protocol', action='store',default=None, + help='destination protocol to direct tests towards', choices=['http','https']) def pytest_generate_tests(metafunc): """ @@ -86,7 +107,7 @@ def pytest_generate_tests(metafunc): if metafunc.config.option.ruledir: rulesets = util.get_rulesets(metafunc.config.option.ruledir, False) if metafunc.config.option.ruledir_recurse: - rulesets = util.get_rulesets(metafunc.config.option.ruledir_recurse, True) + rulesets = util.get_rulesets(metafunc.config.option.ruledir_recurse, True) if metafunc.config.option.rule: rulesets = util.get_rulesets(metafunc.config.option.rule, False) if 'ruleset' in metafunc.fixturenames and 'test' in metafunc.fixturenames: diff --git a/test/test_default.py b/test/test_default.py index 8ff8f03..a51f97c 100644 --- a/test/test_default.py +++ b/test/test_default.py @@ -1,16 +1,20 @@ import pytest from ftw import testrunner, errors -def test_default(ruleset, test, destaddr): +def test_default(ruleset, test, destaddr, port, protocol): """ Default tester with no logger obj. Useful for HTML contains and Status code Not useful for testing loggers """ - runner = testrunner.TestRunner() + runner = testrunner.TestRunner() try: for stage in test.stages: if destaddr is not None: stage.input.dest_addr = destaddr + if port is not None: + stage.input.port = port + if protocol is not None: + stage.input.protocol = protocol runner.run_stage(stage, None) except errors.TestError as e: e.args[1]['meta'] = ruleset.meta