v1.47.0

Console

For #281, Added created and last_updated fields for the models:

• Configuration
• Engine
• EnginePool
• GloballyExcludedTarget
• Scan
• ScanCommand
• Site

The created and last_updated fields are also exposed in the API

import scantron_api_client

sc = scantron_api_client.ScantronClient()

response = sc.retrieve_site(1)

print(response.json())

{
        "id": 1,
        "site_name": "Test",
        "description": "",
        "targets": "192.168.1.10 192.168.1.11 192.168.1.12",
        "excluded_targets": "",
        "scan_command": 1,
        "scan_engine": 1,
        "scan_engine_pool": null,
        "email_scan_alerts": false,
        "email_alert_addresses": "",
        "email_scan_diff": true,
        "email_scan_diff_addresses": "test@localhost",
        "created": "2021-08-09T16:02:39.892926-05:00",
        "last_updated": "2021-08-09T16:03:56.611772-05:00"
    }

v1.46.0

Console

• Re-version from "1.46" to "1.46.0"
• Updated extract_targets.py to be more robust and accurate #271
• Updated base.txt requirements, except psycopg2-binary, because of the error it generates: "AssertionError: database connection isn't set to UTC". See psycopg/psycopg2#1293 (comment)
• Disabled auto reboot at the end of the console Ansible playbook.

Engine

• Aesthetic change only. Removed "f" in 2 print() statements since no variables needed to be converted to strings
• Re-version from "1.03" to "1.0.4" #272
• Re-built standalone engine binary and updated engine/standalone_engine_binary.txt hashes
• Disabled auto reboot at the end of the engine Ansible playbook.

API Client

• moved utility.py functions into scantron_api_client.py to simplify #273
• Replaced space (" ") with "_" in some of the globally excluded target URL paths
• Re-version from "0.0.6" to "1.0.7"

v1.45

Console

• Configuration model added to control if scan retention should be enabled, and if so, how long scan data and files should be retained. Disabled by default.

• Added filtering for some models in the admin portal

• Bumped rq version to 1.8.1 in response to rq/rq#1469

API Client

• Updated API client to support new /api/configuration endpoint

v1.44

Console

Highly recommend a fresh build of the console! The new scan scheduling capability relies on ScanSchedule model fields that won't exist until you re-save each Scan.

• Added supported for scheduling hourly scans.
• Updated scan_scheduler.py logic to handle new hourly scan frequency capability.
• Added console/scan_scheduler_visualizer.py to assist in visualizing scan start times.
• Added /api/server_time API endpoint to return a datetime string from the server to assist with scan scheduling.
• Added 2 different timezone settings in ansible-playbooks/group_vars/all
timezone_server - Set this to be the timezone you want the server to be in, usually UTC.
  timezone_django - Set this to be your local timezone. It makes dealing with dates, times, and scheduling easier.
• Added supported_scan_binaries key to ansible-playbooks/roles/engine/templates/engine_config.json as part of #265 Thank you @albertcp!

API Client

• Added retrieve_server_time() function to retrieve a datetime string of the server to assist with scan scheduling.
• Added retrieve_all_masscan_targets_and_open_ports_from_scan_id() function to retrieve all the targets and open ports given a scan ID for masscan only.
• Added "pooled" as a valid file_type for some of the functions that retrieve scan files.

v1.43

Console

• Added pyndiff (https://github.com/rackerlabs/pyndiff) support to email human-readable ndiff output when comparing 2 Nmap scans
• Bumped Python module versions in scantron/console/requirements/base.txt
• Bumped uWSGI system version to 2.0.19.1

v1.42

Console

• Bumped Python module versions in scantron/console/requirements/base.txt

API Client

• Bumped Python module versions in scantron/scantron_api_client/requirements.txt

v1.41

Console

• Email scan alerts (sent from console/utility.py) now contain the entire scheduled_scan_dict

{
    "id": 27,
    "site_name": "Test",
    "start_time": null,
    "scan_engine": "engine1",
    "start_datetime": "2020-11-24 19:51:00+00:00",
    "scan_binary": "nmap",
    "scan_command": "--top-ports 20 -sV -Pn -n --open",
    "targets": "192.168.1.1 192.168.1.2",
    "excluded_targets": "",
    "scan_status": "started",
    "completed_time": null,
    "result_file_base_name": "test__engine1__20201124_1351",
    "pooled_scan_result_file_base_name": "",
    "scan_binary_process_id": 0
}

v1.40

Console

• Pooled scan results now combined into 1 .json file (for masscan) or 1 .xml file (for nmap) after all the pooled scans complete.

API Client

• Added scan engine pool CRUD functions.

v1.39

Console

• Requires engine version 1.03!
• scan_results/nmap_to_csv.py and scan_results/masscan_to_csv.py moved from the console's root cronjob to module imports in console/utility.py. This means you don't have to wait for at most a minute to download the scan results. They will be available immediately.
• Scheduled scans "Results File" column renamed to "Result Files"
• Scheduled scans sorted by descending scan ID

Engine

• Requires console version 1.39!
• Removed the onus of moving files from scan_results/pending to either 1) scan_results/complete or 2) scan_results/cancelled from the engine. It is now handled by the console.
• Engine no longer passes completed_time to the console. The console handles that now and is the source of truth.
• Engine no longer passes result_file_base_name. Key was not utilized by the console.
• Engine will create a /.logs directory if one does not exist. Aims to minimize the amount of dependencies usually handled by Ansible.

v1.38

Console

• When adding a scan, the sites will be sorted alphabetically (#222)
• Multiple __ will be removed from site names. Ensures __ can be used as a delimiter to extract site name, engine, and timestamp in the console/scan_results/masscan_json_to_csv.py and console/scan_results/nmap_to_csv.py scripts.
• Standardized and re-named column names for console/scan_results/masscan_json_to_csv.py and console/scan_results/nmap_to_csv.py. For now, an "openports" column was also added for Rackspace's purposes that won't make sense in normal cases, but it's innocuous. Note that these scripts should be customized based on your organization's use of Scantron data. A more thorough explanation can be found in the scripts' comments. The plan is to hopefully retire it or require a script switch to optionally add the column.
• Added more verbose Scheduled Start Time for "Scheduled Scans" view (#153)
• Removed redundant "Scan Binary" column since it is already in the "Scan Command" column.

Engine

• Added capability in engine Ansible playbook to pull down additional NSE scripts by specifying URLs in the nse_script_urls variable.

API Client

• Changing versioning to be PyPI compliant for a future pip install capability. New version is now: 0.0.1
• Utilizing requests-toolbelt for request debugging.