Release 5.1.0

Version 5.1.0

Added

• Bump supported versions:
  • Jira LTS from 8.5.15 to 8.5.18
  • Jira LTS from 8.13.7 to 8.13.10
  • Jira Service Management LTS from 4.5.15 to 4.5.18
  • Jira Service Management LTS from 4.13.7 to 4.13.10
  • Confluence LTS from 7.4.9 to 7.4.11
  • New Confluence LTS 7.13.0
  • Bitbucket LTS from 6.10.11 to 6.10.13
  • Bitbucket LTS from 7.6.7 to 7.6.9
  • Crowd from 4.3.0 to 4.3.5
• Retry logic for Confluence data preparation script.
• Check for toolkit updates.

Fixed

• Documentation updates and improvements.
• JSM Selenium actions stability improvements.
• Error handling in case of incorrect force flag usage for util scripts.
• Crowd base_url propagation.

Changed

• Updated Bitbucket user guide.
• Bump bzt version up to 1.15.4.
• Bump locust version to 1.6.0.
• Bump chromedriver version up to 93.0.4577.63 to support Chrome browser version 93.
• Bump downstream libraries versions.

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 5.0.0

Version 5.0.0

Added

• Crowd DC support.
• Bump supported versions:
  • Jira LTS from 8.5.12 to 8.5.15
  • Jira LTS from 8.13.4 to 8.13.7
  • Jira Service Management LTS from 4.5.12 to 4.5.15
  • Jira Service Management LTS from 4.13.4 to 4.13.7
  • Confluence LTS from 7.4.8 to 7.4.9
  • Bitbucket LTS from 6.10.9 to 6.10.11
  • Bitbucket LTS from 7.6.4 to 7.6.7
• Retry logic for JSM data preparation script.

Fixed

• Documentation updates and improvements.
• Selenium and JMeter actions stability.
• Dockerfile issue with bintray repo.
• Invalid XML character error for JSM.
• Confluence dataset to make view_blog action timings more stable.
• Run as specific user logic for Locust scripts.

Changed

• Updated Locust scripts according to the latest Jira and Confluence releases.
• Bump chromedriver version up to 91.0.4472.101 to support Chrome browser version 91.
• Bump downstream libraries' versions.

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 4.2.0

Version 4.2.0

Added

• Ability to run app_specific_action as a specific_user for JMeter.
• Log active node_id for selenium scripts.
• Improved pacing logic for JMeter scripts.
• Scenario status on the charts.

Fixed

• Documentation updates and improvements.
• Clear selenium browser cookies for each iteration.
• Improved index_synk.sh script to handle log rotation edge case.
• Updated JMeter actions to the latest product updates.
• Fixed total git operation count for Bitbucket.
• Fixed success rate calculation logic for selenium scenarios.
• Fixed selenium login for confluence in case of a server.
• Selenium and JMeter actions stability.

Changed

• Actions percentages for all products to make distribution better.
• Removed Python 3.6 support
• Bump bzt version up to 1.15.3.
• Bump locust version to 1.4.4.
• Bump chromedriver version up to 90.0.4430.24 to support Chrome browser version 90.
• Bump downstream libraries' versions.

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 4.1.0

Version 4.1.0

Added

• Bump supported versions:
  • Jira LTS from 8.5.11 to 8.5.12
  • Jira LTS from 8.13.3 to 8.13.4
  • Jira Service Management LTS from 4.5.10 to 4.5.12
  • Jira Service Management LTS from 4.13.2 to 4.13.4
  • Confluence LTS from 7.4.6 to 7.4.8
  • Bitbucket LTS from 6.10.7 to 6.10.9
  • Bitbucket LTS from 7.6.2 to 7.6.4
• Automatically zipping result directories after running chart generator script.
• Possibility to run app_specific_action as specific_user for Locust and Selenium.
• Catalina options for Confluence provisioning to increase Indexing performance.
• Auto-disable of Bitbucket pull request auto-decline feature.
• App-specific labeling on the resulting chart.

Fixed

• Custom dataset support for Jira Service Management.
• Documentation for toolkit local environment setup.
• Unicode support to the Locust scripts.
• Selenium login and logout actions logic for all supported products.
• Error handling for analytics scripts.
• Selenium actions stability.

Changed

• Bump bzt version up to 1.15.2.
• Bump locust version to 1.4.3.
• Bump chromedriver version up to 89.0.4389.23 to support Chrome browser version 89.
• Bump downstream libraries' versions.

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 4.0.0

Version 4.0.0

Added

• Jira Service Management support
  • JSM LTS 4.5.10 and 4.13.2
  • User Guide for JSM
  • JSM “large“ and “small“ datasets
• Bump supported versions:
  • Jira LTS from 8.5.8 to 8.5.11
  • New Jira LTS 8.13.3
  • Confluence LTS from 7.4.4 to 7.4.6
  • Bitbucket LTS from 6.10.5 to 6.10.7
  • New Bitbucket LTS 7.6.2
• Util script start_jmeter_ui.py to simplify usage of JMeter UI for debugging and app-specific actions development.
• Average action timings into results_summary.log.
• Warning if basic action timing exceeds 20 seconds threshold.
• Recommended storage amount to the execution environment section of the User Guide.
• AWS vCPU limit note in the User Guide.
• How to stop the database instructions in the User Guide.

Removed

• Support deprecated:
  • EOL Jira 7.13.x and 8.0.x
  • EOL Confluence 6.13.x

Fixed

• Confluence index-snapshot.sh script to properly check if an index snapshot generation is finished without a need to re-run the script.
• Jira get index nodes logic to count only active and alive nodes.
• Confluence re-index instructions in the User Guide.
• Explanation of the acceptable error rate for full-scale runs in the User Guide.
• Bitbucket language check header.
• Bitbucket selenium_create_pull_request action selector to make the action more stable.
• The naming convention for all actions.
• Confluence JMeter create_page action edge case when the page has an incorrect name.

Changed

• Pin supported Python versions to 3.6-3.8.
• Bump bzt version up to 1.15.1.
• Bump locust version to 1.3.1.
• Bump chromedriver version up to 88.0.4324.27 to support Chrome browser version 88.
• Bump downstream libraries' versions.

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 3.2.0

Version 3.2.0

Added

• Bump supported versions:
  • Jira LTS from 8.5.6 to 8.5.8
  • Confluence LTS 7.4.4 (new)
• Custom dataset filtering feature.
• Support of self-signed SSL certificate - secure: True/False toggle in .yml configuration files.
• Check if a run has app-specific actions in results_summary.log.
• Different levels of warnings and errors logging into bzt.log.
• Check if the toolkit executed against Bitbucket Server.
• Force set a language for Selenium scripts to English.

Fixed

• Locust scripts error handling.
• Fail immediately subsequent locust actions if login action failed.
• Bitbucket Selenium scripts case when a pull request has conflicts.
• Confluence JMeter and Locust scripts to run with a postfix.
• Confluence Selenium write_content method to not rely on a <p> tag.
• Removed hipchat requests from Confluence JMeter and Locust scripts.
• Edge case when project key matched JQL reserved words.

Changed

• Redesigned User Guides:
  • updated workflows
  • improved sections order
  • development and execution environment
  • new examples for JMeter, Locust, and Selenium app-specific actions development
• Bump bzt version up to 1.15.0.
• Bump locust version to 1.2.3.
• Bump chromedriver version up to 86.0.4240.22 to support Chrome browser version 86.
• Bump downstream libraries' versions.

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 3.1.0

Version 3.1.0

Added

• Bump supported versions:
  • Jira LTS from 7.13.6 to 7.13.15
  • Jira LTS from 8.5.0 to 8.5.6
  • Confluence LTS from 6.13.8 to 6.13.13
  • Confluence Platform from 7.0.4 to 7.0.5
  • Confluence LTS 7.4.x is not supported yet (work in progress, support will be added in upcoming releases)
  • Bitbucket LTS from 6.10.0 to 6.10.5
  • Bitbucket Platform from 7.0.0 to 7.0.5
• Instructions on how to stop cluster nodes into User Guides.
• Check if the user stated in the .yml configuration files has sufficient permissions for the run.
• Ability run tests with both RTE enabled and RTE disabled for Jira.

Fixed

• Bug in verify response logic for correct error handling.
• Get Jira nodes count logic to filter only active nodes.
  Confluence index-snapshot.sh script to handle multi snapshot case.

Changed

• Recommended AWS Quickstart CloudFormantion templates parameters to reduce AWS costs.
• Prepare dataset by the performance users, but not by the admin user to reduce data permission issues.
• Read WEBDRIVER_VISIBLE from .yml configuration file instead of the corresponding environment variable.
• Bump ChromeDriver version up to 84.0.4147.30 to support Chrome browser version 84.
• Bump downstream libraries' versions.

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 3.0.1

Version 3.0.1

Added

• Get version functionality in case Jira Server or Confluence Server

Fixed

• Language check for Confluence and Bitbucket in case the host uses https protocol
• Selenium selector when no results found by JQL search

Changed

• Documentation about Jira Index was updated to be more clear
• Recommended Jira Database instance class for better Jira Full Re-Index timing consistency
• Recommended Confluence Database instance class to use the latest generation
• Removed hipchat endpoint from jira.jmx

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 3.0.0

Version 3.0.0

Added

• locust executor and Locust scripts for Jira and Confluence (default load executor is still JMeter).
• Official Docker container with the Toolkit.
• Instruction how to set up and run the Toolkit on an EC2 instance in a Docker container.
• Automatic Base URL update after database restore.
• Nodes count information into results_summary.log.
• Dataset information into results_summary.log.
• Check if the script is running not on the NFS server for Bitbucket.
• Check the product language before the start.
• Check Collaborative Editing is enabled for Confluence.
• Check if the script is running on a bastion host.
• Extra logging into bzt.log.

Fixed

• Bug when Jira DC has more than 8000 projects.
• Utility functions were removed from conftest.py.
• Error handling in case the product version was not found.
• Error handling in case the product URL is invalid.
• Bug with an infinite loop on user creation for Confluence.
• Bug with random project selection for Jira.
• Bug when JMeter script fails on Bitbucket with postfix.

Changed

• print_time decorator was refactored for better readability and simplicity.
• Do not override License on database restore.
• bzt version bump up to 1.14.2.
• ChromeDriver version bump up to 83.0.4103.39 to support Chrome browser version 83.
• Downstream library's versions bump.
• Skip the whole test if login failed.

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Locust executor

In release 3.0.0 new locust executor and Locust scripts were added for Jira and Confluence. Locust (https://locust.io) is an open-source performance tool based on Python requests library. The benefit of using locust executor over jmeter executor is simplicity, extensibility and debugging flexibility. Locust scripts are a simple Python code that is straightforward to run, debug and extend.
jmeter executor is set by default in jira.yml and confluence.yml files. But if you want to give Locust a try or think that JMeter UI is too complicated for issues debugging or new app-specific actions creation just change load_executor value to locust in .yml configuration file.
More details about Locust could be found in README.md files.

App-specific actions change required

DCAPT framework uses print_timing decorator to measure timings of all Selenium actions and sub-actions.
In release 3.0.0 print_timing decorator was refactored for better readability and convenience of use. The signature of the decorator was changed, so if you have app-specific actions created for a previous version of the toolkit you need to update it as shown on an example below. Also, now there is no need to pass interaction into every selenium webdriver step.
Old decorator signature and usage example:

def custom_action(webdriver, datasets):
    page = BasePage(webdriver)
    @print_timing
    def measure(webdriver, interaction):
        @print_timing
        def measure(webdriver, interaction):
            page.go_to_url(f"{JIRA_SETTINGS.server_url}/plugin/report")
            page.wait_until_visible((By.ID, 'report_app_element_id'), interaction)

        measure(webdriver, 'selenium_app_custom_action:view_report')

        @print_timing
        def measure(webdriver, interaction):
            page.go_to_url(f"{JIRA_SETTINGS.server_url}/plugin/dashboard")
            page.wait_until_visible((By.ID, 'dashboard_app_element_id'), interaction)

        measure(webdriver, 'selenium_app_custom_action:view_dashboard')
    measure(webdriver, 'selenium_app_custom_action')

New decorator signature and usage example:

def custom_action(webdriver, datasets):
    page = BasePage(webdriver)

    @print_timing("selenium_app_custom_action")
    def measure():

        @print_timing("selenium_app_custom_action:view_report")
        def sub_measure():
            page.go_to_url(f"{JIRA_SETTINGS.server_url}/plugin/report")
            page.wait_until_visible((By.ID, 'report_app_element_id'))
        sub_measure()

        @print_timing("selenium_app_custom_action:view_dashboard")
        def sub_measure():
            page.go_to_url(f"{JIRA_SETTINGS.server_url}/plugin/dashboard")
            page.wait_until_visible((By.ID, 'dashboard_app_element_id'))
        sub_measure()
    measure()

Release 2.0.0

Version 2.0.0

Added

• Support for Bitbucket 7.0.0
• Results_summary.log file with detailed information of the run
• Scenario summary report
• Check if git installed for Bitbucket
• Post-run check for JMeter health
• Post-run artifacts folder cleanup

Fixed

• Documentation typos
• Bitbucket create_pull_request selenium action to be more realistic

Changed

• Refactored all Selenium code for Jira, Confluence, and Bitbucket to Page Object pattern
• ChromeDriver version bump to 80.0.3987.106 to support Chrome version 80
• Downstream libraries versions bump

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt