Skip to content

AaronPLopez/SystemLogParser

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 

Repository files navigation

System Log Parser

Version 0.12.25.0

Date: 2023/09/01


Download Latest Release

System Log Parser Download

System Log Parser GUI

Description

System Log Parser is an ArcGIS for Server (10.1+) log query and analyzer tool to help you quickly quantify the GIS usage in your deployment. When run, it connects to an ArcGIS for Server instance on port 443/6443/6080/80 as a publisher (or an administrator), retrieves the logs from a time duration (specified as an input), analyzes the information then produces a spreadsheet version of the data that summarizes the service statistics. However, System Log Parser can also read access logs from other sources such as: Microsoft IIS, Apache Tomcat, Amazon ELB/ALB, Amazon CloudFront and Microsoft Azure.

System Requirements

  • 64bit Windows Operating System:

    • Windows (Workstation): 7, 8*, 8.1**, 10**, 11**
    • Windows (Server): 2008, 2012, 2016, 2019, 2022
  • Processor: AMD64/Intel64

  • RAM: 8GB

  • Disk Space: 2GB free when using the new Optimized Analysis Type

  • Security Protocol***: TLS 1.0, TLS 1.1, TLS 1.2, or TLS 1.3

  • Microsoft .NET Framework 4.8 (Full)

  • Publisher (or administrative) access to ArcGIS Server's REST API Admin endpoint in order to query the logs

  • ArcGIS Server log level set to FINE (from within the ArcGIS Server Manager) before using this tool

  • A deployment running ArcGIS Server 10.1 or higher

  • If performing Web (IIS) log analysis, it is recommended to have the Log File Rollover Schedule set to Hourly or Daily (from the Internet Information Services Manager)

  • *If using Windows 8, .NET Framework 3.5 (in addition to 4.8) must be enabled per Microsoft documentation

  • **If using Windows 8.1 or Windows 10, .NET Framework 3.5 (in addition to 4.8) must be enabled per Microsoft documentation

  • ***Not relevant if logs are consumed directly from the file system (e.g. reading the local disk or via CIFS share)

System Recommendations:

  • Microsoft Excel 2010 or higher (or appropriate xlsx viewer)
  • RAM: 16GB
  • Disk Space: 6GB free when using the new Optimized Analysis Type
  • Security Protocol*: TLS 1.2 or TLS 1.3
  • If performing Microsoft IIS log analysis, it is recommended to have the Log File Rollover Schedule set to Hourly (from the Internet Information Services Manager); Hourly creates more, smaller files but allows for a finer grain search
  • If querying through the web, an user with administrative access is recommended as they can gather details on all service types
  • *Not relevant if logs are consumed directly from the file system (e.g. reading the local disk or via CIFS share)

Esri Community Articles on Using System Log Parser


CHANGELOG

Build 0.12.25.0 (Prerelease)

  1. Fixed an issue where requests for Linear Referencing services would have their Capability set as MapServer instead of LRServer
  2. Added support to pickup the WPSServer service capability so the report can group accordingly for improved analysis
  3. Fixed an issue where Wait Timeouts could get counted as an elapsed time for ArcGIS reports with an Analysis Type of: Simple, WithOverviewCharts and Complete
  4. Fixed an issue when running slp.exe that was preventing the command-line option of "-validate" to be true by default; setting "-validate true" is necessary to see console messages printed to the screen
  5. Added some more throughput initialization to the ArcGIS Site object to help prevent the Provider summary in the ServiceDetails report from being empty
  6. Removed the Cloud Front screen from the System Log Parser GUI; cloudfront can still be selected from the command-line (e.g., slp.exe)
  7. Added an Application Log Level option to the ArcGIS Server (Web) screen; this provides the same capability as using "-apploglevel" from the command-line; this only creates a log for ArcGIS Server (web) log queries
  8. Removed the sanity check for End and Start times when selecting an Analysis Type of ServiceDetails; The ServiceDetails report does not make use of End or Start times
  9. Removed the "System Log Parser Installation Guide.docx" from the zip file; this doc was outdated and from an older release of System Log Parser
  10. In the GUI, the label of "Elapsed Time Minimum" has been changed to "Elapsed Time Minimum (Complete Analysis Only)" for clarity
  11. Enhanced the optional regular expression pattern to search against the source and operation fields; previous releases only search the source field (e.g., the resource or service name)
  12. Fixed an issue where using a regular expression for a ArcGIS Server (FS) log query could cause System Log Parser to crash
  13. For WithOverviewCharts, Complete and Optimized reports, a Wait Pct column has been added to the "Wait Time (Queue Time)" worksheet
  14. Fixed an issue with Optimized reports, where the Service Requests count could be listed as 0
  15. For ArcGIS Server Web log queries using an Optimized report, the Server Version and Server LogLevel fields were added to the Summary page
  16. For Optimized, Simple, WithOverviewCharts and Complete (spreadsheet) reports, the Max column has been moved immediately after the Min column for "Statistics By Resource", "Statistics By User", "Wait Time (Queue Time)" and "Instance Creation Time" worksheets
  17. Fixed an issue that would leave the "Arrival Rate Per Service" worksheet blank for Optimized reports using an ArcGIS Server log queries
  18. Removed the "Server LogLevel (FileAge)" properties listing for spreadsheet reports generated through ArcGIS Server (FS) log queries
  19. Enhanced some of the internal reporting of the ArcGIS Site object's contents

Build 0.12.24.0 (Prerelease)

  1. For (Analysis Type) Complete reports, the Path column has been added to the table on the "Elapsed Time - All Resources" worksheet
  2. Fixed an issue with the (new) "Arrival Rate Per Service" worksheet to group specifically by service name; the previous build used the service name with capability so instead of counting requests for "PostgreSQL/Branch_Spatial1", the table would separate it by "PostgreSQL/Branch_Spatial1/FeatureServer", "PostgreSQL/Branch_Spatial1/MapServer" and "PostgreSQL/Branch_Spatial1/VersionManagementServer"; while this was correct, that extra separation was not originally intended
  3. In addition to "Total Arrival Request Entries" counts on the arrival worksheets, "Data Rows Captured" are now listed as well
  4. For Simple, WithOverviewCharts, Complete and Optimized reports, the data Start Time and End Time (in Coordinated Universal Time) are now listed
  5. Improved request breakdown analysis has been added to IIS log parsing for capabilities such as NetworkDiagramServer

Build 0.12.23.0 (Prerelease)

  1. For Spreadsheet reports that pull log data from a local or network path, a "Log Location" field containing the passed in value has been added
  2. The command-line option of validate is now true by default; when running slp.exe, for previous versions, this had to be set to true in order to see any messages or warnings that were printed to the console
  3. Added a wait time minimum option to slp.exe for ignoring entries less than the passed in value; the default value is 1 (ms); this can be changed with the "-wtm" flag and the passed in value is assumed to be in milliseconds; for example: "-wtm 2" would be 0.002 seconds and "-wtm 2000" would be 2.0 seconds
  4. Fixed issue for ArcGIS Server File System log queries where intermediate elapsed time entries were being included in the statistics by resource summary
  5. For ArcGIS Server log source queries using an Analysis Type of Optmized, the worksheet "Arrival Rate" has been changed to "Arrival Rate Per User"
  6. For ArcGIS Server log source queries using an Analysis Type of Optmized, a worksheet called "Arrival Rate Per Service" has been added which shows the amount of requests for services (per second)...this table does not additionally break down the data by user and instead just focuses on what was being requested at the service-level

Build 0.12.22.0 (Prerelease)

  1. For reports of type ServiceDetails, the internal counting logic had been adjusted to better differentiate between the service instance types (e.g. dedicated, hosted, shared). This was done to help validate the same counts on the "Site Details" worksheet and the "Service Provider Stats" table and the bottom of the "Summary" worksheet. A few things to keep in mind, to help keep things simple, even though it is a dedicated service, System Log Parser labels "System/DynamicMappingHost.MapServer" as "Shared" since it is responsible for handling requests for shared services. Instances minimiums and maximums for Shared and Hosted services are set to 0 in the report...for some ArcGIS Server releases, the REST Admin API may list values greater than 0 (in such cases, these values are an artifact that do not have a functional impact to the service). Going forward, System services will be listed as System_Dedicated and Utility services will be listed as Utilities_Dedicated.
  2. Several columns in the "Site Details" worksheet have been group closer together for easier analysis (e.g. "Provider", "Instance Type", "Min Instances (Node)", "Max Instances (Node)", "Configured State")
  3. In the GUI, the size of the LogsLocation field (for the AGSFS) has been increased from 300 characters to 3000
  4. For listing the help in the command-line from slp.exe, the following commands can now be used: slp.exe, slp.exe -h, slp.exe --h and slp.exe -help
  5. File signing now includes System Log Parser EXEs and DLLs
  6. Corrected typo where the Header comments for the "Server Timeout" worksheet table lists the column as wait times
  7. Adjusted the Header comments for the Service Requests column (in the "Wait Time (Queue Time)", "Instance Creation Time", "Server Timeout" and "Wait Time Expired" worksheets) to denote the values are for successful requests; in some situations, a higher number of wait times for a services can be listed than service requests; in such a case, the higher count of wait time suggests than many requests failed and were never logged as "successful"
  8. Added a Report Name text field in the GUI for ArcGIS Server log queries (File System); when scanning large sites (multiple backend machines) over the network (e.g., several log locations), the report name that is automatically generated can become quite large and may have issues in Windows; this allows a convenient way to shorten it from the GUI
  9. Fixed issue when performing ArcGIS Server log queries (File System) where the passed in regular expression string would remove all services from the "Wait Time (Queue Time)" worksheet instead of only showing values for the specific service of interest

Build 0.12.21.1 (Prerelease)

  1. Fixed issue where not all EXEs were properly signed

Build 0.12.21.0 (Prerelease)

  1. For the "Arrival Rate" worksheet (ArcGIS Server log sources), the total number of request arrivals is now listed at the top; this is an important statistic since the tabular view of the data will only show 200,000 rows (for performance and practical reasons) so understanding the original total is valuable; this total is available for the WithOverviewCharts, Complete and Optimized reports
  2. For the ArcGIS Server ErrorsOnly reports (SpreadSheet or Text file output), the word "Errors" is not inserted into the filename of the generated report
  3. Fixed an issue with the resource regular expression where you could not capture entries for a particular service name when using the Optimized report with an ArcGIS Server Web log source
  4. Unified the tooltip text for the Regular Expression Pattern input box in the GUI
  5. Changed the worksheet "Arrival Rate" to "Arrival Rate Per User"
  6. Added a worksheet called "Arrival Rate Per Service" which shows the amount of requests for services (per second) but does not additionally break it down by user
  7. Fixed issue in the GUI where toggling the "Add Service Details to Report (Enhanced Site Report)" option in the "ArcGIS Server Log Query (Web)" panel would force the Analysis Type to reset to Simple
  8. Consolidated messages listed on ErrorsOnly reports by removing the user field from the internal error message dictionary (the user column was not being shown)
  9. Updated the Copyright to 2023 for System Log Parser DLLs and EXEs

Build 0.12.20.0 (Prerelease)

  1. Fixed an issue where a malformed requestID GUID in an ArcGIS Server log entry could cause System Log Parser to stop reading the logs over the web
  2. For the "Elapsed Time - All Resources" worksheet in the Complete report, the auto-fit column width for all columns has been removed to make filtering easier (specifically for the "Query String (Decoded)" and "Request Url" columns); several columns like "Date Time (Local Time)", "Epoch Time" and "Resource" still have the auto-fit sizing
  3. For IIS, Azure and Tomcat, the default "ArcGIS Type" value has been changed from "-" to ""
  4. For the "Elapsed Time - All Resources" worksheet in the Complete report, the "?-" string has been removed from the end of the "Request Url" column
  5. For the "Elapsed Time - All Resources" worksheet in the Complete report, if a token key/value pair was detected in the query string, it is now seperated out into its own column, this makes it easier to read the other data in the query string
  6. For (some) performance and load testing support, the Complete report has been exteneded with the following additional columns: Test_TestPlanName, Test_ThreadNumber, Test_PortalMember, Test_TransactionName; many testing frameworks have the ability to add these informative (but benign) components to requests in a test; if these key/value pairs are detected in the querystring (IIS, ALB/ELB and Azure log sources), they are extracted out into their own column to assist with filtering and analysis
  7. For the "Elapsed Time - All Resources" worksheet in the Complete report, the "ALB TargetGroup" and "CloudFront Edge Result Type" columns have been removed due to low usage

Build 0.12.19.0 (Prerelease)

  1. Improved request breakdown analysis has been added to IIS log parsing for capabilities such as VersionManagementServer, UtilityNetworkServer and LRServer
  2. Added a command-line option to slp.exe called "fsp" to force server parsing of logs even if the passed in URL has indications of a Portal instance; the default is false; active with: "-fsp true"
  3. For the "Site Details" worksheet, the Service Name has been changed from the REST Admin link to just the name of the service to prevent Excel from blocking all the contents when it is in read-only mode
  4. When using a pre-generated token with an ArcGIS Server (Web) log query from the GUI, the tool tip added clarification to create one that is Webapp URL based: "Manually provide an ArcGIS Server Token or ArcGIS Portal Token (Webapp URL based)"; for example, from https://myserver.domain.com/portal/sharing/rest/generateToken, select "Webapp URL" (where the URL would be something like: https://myserver.domain.com/portal); the command-line help has also been clarified

Build 0.12.18.0 (Prerelease)

  1. Added the build number to the title bar of the window's GUI
  2. Changed the System Log Parser URL link on the About screen to https://github.com/AaronPLopez/SystemLogParser
  3. Updated the Title and Product Assembly Information for all SLP DLLs and Exes

Build 0.12.17.0 (Prerelease)

  1. Added a listing of the IsSchemaLocking property for each service (where is exists) on the "Site Details" worksheet; This worksheet is found through: the ServiceDetails Analysis Type, enabling the "Add Service Details to Report (Enhanced Site Report)" option in the GUI, or setting the "-esr" parameter to true on the command-line; This worksheet is not available with the Optimized Analysis Type

Build 0.12.16.0 (Prerelease)

  1. Added Wait Time and Instance Creation Time statistics to the TextFile report
  2. Fixed an issue with the JSON Report Type where the complete output would not be available
  3. Corrected a typo on the command line help where the JSON Report Type was listed as an Analysis Type
  4. Added Wait Time and Instance Creation Time statistics to the JSON report

Build 0.12.15.0 (Prerelease)

  1. Added the Optimized Analysis Type as an option for AGS -- ArcGIS Server Log Query (Web); this is the recommended Analysis Type as the memory savings over Simple, WithOverviewCharts and Complete reports is significant; an increase in performance might be minimal as the ArcGIS Server Admin API will most likely be a bottleneck in the speed
  2. Updates to the command line help (show help with "slp.exe -h" or "slp.exe -help")
  3. Changed the GUI button "Analyze Logs" to just "Analyze"

Build 0.12.14.2 (Prerelease)

  1. When generated an enhanced site report, SLP skips over the examination of service datasources that cannot be parsed
  2. Added several internal checks to avoid crashes by SLP when the siteDetails.ArcGISServiceDatasourceDetails object contains unexpected data
  3. Increased the apploglevel verbosity of several internal methods

Build 0.12.14.1 (Prerelease)

  1. Package was unable to include signed binary files created by Esri; this has been corrected

Build 0.12.14.0 (Prerelease)

  1. If System Log Parser was generating "enhanced site report" worksheet, the reading of the site configuration data could be interrupted if "corrupt service" was encountered; slp now tries to continue if such a condition is met
  2. Added some details to the application debug log on timezone and utc offset from the machine running System Log Parser
  3. Added an Analysis Type of ServiceDetails which only connects to the ArcGIS web endpoint to retrieve service and datasource information (e.g. no log statistics)
  4. Added comments to the Min Instances (Node) and Max Instances (Node) colummn headers on the "Site Details" worksheet; comments are to clarify that although some DMaps (Shared) and SDS (Hosted) services may contain a configuration with an unexpected instances value, it has not impact to the respective service
  5. Provided an update to the Get-SLPReport.ps1 PowerShell automation script; this script can be used to quickly download large amounts (e.g. 1 month) of (ELB) log data using multiple copy threads in parallel (outside of SLP), then SLP is invoked to read all the log data from the local disk and quickly create an Optimized report

Build 0.12.13.0 (Prerelease)

  1. Added the version of System Log Parser to the "Summary" worksheet
  2. Added "Start Time for Query (Coordinated Universal Time)" and "End Time for Query (Coordinated Universal Time)" to the "Summary" worksheet
  3. For clarity, the "local time" string in the "Summary" worksheet was changed in "Start Time for Query (local time)" and "End Time for Query (local time)" to the actual local zone name of where System Log Parser is being run
  4. In the Service Provider Stats table ("Summary" worksheet) for the "Hosted (SDS)" row, the "Minimum Instances Per Node (Started)" and "Maximum Instances Per Node (Started)" values have been set to 0 despite their actual definition; this was done to avoid confusion; in some cases, a hosted feature service can take a value for this option but in actuality, it has no affect on the service
  5. In the Service Provider Stats table "Summary" worksheet for the "Shared ArcGIS Pro (DMaps)", the "Minimum Instances Per Node (Started)" and "Maximum Instances Per Node (Started)" values have been set to Site value for the Shared Instance Setting (e.g. Number of shared instances per machine)
  6. Corrected an incorrect labeling of the file and product version for the SystemLogsGUI executable

Build 0.12.12.0 (Prerelease)

  1. Added a Service Provider Stats table to the "Summary" worksheet

Build 0.12.11.2 (Prerelease)

  1. Fixed issue when parsing ArcGIS Server logs from the file system where incomplete entry would stop SLP from reading the rest of the files

Build 0.12.11.1 (Prerelease)

  1. Fixed issue when parsing IIS logs where SLP could crash if the header and data column count did not match on a line and if an Optimized report was being created

Build 0.12.11.0 (Prerelease)

  1. For ArcGIS Server (web) log queries that have the "Add Service Details to Report" option selected (also known as the enhanced site report option), the ArcGIS "provider" field is listed for each service on the "Site Details" worksheet

Build 0.12.10.2 (Prerelease)

  1. Fixed a performance issue with the "Arrival Rate" worksheet for WithOverviewCharts and Complete reports
  2. Fixed an issue the "Arrival Rate" worksheet for WithOverviewCharts and Complete reports (ArcGIS Server log queries) where under some conditions the displayed data object could still exceed the maximum of rows allowed in the table
  3. To help keep the file size more manageable for WithOverviewCharts and Complete reports (ArcGIS Server log queries), the maximum table size of the "Arrival Rate" worksheet is now set to 200,000 rows
  4. Minor fixes to application debug log statements

Build 0.12.10.1 (Prerelease)

  1. Fixed an issue with the Optimized report on "Statistics By Resource" worksheet where if the listed Methods ended the word "Server", the word "Server" would be omitted
  2. Fixed a presentation issue with the "Elapsed Time - All Resources" worksheet where the Epoch Time values would be formatted with a comma
  3. Fixed issue with the "Arrival Rate" worksheet for WithOverviewCharts and Complete reports (ArcGIS Server log queries) where the number of items in the table could exceed the maximum allowable for a spreadsheet and crash System Log Parser; the table will now automatically cutoff the remaining data at 1,000,000 rows
  4. Adjusted the "Request Throughput" worksheet for optimized reports to show Minutes if the available data time span is 24 hours or less; if over 24 hours, the worksheet will instead show throughput across Hours

Build 0.12.10.0 (Prerelease)

  1. Fixed an issue with the Optimized report for ArcGIS Server file system log queries where long timespans (or very busy Sites) would contain enough data for the "Arrival Rate" worksheet that exceeded the maximum table size in Excel; the maximum number of rows for tables in the Optimized report is set to 200,000
  2. For the Optimized report, the "Client IP", "Users", "Wait Time (Queue Time)", "Instance Creation Time" and "Request Throughput" worksheets have a maximum number of rows for tables set to 200,000
  3. slp.exe now prints an error message if an ArcGIS Server (web) log query is used with the Optimized Analysis Type; currently the only "ArcGIS Server" log query that is supported with the Optimized Analysis Type is "File System"
  4. Fixed issue with the "Request Throughput" worksheet where it would show Minutes insteads of Hours; this was greatly increasing the size of the report; Minutes are only supposed to be shown for query spans less than or equal to 3 hours
  5. Added option to slp.exe called "-optimizedextras" which can toggle the inclusion of some of the worksheets; the specific worksheets are: Client IP, Users, Arrival Rate, Wait Time (Queue Time), Instance Creation Time; some of the table contents for these worksheets can grow very large as the presentation is not a statistic view of the data; this option allows you to omit them in order to use less memory when performing large queries
  6. Increased the application debug log verbosity for some functions
  7. Updated the slp.exe command-line Help
  8. Adjusted the logic for handling the log source directory path(s) for ArcGIS Server file system log queries (Optimized, Simple, WithOverviewCharts, or Complete reports)
  9. Minor formatting adjustments of the "Request Throughput" worksheet

Build 0.12.9.0 (Prerelease)

  1. Removed the elb.exe and elb.exe.config files from the System Log Parser package; these files were kept for backwards compatibility to support early versions of ArcGIS Monitor extensions; the slp.exe (and slp.exe.config) provides the same functionality and support

Build 0.12.8.0 (Prerelease)

  1. Minor adjustments to the reading concurrency logic for Optimized Analysis Type IIS log queries
  2. Added the Optimized Analysis Type as an option for AGSFS (ArcGIS Server File System) log queries; this is the recommended Analysis Type as the memory savings and performance increase over Simple, WithOverviewCharts and Complete reports are significant
  3. Minor adjustments to the column descriptions on the "Wait Time (Queue Time)" and "Instance Creation Time" worksheets for IIS log queries using an Analysis Type of "WithOverviewCharts" or "Complete"
  4. Added column description detail to the "Arrival Rate" worksheet for AGS (web) and AGSFS log queries (Analysis Type of WithOverviewCharts and Complete); this was done to match the "Arrival Rate" worksheet from the new Optimized Analysis Type
  5. For AGS (web) and AGSFS log queries, the "Arrival Rate" worksheet, a label is appended to the end of the table noting that the arrival of hosted service requests can be pulled from the logs but their respective elapsed time information is not found on the "Statistics By Resource" worksheet
  6. The Optimized Analysis Type is now the default for ArcGIS Server (FS) log queries from the GUI

Build 0.12.7.1 (Prerelease)

  1. Removed several command-line options that were utilized to support ArcGIS Monitor (Classic) functionality
  2. Added logic for elbfs log queries to skip files of zero size
  3. Fixed an issue where querying IIS logs from the GUI did not fully take advantage of the all the memory improvements of the Optimized Analysis Type; slp.exe (and elb.exe) were not affected by this issue

Build 0.12.7.0 (Prerelease)

  1. Fixed issue with Simple, WithOverviewCharts and Complete Analysis Types where HTTP Codes would appear in a non-sorted order
  2. Added some small memory savings for Simple, WithOverviewCharts and Complete Analysis Types generated from IIS log queries
  3. Added the Optimized Analysis Type as an option for IIS log queries; this is the recommended Analysis Type as the memory savings and performance increase over Simple, WithOverviewCharts and Complete reports are significant
  4. The Optimized Analysis Type is now the default for ELB and IIS log queries from the GUI
  5. Improved error handling and message verbosity for various log query methods
  6. Added fix to automatically remove "info.log" if it was added to the path for ArcGIS Server file system log queries
  7. ArcGIS Monitor (Classic) support officially dropped
  8. System Log Parser now includes a PowerShell script (Get-SLPReport.ps1) to assist downloading ELB/ALB logs and uncompressing gzip logs outsite of SLP; this adds benefit of using Get-S3Object to quickly download multiple logs simultaneously and then call 7zip directly to uncompress the logs from a script (as oppose having SLP call an external process) In this script, the environment inherits credentials from an IAM Role and downloads 30 days worth of logs locally for analysis by SLP
  9. Support for calling 7zip as an external process has been removed; see the Get-SLPReport.ps1 script as an example of more securely utilizing a specific release of 7zip for *.log.gz file decompression
  10. Added several updates to the command-line Help

Build 0.12.6.0 (Prerelease)

  1. For IIS log queries that use an Analysis Type of WithOverviewCharts or Complete, a worksheet called "Service Referer" has been added which lists requested services and the associated referer header along with a count; static "REST", "SOAP", "Portal" and "OtherRequests" are not listed in this worksheet table
  2. Added automatic filtering support for the TopographicProductionServer (Topographic Mapping Server) capability; requests for such resources are placed in the (ArcGIS) Server group

Build 0.12.5.1 (Prerelease)

  1. System Log Parser executable files (SystemLogsGUI.exe, slp.exe, and elb.exe) include Digitial Signatures with "Environmental Systems Research Institute, Inc." as the Name of signer

Build 0.12.5.0 (Prerelease)

  1. Added option to decompress ELB/ALB GNU zip (*.gz) logs with a "local" instance of 7-zip (must be previously downloaded and installed); this feature is only available from the command line (slp.exe) and can be utilized with the "-7zippath [PATH_TO_7ZIP.EXE]" option
  2. Added data labels to all charts generated from reports with the WithOverviewCharts or Complete Analysis Type
  3. Adjusted the data accuracy presented on the Response and Request Sizes worksheet to 3 decimal places (was 6)

Build 0.12.4.0 (Prerelease)

  1. Removed a runtime check that prevented some methods from executing for non-spreadsheet report types
  2. Included milliseconds in the app log date timestamps
  3. Added the Source Server field to the Optimized report
  4. Improved the query statistic collection and display for the Optmized report; a worksheet called, "Query Statistics" is appended to the report if any logs cannot be read
  5. Added TRACE as an app log level
  6. Some internal code cleanup
  7. Added addtional checks for ELB/ALB queries (simple, withoverviewcharts, complete, optimized, downloadlogsonly) to help prevent SLP from stopping in the event it could not read certain logs
  8. Added app log verbosity statements to ELB/ALB log queries
  9. Added a fix for the command-line (slp.exe) to allow the Report Type of DownloadLogsOnly to override and download logs even if the Analysis Type of Optimized was passed in as an argument
  10. Minor updates to the command-line help

Build 0.12.3.0 (Prerelease)

  1. Added a new Report Type of "JSON"; this report type option is only available from the command-line (using the "-r json" switch) as the output is printed to the screen (STDOUT)
  2. Add the Response Time Histogram summary to the Text Report
  3. Minor formatting adjustments to the Text Report
  4. Added a fix where System Log Parser would stop reading the logs if entries with a code of 9000 were detected that did not have a service name in the message
  5. Adjusted the log filters to better catch errors SEVERE (errors) with a code of 9000
  6. Removed the Server IP field as a required column of data for IIS logs; although recommended, it is no longer required
  7. Added a fix to address a NullReferenceException that could be encounted when generating an ErrorsOnly report against log entries that were of type SEVERE with a code of 50000

Build 0.12.2.0 (Prerelease)

  1. Corrected a display error on the "Wait Time (Queue Time)" and "Instance Creation Time" worksheets where the overall values in the bottom table did not reflect a recent column reordering change
  2. Added the "Server Timeout" worksheet for ArcGIS log queries (web and fs, not ageok...no values will be retrieved for ageok reports)
  3. Added the "Wait Time Expired" worksheet for ArcGIS log queries (web and fs, not ageok...no values will be retrieved for ageok reports)
  4. Fixed some internal reporting logic for ArcGIS Enterprise for Kubernetes (ageok) log queries

Build 0.12.1.0 (Prerelease)

  1. Added support for ArcGIS Enterprise on Kubernetes (AGEOK); require version and built of at least: 10.9.1.1357 Currently, only command-line support exists; to utilize this functionality, pass in "ageok" to the -f command-line switch parameter Note: Not all functionality available from AGS is available yet from AGEOK (e.g. "Windows Authentication" and the "Enhanced Site Report")
  2. Minor updates to the command-line help

Build 0.12.0.0 (Prerelease)

  1. Updated AWS libraries (AWSSDK.Core, AWSSDK.S3, AWSSDK.IdentityManagement)
  2. Updated Azure libraries (Microsoft.Azure.KeyVault.Core, Microsoft.Rest.ClientRuntime, Microsoft.Azure.Storage.Common, Microsoft.Azure.Storage.Blob
  3. Removed EPPlus library; added PerfectXL.EPPlus library
  4. Updated Newtonsoft.Json library
  5. Updated the CommonServiceLocator library
  6. Updated the Unity library
  7. Updated the NLog library
  8. Added support for TLS 1.3
  9. The System Log Parser "package" no longer includes the xml file associated with each dll

Build 0.11.3.0 (Prerelease)

  1. Added command-line support for AWS ELB/ALB connections through Inherited Role Permissions; under this authentication scenario the Access Key, Secret Key and Region can be omitted from the command line as long as permission is granted to a specific machine using an IAM role and Instance Profile. For example, the following could be granted for S3 bucket access (this is not performed from System Log Parser): { "Effect": "Allow", "Action": [ "s3:ListBucket", "s3:GetObject" ], "Resource": [ "arn:aws:s3:::", "arn:aws:s3:::/*" ] }

    For more information: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html

  2. Fixed a formatting issue with the "Request Performance" worksheet

  3. Fixed a formatting issue with the "Request Status (HTTP 500)" worksheet

  4. For Optimized Analysis Type reports, a "Client IPs" worksheet has been added that lists the IPv4 Addresses and occurrences of connecting machines

  5. For Optimized Analysis Type reports, a "Request Throughput" worksheet has been added that lists the number of requests per hour (or per minute if query time span hour component is 3 hours or less)

Build 0.11.2.1 (Prerelease)

  1. Fixed an issue when parsing Azure Application Gateway v2 logs where the resource string was taken from requestUri instead of originalRequestUriWithArgs; originalRequestUriWithArgs is favored as it contains the url path that is closer to what the user called
  2. Fixed an issue when parsing Azure Application Gateway v2 logs where the resource string included the url parameters; in some cases this would make the resource extremely long and could the log parsing to fail with certain Tile requests
  3. Optimized Azure Application Gateway v2 resource analyzer
  4. Optimized the VectorTileServer service type definitions

Build 0.11.2.0 (Prerelease)

  1. For IIS log queries, any encountered malformed log entries are now just counted and the process is allowed to continue; the malformed log entry count is displayed in the report on the "Summary" worksheet under "Total Malformed Log Entries"
  2. Added "Arrival Rate" worksheet (and Request Arrival Time table) for ArcGIS Server (web and file system) log source queries; The Arrival table can help highlight which services might be consistently receiving more requests simultaneously than the available hardware can support; if the service is a tradition one (e.g. dedicated) it may be a good candidate for increasing the ArcSOC instances and/or adding more hardware resources
  3. Fixed several formatting issues with the "Throughput per Minute" worksheet
  4. The "Statistics By Resource", "Statistics By User", "Wait Time (Queue Time)", and "Instance Creation Time" worksheets are now sorted by Sum; Categorizing the values by Sum helps show which resources are taking the most time overall to work on by the server

Build 0.11.1.1 (Prerelease)

  1. Improved support for Azure Application Gateway v2 logs

Build 0.11.1.0 (Prerelease)

  1. Added support for Azure Application Gateway log v2
  2. Support for ArcGIS Monitor is now deprecated within System Log Parser. The functionality to upload log queries into ArcGIS Monitor will be removed in the near future.
  3. When using slp.exe (the command-line version), the ability has been added to override the auto-generated filename of the report through the "-n" switch (e.g. -n "myreport.xlsx")
  4. Expanded the regular expressions for ELB and IIS log sources to include group by tilemap and portal content listing requests
  5. For AGSFS log queries, any encountered malformed log entries are now just counted and the process is allowed to continue; the malformed log entry count is displayed in the report on the "Summary" worksheet under "Total Malformed Log Entries"
  6. For AGSFS log queries, log entries containing the requestID field are now parsed in a safer manner
  7. Changed the default spreadsheet column formatting from "auto" to a min and max characters of 5 and 75, respectively; this was done as some columns of data for some sites can have values that are extraordinarily long making the manual adjustment for screen readability cumbersome
  8. Several updates to the command-line Help

Build 0.11.0.0 (Prerelease)

  1. In the interest of greatly reducing memory requirements (querying busy sites or executing a query with a long time duration) the Analysis Type of "Optimized" has been introduced. This new "Optimized" Analysis Type is currently only available for ELB log queries. Memory savings with "Optimized" have been shown to be up to 120 times less than "WithOverviewCharts"! This allows for running "bigger" log queries which cover a longer time span duration (note: the time to download load the logs from the remote source remains the same).
    The analysis with "Optimized" is also significantly faster than "WithOverviewCharts". The "Optimized" report output will be similar to the "WithOverviewCharts" report but not all all worksheets are available. The speed and memory savings of the new "Optimized" Analysis Type is achieved by internally transforming the raw log data into a compressed format that is more efficient on system resources to analyze.
  2. Added an automatic filter for tile cache requests for traditional ELB log queries were the resource is rewritten to (more efficiently) just count the calls per map scale

Build 0.10.6.1 (Prerelease)

  1. Minor code cleanup and removal of some unused logic

Build 0.10.6.0 (Prerelease)

  1. Added additional memory utilization and performance enhancements
  2. Adjusted the formatting of the HTTP Method section in the "Summary" portion of the report
  3. Fixed issue where some IIS log queries against many large files would result in an "Unhandled Exception: OutOfMemoryException"
  4. Added improvements to security for working with passwords in memory at "rest"
  5. Adjusted the formatting of the CloudFront section in the "Summary" portion of the report
  6. Adjusted the formatting of the HTTP Code section in the "Summary" portion of the report

Build 0.10.5.0 (Prerelease)

  1. Added some memory utilization enhancements to various System Log Parser functions
  2. Added some format adjustments to the HTTP Code summary sections for both the spreadsheet and textfile reports
  3. Added an HTTP Method section to the "Summary" worksheet in the report
  4. Added a CloudFront section to the "Summary" worksheet in the report which details some CDN information

Build 0.10.4.2 (Prerelease)

  1. Adding numerical formatting to the integer values of the "Capabilities" worksheet which was missing
  2. Fixed an issue that corrected the reporting on the actual number of HTTP 500 requests for the "Request Status" worksheet if the total exceeded the allowed maximum of 1,000,000
  3. The "Throughput per Minute" worksheet has been moved back the "WithOverviewCharts" Analysis Type after additional testing revealed the increase in file size is not drastic; this worksheet enhances the "WithOverviewCharts" analysis while leaving "Complete" to show details on every request

Build 0.10.4.1 (Prerelease)

  1. To help keep the file size down for most reports, the "Throughput per Minute" worksheet has been moved back to the "Complete" Analysis Type
  2. Removed some unnecessary formatting in the "Request Status" worksheet

Build 0.10.4.0 (Prerelease)

  1. Fixed an issue where floating point numbers would not display properly in a Culture other than "en-US"
  2. Increased the verbosity of the System Log Parser application debug log for several methods and objects
  3. Fixed an issue where the service requests for the corresponding item in the "Wait Time" workbook would always be listed as 0
  4. Fixed a KML request identification issue for System Log Parser to ArcGIS Monitor uploads
  5. Fixed an issue that could cause System Log Parser to stop reading malformed ALB logs
  6. Added an optimization to elb log queries with a Report Type of "Downloadlogsonly" that the logs are just saved to disk and not also read into memory
  7. Added optimizations to elbfs to read all the logs files in the specified directory regardless of file timestamps and to read the log files in parallel
  8. Removed the "ArcGIS Capability" stats on the "Summary" worksheet of the spreadsheet and text report
  9. Extended the Analysis Type of "WithOverviewCharts" and "Complete" to include worksheets summarizing requests based on ArcGIS Server Service Types and ArcGIS Server Capabilities This new analysis assigns a capability to every request that is based on the requested resource For example: MapServer, ImageServer, NAServer, WMSServer, FeatureServer, HostedFeatureServer, VectorTileServer, MapServerDefinition, IMAGESERVERDEFINITION, PortalContentData, REST, OtherRequests, etc... These requests are also shown with respect to an ArcGIS Server function capability grouping (availability depends on selected log source) For example: ArcGIS Server, ArcGIS Server Extensions, OGC, Tiles, Portal for ArcGIS, Server Definition, REST and SOAP, Other Requests
  10. Extended the Analysis Type of "WithOverviewCharts" and "Complete" to include worksheets summarizing the top 500 slowest requests as well as all HTTP 500 errors (up to the 1,000,000 entries) for the given time span of the query (availability depends on selected log source)
  11. Enhanced the HTTP Status information on the Summary worksheet to include details such the specific code as well as its specific definition and number of occurences
  12. For easier reading of the spreadsheet report, some worksheet groups were assigned a specific code
  13. The "Throughput per Minute" worksheet has been moved the "WithOverviewCharts" Analysis Type

Build 0.10.3.2 (Prerelease)

  1. Increased the warning message verbosity when an IIS log is read that does not meet the required criteria (e.g. header format, necessary fields)
  2. Fixed an issue in the System Log Parser application debug log that would incorrectly report the number of messages per page (pageSize) returned in the response from ArcGIS Server
  3. Fixed an issue with ArcGIS Server (web) queries where an error would be thrown if the hostname of the server contained the same word as the instance name (e.g. arcgis1.mydomain.com:6443/arcgis)

Build 0.10.3.1 (Prerelease)

  1. Provided clarification of the stated Security Protocol requirement; the Security Protocol requirement only pertains to connections over HTTPS and not queries where the logs are consumed directly from the file system (e.g. reading the local disk or via CIFS share)

Build 0.10.3.0 (Prerelease)

  1. When using slp.exe (the command-line version), the ability has been added for an ArcGIS Server (FS) query to read from multiple server log directory sources (presumably from the same Site) all at once; the capability can be achieved by wrapping the specified input paths in quotes and separating each ArcGIS Server log folder by a comma, for example: '-i "\myserver1.domain.com\c$\arcgisserver\logs\MYSERVER1.DOMAIN.COM,\myserver2.domain.com\c$\arcgisserver\logs\MYSERVER2.DOMAIN.COM"'
  2. For ArcGIS Server (FS) queries, fixed an issue with reading certain ArcGIS Server 10.8 log files where the file would contain an xml header but no data; this is a regression bug from versions 0.8.x.x
  3. For TimeSpan queries that exceed 7days with an Analysis Type of "WithOverviewCharts" or "Complete", the "Time" worksheet will now generate charts for the most recent 7days and omit the rest, previous builds will only generate charts of the TimeSpan was 7days or less
  4. When using slp.exe (the command-line version), the ability has been added for an IIS query to read from multiple server log directory sources (presumably from the same Site) all at once; the capability can be achieved by wrapping the specified input paths in quotes and separating each IIS log folder by a comma, for example: '-i "\myserver1.domain.com\c$\inetpub\logs\LogFiles\W3SVC1,\myserver2.domain.com\c$\inetpub\logs\LogFiles\W3SVC1"'
  5. When using slp.exe (the command-line version), the ability has been added for a Tomcat query to read from multiple server logs directory sources (presumably from the same Site) all at once; the capability can be achieved by wrapping the specified input paths in quotes and separating each Tomcat log folder by a comma, for example: '-i "\myserver1.domain.com\c$\tomcat\logs,\myserver2.domain.com\c$\tomcat\logs"'
  6. Newer System Log Parser builds only support System.Net.SecurityProtocolType of Tls11 and Tls12
  7. For backward compatibility with older deployments, support for TLS 1.0 was readded; this had been automatically removed with the 0.10.x.x branches migration to .NET Framework 4.8; although System Log Parser supports connecting to deployments that are running older security protocols such as TLS 1.0 or TLS 1.1, it is highly recommended to use TLS 1.2 instead

Build 0.10.2.0 (Prerelease)

  1. Added the ability of a Tomcat query to read server logs even if they are in use (by ArcGIS Server); in the case of reading logs that are in use, the data can be read by System Log Parser as soon as it has been flushed to the disk by ArcGIS Server
  2. Added internal optimization with ArcGIS Server (web) queries by adjusting the Start time and End time ordering
  3. Added additional application debug logging for several query meta data elements and adjusted the details of several existing logging events

Build 0.10.1.0 (Prerelease)

  1. Added additional application debug logging for ArcGIS Server (web) log queries; the application debug logging can only be utilized when using slp.exe (the command-line version) and passing in the "-apploglevel INFO" or "-apploglevel DEBUG" command-line option; the default log level is OFF
  2. Added the ability of an ArcGIS Server (FS) query to read service logs even if they are in use (by ArcGIS Server); in the case of reading logs that are in use, the data can be read by System Log Parser as soon as it has been flushed to the disk by ArcGIS Server
  3. Added the ability of an ArcGIS Server (FS) query to read server logs even if they are in use (by ArcGIS Server); in the case of reading logs that are in use, the data can be read by System Log Parser as soon as it has been flushed to the disk by ArcGIS Server
  4. Fixed issue where some error messages upon startup of System Log Parser were silently discarded
  5. Added the ability to pass in a value to specify the folder where the application debug log is written; the default location is "C:\Users[USERNAME]\Documents\System Log Parser\Logs"; this can now be overridden with the already existing "-downloadlocation" command-line option; so '-downloadlocation "D:\some\folder\with\write\access"' would save all logs to D:\some\folder\with\write\access\
  6. Added logic to skip any instance creation time log entries where the elapsed time is 0; although it is not anticipate to exist, adding in zeros would throw off the "accuracy" of the statistical representation of this data
  7. Clarified the description on the "Wait Time" and "Instance Creation Time" worksheets that the statistical representation does not include entries where the value was 0 (e.g. 0 seconds)
  8. Changed the page title of the "Instance Creation Time" worksheet from "Instance Creation Statistics" to "Instance Creation Time Statistics"
  9. Changed the tab title of "Wait Time" worksheet from "Wait Time" to "Wait Time (Queue Time)"
  10. For a better user experience, the System Requirements for RAM was increased from 4GB to 8GB
  11. Added the ability of an IIS query to read the logs even if they are in use (by IIS); in the case of reading logs that are in use, the data can be read by System Log Parser as soon as it has been flushed to the disk by IIS
  12. Added additional application debug logging for ArcGIS Server (file system) log queries

Build 0.10.0.0 (Prerelease)

  1. Added enhanced statistics to the generated reports from System Log Parser; enhanced stats included additional information between the 95th and Max percentiles
  2. Added some application debug logging for Azure
  3. Fixed issue where System Log Parser would not read in any wait time and instance creation time log entries with an ArcGIS Server file system query
  4. Added some application debug logging for the "Datasource Information" and "Datasource Analysis" worksheets (ArcGIS Server web queries)
  5. Updated the System Log Parser URL in About System Log Parser to point back to the original System Log Parser "home page": https://www.arcgis.com/home/item.html?id=90134fb0f1c148a48c65319287dde2f7
  6. Moved System Log Parser to .NET 4.8 Framework
  7. Updated AWS libraries; System Log Parser now supports the following AWS regions: SLP Input Name Summary useast1 Amazon.RegionEndpoint.USEast1 The US East (Virginia) endpoint useast2 Amazon.RegionEndpoint.USEast2 The US East (Ohio) endpoint uswest1 Amazon.RegionEndpoint.USWest1 The US West (N. California) endpoint uswest2 Amazon.RegionEndpoint.USWest2 The US West (Oregon) endpoint eunorth1 Amazon.RegionEndpoint.EUNorth1 The EU North (Stockholm) endpoint euwest1 Amazon.RegionEndpoint.EUWest1 The EU West (Ireland) endpoint euwest2 Amazon.RegionEndpoint.EUWest2 The EU West (London) endpoint euwest3 Amazon.RegionEndpoint.EUWest3 The EU West (Paris) endpoint eucentral1 Amazon.RegionEndpoint.EUCentral1 The EU Central (Frankfurt) endpoint apeast1 Amazon.RegionEndpoint.APEast1 The Asia Pacific (Hong Kong) endpoint apnortheast1 Amazon.RegionEndpoint.APNortheast1 The Asia Pacific (Tokyo) endpoint apnortheast2 Amazon.RegionEndpoint.APNortheast2 The Asia Pacific (Seoul) endpoint apnortheast3 Amazon.RegionEndpoint.APNortheast3 The Asia Pacific (Osaka-Local) endpoint apsouth1 Amazon.RegionEndpoint.APSouth1 The Asia Pacific (Mumbai) endpoint apsoutheast1 Amazon.RegionEndpoint.APSoutheast1 The Asia Pacific (Singapore) endpoint apsoutheast2 Amazon.RegionEndpoint.APSoutheast2 The Asia Pacific (Sydney) endpoint saeast1 Amazon.RegionEndpoint.SAEast1 The South America (Sao Paulo) endpoint usgovcloudeast1 Amazon.RegionEndpoint.USGovCloudEast1 The US GovCloud East (Virginia) endpoint usgovcloudwest1 Amazon.RegionEndpoint.USGovCloudWest1 The US GovCloud West (Oregon) endpoint cnnorth1 Amazon.RegionEndpoint.CNNorth1 The China (Beijing) endpoint cnnorthwest1 Amazon.RegionEndpoint.CNNorthWest1 The China (Ningxia) endpoint cacentral1 Amazon.RegionEndpoint.CACentral1 The Canada (Central) endpoint mesouth1 Amazon.RegionEndpoint.MESouth1 The Middle East (Bahrain) endpoint
  8. Updated EPPlus library
  9. Updated Newtonsoft.Json library
  10. Removed dependency of WindowsAzure.Storage library
  11. Added support for Microsoft.Azure.Storage.Blob and Microsoft.Azure.Storage.Common libraries
  12. Added support for Microsoft.Rest.ClientRuntime and Microsoft.Rest.ClientRuntime.Azure libraries
  13. Updated Microsoft.Azure.KeyVault.Core library
  14. Minor code clean-up
  15. This version (0.10.x.x) will be last System Log Parser release to support log query uploads into ArcGIS Monitor 10.7.x/10.8.x

Build 0.9.6.0 (Prerelease)

  1. Fixed some additional issues with the ArcGIS Server file system log queries where an Analysis Type of "ErrorsOnly" could not parse certain entries
  2. Fixed an issue with ArcGIS Server file system log queries where under certain conditions an "ErrorsOnly" Analysis Type report could not be created with either a Text file or Spreadsheet file output
  3. For AWS environments, added support and command line help documentation for the "-printfolders" boolean option
  4. Fixed an issue where some error messages where silenced when run from the GUI
  5. For ArcGIS Server web log queries, the "Add Service Details to Report" option has been slightly optimized to issue less requests
  6. Added P95 and P99 (95th and 99th percentile statistics) to the "Throughput per Minute" worksheet table for reports with an Analysis Type of "Complete"
  7. Enhanced the table header descriptions for the "Throughput per Minute" worksheet table
  8. Added a notification warning if no log entries were found for a report with an Analysis Type of "ErrorsOnly"
  9. For IIS log queries, support has been added for the X-Forwarded-For field to use in place of the c-ip field for the Client IP value; If the X-Forwarded-For field is not present or only contains an empty value, the "origial" c-ip field will be used instead
  10. Fixed an issue with ArcGIS Server file system log queries where the elapsed time of some entries where numerically presented based on the locale of the machine running System Log Parser; from the file system, all ArcGIS Server elapsed time entries are consumed and analyzed in a consistent manner
  11. For ArcGIS Server file system log queries from the GUI, the maximum Start Time has been increased from 90day(s) to 120day(s)
  12. Added an option to toggle the inclusion of the "Statistics By User" worksheet for ArcGUS Server Web, ArcGIS Server file system and IIS log queries (by default this is now false)
  13. Added a command line switch (e.g. the -sbu option) to toggle the inclusion of the "Statistics By User" worksheet (by default this is now false)
  14. From the System Log Parser GUI, the name "Output Folder" has been changed to "Report Folder" for all log queries
  15. Corrected a minor tab ordering issue in the System Log Parser GUI

Build 0.9.5.0 (Prerelease)

  1. Fixed an issue with ArcGIS Server file system log queries where certain entries could not parsed
  2. Fixed an issue with the ArcGIS Server (web and file system) log query where an Analysis Type of "ErrorsOnly" could crash System Log Parser when attempting to generate the report
  3. Added support to the "Site Details" worksheet for SceneServer and WorkspaceServer service types
  4. Fixed an issue that prevented the System Log Parser GUI interface from allowing the ability to select an Analysis Type ErrorsOnly report with a Text file output for ArcGIS Server web and ArcGIS Server file system log queries
  5. For ArcGIS Server log query reports with an Analysis Type of "WithOverviewCharts" or "Complete", the "Queue Time" worksheet has been renamed to "Wait Time" to better match the naming convention of ArcGIS Server
  6. For ArcGIS Server log query reports that upload to ArcGIS Monitor the Counter Instances of "Tr/Interval Service Queueing" and "Tr/Interval Service Queueing Histogram (%)" have been changed to "Tr/Interval Service Wait Time" and "Tr/Interval Service Wait Time Histogram (%)", respectively; their associated Counter Names have also been adjusted to reflect this change

Build 0.9.4.0 (Prerelease)

  1. Added some memory clean enhancements to SLP to help improve resource consumption after repeated GUI runs
  2. Added ELBFS support to the interval SetSourceTypeFolder method
  3. Enhanced the LogPrefix logic to automatically append "/AWSLogs" if this was omitted
  4. Altered ELB log query logic by removing the requirement of the AWS ListBuckets method
  5. Added command line help documentation for the "-listbuckets" boolean option
  6. Separated the logic for the listbuckets routine. This is no longer part of the log query for a specific bucket.
  7. Added a command line option boolean switch called "-listaccountid" (default is false) to list the AWS account id associated with the given awsAccessKeyId, awsSecretAccessKey and RegionEndpoint
  8. Added command line help documentation for the "-listaccountid" boolean option
  9. Expanded the supported security protocols for ELB/ALB log queries to include TLS 1.1 and TLS 1.2
  10. Expanded the supported security protocols for CFRONT (CloudFront) log queries to include TLS 1.1 and TLS 1.2
  11. Expanded the supported security protocols for Azure log queries to include TLS 1.1 and TLS 1.2
  12. Added an "Instance Creation Time" worksheet to ArcGIS Server log query reports with an Analysis Type of "WithOverviewCharts" or "Complete". The "Instance Creation Time" worksheet list the times required for an service to start up if a request was made and an instance was not available (and the maximum number of concurrent instance was not yet reached).
  13. Corrected an issue where some POST requests for ArcGIS Server log queries may not get counted in the statistical summaries
  14. Removed the ClientIPs worksheet from ArcGIS Server log query reports with an Analysis Type of "WithOverviewCharts" or "Complete"
  15. Added the Users worksheet in addition to the ClientIPs worksheet to IIS log query reports with an Analysis Type of "WithOverviewCharts" or "Complete"
  16. Fixed an issue where Windows Authentication could not be used for some ArcGIS Server log queries
  17. For a report generated from all log queries, the "Statistics" worksheet has been renamed to "Statistics By Resource" and an additional worksheet named "Statistics By User" has been added (For ArcGIS and IIS log queries) which lists resources with respect to the user who requested them
  18. Improved the accuracy of the service type auto detection of Hosted, Dedicated and Shared services for the "Site Details" worksheet (for ArcGIS Server Web based log queries)
  19. Added an ArcGIS Server capability count table to the Summary worksheet
  20. Bring the Summary content of both the Text and Spreadsheet output closer together
  21. Reduced the number of returned ArcGIS Server log entries chunks per web query from 9999 to 1000 to help improve accuracy and consistency
  22. Added the "Domain" and "Server Machine" columns to the "Elapsed Time - All Resources" worksheet
  23. Expanded the supported security protocols for ArcGIS Monitor log uploads to include TLS 1.1 and TLS 1.2

Build 0.9.3.0 (Prerelease)

  1. Added support to run ELB log queries against gz compressed log files residing on the local disk through the -f ELBFS command line switch
  2. Added support to run ELB log queries against plaintext log files residing on the local disk through the -f ELBFS command line switch
  3. Expanded the command line help to include information on the "esr" switch; the "esr" switch toggles the "IsEnhancedSiteReport" option; In the GUI, this boolean option is presented as "Add Service Details to Report"
  4. Added support for manually setting the ArcGIS Server Token Expiration and/or Request Timeout value from the command line
  5. Expanded the command line help to include information on manually setting the an ArcGIS Server Token Expiration or Request Timeout value
  6. The Vector Tile request signature categorization for ArcGIS Monitor has been changed from "vectortileserver" to "vectortileserver/tile"
  7. Expanded verbosity of error details if a specific log file or log line cannot be read from an elb or alb source
  8. Improved the robustness of reading elb and alb logs originating from HTTP/2 gzip sources
  9. Expanded verbosity of error details if a specific log file or log line cannot be read from an iis source
  10. Improved the robustness of detecting the required iis w3c headers that could be missing or incomplete
  11. Removed iis log source restriction: the specified target directory no longer needs to contain "w3svc" or "inetpub"
  12. Expanded verbosity of error details if a specific log file or log line cannot be read from an azure source
  13. Expanded verbosity of error details if a specific log file or log line cannot be read from a tomcat source
  14. Expanded verbosity of error details if a specific log file or log line cannot be read from a cloudfront source
  15. Expanded verbosity of error details if a specific log file or log line cannot be read from an arcgis server web source
  16. Expanded verbosity of error details if a specific log file or log line cannot be read from an arcgis server file system source
  17. Fixed several issues with the "Throughput per Minute" worksheet which could cause the report generation to fail
  18. Changed the default ArcGIS Server URL from "https://gisserver.domain.com:6443/arcgis" to "https://gisserver.domain.com/server"

Build 0.9.2.0 (Prerelease)

  1. For log query uploads to ArcGIS Monitor, a "Response Time per Service Type (Sec)" counter has been added

Build 0.9.1.0 (Prerelease)

  1. Further enhanced the resource id/GUID name consolidation to support additional raster type requests
  2. Fixed an issue with ELB and ALB log queries where a "bad" log entry could stop the progress of the read
  3. In addition to working with the existing Azure CloudBlockBlob storage, SLP now supports the newer CloudAppendBlob storage format (for Azure log queries)
  4. Due to the CloudAppendBlob support, SLP is also now compatible with the newer JSON Lines log format (for Azure log queries)
  5. Added several checks to help avoid a potential "division by zero" error for reports with an Analysis Type of "WithOverviewCharts" or "Complete"
  6. Fixed issue with some log queries where certain services would incorrectly be marked as a "Hosted service" if the web adaptor instances of the site was called "hosted"

Build 0.9.0.0 (Prerelease)

  1. Fixed issue where "Client IP" column in (from a Complete Analysis report) may not be listed with a CloudFront log query in the "Elapsed Time of All Resources" worksheet
  2. Fixed issue where the "Users" worksheet (from a Complete Analysis report) may be listed instead of the the "ClientIPs" worksheet with a CloudFront log query
  3. With a Complete Analysis report, the "ArcGIS Log File" column was changed to "Log Source"
  4. With a Complete Analysis report, the "Elapsed Time of All Resources" worksheet has been renamed to "Elapsed Time - All Resources"
  5. Fixed some of the ELB and CloudFront tooltips which did not provide the correct information
  6. Added the "-help" option in addition to the "-h" option for displaying "help" from the command line (e.g. with slp.exe)
  7. Added ability to read Azure logs and perform queries
  8. Added timespan details in the generated report for both the query time range and log data time range
  9. Improved log datetime filtering accuracy for certain sources where the some files may contain a few enteries of the previous hour
  10. Changed the maximum allowable lines of generated log data (per worksheet) in the spreadsheet report to be 1,000,000
  11. Added the ability of SLP to automatically create a text file of the data for a worksheet (e.g. "Throughput per Minute", Elapsed Time - All Resources) that exceeds the maximum allowable lines
  12. Added some report generation progress feedback to the SLP GUI
  13. Added a check in the SLP GUI to help ensure that only the Analysis Type of "Simple" can be used with a Report Type of "TextFile"
  14. Increased the logs detail of information for traditional Feature Server requests from a VERBOSE report
  15. For ArcGIS Monitor log uploads, items like tiles and service definition requests are now automatically saved to the repository
  16. Optimized the VerboseMode and ErrorsOnly Report column formatting to improve creation times
  17. System Log Parser now consolidates jobid GUIDs within specific ArcGIS-type requests in order to help make the generated statistics easier to read. The exception to this are requests to ArcGIS Portal resources (e.g. "/sharing/rest/content/items/[guid]")...these are left in their original form.
  18. Added ALB/ELB fixes for malformed requests
  19. Added improvements to multistream gzip decompression for incomplete files
  20. Added a fix to ArcGISFS log queries to improve accuracy of read based on certain entries
  21. Added a fix to ArcGISFS log queries for an Analysis Type of ErrorsOnly...previous builds would not find any WARNING or SEVERE log entries, under this condition
  22. Fixed an issue when creating the Statistics worksheet where it could take a long time if the log query resulted in many different resources
  23. Added a "Response|Request Sizes" worksheet for reports of Analysis Type "WithOverviewCharts and "Complete" which show the top resources for the largest response and request sizes

Build 0.8.19.0 (Prerelease)

  1. Adjusted the queueing counts in the ArcGIS Server report to not include entries with an elapsed time of 0. With this adjustment the count now reflect requests that queued (instead of any request).
  2. Added "Service Requests" count column to the "Queue Time" worksheet to given an improved perspective of the impact of the queue time entries
  3. Fixed issue with the SiteDetail worksheet where some services would not be listed if no extensions were enabled
  4. Added an "Is Enabled" column to the SiteDetail worksheet to help filter out extensions that are enabled or disabled
  5. For ArcGIS Server (Web) log queries that are uploaded to ArcGIS Monitor, the following metrics have been added under "Tr/Interval Service Queueing": TotalSecondsQueued TotalRequestsQueued TotalServicesQueued
    Tr/Interval Service Queueing Histogram (%)
  6. For IIS log queries that are uploaded to ArcGIS Monitor, the domain and username information is reported on IIS logs will contains this information if the "User Name (cs-username)" is enabled (in IIS) and the ArcGIS site's Web Adaptor is configured for Windows Authentication and authenticated users made requests during the collection interval
  7. Fixed issue with ArcGIS Server (Web) log queries uploaded to ArcGIS Monitor where the values of the "Tr/Interval Response Time Histogram (%)" counters were the counts instead of the percentage of counts
  8. Added ability to read Tomcat access logs and perform queries (command line only). To enable Tomcat access logs (Note: with this setting, the logs will rotate every hour): Find the server.xml file within the appropriate Tomcat installation Make backup copy of original server.xml file Open server.xml in text editor Replace the following line:

About

A reporting tool specifically designed for analyzing ArcGIS server and service logs to help quantify the GIS usage in the deployment

Topics

Resources

Stars

Watchers

Forks

Packages

No packages published