Skip to content

Releases: MobilityData/gtfs-validator

v5.0.1

20 Mar 21:46
@jcpitre jcpitre
v5.0.1
3fc961b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v5.0.1 Latest
Latest

This minor release was created to fix a key bug that was preventing feed URLs from running successfully in the Web UI. This bug only impacted the Web UI, not the desktop app or CLI tool.

For more details on our 5.0.0 release, you can read about them here.

PR List

  • chore: remove credentials on ui build by @davidgamez in #1700
  • chore(deps-dev): bump follow-redirects from 1.15.5 to 1.15.6 in /web/client by @dependabot in #1717
  • chore(deps-dev): bump vite from 4.3.9 to 4.5.2 in /web/client by @dependabot in #1705
  • chore(deps-dev): bump word-wrap from 1.2.3 to 1.2.5 in /web/client by @dependabot in #1697
  • docs: Automated update of NOTICE_MIGRATION.md by @github-actions in #1716
  • hotfix: jobId incorrectly set in the UI by @cka-y in #1723

Full Changelog: v5.0.0...v5.0.1

Contributors

davidgamez, dependabot, and cka-y
Assets 12

v5.0.0

15 Mar 00:55
@jcpitre jcpitre
v5.0.0
4029a37
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v5.0.0

Highlights

Reducing the noisiness of errors

There have been repeated issues with the sensitivity of
trip_distance_exceeds_shape_distanceand equal_shape_distance_diff_coordinates. No more! We did an extensive review of real life GTFS data from the Mobility Database and decided on thresholds for these two errors. Now data producers will only see these errors when they’re a significant issue for data quality. Below the thresholds, these issues will generate a warning instead.

Breaking changes: JSON report improvements

  • Issue #1652: Use ISO_ZONED_DATE_TIME for validatedAt field to be machine readable.
  • Issue #1685 Change gtfsComponents label to gtfsFeatures.

Notice changes

  • equal_shape_distance_diff_coordinates: Error is triggered when there's 1.11 metres or more distance between the different shape points. Discussion here.
  • trip_distance_exceeds_shape_distance: Error is triggered when there's 11.1 metres or more distance between the last stop point and the last shape point. Discussion here.
  • stop_without_zone_id changed from 🔴 ERROR to ⚪ INFO and updated so it is only triggered when there are route-based fare related fields in fare_rules.txt . Discussed in the spec in google/transit#432.

New notices

  • route_networks_specified_in_more_than_one_file: New 🔴 ERROR for Fares v2 data. Added to spec in google/transit#405.
  • equal_shape_distance_diff_coordinates_distance_below_threshold: New 🟡 WARNING that is triggered when the distance between the different shape points is less than 1.11 metres.
  • trip_distance_exceeds_shape_distance_below_threshold: New 🟡 WARNING that is triggered when the distance between the last stop point and the last shape point is less than 11.1 metres.

Impacts on existing data

You can find the impact of this release on all existing datasets we have in the Mobility Database in this spreadsheet.

Other notable improvements

New Contributors

PR List

  • docs: Update NOTICE_MIGRATION.md by @emmambd in #1599
  • Docs: Add mention to update version number on gtfs.org by @isabelle-dr in #1600
  • feat: Support injection of arbitrary types via ValidationContext by @aababilov in #1607
  • fix: Add label to step that creates an issue by @fredericsimard in #1610
  • fix: cypress error scenario on downloading rules definitions file by @davidgamez in #1617
  • feat: Update references to Dataset Publishing Guidelines from #1601 by @emmambd in #1609
  • Update release process so we run analytics before release is launched by @emmambd in #1612
  • fix: validator version displayed in validator report summary by @jcpitre in #1627
  • fix: Correct links to rule documentation by adding -rule by @bradyhunsaker in #1629
  • fix: decreasing_or_equal_stop_time_distance includes prevShapeDistTraveled by @cka-y in #1631
  • feat: #1619 Added a bit of documentation to refer to gtfs-validator-example. by @jcpitre in #1635
  • fix: Feed generates error to authorize upload by @davidgamez in #1633
  • refactor: Replace CurrentDateTime with DateForValidation and use LocalDate by @bradyhunsaker in #1636
  • docs: Update PULL_REQUEST_TEMPLATE.md by @emmambd in #1622
  • fix: authorize upload error on create job endpoint by @davidgamez in #1642
  • fix: Corrected a problem where the version was null in the desktop version. by @jcpitre in #1643
  • feat: Add an argument for the cli (--date) that specifies the date to use during verification. by @bradyhunsaker in #1628
  • fix: Corrected a problem where code was not multithread safe by @jcpitre in #1653
  • Feat: 1640 add remaining gtfs features to the validator by @qcdyx in #1656
  • feat: removed route name and agency information components by @qcdyx in #1659
  • Minor documentation changes by @jcpitre in #1658
  • feat: added GtfsNetworkSchema by @qcdyx in #1660
  • feat: Added features document. Some corrections to components presence by @jcpitre in #1662
  • fix: prevent exception raised when serviceId is missing from calendar file by @davidgamez in #1646
  • Fix: replace GitRunners with GitHub's own runners by @fredericsimard in #1664
  • Fix: replaced Gitrunners with GitHub's own servers by @fredericsimard in #1673
  • fix: reformatted doc using HTML table format by @qcdyx in #1669
  • fix: update gradle wrapper by @qcdyx in #1677
  • feat: generated CLI package installer by @cka-y in #1680
  • feat: filter out acceptance tests from validator usage report by @cka-y in #1683
  • chore: add skip version check to the master build by @davidgamez in #1684
  • feat: validation rules for networks and route_networks by @cka-y in #1671
  • fix: cypress tests pointing to production are triggered on PRs checks by @davidgamez in #1687
  • feat: 1651 web validator retries with bad zip file 2 by @qcdyx in #1686
  • fix: table not properly rendered in notices by @cka-y in #1695
  • feat: used the standard date time formatter by @qcdyx in #1691
  • feat: replace GTFS Components language to GTFS features by @cka-y in #1696
  • feat: threshold of 1.11m on equal_shape_distance_diff_coordinates by @cka-y in #1675
  • fix: Added a execution result file by @jcpitre in #1690
  • feat: reformat trip and shape dist validator by @cka-y in #1676
  • feat: display ERRORS first on validator documentation by @cka-y in #1667
  • fix: make stop_without_zone_id conditional on fare rule type (#1663) by @michaelandrewkearney in #1693
  • fix: correct field name in missing_recommended_field_notice (#1574) by @michaelandrewkearney in #1706
  • fix: 1707 web validator waits for validation forever by @jcpitre in #1711
  • docs: update deprecated architecture information by @michaelandrewkearney in #1694
  • fix: setting package installer java version to 17.0 by @cka-y in #1709
  • feat: 1689 web validator app consumes execution results json by @davidgamez in #1712
  • feat: Add examples table for mixed_case_recommended_field by @emmambd in #1708

Full Changelog: v4.2.0...v5.0.0

Contributors

aababilov, davidgamez, and 8 other contributors
Assets 12

v4.2.0

07 Nov 17:13
@jcpitre jcpitre
v4.2.0
fdd6c9b
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.2.0

Overview

This release includes

  • Support for time variable fares that was adopted in GTFS in July 2023
  • A new summary section of the validation report, including tags to see if a feed has GTFS components like Blocks, Frequencies, or Fares v2
  • Automatic generation of the RULES documentation on https://gtfs-validator.mobilitydata.org/rules.html
  • New releases published to Maven

New notices

Notices for Fares v2 - time variable fares, added to the spec in google/transit#357

  • timeframe_overlap 🔴 ERROR
  • timeframe_only_start_or_end_time_specified 🔴 ERROR
  • timeframe_start_or_end_time_greater_than_twenty_four_hours 🔴 ERROR

Changed notices

  • Downgraded platform_without_parent_station 🟠WARNING → ⚪ INFO
  • Replaced missing_timepoint_column 🟠WARNING → missing_recommended_column 🟠WARNING

Impacts on existing data

You can find the impact of this release on all existing datasets we have in the Mobility database in this spreadsheet.

Full Changelog: v4.1.0...v4.2.0

New Contributors

What's Changed

Rules

  • fix: Downgraded platform_without_parent_station notice to info by @jcpitre in #1444
  • feat: Add a rule that stations (location_type 1) must be the parent_station of some stop (location_type 0). by @bradyhunsaker in #1493
  • feat: Added missing_recommended_column notice by @jcpitre in #1470
  • feat: Validation for timeframes.txt by @bdferris-v2 in #1518
  • feat: 1423 only warn if feed contact email or url are both unset by @qcdyx in #1514
  • feat: validator for ferry trips + bikes allowance by @cka-y in #1510
  • feat: 1504 add new error if a gtfs dataset has gtfs files in sub folders by @qcdyx in #1535
  • feat: trip vs shape distance validation by @cka-y in #1553
  • fix: comments on #1587 by @cka-y in #1592

Bug fixes

New summary section of report

Generate documentation automatically

  • fix: Refactor MainTest to avoid stubbing System.exit() by @bdferris-v2 in #1410
  • feat: Add @GtfsValidationNotice annotations to all notice classes by @bdferris-v2 in #1366
  • feat: Add unit-test check for @GtfsValidationNotice annotation and notices that I missed by @bdferris-v2 in #1413
  • feat: Include section references in the notice output schema. by @bdferris-v2 in #1416
  • feat: Add Notice documentation header check. by @bdferris-v2 in #1422
  • feat: Change Notice field comment format and unit-test to enforce they exist by @bdferris-v2 in #1421
  • Fix: Switch javadoc aggregation plugin to non-jar version. by @bdferris-v2 in #1432
  • feat: Add a few missing "Affected files" references by @bdferris-v2 in #1427
  • fix: Use correct import for @nullable annotation. by @bdferris-v2 in #1438
  • Feat: Bulk update Notice Javadoc to include content from RULES.md. by @bdferris-v2 in #1428
  • feat: Add documentation update + unit-test for new notice doc conventions. by @bdferris-v2 in #1429
  • feat: Split the notice doc comment into a short summary string and additional documentation. by @bdferris-v2 in #1452
  • feat: Additional test to enforce documentation comment conventions by @bdferris-v2 in #1450
  • Fix: bug in Notice documentation comment splitting. by @bdferris-v2 in #1479
  • feat: Resolve notice severity level from @GtfsValidationNotice annotation. by @bdferris-v2 in #1475
  • Feat: Remove SeverityLevel from ValidationNotice constructor by @bdferris-v2 in #1489
  • feat: Remove unit-test that RULES.md is up-to-date with Notice classes. by @bdferris-v2 in #1517
  • Remove unit-test that RULES.md is up-to-date with Notice classes by @qcdyx in #1522

Web-based validator improvements

  • feat: integrate Cypress for E2E testing by @mijogu in #1396
  • chore: ignore web packages on acceptance and e2e tests by @davidgamez in #1403
  • feat: report link sharing by @...
Read more

Contributors

briandonahue, Whoops, and 17 other contributors
Assets 12

v4.1.0

26 Apr 21:43
@isabelle-dr isabelle-dr
v4.1.0
80550f3
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.1.0

Overview

This release includes the addition of the Web version available at https://gtfs-validator.mobilitydata.org/, new notices, support for Fare Media that was adopted in GTFS in March 2023, and initial support to programmatically generate the notice documentation.
See the notice mapping between releases in NOTICE_MIGRATION.md.

New notices

  • invalid_currency_amount 🔴 ERROR
  • missing_stop_name 🔴 ERROR
  • stop_without_location 🔴 ERROR
  • too_many_rows 🔴 ERROR
  • duplicate_fare_media 🟠 WARNING
  • expired_calendar 🟠 WARNING
  • missing_feed_info_date 🟠 WARNING
  • mixed_case_recommended_field 🟠 WARNING
  • route_long_name_contains_short_name 🟠 WARNING
  • trip_coverage_not_active_for_next7_days 🟠 WARNING

Impacts on existing data

You can find the impact of this release on all existing datasets we have in the Mobility database in this spreadsheet.

What's Changed

Web-based validator

Bug fixes

Rules

Documentation updates

Generate the doc programmatically

  • feat: Initial entry for @GtfsValidationNotice annotation by @bdferris-v2 in #1361
  • feat: Update Notice source code such that field name comments match documentation in RULES.md by @bdferris-v2 in #1346
  • feat: Add unit-test to check consistency of Notice field names. by @bdferris-v2 in #1345
  • fix: Inconsistencies in RULES.md against underlying source by @bdferris-v2 in #1341
  • feat: Add unit test to verify that every Notice has an entry in RULES.md by @bdferris-v2 in #1331
  • feat: Extract notice docs from source-code for in-app use and schema export by @bdferris-v2 in #1365

Acceptance tests improvements

  • feat: Update acceptance test report to include changes in WARNINGS. by @bdferris-v2 in #1354
  • feat: Fail acceptance tests on significant change in warnings by @bdferris-v2 in #1377

General maintenance + other good stuff

  • Use smaller integer field types for GTFS entity classes by @aababilov in #1273
  • Introduce GtfsEntityBuilder interface by @aababilov in #1286
  • Use int instead of long to keep CSV row number by @aababilov in #1287
  • Generate registries of tables and validators by @aababilov in #1290
  • Avoid unchecked conversion in table and validator registries by @aababilov in #1293
  • build(deps): bump certifi from 2021.5.30 to 2022.12.7 in /scripts/mobility-database-harvester by @dependabot in #1299
  • Separate table loading logic from table structure by @aababilov in #1284
  • Do not invoke single file validators if table indices were not generated by @aababilov in #1300
  • Always invoke single file validators (revert #1300) by @aababilov in #1303
  • Discover tables and validators in runtime using io.github.classgraph by @aababilov in #1311
  • Initialize one field per line in the generated code by @aababilov in #1314
  • feat: Refactor JSON report comparison for better diffing in unit-tests. by @bdferris-v2 in #1352
  • feat: Move key annotation and entity interfaces into a new model module by @bdferris-v2 in #1362
  • feat: Initial support for GTFS Fares v2 - fare media by @bdferris-v2 in #1305
  • feat: Add a more descriptive test failure message to NoticeFieldsTest. by @bdferris-v2 in #1369
  • chore: Add unit tests to AnyTableLoader by @davidgamez in #1380

New Contributors

Full Changelog: v4.0.0...v4.1.0

Contributors

briandonahue, KClough, and 7 other contributors
Assets 6

v4.0.0

25 Oct 01:55
@isabelle-dr isabelle-dr
v4.0.0
35bf825
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.0.0

Overview

This is tagged as a major release because adding the rules for Fares v2 base implementation is affecting the validity of existing datasets. We started to evaluate the impact of new rules on production data in 2022 with the acceptance tests, which runs this validator on all datasets from the Mobility Database, and reports if additional errors appear. When new errors appear, we publish a major release.

New notices

GTFS-Fares v2 base implementation, added in google/transit/pull/286

  • fare_transfer_rule_duration_limit_type_without_duration_limit 🔴 ERROR
  • fare_transfer_rule_duration_limit_without_type 🔴 ERROR
  • fare_transfer_rule_invalid_transfer_count 🔴 ERROR
  • fare_transfer_rule_missing_transfer_count 🔴 ERROR
  • fare_transfer_rule_with_forbidden_transfer_count 🔴 ERROR
  • invalid_currency_amount 🔴 ERROR

Transfers, changes in google/transit/pull/303 and google/transit/pull/284

  • transfer_with_invalid_stop_location_type 🔴 ERROR
  • transfer_with_invalid_trip_and_route 🔴 ERROR
  • transfer_with_invalid_trip_and_stop 🔴 ERROR
  • transfer_with_suspicious_mid_trip_in_seat 🟠 WARNING

Best Practice rules

  • missing_recommended_file (applies to feed_info.txt) 🟠 WARNING
  • missing_recommended_field (applies to feed_info.txt start_date, end_date, email, URL, version) 🟠 WARNING)
  • feed_expiration_date_7_days (replacing feed_expiration_date) 🟠 WARNING
  • feed_expiration_date_30_days (replacing feed_expiration_date) 🟠 WARNING

Impacts on existing data

On all datasets MobilityData ran the analytics on, we found that 73 sources would be affected by this release. The datasets affected mostly contain earlier versions of Fares v2 that don't match what was adopted in the spec, and uncertain cases, future versions.
The detailed analysis is in the Fares v2 base implementation for the validator document.

Producers affected

  • Trillium Transit
  • San Diego Metropolitan Transit System

Complete list

See the complete list of datasets affected 1/2
URL Notices
http://data.trilliumtransit.com/gtfs/nts-ca-us/nts-ca-us.zip duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/victorville-ca-us/victorville-ca-us.zip duplicate_key,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/tracy-ca-us/tracy-ca-us.zip duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/downey-ca-us/downey-ca-us.zip duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/weaverville-ca-us/weaverville-ca-us.zip duplicate_key,foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/laketransit-ca-us/laketransit-ca-us.zip duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/auburntransit-ca-us/auburntransit-ca-us.zip duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/cityofridgecrest-ca-us/cityofridgecrest-ca-us.zip duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
https://www.sdmts.com/google_transit_files/google_transit.zip duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/lagunabeach-ca-us/lagunabeach-ca-us.zip invalid_currency_amount
http://data.trilliumtransit.com/gtfs/goldcountrystage-ca-us/goldcountrystage-ca-us.zip duplicate_key,fare_transfer_rule_with_forbidden_transfer_count,foreign_key_violation,invalid_currency_amount
https://transitfeeds.com/p/santa-cruz-metro/343/latest/download duplicate_key,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/redding-ca-us/redding-ca-us.zip duplicate_key,foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/corona-ca-us/corona-ca-us.zip fare_transfer_rule_missing_transfer_count,foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/gtrans-ca-us/gtrans-ca-us.zip fare_transfer_rule_with_forbidden_transfer_count,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/vctc-ca-us/vctc-ca-us.zip fare_transfer_rule_with_forbidden_transfer_count,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/sanbenitocounty-ca-us/sanbenitocounty-ca-us.zip fare_transfer_rule_with_forbidden_transfer_count,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/delnorte-ca-us/delnorte-ca-us.zip fare_transfer_rule_with_forbidden_transfer_count,foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/plumas-ca-us/plumas-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/sagestage-ca-us/sagestage-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/bigbear-ca-us/bigbear-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/mercedthebus-ca-us/mercedthebus-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/thousandoaks-ca-us/thousandoaks-ca-us.zip foreign_key_violation,invalid_currency_amount
See the complete list of datasets affected 2/2
URL Notices
http://data.trilliumtransit.com/gtfs/lassen-ca-us/lassen-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/banning-ca-us/banning-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/morongobasin-ca-us/morongobasin-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/amador-ca-us/amador-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/glenn-ca-us/glenn-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/...
Read more

Contributors

KClough, themightychris, and 6 other contributors
Assets 8
2 Join discussion

v3.1.1

28 Jun 23:21
@isabelle-dr isabelle-dr
v3.1.1
ceb2a86
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v3.1.1

Bug fixes

  • fix: Bug with ssl network connections + Java module permissions. by @bdferris-v2 in #1182
  • fix: Temporary fix for MacOS validator app corruption by @bdferris-v2 in #1186

Enhancements

New Contributors

Full Changelog: v3.1.0...v3.1.1

Contributors

KClough and bdferris-v2
Assets 10

v3.1.0

03 Jun 21:06
@isabelle-dr isabelle-dr
v3.1.0
da57dbf
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v3.1.0

📣 New feature alert! Users can now run the validator as a packaged app and see the results in the browser. This work is described in the epic issue #1124

Enhancements

Bugs fixed

Documentation improvements

Maintenance and upkeep improvements

Contributors to this release

Full Changelog: v3.0.1...v3.1.0

Contributors

barbeau, ed-g, and 4 other contributors
Assets 8

v3.0.1

16 Mar 20:21
@isabelle-dr isabelle-dr
v3.0.1
b8debd9
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v3.0.1

Main update

The main difference between the v3.0.0 and v3.0.1 is the modification of ShapeIncreasingDistanceValidator. After a discussion in #1070, we realized that we had a problem with the notice DecreasingOrEqualShapeDistanceNotice (previously an ERROR): because of the way scheduling software produce data, we can have records in shapes.txt that have equal coordinates, and equal values for shape_dist_traveled. This can happen when two shape points are really close to one another. It creates a somewhat duplicative record, and this should not be an ERROR. A true ERROR happens if values of shape_distance_traveled are equal for two shape points that have different coordinates.

We decided to replace this notice with three new ones (in PR #1083):
1- DecreasingShapeDistanceNotice as an ERROR: if two consecutive shape points have decreasing values for shape_dist_traveled.
2- EqualShapeDistanceDiffCoordinatesNotice as an ERROR: if two consecutive points have equal shape_dist_traveled and different lat/lon coordinates in shapes.txt.
3- EqualShapeDistanceSameCoordinatesNotice as a WARNING: if two record have the same values for shape_dist_traveled and lat/lon coordinates in shapes.txt.

List of merged PRs

New Contributors

Full Changelog: v3.0.0...v3.0.1

Contributors

aababilov, barbeau, and 3 other contributors
Assets 5

v3.0.0

10 Jan 16:13
@isabelle-dr isabelle-dr
v3.0.0
fed4bad
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v3.0.0

We're excited to bring you the 3.0.0 release, it's a big one!

We three had main goals for this release:

➡️ Solve discrepancies with the official GTFS specification.
26 new rules have been added in total:

  • New validation rules were added for the adopted GTFS extensions pathways.txt, levels.txt, translations.txt
  • New custom rules were added for parts of the spec that were missing from this validator
  • Some existing rules were improved to better reflect the specification
  • The severity of some rules was updated to reflect the specification after the update to RFC 2119 has been done

Please refer to the V2.0.0 to V3.0.0 rule mapping available in NOTICE_MIGRATION.md for more details on the rule updates.

➡️ Implement automated tests that will evaluate if a new rule results in datasets from the mobility database getting additional errors. If so, the Pull Request is blocked and a report will be generated for further investigation. This ensures the stability and reliability of this validator, while still being updated with updates and extensions in the specification.

➡️ Make sure MobilityData's work has related public communication to facilitate community contribution (issues opened, public sprint board and bug tracking board, roadmap, ways for users to voice what they'd like to see next and for contributors to see how to get involved).

There are also new command line parameters that can be found in USAGE.md.

There is a lot more work included in this release, thanks to our amazing contributors. See details below!

Work ⚙️

Solve discrepancies with the spec

  • chore: downgrade LeadingOrTrailingWhitespacesNotice to WARNING in #929
  • chore: upgrade EmptyColumnNameNotice to ERROR in #1019
  • chore: new rule - additional requirements urls (stops.txt, routes.txt, agency.txt) in #932

routes.txt

  • feat: Reduce severity of SameNameAndDescriptionForRouteNotice to WARNING in #917

stops.txt

  • chore: check stop name and desc in #937
  • chore: check stops.zone_id presence in #936

stop_times.txt

  • chore: new rule - additional check on stops.location_type in #938
  • chore: upgrade LocationWithUnexpectedStopTimeNotice to ERROR in #1021

pathways.txt and levels.txt

  • feat: Validate how locations are referenced from stop_times.stop_id in #960
  • feat: Validate that pathway endpoints have correct types in #961
  • feat: verify conditional requirement of levels.txt presence in #966
  • feat: additional checks on pathways.txt and stops.txt in #967
  • feat: Validate that locations are reachable if a station has pathways in #997
  • feat: Validate that a pathway is not a loop in #999
  • feat: Validate field presence and key integrity for translations.txt in #968
  • chore: upgrade PathwayUnreachableLocationNotice to ERROR in #1028

translations.txt

  • chore: upgrade TranslationForeignKeyViolationNotice to ERROR in #1027

Automated test to see if a PR results in additional errors

Other work included

Better notice schema export

Better documentation

Bugs

  • fix: Prevent NullPointerException when Throwable.getMessage returns null in #871
  • fix: Catch any runtime exception when parsing a GTFS field in #1012
  • fix: Swap arguments of isAssignableFrom in #872
  • fix: bump debian version used in docker image in #891
  • fix: Protect against start_date and end_date in wrong order in #897
  • fix: Raise IllegalArgumentException for invalid date in #977
  • fix: change limit on notice count during validation to 100_000 in #1018
  • fix: validate route, stops and agency consistency urls in #1003
  • feat: Prevent OOM in NoticeContainer and speed up hasValidationErrors in #895
  • fix: Skip phone number validation when country code is unknown in #1062
  • fix: False positives for StopTimeTimepointWithoutTimesNotice in #1044

Other good stuff

  • ci: check code formatting on pull requests in #953
  • ci: publish a new docker image on every push on master in #913
  • ci: verify gradle wrapper in #964
  • ci: bug triage in #982
  • feat: Provide enum setters and clearFIELD methods in #843
  • feat: make report names user configurable via CLI args in #852
  • feat!: deprecate -f in #851
  • feat: Pass severityLevel to constructor of some notices in #863
  • feat: Propagate InterruptedException during feed loading in #869
  • feat: Catch CSV parse errors and add validation notices in #874
  • feat: print help menu in #888
  • feat: Export infinity and NaN values in JSON in #889
  • feat: Treat empty files as parsed unsuccessfully in #898
  • feat: introduce -p CLI parameter to output beautified reports in #901
  • feat: Add byPrimaryKey and getKeyColumnNames methods to GtfsTableContainer in #1000
  • chore: upgrade to gradle wrapper to 7.2 in #965
  • chore: Upgrade to Java 11 in #1014

Contributors 🙏

Massive shoutout to:
🎉 @aababilov
🎉 @asvechnikov2
🎉 @barbeau
🎉 @lionel-nj
🎉 @maximearmstrong
🎉 @isabelle-dr

Contributors

aababilov, barbeau, and 4 other contributors
Assets 5

v2.0.0

24 Mar 20:05
@lionel-nj lionel-nj
v2.0.0
131ab53
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v2.0.0

What's new 🆕

We're excited to bring you this v2 release of the gtfs-validator! A lot has changed since v1, and we think you'll appreciate the new features.

  • Faster 🚀 and stronger 💪 thanks to a new multi-threaded architecture (thank you @aababilov!) (PR #486)
    • Supports the largest known real-world GTFS datasets. For example, the Netherlands GTFS previously failed with an OutOfMemory error, and now it can be validated in 2 minutes and 12 seconds on an ordinary laptop!
    • Validates datasets much faster. For example, MBTA used to take 38 seconds to validate, and now it only takes 14 seconds - less than half the time!
  • ⏬ Reduces the amount of code needed for adding new validation rules and support for new GTFS files and fields by leveraging Java annotations and AutoValue
  • 📝 Notice definitions for validation rules have been updated
    • Notice severities now align with the canonical GTFS spec (e.g., issues #472, #417). ERRORS are for items that the spec explictly requires or prohibits, and WARNINGS are for items that will impact the quality of GTFS datasets but aren't strictly required (suggestions or best practices).
    • Notices have ported to the new architecture (e.g., PR #592, PR #710). See the "Rules" documentation for the new v2 notice definitions and the "v1 to v2 Migration Guide" for a table mapping v1 notices to v2 notices.
  • 🛠 Better error handling and reporting
    • In case of problems with the validator software itself, an error report is now written to a file system_errors.json to help with troubleshooting (PR #596). Normal validation results can be found in report.json.
  • 🗺 New project roadmap
    • Check out the project wiki. Please feel free to add things you are interested in!
Assets 5