Summary
This epic tracks the work required to build a Change Tracker for GTFS validation report differences.
The Change Tracker is intended to address the risk of silent updates, where important changes in validation results go unnoticed until they cause downstream issues. By identifying and showing differences between validation reports, this work will provide greater operational transparency for agencies and a clearer view of how data quality changes over time for researchers and other consumers.
This PR is a complement of https://github.com/MobilityData/product-tasks/issues/189
This effort includes the core dataset comparison logic, a user-facing dashboard to explore changes, and a notification system( addressed in https://github.com/MobilityData/product-tasks/issues/196) for relevant updates.
Problem statement
When a GTFS validation report changes, it is not always clear what changed. New notices can appear, existing notices can disappear, and counts can go up or down without anyone noticing until something breaks. This makes it harder for developers to fix problems, makes data users less confident, and makes it difficult for agencies and researchers to see how data quality changes over time.
A Change Tracker would let people quickly see what is different between two validation reports: what was added, what was removed, and what changed.
Goals
- Find the real differences between two validation reports
- Show those differences in a way that is easy to understand for both technical and non-technical users
- Help people see how validation results change over time
- Make it faster to figure out what caused a problem after a validation update
- Let users know when something important has changed
Scope
This epic covers the work needed to design, build, and improve the Change Tracker(validation results), including:
- Building the logic that compares two validation reports and finds the differences
- Deciding which report changes are worth showing
- POC a simple dashboard where users can explore what changed
- Making it easy for non-technical users to browse changes
- Propose types of notifications so users can be alerted when something important changes
- Sharing the tool with real agencies and improving it based on their feedback
- Propose a strategy on how to integrate the validation report differences with the file differences implemented in (https://github.com/MobilityData/product-tasks/issues/189)
Expected outcomes
- Users can compare validation reports and clearly see what changed
- Agencies can better keep track of changes in their validation results
- Researchers and data users can follow how validation results evolve over time
- Non-technical users can explore changes through a dashboard
- Users who opt in can get alerts when important changes happen
Summary
This epic tracks the work required to build a Change Tracker for GTFS validation report differences.
The Change Tracker is intended to address the risk of silent updates, where important changes in validation results go unnoticed until they cause downstream issues. By identifying and showing differences between validation reports, this work will provide greater operational transparency for agencies and a clearer view of how data quality changes over time for researchers and other consumers.
This PR is a complement of https://github.com/MobilityData/product-tasks/issues/189
This effort includes the core dataset comparison logic, a user-facing dashboard to explore changes, and a notification system( addressed in https://github.com/MobilityData/product-tasks/issues/196) for relevant updates.
Problem statement
When a GTFS validation report changes, it is not always clear what changed. New notices can appear, existing notices can disappear, and counts can go up or down without anyone noticing until something breaks. This makes it harder for developers to fix problems, makes data users less confident, and makes it difficult for agencies and researchers to see how data quality changes over time.
A Change Tracker would let people quickly see what is different between two validation reports: what was added, what was removed, and what changed.
Goals
Scope
This epic covers the work needed to design, build, and improve the Change Tracker(validation results), including:
Expected outcomes