2025-10-09 Engineering Team

[jpn--]

@ActivitySim/engineering

Agenda

• PopulationSim release prep: what version? Latest release was 0.5.1, pyproject.toml now says 0.9.2
• Implementation review on TransLink & ODOT's park-and-ride solution ActivitySim#965 ActivitySim#1001
• Recommend next steps (cost, schedule, expected value) on EET.
  • random number generation
  • simple random sampling for destinations
  • stratified random sampling for destinations
  • other?
• Updates on 11B tasks
  • Consolidated Error Definitions (CS) -- Opened ActivitySim#1000
  • Trace files consistent hash (CS)
  • UV Support (WSP + All)
  • Clean up loggging (CS)
  • Consistent Alternatives File Formatting (RSG)
  • Trip Scheduling Choice Reproducible Random (RSG)
  • Fix Shadow Price Zones Reopening (RSG)
  • Global Option to Skip Households on Fail (WSP)
  • Example SANDAG Model Update (WSP) -- Opened sandag-abm3-example#37
  • SANDAG Sharrow Performance Optimization (DL)

Meeting Notes: Zephyr Foundation Engineering Call

Date: 2025-10-09
Duration: 1 hour 1 minute
Attendees: Joe Castiglione, Sijia Wang, Jeff Newman, David Hensle, Stefan Coe, Sumit, Joel Freedman, Guy Rousseau, Jilan Chen, Cherry Liu

1. Population Sim Release Version Discussion

Discussion

• Last consortium release was numbered 0.5.1
• Recent pull request from RSG shows version 0.9.2 in pyproject.toml
• Concern about version number conflicts and maintaining clarity across users

Decision

• Skip to version 0.10 for next release to avoid confusion
• Release notes will explain the version number skip
• No objections from team members

2. Explicit Error Term (EET) Runtime Optimization

Current Issues

• Runtime is the primary concern with EET implementation
• Location choice models now account for disproportionate share of overall runtime
• EET destination choice models require random number draws plus two exponentiations per alternative (22,000 alternatives at MAZ level)

Three Proposed Approaches

A. Random Number Generation Modernization

• Current system uses expensive re-seeding method (Mersenne Twister)
• Modern NumPy generators use only 128-bit state (2 doubles) vs. kilobyte-sized state
• Could store random number generator state per tour/household/trip
• Status: Prototyping and testing needed
• Concerns: Memory implications need assessment; implementation complexity

B. Sampling Method Changes

• Switch from importance sampling to simple random sampling or stratified random sampling
• Expected to significantly reduce runtime for destination choice
• Status: Joel indicates high confidence this would solve EET runtime issues
• Concerns:
  • Not just a software issue - requires extensive testing for model sensitivities
  • Implications for existing calibrated models unclear
  • Could require more substantial changes than anticipated

C. Disaggregate Accessibilities Investigation

• Address unexplained changes in unaffected zones
• Status: Lower priority; needs scoping

External Input

• Sumit reported Bentley (Momo conference) claims to have solved runtime explosion issue
• Team will reach out to inquire about their approach before committing to direction

Action Items

• Engineering team to write discrete scopes for:
  1. Random number generation prototyping (3-month timeframe)
  2. Sampling method investigation and testing
  3. Disaggregate accessibilities (if prioritized)
• Submit scopes to product/community team and executive team for prioritization
• Contact Bentley/Peter about their EET implementation approach

Consensus

• Random number generation: Broadly supported for prototyping
• Sampling methods: High potential but requires careful testing of model impacts
• Disaggregate accessibilities: Deferred pending scoping and resource availability

3. Park and Ride (P&R) Methodology and Implementation

Presenter: David Hensle

Current P&R Implementation (Most ActivitySim Models)

• Assignment software handles P&R via hyperpaths across multiple lots
• Cannot distinguish actual parking location
• Generates expensive inbound/outbound P&R skims (doubled transit skims)
• Capacity constraints difficult/impossible to implement in demand model
• Stops not allowed on P&R tours (car location unknown)
• Trip mode choice either restricts to P&R or allows inconsistent modes

Proposed New Methodology

Core Concept

• Add P&R lot choice model before tour mode choice
• Explicitly select parking lot based on:
  • Auto time from origin to lot
  • Walk transit time from lot to destination
  • Calculated for both directions and all P&R locations

Benefits

• Eliminate dedicated P&R skims (use auto + walk transit skims instead)
• Direct control over utility specifications (terminal time, costs, etc.)
• Enable capacity tracking and constraints
• Support iterative capacity enforcement

Capacity Constraint Mechanism

1. Run P&R lot choice to determine which lot would be used
2. Calculate tour mode choice utilities using selected lot
3. Count tours actually selecting P&R at each lot
4. Turn off capacitated lots
5. Re-simulate tours that can't fit (switch to alternative mode or lot)
6. Iterate until convergence

Time-of-Day Handling

• Tours simulated simultaneously throughout day
• When lot fills, tours departing after fill time are re-simulated
• Lots remain full for entire day (don't reopen) - reasonable for planning purposes per Jeff Newman

Implementation Details

• Filter destinations without transit access to reduce computation
• Land use file specifies formal and informal parking capacity per zone
• Iteration settings include:
  • On/off toggle for capacity constraints
  • Maximum iterations before timeout
  • Tolerance threshold for convergence
  • Resample strategy (latest tours vs. random)
• Output: Park and ride zone ID in tours table

Questions Raised

Joe (Zephyr Foundation)

• Runtime implications need confirmation, especially with re-simulation
• Time-of-day mechanics need clearer documentation
• Convergence properties critical (referenced DaySim implementation that never converged - ended up using arbitrarily high capacities)
• Need path to integration testing with example dataset

Guy Rousseau

• Cardinal orientation handled through utilities (drive time + transit time naturally optimizes location selection)

Stefan Coe

• Behavioral approach more sensible than shadow pricing
• P&R zones can be own MAZ or within existing MAZ

Jilan Chen

• Lot choice runs for each tour initially, then only for re-simulation
• Informal parking can be included via land use fields (formal + informal capacity)

Sijia Wang

• Transit mode variety between lots (walk premium vs. walk local) - David confirmed both would be in utilities like regular tour mode choice

Cherry Liu

• In/out tracking - Clarified lots stay full for day once capacitated (planning vs. operational model)

Limitations

• Tours can only change mode during iteration (not destination or time-of-day)
• Minimal runtime impact expected (only re-simulating constrained tours)
• Requires running P&R lot choice during logsum calculations

Status

• Implementation complete
• Full-scale testing in progress
• Need to validate: convergence, runtime performance, lot filling behavior

Next Steps

• Continue extensive testing
• Discuss integration into consortium examples
• Future session: utility building details

4. 11B Tasks Status Check (Deferred)

Decision

• Not enough time to cover in this meeting
• Next week's meeting (Thursday) canceled - Jeff Newman and David Hensle unavailable
• Team members to provide asynchronous status updates on 11B tasks via meeting thread
• Updates encouraged before next scheduled meeting

Action Items Summary

Owner	Action	Timeline
Engineering Team	Write scopes for EET tasks (random number generation, sampling methods, disaggregate accessibilities)	TBD
Team	Contact Bentley/Peter about EET runtime solution	ASAP
David Hensle	Continue full-scale P&R testing (convergence, runtime, behavior)	Ongoing
All	Post 11B task status updates to meeting thread	Before next meeting
Joe (Zephyr)	Cancel next week's engineering call	Complete

Next Meeting

• Next scheduled call: Two weeks from this meeting
• Format: Regular synchronous meeting

[JoeJimFlood]

Update on the error consolidation task: I have inventoried every RuntimeError in the v1.5.0 codebase (skipping some tests and the old PSRC example) and put the results in an Excel file that includes the file, line number, message, and some context about what is happening (I then added fields for the suggested error to replace RuntimeError with). The results so far are in the draft PR #1000. Between replacing RuntimeError with new and existing exceptions, we're down to 13 Runtime errors, and I feel like there's more that can be removed.

[jpn--]

Here are alternative meeting notes generated by Claude Sonnet 4.5. I didn't find anything technically incorrect with the above notes, but I find this more conversational format easier to digest...

ActivitySim Engineering Meeting Notes

Meeting Overview

Agenda

• PopulationSim version numbering decision
• Explicit Error Term (EET) next steps and prioritization
• Park and Ride solution methodology review
• 11B task status updates (deferred)

Participants

• Joe Castiglione, Zephyr Foundation (@joecastiglione)
• Jeff Newman, Driftless (@jpn--)
• David Hensle, RSG (@dhensle)
• Joel Freedman, RSG (@jfdman)
• Sijia Wang, WSP (@i-am-sijia)
• Stefan Coe, PSRC (@stefancoe)
• Guy Rousseau, ARC (@guyrousseau)
• Jilan Chen, SEMCOG (@JilanChen)
• Cherry Liu, SEMCOG
• Sumit Bindra, TransLink

Duration

Approximately 60 minutes

Executive Summary

The team decided to release PopulationSim as version 0.10 to avoid confusion with intermediate version numbers. Discussion on EET runtime improvements identified three potential approaches: random number generation updates, simple random sampling, and stratified random sampling for destination choice models. The team will reach out to Bentley for insights and develop scoped proposals for the random number generation and sampling investigations. David Hensle presented a new park and ride methodology that selects lots before tour mode choice, eliminates the need for park and ride skims, and implements capacity constraints through iterative resimulation.

Meeting Notes

PopulationSim Version Numbering

The team discussed version numbering for the upcoming PopulationSim release. The last consortium release was version 0.5.1, but RSG's pull request showed version 0.9.2 in the pyproject.toml file. To avoid confusion with any intermediate versions that may have been used by individual agencies, the team decided to skip ahead to version 0.10 for the next release. A note will be included in the release explaining the version number gap.

Explicit Error Term (EET) Development

Runtime performance remains the primary concern for EET implementation, with destination choice models accounting for most of the additional runtime. Three potential approaches were discussed for improving performance:

Random Number Generation: The current implementation reseeds the random number generator for each use, which is expensive. Modern generators like NumPy's default (128-bit state) could store state efficiently as two doubles per entity. This would require careful implementation but could benefit all models, not just EET. Joel noted that similar approaches were used in CTRAMP with arrays to track random number advancement.

Simple Random Sampling: Moving from importance sampling to simple random sampling for destination choice could significantly reduce runtime since it eliminates the need for two exponentiations per alternative. This is primarily a software implementation issue.

Stratified Random Sampling: Similar to approaches used in SFCHAMP, this could solve the runtime problem but would require extensive testing to ensure model sensitivities remain stable.

Sumit reported that Bentley presented at the Momo conference claiming they solved the runtime explosion problem. The team agreed to reach out to Bentley to learn about their approach before proceeding with development. Joel noted that Peter has been investigating simple random sampling as a potential solution.

Joe expressed support for investigating both random number generation and sampling methods. He noted concerns about the implications of changing sampling methods for existing models, suggesting it should be implemented as a switch. He questioned whether the disaggregate accessibilities investigation should be prioritized, noting unexplained anomalies in unaffected zones but expressing uncertainty about resource allocation.

Joel emphasized that testing stratified random sampling has both software and modeling components requiring extensive sensitivity testing, while random number generation updates are primarily software issues. The engineering team will develop discrete scopes with cost estimates for these three elements for consortium review and prioritization.

Park and Ride Solution Methodology

David Hensle presented a comprehensive new approach to park and ride modeling in ActivitySim.

Current Implementation Limitations: Most ActivitySim models rely on assignment software to generate park and ride skims as hyperpaths across multiple lots. This makes it impossible to track individual lot choices, difficult to implement capacity constraints, and typically prohibits stops on park and ride tours. The park and ride skims are expensive, effectively doubling transit skim requirements.

Proposed Methodology: The new approach adds park and ride lot choice as a model component that runs before tour mode choice. The model calculates utilities using auto times from origin to lot and walk-transit times from lot to destination. This explicit lot selection enables several improvements:

• Eliminates the need for park and ride skims, using only auto and walk-transit skims
• Provides direct control over utility specifications including terminal time and lot-specific costs
• Enables capacity tracking and constraint enforcement
• Allows for both formal and informal parking capacity designation

Capacity Constraint Implementation: When lots reach capacity, the model identifies tours arriving after the lot fills based on time of day and resamples only those tours. These tours can then select alternative lots or switch to different modes. The implementation uses a simulation-based constraint mechanism similar to location choice shadow pricing but counts actual demand rather than applying shadow prices. Tours are only constrained to change mode, not destination or time of day.

Key Design Considerations: Park and ride lot choice runs during both tour mode choice and logsum calculations. Lots that fill remain closed for the entire day rather than reopening when vehicles depart, which is appropriate for planning purposes given that demand after morning peak is minimal. The model filters out destinations without transit access to reduce unnecessary calculations.

Guy asked about cardinal orientation (i.e. geography), which is handled through the utility calculations where longer drive or transit times create disadvantages. Joe questioned the time of day implementation, noting that tours are simulated simultaneously rather than sequentially through the day. David confirmed that capacity enforcement considers arrival time when determining which tours to resample.

Stefan noted the behavioral advantage of this approach over pure shadow pricing, as later arrivals are less likely to find spaces. Jeff reinforced that planning model requirements differ from operational models, making the simplified daily closure approach appropriate and cost-effective.

Jilan asked about informal parking, which can be designated in separate land use fields and included in total capacity calculations. Sijia questioned whether alternative design options were considered; David explained the team focused on iteration scope, deciding that iterating only tour mode choice provides the right balance of responsiveness and implementation complexity.

Cherry asked about tracking in/out flows and whether lot choice would be skipped when capacity is met. David clarified that lots remain closed once full, and when no capacity exists anywhere, park and ride becomes unavailable through tour mode choice availability conditions.

Stefan asked whether park and ride zones can be independent MAZs; David confirmed any valid zone in the land use table with capacity greater than zero is eligible for selection.

Implementation Status: The methodology has been implemented but full-scale testing has not been completed. Next steps include convergence testing, runtime validation, and model sensitivity verification. The team acknowledged the need to establish how this functionality will be incorporated into example models for integration testing.

11B Task Status Updates

Due to time constraints, the team deferred discussion of 11B task status updates. The next week's engineering call was cancelled, with participants asked to provide asynchronous status updates on the meeting thread.

Action Items

• Engineering Team: Develop discrete scopes and cost estimates for three EET improvement approaches: random number generation updates, simple random sampling investigation, and stratified random sampling investigation
• Engineering Team (Jeff Newman or designee): Contact Bentley to inquire about their EET runtime solution presented at MoMo conference
• Engineering Team: Include assessment of model outcome impacts when scoping sampling method changes
• Product/Community/Executive Teams: Review and prioritize scoped EET improvement proposals once submitted
• RSG (David Hensle): Complete full-scale testing of park and ride implementation including convergence, runtime, and sensitivity analysis
• Consortium: Determine approach for incorporating park and ride functionality into example models
• All Teams with 11B Tasks: Provide asynchronous status updates on meeting thread
• Joe Castiglione: Cancel next week's engineering meeting (October 16th)