feat(T4MetricV2): add distance-based evaluation to 3D detection #162
Conversation
…thub.com:tier4/AWML into feat/add_distance_based_evaluation_to_t4metricv2
There was a problem hiding this comment.
Pull request overview
This PR enhances T4MetricV2 to support distance-based evaluation by enabling multiple evaluators for different BEV (Bird's Eye View) distance ranges in 3D object detection. The key change transforms the metric system from single-range evaluation to multi-range evaluation, allowing metrics to be computed separately for different distance buckets (e.g., 0-60m, 60-90m, 90-121m, and overall 0-121m).
Key Changes:
- Modified T4MetricV2 to create multiple evaluators based on BEV distance ranges
- Updated configuration files to use list-based min/max distances instead of single values
- Removed the
critical_object_filter_configfrom several configuration files, setting it toNone
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| autoware_ml/detection3d/evaluation/t4metric/t4metric_v2.py | Core implementation of multi-evaluator support with distance-based filtering |
| autoware_ml/configs/detection3d/dataset/t4dataset/*.py | Updated metric configs to use distance range lists instead of single values |
| projects/CenterPoint/configs/t4dataset/Centerpoint/*.py | Removed critical_object_filter_config and updated dataset names |
| projects/CenterPoint/configs/t4dataset/Centerpoint/second_secfpn_4xb16_121m_j6gen2_v6_amp_rfs.py | New configuration file for frame sampling with distance-based object samplers |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| previous_scene_id = None | ||
| previous_perception_frame_result = None | ||
| for perception_frame_preprocessing_result in perception_frame_preprocessing_results: | ||
| # When the scene id is different from the previous frame scebe id, it's the first frame of the scene or when the previoous_scene_id is None |
There was a problem hiding this comment.
Corrected spelling of 'previoous_scene_id' to 'previous_scene_id' in the comment on line 685.
| # When the scene id is different from the previous frame scebe id, it's the first frame of the scene or when the previoous_scene_id is None | |
| # When the scene id is different from the previous frame scebe id, it's the first frame of the scene or when the previous_scene_id is None |
| previous_scene_id = None | ||
| previous_perception_frame_result = None | ||
| for perception_frame_preprocessing_result in perception_frame_preprocessing_results: | ||
| # When the scene id is different from the previous frame scebe id, it's the first frame of the scene or when the previoous_scene_id is None |
There was a problem hiding this comment.
Corrected spelling of 'scebe' to 'scene' in the comment on line 685.
| # When the scene id is different from the previous frame scebe id, it's the first frame of the scene or when the previoous_scene_id is None | |
| # When the scene id is different from the previous frame scene id, it's the first frame of the scene or when the previoous_scene_id is None |
| previous_perception_frame_result: Optional[PerceptionFrameResult], | ||
| ) -> PerceptionFrameMultiProcessingResult: | ||
| """ | ||
| Wrapper to apply an evaluator to a a pair of PerceptionFrameResults. |
There was a problem hiding this comment.
Removed duplicate article 'a' in the docstring.
| Wrapper to apply an evaluator to a a pair of PerceptionFrameResults. | |
| Wrapper to apply an evaluator to a pair of PerceptionFrameResults. |
|
|
||
| assert isinstance(perception_evaluator_configs["evaluation_config_dict"]["min_distance"], list) and isinstance( | ||
| perception_evaluator_configs["evaluation_config_dict"]["max_distance"], list | ||
| ), f"min_distance and max_distance must be a list, got: {type(perception_evaluator_configs['min_distance'])} and {type(perception_evaluator_configs['max_distance'])}" |
There was a problem hiding this comment.
The error message references incorrect dictionary keys. It should use evaluation_config_dict nested keys instead of top-level keys that don't exist.
| ), f"min_distance and max_distance must be a list, got: {type(perception_evaluator_configs['min_distance'])} and {type(perception_evaluator_configs['max_distance'])}" | |
| ), ( | |
| "min_distance and max_distance must be a list, got: " | |
| f"{type(perception_evaluator_configs['evaluation_config_dict']['min_distance'])} and " | |
| f"{type(perception_evaluator_configs['evaluation_config_dict']['max_distance'])}" | |
| ) |
| # selected_scenes = {} | ||
| # for key in list(scenes.keys())[:2]: | ||
| # selected_scenes[key] = scenes[key] |
There was a problem hiding this comment.
Remove commented-out debugging code that is not needed in production.
| # selected_scenes = {} | |
| # for key in list(scenes.keys())[:2]: | |
| # selected_scenes[key] = scenes[key] |
Summary
Write what you change and why you change
Change point
Explain for detail
If you add new feature or fix for core library
If you add new feature or fix for projects
If you change the thing without program
Note
Write note like influences for other projects.
Test performed