Add section on obtaining vision measurements for pose estimation #3137
Conversation
Adds comprehensive "Obtaining Vision Measurements" section explaining how to get robot pose estimates from vision systems like PhotonVision and Limelight for use with AddVisionMeasurement(). Fixes wpilibsuite#1413
| .. important:: When using vision measurements, it's critical to: | ||
|
|
||
| - Use the **timestamp** from when the image was captured, not when it was processed | ||
| - Scale the **standard deviations** based on factors like distance from tags, number of tags seen, and tag ambiguity |
There was a problem hiding this comment.
This feels like it should be in the tuning pose estimators below.
|
Thinking about this more, I think most of this probably belongs in the vision processing docs, rather then here. |
Per review feedback, moved the detailed vision measurement information from the pose estimator docs to a new article in the vision processing section. Changes: - Created new apriltag-pose-estimation.rst in vision-processing/apriltag/ - Comprehensive guide covering PhotonVision, Limelight, and custom solutions - Detailed sections on timestamps, standard deviations, and rejecting bad measurements - Code examples in Java, C++, and Python for both libraries - Updated apriltag/index.rst to include new article - Replaced detailed section in state-space-pose-estimators.rst with a seealso link The pose estimator docs now focus on the API usage, while the vision processing docs explain how to obtain the measurements. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Changed links in "See Also" section to anonymous hyperlinks (double underscores) to avoid conflicts with similar link text used earlier in the document. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
…d-layout file Removed references to apriltag-field-layout.rst which doesn't exist in this PR: - Removed from toctree in index.rst - Removed from "See Also" section in apriltag-pose-estimation.rst The apriltag-field-layout article is being created in PR wpilibsuite#3138. Once that PR merges, the references can be added back. This fixes the build errors: - "toctree contains reference to nonexisting document" - "unknown document: 'apriltag-field-layout'" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
|
|
||
| ## Using Vision Libraries | ||
|
|
||
| Most teams use existing vision processing libraries that handle the complex mathematics for you: |
There was a problem hiding this comment.
This jumps right into the robot code side, but there should probably be some mention about needing to set-up photonvision and limelight for April tag tracking.
|
|
||
| It's critical to use the **timestamp from when the image was captured**, not when it was processed. Vision processing introduces latency (typically 20-100ms), and the pose estimator needs the actual capture time to properly fuse the measurement with odometry data. | ||
|
|
||
| Most vision libraries provide this timestamp: |
| int tagCount = /* number of tags seen */; | ||
| // More tags = more trust, greater distance = less trust | ||
| double xyStdDev = 0.5 * Math.pow(distance, 2) / tagCount; |
There was a problem hiding this comment.
This formula seems suspect. If you're 1 meter away with 1 tag, it would give a standard deviation of half a meter. But if you're 10 meters away, it would give a standard deviation of 50 meters.
| // More tags = more trust, greater distance = less trust | ||
| double xyStdDev = 0.5 * Math.pow(distance, 2) / tagCount; | ||
| double thetaStdDev = 999999.9; // Don't trust rotation from single tag |
There was a problem hiding this comment.
This comment would imply that there should be a conditional at use a large tehetaStdDev if there's only a single tag and a different one if there's more then one tag visible
| var estimatedPose = result.get(); | ||
| // Check if pose is reasonable (within field boundaries) | ||
| if (estimatedPose.estimatedPose.getX() >= 0 && |
There was a problem hiding this comment.
It feels like you wouldn't want to reject a tag that's on the edge of the field if it was reported as just outside the field, as that would still be a reasonable measurement.
| - **Unrealistic poses**: Reject measurements that are outside the field boundaries or far from your current estimate | ||
| - **During rapid motion**: Vision measurements may be less reliable during fast turns or acceleration | ||
|
|
||
| Example rejection logic: |
There was a problem hiding this comment.
This should specify what checks the code below does and what are left as exercise for the readers. This looks like it only handle the no tags detected and outside field boundaries cases.
2 participants
Summary
Creates a comprehensive guide for using AprilTags for robot pose estimation, addressing the lack of documentation on obtaining vision measurements for pose estimators.
Changes
New file:
apriltag-pose-estimation.rstin vision processing sectionModified:
state-space-pose-estimators.rstModified:
apriltag/index.rstOrganization
Per review feedback, vision measurement content now lives in vision processing docs (where teams look for vision guidance) rather than pose estimator docs (which focus on the API).
Fixes #1413