Skip to content

Add a benchmark scene with multiple rigid objects #798

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Nov 13, 2025
Merged

Add a benchmark scene with multiple rigid objects #798

merged 8 commits into from
Nov 13, 2025

Conversation

@Kenny-Vilella
Copy link
Collaborator

@Kenny-Vilella Kenny-Vilella commented Oct 27, 2025

  • Added a new Apollo scene with a box of rigid objects (cubes)

@Kenny-Vilella
Copy link
Collaborator Author

Kenny-Vilella commented Oct 27, 2025

@erikfrey I have been told by @adenzler-nvidia that we should add benchmark scenes that are important for us.
This is one of the representative scene that are currently critical for us.

There is some jittering in the cubes collision, but not sure how to make it consistently stable.

@erikfrey
Copy link
Collaborator

erikfrey commented Oct 27, 2025

@Kenny-Vilella this is awesome, thanks. Yes let's add it. Let me study the XML a bit before merging just to make sure the defaults are sensible, etc. Will get to it soon.

hartikainen
hartikainen reviewed Oct 28, 2025
View reviewed changes
@hartikainen hartikainen mentioned this pull request Nov 1, 2025
Merged
@Kenny-Vilella
Copy link
Collaborator Author

Kenny-Vilella commented Nov 6, 2025

After the merge, I cannot include the apollo anymore.
It seems that the replicate clash with the keyframe in the apollo model.
If I remove the replicate and add individual objects, then I guess the keyframe from the apollo overwrite the objects body pose and everything start at the origin.
If I do not put a position in the body but at the geom level, then it is highly unstable.

What is the proper way to deal with this?

@yuvaltassa
Copy link
Collaborator

yuvaltassa commented Nov 10, 2025

@Kenny-Vilella Apologies for the delay. Can you please describe more precisely what you are seeing?

keyframe from the apollo overwrite the objects body pose and everything start at the origin

Are you saying that pose information from the robot is "leaking" into the poses of objects?

@Kenny-Vilella
Copy link
Collaborator Author

Kenny-Vilella commented Nov 11, 2025

@yuvaltassa I didn't track deep into the code, but it looks like something from the robot is leaking into the poses of objects.
I should have some time today or tomorrow to look deeper in what is happening.

Summary of my tests:
(1) Adding to the new xml
<include file="apptronik_apollo.xml" />
leads to the following error:

File "/home/kvilella/Documents/Codes/mujoco_warp/mujoco_warp/viewer.py", line 83, in _load_model
    spec = mujoco.MjSpec.from_file(path.as_posix())
ValueError: XML Error: Error: Keyframe 'stand' has invalid qpos size, got 26, should be 33
Element 'replicate', line 29

26 qpos are 7 for the root + 19 joints from the apollo.
It looks like it expects the object pose to be present in the keyframe of the robot.
If I remove the keyframe from the robot everything works as expected.

(2) Same as (1) except that I replaced all the replicate by adding directly the objects:
Just adding the 4 spheres is enough:

    <body name="sphere_1" pos="0.6 -0.1 0.81">
        <freejoint name="sphere_1"/>
        <geom name="geom_sphere_1" type="sphere" size="0.05" contype="1" conaffinity="1" rgba="1 0 0 1"/>
    </body>
    <body name="sphere_2" pos="0.675 -0.025 0.81">
        <freejoint name="sphere_2"/>
        <geom name="geom_sphere_2" type="sphere" size="0.05" contype="1" conaffinity="1" rgba="1 0 0 1"/>
    </body>
    <body name="sphere_3" pos="0.75 0.05 0.81">
        <freejoint name="sphere_3"/>
        <geom name="geom_sphere_3" type="sphere" size="0.05" contype="1" conaffinity="1" rgba="1 0 0 1"/>
    </body>
    <body name="sphere_4" pos="0.825 0.125 0.81">
        <freejoint name="sphere_4"/>
        <geom name="geom_sphere_4" type="sphere" size="0.05" contype="1" conaffinity="1" rgba="1 0 0 1"/>
    </body>

With this, I can load the model, but all the spheres start at the origin, instead of the indicated body pose.
If I remove the keyframe from the apollo model, again everything works as expected.
So yes I think that the keyframe information from the robot is leaking into the scene.

@Kenny-Vilella
Copy link
Collaborator Author

Kenny-Vilella commented Nov 11, 2025

For the first case, it looks like that when we replicate bodies, we use the function mjs_attach (here), which ultimately call this operator. And in this operator we call StoreKeyframes, which throw an error because we did not first expand the keyframe. Does it looks like an edge case that was not accounted for?

For the second case, it looks like that the behavior is "expected". The ExpandKeyframe function is just filling 0.0 for missing qpos in the keyframe. I expected that the body position in the xml would be used in case of missing qpos in the keyframe, but it's probably not that easy ^^

@thowell
Copy link
Collaborator

thowell commented Nov 12, 2025

@Kenny-Vilella i submitted a pr to your branch that should resolve this issue Kenny-Vilella#2

the fix was adding apptronik_apollo.xml as a model asset and then adding it to the scene with attach.

scene_objects_attach.mov

@thowell
Copy link
Collaborator

thowell commented Nov 12, 2025

@Kenny-Vilella @hartikainen this scene helped identity an issue with attach + replicate + keyframe w/ ctrl. thanks!

@Kenny-Vilella
Copy link
Collaborator Author

Kenny-Vilella commented Nov 13, 2025

It's working perfectly now, thanks @thowell !

@thowell thowell merged commit 37a5a51 into google-deepmind:main Nov 13, 2025
6 checks passed
@Kenny-Vilella Kenny-Vilella deleted the dev/kvilella/add_benchmark_rigid_objects branch November 18, 2025 01:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@thowell thowell thowell approved these changes

+1 more reviewer

@hartikainen hartikainen hartikainen left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Kenny-Vilella @erikfrey @yuvaltassa @thowell @hartikainen