Skip to content

[MISC] Picking boxes and Panda links with a MouseSpring #1443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

gasnica
Copy link
Collaborator

@gasnica gasnica commented Jul 21, 2025

Description

Added:

  • maths for object picking, with spherical inertia only
    • implemented just applying external force & torque, outside of solver step
    • will add non-spherical inertia handling, and motion damping in later pass
  • cleaned up & refactored a bit:
    • MouseSpring & ViewerInterction refrence held_link, instead of geom or entity
    • raycasting functions always return a RayHit, and it holds hit Geometry
  • thread locking was needed when running viewer in threaded mode. On a single debugging attempt, it seemed window-event-callbacks were called on a different thread than simulaton-update callback (which makes sense)
  • simple placeholder OBBs are used to raycast against and pick up geometries/links of Panda robot
  • split out OBB from AABB for maybe slightly cleaner code
  • some linting added
  • also there's the thought of adding pose property to RigidLinks, but that's just an idea

Motivation and Context

Allow interaction with the scene.

How Has This Been / Can This Be Tested?

Manual testing only ST/MT

Screenshots (if appropriate):

721.Interaction.with.Boxes.and.Panda.mp4

Checklist:

  • I read the CONTRIBUTING document.
  • I followed the Submitting Code Changes section of CONTRIBUTING document.
  • I tagged the title correctly (including BUG FIX/FEATURE/MISC/BREAKING)
  • I updated the documentation accordingly or no change is needed.
  • I tested my changes and added instructions on how to test it for reviewers.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@@ -79,3 +72,33 @@ def from_center_and_size(cls, center: Vec3, size: Vec3) -> 'AABB':
max = center + 0.5 * size
bounds = np.stack((min.v, max.v))
return cls(bounds)


class OBB(AABB):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does not make sense. You should probably have some BaseBoundingBox abstraction.

self.pose = pose

def raycast(self, ray: Ray) -> RayHit:
inv_pose = self.pose.get_inverse()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not compute inverse explicit. This is slow. You should rather implement inv_transform_point, inv_transform_direction

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, I think, pose.get_inverse has the same cost as transform_point -- one rotation of vector by quaternion being the main cost.

But yes, with the extra inv_pose, it's 3 such function calls instead of 2. Fixing

def from_center_and_size(cls, size: Vec3, pose: Pose) -> 'OBB':
min = - 0.5 * size
max = + 0.5 * size
bounds = np.stack((min.v, max.v))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always specify on which access you stack for readability.


def apply_force(self, control_point: Vec3, delta_time: float):
_ensure_torch_imported()

# note when threaded: apply_force is called before attach!
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you confirm if it is fixed and clean these notes (which does NOT mean removing them entirely!)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • I don't think I can confirm with threaded nondeterministic execution (?)
  • the early return here was the first work around, and it fixed the crash
  • later I added locks, which might also fix the cause of the earlier crash
  • I have not tested removal of the original work around, but it's nondeterministic, so that wouldn't confirm anything, so I'm just leaving it here, with comments.

Would you like me to add the above notes to the source, or do you want me to write something else specific instead?

link: RigidLink = self.held_geom.link
link_pos: Vec3 = Vec3.from_tensor(link.get_pos())
link: RigidLink = self.held_link
assert link
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes no sense if the type of link is RigidLink. Either the assert should be removed or typing should be replaced by RigidLink | None

delta_3d_pos: Vec3 = new_mouse_3d_pos - self.prev_mouse_3d_pos
self.prev_mouse_3d_pos = new_mouse_3d_pos

use_force: bool = True
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same. But you can keep them if you like them x)

closest_hit = RayHit.no_hit()
for link in entity.links:
# how to check if the link is dynamic ??
if 0.0 < link.get_mass():
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if link.get_mass() > 0.0: is more natural for me.

else:
closest_hit = RayHit.no_hit()
for link in entity.links:
# how to check if the link is dynamic ??
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this comment means? Whether it can move ? If so, you need to change for its parent index. If it is "-1", then it is fixed wrt world (95% sure). Another option is to check the value of is_fixed, which is arguably much clearer.

return Pose.from_geom(entity.geoms[0])
def _get_geom_placeholder_obb(self, geom: 'RigidGeom') -> OBB:
pose = Pose.from_geom(geom)
size = 0.125 * Vec3.one()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add def full(fill_value): ? That would be more convenient and efficient.

quat = Quat.from_tensor(geom.get_quat())
aabb = AABB.from_center_and_size(pos, size)
if obb:
aabb: AABB = AABB.from_center_and_size(obb.pose.pos, obb.extents)
aabb.expand(0.01)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use keyword arguments when passing constants to methods. This makes the code more self-explanatory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants