FoundationPose: score always 0.0 and multiple object support

Robotics & Edge Computing Robotics - Isaac Isaac ROS
1

Hi, I’m using the original NVIDIA Isaac ROS FoundationPose package on a Jetson Orin 64 GB (ROS2 Humble).

I have two questions:

The pose estimation (translation and rotation) is accurate, but the returned pose score is always exactly 0.0.

Here’s an example of the output I’m getting:

vision_msgs.msg.Detection3DArray(
  header=std_msgs.msg.Header(
    stamp=builtin_interfaces.msg.Time(sec=1752675615, nanosec=287351217),
    frame_id='tf_camera'
  ),
  detections=[
    vision_msgs.msg.Detection3D(
      header=std_msgs.msg.Header(
        stamp=builtin_interfaces.msg.Time(sec=1752675615, nanosec=287351217),
        frame_id='tf_camera'
      ),
      results=[
        vision_msgs.msg.ObjectHypothesisWithPose(
          hypothesis=vision_msgs.msg.ObjectHypothesis(
            class_id='',
            score=0.0
          ),
          pose=geometry_msgs.msg.PoseWithCovariance(
            pose=geometry_msgs.msg.Pose(
              position=geometry_msgs.msg.Point(
                x=-0.12443303316831589,
                y=0.09022021293640137,
                z=0.7088607549667358
              ),
              orientation=geometry_msgs.msg.Quaternion(
                x=0.11492790604863064,
                y=-0.02546156634882348,
                z=-0.417520445400986,
                w=0.9010105230919914
              )
            ),
            covariance=array([
              0., 0., 0., 0., 0., 0.,
              0., 0., 0., 0., 0., 0.,
              0., 0., 0., 0., 0., 0.,
              0., 0., 0., 0., 0., 0.,
              0., 0., 0., 0., 0., 0.,
              0., 0., 0., 0., 0., 0.
            ])
          )
        )
      ],
      bbox=vision_msgs.msg.BoundingBox3D(
        center=geometry_msgs.msg.Pose(
          position=geometry_msgs.msg.Point(
            x=-0.12443303316831589,
            y=0.09022021293640137,
            z=0.7088607549667358
          ),
          orientation=geometry_msgs.msg.Quaternion(
            x=0.11492790604863064,
            y=-0.02546156634882348,
            z=-0.417520445400986,
            w=0.9010105230919914
          )
        ),
        size=geometry_msgs.msg.Vector3(
          x=0.06567460298538208,
          y=0.06567499786615372,
          z=0.03480000048875809
        )
      ),
      id=''
    )
  ]
)

I also noticed the returned message always has an empty class_id (‘’). Could this be the reason why the score is always 0.0?

  1. Could you please explain why the final pose score is always zero and how to resolve this?

  2. My second question is whether FoundationPose can be used to detect multiple objects, not just one — it seems possible by launching a separate ROS node per object, but this consumes a lot of VRAM.

Thanks!