auto_assembly

Install

• ROS noetic

• realsense

  • https://qiita.com/porizou1/items/9d44053dbce648f1470d
  • https://github.com/realsenseai/librealsense/blob/master/doc/distribution_linux.md

• nvidia driver

• workspace setting

  cd personal_ws/cheng/master/code/dobot_ws
  catkin init
  catkin build  # for init
  

• apt

  sudo apt install python3-opencv  # for aruco
  

• git pkg

  cd src
  git clone git@github.com:IFL-CAMP/easy_handeye.git
  git clone git@github.com:asanolab/dobot_magician_ros.git
  

  • memo
    • dobot_magician_ros is a customized version of below (original pkg)
      • git@github.com:HKUArmStrong/dobot_magician_ros.git

• python

  python3 -m pip install pyserial
  python3 -m pip install numpy==1.23.5  # 1.23以下が安定 
  

  • memo
    • pipでopencvは入れずaptで入れる. aruco関係が不安定

• yolo

  python3 -m pip install ultralytics
  

  • see https://github.com/ultralytics/ultralytics

• build pkgs

  cd src
  catkin build
  

Demo

• magician

roslaunch magician display.launch
rosrun magician_control joint_state_feedback.py
rosrun tf2_ros static_transform_publisher -0.06 -0.059 0 0 0 0 link_6 tcp  #(optional)

• realsense and aruco

roslaunch realsense2_camera rs_camera.launch align_depth:=true
rosrun dobot_magician_demo aruco_detect.py _dict:=DICT_5X5_1000 _marker_id:=0 _marker_length:=0.018 _image:=/camera/color/image_raw _camera_info:=/camera/color/camera_info _marker_frame:=aruco_marker_0

• coordinates

  • pick first tf data for stable (fixed) transformation. (avoid vibrated result of recognition)

  rosrun tf2_ros static_transform_publisher 0.072 0.072 -0.057 0 0 0  aruco_marker_0 base_link
  

  tf_echo the camera link with base_link by the below, and copy xyz and RPY

  rosrun tf tf_echo /camera_link /base_link
  

  then, kill “rosrun tf2_ros static_transform_publisher 0.072 0.072 -0.057 0 0 0 aruco_marker_0 base_link”
  then, execute below with copied xyzRPY

  rosrun tf2_ros static_transform_publisher x y z R P Y camera_link base_link
  

• yolo

  rosrun yolov8_detector yolov8_multi_tf_node.py _model:="path to ur weights" _image:=/camera/color/image_raw _camera_info:=/camera/color/camera_info _device:=0 _plane_z:=-0.008 _classes:=peltier,motor,gear,temp,sink _kpt_head_idx:=0 _kpt_tail_idx:=1 _debug_image:=/yolo/module/debug_image
  

• start demo

  rosrun dobot_magician_demo grasp_from_tf.py
  

  • memo
    • デモを実行する前にmotor, temp, sink, peltierの４つを同時に認識しtfがつながっている必要がある。していないと以下のエラーがでる

      [ERROR] [1774930559.857751]: TF not found: base_link -> temp. Check frame name and publisher.
      

    • tf_treeは以下で確認できる

       $ rosrun rqt_tf_tree rqt_tf_tree 
      

Acknowledgement

auto_assembly was initially implemented by @icybarbarian (Cheng Xingyan)