Hello!
I have an application where an object traveling vertically needs to be identified and picked. Then a replacement object put Exactly Back in its place. The objects vertical position will vary by +/- 7"
looking for input on how to achieve this
pardon my ignorance.
new to the bots and vision.
experienced with cnc electronics and arduino.
If you're using vision that means a video camera. No lidar involved.
Classical machine learning way would probably be to use YOLO or some other machine policy to identify the where the object is in the frame (in terms of pixel position). Once you know where the object is in the frame, you would use matrix transformations to go from the camera frame to Cartesian space (where the object is in space in millimeters). Once you know that you command the arm to move the calculated position. You can take into account object velocity so you grab where you expect the object to be based on last known location and velocity.
New way might be to use behavioral cloning from human demonstrations. Here you would teleoperate the arm to produce a number of demonstration of how the movement should be done. Then you train on these examples with an algorithm like ACT (Action Chunking with Transformers). The result is a policy that you can deploy.
New new way would be to use a vision-language policy (like RT-X or Octo). Here you describe what you want the robot to do in English and the robot does it.
