Abstract

To reliably manipulate previously unknown objects in semi-structured environments, robots require rapid deployments and seamless transitions in pose estimation and grasping. This work proposes a novel two-stage robotic grasping method that instantly achieves accurate grasping without prior training. At the first stage, depth information and structured markers are utilized to construct compact templates for packaged targets, reducing noise and automating annotations. Then, we conduct coarse matching and design a new variant of the iterative closest point algorithm, named adaptive template-based RANSAC and iterative closest point (ATSAC-ICP), for precise point cloud registration. The method extracts locally well-registered pairs, regresses and optimizes six-degree-of-freedom (6-DOF) pose to satisfy confidence probability and precision threshold. The second stage normalizes the target pose for consistent grasp planning, which is based on scene and placement patterns. The proposed method is evaluated by several sets of experiments using various randomly selected textured objects. The results show that the pose errors are approximately ±2 mm, ±3 deg, and the successful grasping rate is over 90%. Physical experiments, conducted in different lighting conditions and with external disturbances, demonstrate effectiveness and applicability in grasping daily objects.

References

1.
Liu
,
H.
,
Zhou
,
L.
,
Zhao
,
J.
,
Wang
,
F.
,
Yang
,
J.
,
Liang
,
K.
, and
Li
,
Z.
,
2022
, “
Deep-Learning-Based Accurate Identification of Warehouse Goods for Robot Picking Operations
,”
Sustainability
,
14
(
13
), p.
7781
.
2.
Chen
,
S.
,
Hong
,
J.
,
Liu
,
X.
,
Li
,
J.
,
Zhang
,
T.
,
Wang
,
D.
, and
Guan
,
Y.
,
2019
, “
A Framework for 3D Object Detection and Pose Estimation in Unstructured Environment Using Single Shot Detector and Refined Linemod Template Matching
,”
2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA)
,
Zaragoza, Spain
,
Sept. 10–13
, IEEE, pp.
499
504
.
3.
Xiang
,
Y.
,
Schmidt
,
T.
,
Narayanan
,
V.
, and
Fox
,
D.
,
2018
, “
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
,”
14th Conference on Robotics – Science and Systems
,
Pittsburgh, PA
.
4.
Prankl
,
J.
,
Aldoma
,
A.
,
Svejda
,
A.
, and
Vincze
,
M.
,
2015
, “
RGB-D Object Modelling for Object Recognition and Tracking
,”
2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
,
Hamburg, Germany
,
Sept. 28–Oct. 3
, pp.
96
103
.
5.
Wu
,
Y.
,
Zhang
,
F.
, and
Fu
,
Y.
,
2021
, “
Real-Time Robotic Multigrasp Detection Using Anchor-Free Fully Convolutional Grasp Detector
,”
IEEE Trans. Ind. Electron.
,
69
(
12
), pp.
13171
13181
.
6.
Gabellieri
,
C.
,
Angelini
,
F.
,
Arapi
,
V.
,
Palleschi
,
A.
,
Catalano
,
M. G.
,
Grioli
,
G.
,
Pallottino
,
L.
,
Bicchi
,
A.
,
Bianchi
,
M.
, and
Garabini
,
M.
,
2020
, “
Grasp It Like a Pro: Grasp of Unknown Objects With Robotic Hands Based on Skilled Human Expertise
,”
IEEE Rob. Autom. Lett.
,
5
(
2
), pp.
2808
2815
.
7.
Mahler
,
J.
,
Liang
,
J.
,
Niyaz
,
S.
,
Laskey
,
M.
,
Doan
,
R.
,
Liu
,
X.
,
Ojea
,
J. A.
, and
Goldberg
,
K.
,
2017
, “
Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics
,”
13th Conference on Robotics – Science and Systems
,
Cambridge, MA
.
8.
Du
,
G.
,
Wang
,
K.
,
Lian
,
S.
, and
Zhao
,
K.
,
2021
, “
Vision-Based Robotic Grasping From Object Localization, Object Pose Estimation to Grasp Estimation for Parallel Grippers: A Review
,”
Artif. Intell. Rev.
,
54
(
3
), pp.
1677
1734
.
9.
Bormann
,
R.
,
de Brito
,
B. F.
,
Lindermayr
,
J.
,
Omainska
,
M.
, and
Patel
,
M.
,
2019
, “
Towards Automated Order Picking Robots for Warehouses and Retail
,” 12th International Conference on Computer Vision Systems, ICVS 2019, Thessaloniki, Sept. 23–25,
Springer
, pp.
185
198
.
10.
Shi
,
H.
,
Chen
,
J.
,
Pan
,
W.
,
Hwang
,
K.-S.
, and
Cho
,
Y.-Y.
,
2018
, “
Collision Avoidance for Redundant Robots in Position-Based Visual Servoing
,”
IEEE Syst. J.
,
13
(
3
), pp.
3479
3489
.
11.
Morrison
,
D.
,
Corke
,
P.
, and
Leitner
,
J.
,
2018
, “
Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach
,”
14th Conference on Robotics - Science and Systems
,
Pittsburgh, PA
.
12.
Dong
,
H.
,
Prasad
,
D. K.
, and
Chen
,
I.-M.
,
2020
, “
Object Pose Estimation Via Pruned Hough Forest With Combined Split Schemes for Robotic Grasp
,”
IEEE Trans. Autom. Sci. Eng.
,
18
(
4
), pp.
1814
1821
.
13.
Fang
,
H.-S.
,
Wang
,
C.
,
Gou
,
M.
, and
Lu
,
C.
,
2020
, “
Graspnet-1 Billion: A Large-Scale Benchmark for General Object Grasping
,”
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
,
Seattle, WA
,
pp. 11441-11450
.
14.
Francis
,
A. G.
,
Kinman
,
B.
,
Reymann
,
K. A.
,
Downs
,
L.
,
Koenig
,
N.
,
Hickman
,
R. M.
,
McHugh
,
T. B.
, and
Vanhoucke
,
V. O.
,
2022
, “
Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items
,”
2022 International Conference on Robotics and Automation (ICRA)
,
Philadelphia, PA
,
May 23–27
, pp.
2553
2560
.
15.
Calli
,
B.
,
Singh
,
A.
,
Walsman
,
A.
,
Srinivasa
,
S.
,
Abbeel
,
P.
, and
Dollar
,
A. M.
,
2015
, “
The YCB Object and Model Set: Towards Common Benchmarks for Manipulation Research
,”
2015 International Conference on Advanced Robotics (ICAR)
,
Istanbul, Turkey
,
July 27–31
, pp.
510
517
.
16.
Deng
,
X.
,
Geng
,
J.
,
Bretl
,
T.
,
Xiang
,
Y.
, and
Fox
,
D.
,
2022
, “
iCaps: Iterative Category-Level Object Pose and Shape Estimation
,”
IEEE Rob. Autom. Lett.
,
7
(
2
), pp.
1784
1791
.
17.
Lindermayr
,
J.
,
Odabasi
,
C.
,
Jordan
,
F.
,
Graf
,
F.
,
Knak
,
L.
,
Kraus
,
W.
,
Bormann
,
R.
, and
Huber
,
M. F.
,
2023
, “
IPA-3D1K: A Large Retail 3D Model Dataset for Robot Picking
,”
2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
,
Detroit, M
,
Oct. 1–5
, pp.
11404
11411
.
18.
Lindermayr
,
J.
,
Odabasi
,
C.
,
Völk
,
M.
,
Chen
,
Y.
,
Bormann
,
R.
, and
Huber
,
M. F.
,
2023
, “
Synthretailproduct3D (syrepro3D): A Pipeline for Synthesis of 3D Retail Product Models With Domain Specific Details Based on Package Class Templates
,”
14th International Conference on Computer Vision Systems (ICVS)
,
Vienna, Austria
,
Sept. 27–29
, Springer, pp.
230
242
.
19.
Muja
,
M.
, and
Lowe
,
D. G.
,
2009
, “
Fast Approximate Nearest Neighbors With Automatic Algorithm Configuration
,”
4th International Conference on Computer Vision Theory and Applications
,
Lisbon, Portugal
,
Feb. 5–8
.
20.
Segal
,
A.
,
Haehnel
,
D.
, and
Thrun
,
S.
,
2009
, “
Generalized-ICP
,”
Proceedings of Robotics: Science and Systems
,
Seattle, WA
,
June 28–July 1
, p.
435
.
21.
Maken
,
F. A.
,
Ramos
,
F.
, and
Ott
,
L.
,
2019
, “
Speeding Up Iterative Closest Point Using Stochastic Gradient Descent
,”
2019 International Conference on Robotics and Automation (ICRA)
,
Montreal, QC, Canada
,
2019
,
pp. 6395-6401, doi: 10.1109/ICRA.2019.8794011
.
22.
Barath
,
D.
, and
Matas
,
J.
,
2018
, “
Graph-Cut RANSAC
,”
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT
, pp.
6733
6741
.
23.
Chung
,
K.-L.
,
Tseng
,
Y.-C.
, and
Chen
,
H.-Y.
,
2022
, “
A Novel and Effective Cooperative RANSAC Image Matching Method Using Geometry Histogram-Based Constructed Reduced Correspondence Set
,”
Remote Sens.
,
14
(
14
), p.
3256
.
24.
Nakada
,
T.
,
Kagami
,
S.
, and
Mizoguchi
,
H.
,
2010
, “
SIFT-Cloud-Model for Object Detection and Pose Estimation With GPGPU Acceleration
,”
2010 IEEE/RSJ International Conference on Intelligent Robots and Systems
,
Taipei, Taiwan
, pp.
1748
1753
.
25.
Kiatos
,
M.
,
Malassiotis
,
S.
, and
Sarantopoulos
,
I.
,
2020
, “
A Geometric Approach for Grasping Unknown Objects With Multifingered Hands
,”
IEEE Trans. Rob.
,
37
(
3
), pp.
735
746
.
26.
Chiang
,
H.-H.
,
You
,
J.-K.
,
Hsu
,
C.-C. J.
, and
Jo
,
J.
,
2022
, “
Optimal Grasping Strategy for Robots With a Parallel Gripper Based on Feature Sensing of 3D Object Model
,”
IEEE Access
,
10
, pp.
24056
24066
.
27.
Friedrich
,
C.
,
Csiszar
,
A.
,
Lechler
,
A.
, and
Verl
,
A.
,
2017
, “
Efficient Task and Path Planning for Maintenance Automation Using a Robot System
,”
IEEE Trans. Autom. Sci. Eng.
,
15
(
3
), pp.
1205
1215
.
28.
Ten Pas
,
A.
, and
Platt
,
R.
,
2018
, “
Using Geometry to Detect Grasp Poses in 3D Point Clouds
,”
Robotics Research
,
1
, pp.
307
324
.
doi: 10.1007/978-3-319-51532-8_19
.
29.
Jamone
,
L.
,
Bernardino
,
A.
, and
Santos-Victor
,
J.
,
2016
, “
Benchmarking the Grasping Capabilities of the ICub Hand With the YCB Object and Model Set
,”
IEEE Rob. Autom. Lett.
,
1
(
1
), pp.
288
294
.
30.
Chowdhury
,
A. B.
,
Li
,
J.
, and
Cappelleri
,
D. J.
,
2023
, “
Neural Network-Based Pose Estimation Approaches for Mobile Manipulation
,”
ASME J. Mech. Rob.
,
15
(
1
), p.
011009
.
31.
Jiang
,
Y.
,
Moseson
,
S.
, and
Saxena
,
A.
,
2011
, “
Efficient Grasping From RGBD Images: Learning Using a New Rectangle Representation
,”
2011 IEEE International Conference on Robotics and Automation
,
Shanghai, China
, pp.
3304
3311
.
32.
Besl
,
P. J.
, and
McKay
,
N. D.
,
1992
, “
Method for Registration of 3-D Shapes
,”
IEEE Transactions on Pattern Analysis and Machine Intelligence
,
14
(
2
), pp.
586
606
.
33.
Pomerleau
,
F.
,
Magnenat
,
S.
,
Colas
,
F.
,
Liu
,
M.
, and
Siegwart
,
R.
,
2011
, “
Tracking a Depth Camera: Parameter Exploration for Fast ICP
,”
2011 IEEE/RSJ International Conference on Intelligent Robots and Systems
,
San Francisco, CA
, pp.
3824
3829
.
34.
Gualtieri
,
M.
,
Ten Pas
,
A.
,
Saenko
,
K.
, and
Platt
,
R.
,
2016
, “
High Precision Grasp Pose Detection in Dense Clutter
,”
2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
,
Daejeon, South Korea, IEEE
,
pp. 598-605
.
35.
Kumra
,
S.
,
Joshi
,
S.
, and
Sahin
,
F.
,
2020
, “
Antipodal Robotic Grasping Using Generative Residual Convolutional Neural Network
,”
2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
,
Las Vegas, NV
, pp.
9626
9633
.
36.
Kasaei
,
H.
, and
Kasaei
,
M.
,
2023
, “
Mvgrasp: Real-Time Multi-view 3D Object Grasping in Highly Cluttered Environments
,”
Rob. Auton. Syst.
,
160
, p.
104313
.
You do not currently have access to this content.