A Unified Multi-Task Deep Learning Framework for Robotic Bin-Picking of Planar Objects

Online First: 19/08/2026

Các tác giả

Email tác giả liên hệ:

khanh.nt@vgu.edu.vn

DOI:

https://doi.org/10.54644/jte.2026.2401

Từ khóa:

Robotic bin-picking, Pose estimation, Planar object, Digital twin, Deep learning

Tóm tắt

Automating random bin-picking in industrial robotics, where robots handle diverse and cluttered objects, remains challenging due to the complexity of object detection and pose estimation. While many solutions focus on free-form objects, systems specifically designed for planar objects are lacking. Planar objects pose unique challenges, as the commonly used point pair feature approach for free-form objects is ineffective due to their lack of distinctive geometric features. In this study, the proposed framework was implemented and evaluated using USB packs as a representative planar object case study. An innovative approach is introduced for the random bin-picking of planar objects by developing a multi-task model for instance segmentation and keypoint detection in 2D images. Geometric approach is then employed to estimate the 6D object pose for robotic grasping. Furthermore, a grasp candidate selection strategy is proposed to enable reliable grasping in cluttered industrial environments. Experimental results show that the proposed method achieved mAP50 values of 0.954, 0.800, and 0.926 for bounding box detection, instance segmentation, and keypoint detection, respectively, with a processing time of 2.7 ms. Future work will focus on integrating the framework into a digital twin system to support real-time monitoring, simulation, and optimization of automated manufacturing processes.

Tải xuống: 0

Dữ liệu tải xuống chưa có sẵn.

Tiểu sử của Tác giả

The-Thinh Pham, Can Tho University of Technology, Vietnam

The-Thinh Pham received his Bachelor degree in Mechatronic Engineering from Can Tho University of Technology, Vietnam in 2019. He received his Master degree in Mechanical Engineering from National Taiwan University of Science and Technology, Taiwan in 2023. He is currently pursuing a Ph.D. degree with Mechanical Engineering, National Taiwan University of Science and Technology. His current research interests include computer vision, deep learning and robotics.

Email: ptthinh@ctuet.edu.vn. ORCID:  https://orcid.org/0009-0009-4426-0457

Tuan-Khanh Nguyen, Vietnamese-German University, Vietnam

Tuan-Khanh Nguyen received the B.S. and M.S. degrees in electronic telecommunication from Can Tho University, Vietnam in 2010 and Ho Chi Minh University of Technology, Vietnam in 2014, respectively. He got the PhD degree in Electronic and Computer Engineering at National Taiwan University of Science and Technology in 2023. He is currently working at the Vietnamese-German University as a Junior Lecturer in Semiconductor and Microsystems Engineering. His research interests include semiconductors, radio-frequency biomedical sensors, noncontact vital-sign radar sensors, and microwave circuits and modules.

Email: khanh.nt@vgu.edu.vn. ORCID:  https://orcid.org/0000-0002-5162-4417

Chi-Cuong Tran, National Taiwan University of Science and Technology, Taiwan

Chi-Cuong Tran received the B.S. and M.S. degrees in Automation and Control Engineering from Can Tho University, Vietnam, in 2015 and 2017, respectively. He received his Ph.D. degree in Mechanical Engineering from National Taiwan University of Science and Technology (NTUST), in 2023. He is currently working as a postdoctoral researcher at National Taiwan University of Science and Technology, Taipei, Taiwan. His research interests include intelligent robotics, intelligent automation, computer vision, and machine learning.

Email: tccuong09@gmail.com. ORCID:  https://orcid.org/0009-0004-7698-0970

Quang-Huan Dong, Vietnamese-German University, Vietnam

Quang-Huan Dong is a postdoctoral researcher in Business Information Systems at the Vietnamese-German University, Vietnam. He obtained a Bachelor's degree in Information Technology from the University of Science, Vietnam National University Ho Chi Minh City, in 2008. He completed a dual Master's degree in Business Information Systems jointly offered by the Vietnamese-German University, Vietnam, and Heilbronn University, Germany, in 2015. He received his PhD from the TUM School of Engineering and Design, Technical University of Munich, Germany, in 2023, where he conducted his research at the Institute of Automation and Information Systems. Prior to his academic career, he worked as a software engineer, technical lead, and project manager in the software industry for more than eight years. His research focuses on factory automation, software engineering, and digital twins.

Email: huan.dq@vgu.edu.vn. ORCID:  https://orcid.org/0000-0002-3085-3464

Duy-Tan Do, Ho Chi Minh University of Technology and Engineering, Vietnam

Duy-Tan Do received the B.S. degree in Electronics and Telecommunications from the Ho Chi Minh City University of Technology (BK-HCM), Vietnam, in 2010, the M.S. degree in Wireless Communications from the Kumoh National Institute of Technology, South Korea, in 2013, and the Ph.D. degree in Electronic and Telecommunication Engineering from the Autonomous University of Barcelona, Spain, in 2019. He is currently with the Department of Electronics and Information Engineering, HCMC University of Technology and Engineering (HCM-UTE), Vietnam, as an associate professor. His main research interests include resource allocation optimization for wireless networks and advanced coding for reliable systems.

Email: tandd@hcmute.edu.vn. ORCID:  https://orcid.org/0000-0003-4570-0441

Hoang-Vinh-Khang Nguyen, Vietnamese-German University, Vietnam

Hoang-Vinh-Khang Nguyen received his B.S. degree in Electrical Engineering and Information Technology (EEIT, now Electrical and Computer Engineering) from the Vietnamese-German University (VGU) in 2015 and his M.S. degree in Mechatronics and Sensor Systems Technology (MST) from VGU in 2018. As a recipient of the DAAD scholarship, he spent one year at Karlsruhe University of Applied Sciences, Germany, where he conducted research for his master's thesis. His research interests include biomedical signal processing for rehabilitation applications and exoskeleton robot control.

Email: khang.nhv@vgu.edu.vn. ORCID:  https://orcid.org/0000-0003-0256-2237

Quang-Chien Nguyen, Ho Chi Minh University of Technology and Engineering, Vietnam

Quang-Chien Nguyen received the B.S. degree in Automation and Control Engineering from Ho Chi Minh City University of Technology and Engineering (HCM-UTE), Vietnam in 2023. He is currently pursuing a Master's degree in Automation and Control Engineering at HCM-UTE. His research interests include robotics, mobile robot, intelligent control and motion control.

Email: 2431101@student.hcmute.edu.vn. ORCID:  https://orcid.org/0009-0008-1332-7746

Tài liệu tham khảo

W. Abbeloos and T. Goedemé, "Point pair feature based object detection for random bin picking," in Proc. 13th Conf. Comput. Robot Vis. (CRV), 2016.

W. C. Chang and C. H. Wu, "Eye-in-hand vision-based robotic bin-picking with active laser projection," Int. J. Adv. Manuf. Technol., vol. 85, nos. 9–12, pp. 2873–2885, 2016.

M. Y. Liu et al., "Fast object localization and pose estimation in heavy clutter for robotic bin picking," Int. J. Robot. Res., vol. 31, no. 8, pp. 951–973, 2012.

C. Martinez, H. Chen, and R. Boca, "Automated 3D vision guided bin picking process for randomly located industrial parts," in Proc. IEEE Int. Conf. Ind. Technol. (ICIT), 2015.

C. M. Lin et al., "Visual object recognition and pose estimation based on a deep semantic segmentation network," IEEE Sensors J., vol. 18, no. 22, pp. 9370–9381, 2018.

J. Vidal et al., "A method for 6D pose estimation of free-form rigid objects using point pair features on range data," Sensors, vol. 18, no. 8, Art. no. 2678, 2018.

C. M. Bishop, Pattern Recognition and Machine Learning. New York, NY, USA: Springer, 2006.

S. B. Kotsiantis, I. Zaharakis, and P. Pintelas, "Supervised machine learning: A review of classification techniques," Emerg. Artif. Intell. Appl. Comput. Eng., vol. 160, no. 1, pp. 3–24, 2007.

Z. Q. Zhao et al., "Object detection with deep learning: A review," IEEE Trans. Neural Netw. Learn. Syst., vol. 30, no. 11, pp. 3212–3232, 2019.

M. Blum et al., "A learned feature descriptor for object recognition in RGB-D data," in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2012.

E. Brachmann et al., "Learning 6D object pose estimation using 3D object coordinates," in Proc. Eur. Conf. Comput. Vis. (ECCV), 2014.

E. Brachmann et al., "Uncertainty-driven 6D pose estimation of objects and scenes from a single RGB image," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2016.

T. T. Do et al., "Deep-6DPose: Recovering 6D object pose from a single RGB image," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2018.

C. H. Wu, S. Y. Jiang, and K. T. Song, "CAD-based pose estimation for random bin-picking of multiple objects using an RGB-D camera," in Proc. 15th Int. Conf. Control, Autom. Syst. (ICCAS), 2015.

Y. K. Chen et al., "Random bin picking with multi-view image acquisition and CAD-based pose estimation," in Proc. IEEE Int. Conf. Syst., Man, Cybern. (SMC), 2018.

A. Wong et al., "Fast GraspNeXt: A fast self-attention neural network architecture for multi-task learning in computer vision tasks for robotic grasping on the edge," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), 2023.

M. Gilles et al., "MetaGraspNetV2: All-in-one dataset enabling fast and reliable robotic bin picking via object relationship reasoning and dexterous grasping," IEEE Trans. Autom. Sci. Eng., vol. 21, no. 3, pp. 2302–2320, Jul. 2024.

D. Liu et al., "Point pair feature-based pose estimation with multiple edge appearance models (PPF-MEAM) for robotic bin picking," Sensors, vol. 18, no. 8, Art. no. 2719, 2018.

T. T. Le and C. Y. Lin, "Bin-picking for planar objects based on a deep learning network: A case study of USB packs," Sensors, vol. 19, no. 16, Art. no. 3602, 2019.

B. Munkhtulga, K. Gou, and C. Lodoiravsal, "6D pose estimation of transparent objects from a single RGB image for robotic manipulation," IEEE Access, vol. 10, pp. 114897–114906, 2022.

X. Liu et al., "A Pose Estimation Approach Based on Keypoints Detection for Robotic Bin-picking Application," 2021 China Automation Congress (CAC), Beijing, China, 2021, pp. 3672-3677, doi: 10.1109/CAC53003.2021.9727987.

G. Jocher, A. Chaurasia, and J. Qiu, Ultralytics YOLOv8, 2023.

J. Ha, "Probabilistic framework for hand-eye and robot-world calibration AX = YB," IEEE Trans. Robot., vol. 39, no. 2, pp. 1030–1047, 2023.

Q. H. Dong, T. K. Nguyen, C. C. Tran, T. T. Pham, D. T. Do, H. V. K. Nguyen, and Q. C. Nguyen, "Effectiveness of digital twin framework for collaborative robotic manipulation," J. Tech. Educ. Sci., vol. 20, no. 4, 2025.

Q. H. Dong, T. T. Pham, K. Nguyen, C. C. Tran, H. H. Tran, and D. T. Do, "CAD-guided 6D pose estimation with deep learning in digital twin for industrial collaborative robot manipulation," EAI Endorsed Trans. AI Robot., vol. 4, 2025, doi: https://doi.org/10.4108/airo.9676.

Tải xuống

Đã Xuất bản

2026-08-19

Cách trích dẫn

[1]
The-Thinh Pham, “A Unified Multi-Task Deep Learning Framework for Robotic Bin-Picking of Planar Objects: Online First: 19/08/2026”, JTE, tháng 8 2026.

Số

2026: Bài đang xuất bản

Chuyên mục

Bài báo khoa học

Categories

Giấy phép

Bản quyền (c) 2026 Tạp chí Khoa học Giáo dục Kỹ Thuật

Giấy phép Creative Commons

Tác phẩm này được cấp phép theo Giấy phép quốc tế Creative Commons Attribution-NonCommercial-NoDeri Phái sinh 4.0 .

Bản quyền thuộc về JTE.

Crossref
Scopus
Google Scholar
Europe PMC