I am a research scientist at Sony R&D. I received Ph.D. degree from Osaka University at Harada Lab., advised by Professor Kensuke Harada. I obtained my Master's degree from Osaka University and Bachelor's degree from Tianjin University, China.
Recent News
[2023/10] I started working at Sony R&D.
[2023/07] I finished my Ph.D. thesis defense! Thesis / Recording.
Research
A Closed-Loop Bin Picking System for Entangled Wire Harnesses using Bimanual and Dynamic Manipulation
Xinyi Zhang, Yukiyasu Domae, Weiwei Wan, Kensuke Harada
Robotics and Computer-Integrated Manufacturing, 2023
This paper addresses the challenge of industrial bin picking using entangled wire harnesses. Wire harnesses are essential in manufacturing but pose challenges in automation due to their complex geometries and propensity for entanglement. Our previous work tackled this issue by proposing a quasi-static pulling motion to separate the entangled wire harnesses. However, it still lacks sufficiency and generalization to various shapes and structures. In this paper, we deploy a dual-arm robot that can grasp, extract and disentangle wire harnesses from dense clutter using dynamic manipulation. The robot can swing to dynamically discard the entangled objects and regrasp to adjust the undesirable grasp pose. To improve the robustness and accuracy of the system, we leverage a closed-loop framework that uses haptic feedback to detect entanglement in real-time and flexibly adjust system parameters. Our bin picking system achieves an overall success rate of 91.2% in the real-world experiments using two different types of long wire harnesses. It demonstrates the effectiveness of our system in handling various wire harnesses for industrial bin picking.
Learning to Dexterously Pick or Separate Tangled-Prone Objects for Industrial Bin Picking
Xinyi Zhang, Yukiyasu Domae, Weiwei Wan, Kensuke Harada
IEEE Robotics and Automation Letters (RA-L) 2023 / Present at ICRA 2024
Robotic bin picking tasks for tangled-prone parts requires the robot to either lift the untangled objects or perform singulation manipulation when untangled objects do not exist. It is a challenging task due to the high-occluded scenes, elusive entanglement phenomena and skilled manipulation planning. The robot is also required to flexibly select the suitable actions for the current observation. In this paper, we propose an autonomous, effective and generative approach for picking up tangled-prone objects in robotic bin picking. First, we learn a network to decide if the bin contains untangled objects and predict the grasp point for picking. If there are no such objects, we then learn a network to plan motions to separate them. Finally, we propose to drop the tangled objects from dense clutter to a transition area to reduce the degree of the entanglement. Moreover, we embrace the self-supervised learning paradigm using a physics simulator. Results show a significant improvement in various bin picking tasks over baseline methods.
Learning Efficient Policies for Picking Entangled Wire Harnesses: An Approach to Industrial Bin Picking
Xinyi Zhang, Yukiyasu Domae, Weiwei Wan, Kensuke Harada
IEEE Robotics and Automation Letters (RA-L) 2022 / Presented at ICRA 2023
Wire harnesses are essential connecting components in manufacturing industry but are challenging to be automated in industrial tasks such as bin picking. They are long, flexible and tend to get entangled when randomly placed in a bin. This makes the robot struggle to pick a single one from the clutter. Besides, modeling wire harnesses is difficult due to the complex structures of combining deformable cables with rigid components, making it unsuitable for training or collecting data in simulation. In this work, instead of directly lifting wire harnesses, we proposed to grasp and extract the target following circle-like trajectories until it is separated from the clutter. We learn a policy from real-world data to infer the optimal action and grasp from visual observation. Our policy enables the robot to perform non-tangle pickings efficiently by maximizing success rates and reducing the execution time. To evaluate our policy, we present a set of real-world experiments on picking wire harnesses. Results show a significant improvement in success rates from 49.2% to 84.6% over the tangle-agnostic bin picking method. We also evaluate the effectiveness of our policy under different clutter scenarios using unseen types of wire harnesses. The proposed method is expected to provide a practical solution for automating manufacturing processes with wire harnesses.
A Topological Solution of Entanglement for Complex-shaped Parts in Robotic Bin-picking
Xinyi Zhang, Keisuke Koyama, Yukiyasu Domae, Weiwei Wan, Kensuke Harada
IEEE International Conference on Automation Science and Engineering (CASE) 2021
This paper addresses the problm of picking up only one object at a time avoiding any entanglement in bin-picking. To cope with a difficult case where the complex-shaped objects are heavily entangled together, we propose a topology-based method that can generate non-tangle grasp positions on a single depth image. The core technique is the entanglement map, which is a feature map to measure the entanglement possibilities obtained from the input image. We use an entanglement map to select probable regions containing graspable objects. The optimum grasping pose is detected from the selected regions considering the collision between robot hand and objects. Experimental results show that our analytic method provides a more comprehensive and intuitive observation of the entanglement and exceeds previous learning-based work in success rates. Especially, our topology-based method does not rely on any object models or time-consuming training process, so that it can be easily adapted to more complex bin-picking scenes.