Contrastive Prediction and Estimation of Deformable Objects based on Improved Resnet


  • Haipeng Gao
  • Yadong Teng



Contrastive Prediction, Resnet, Deformable


Because the dynamic model of deformable linear object is complex, the learning based on visual model is difficult, and the feature information extraction is insufficient. Therefore, we propose a joint visual representation model using contrast learning of optimized encoder. We start with the encoder, add the residual structure to the encoder, optimize the extraction and compression of its feature information, and control its parameters to 3 million. In this way, we can not only obtain excellent feature information, but also have good efficiency. In the rope experiment, we collect information from the simulated environment without manual marking, extract features through the encoder and transmit them to the downstream task. Experiments show that the evaluation of our model at 135 ° and 45 ° is improved by about 50%.


Download data is not yet available.


Takahiro Wada, Shinichi Hirai, Sadao Kawamura, and Norimasa Kamiji. Robust manipulation of deformable objects by simple positive feedback. In ICRA, 2001.

Dominik Henrich and Heinz Wörn. Robot manipulation of deformable objects. In Springer Science & Business Media, 2012.

John Schulman, Jonathan Ho, Cameron Lee, and Pieter Abbeel. Generalization in robotic manipulation through the use of non-rigid registration. In ISRR, 2013.

John Schulman, Alex Lee, Jonathan Ho, and Pieter Abbeel. Tracking deformable objects with point clouds. In ICRA, 2013.

Yilin Wu, Wilson Yan, Thanard Kurutach, Lerrel Pinto, and Pieter Abbeel Learning to manipulate deformable objects without demonstrations. In arXiv preprint, 2019.

Daniel Seita, Aditya Ganapathi, Ryan Hoque, Minho Hwang, Edward Cen, Ajay Kumar Tanwani, Ashwin Balakrishna, Brijen Thananjeyan, Jeffrey Ichnowski, Nawid Jamali, Katsu Yamane, Soshi Iba, John Canny, and Ken Goldberg. Deep imitation learning of sequential fabric smoothing policies. In arXiv preprint,2019.

Jeremy Martin-Shepard, Marco Cusumano-Towner, Jinna Lei, and Pieter Abbeel. Cloth grasp point detection based on multiple-view geometric cues with application to robotic towel folding. In ICRA, 2010.

Jan Stria, Daniel Prusa, V aclav Hlavac, Libor Wagner, Vladimir Petrik, Pavel Krsek, and Vladimir Smutny. Garment perception and its folding using a dual-arm robot. In IROS, 2014.

Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel. Soft actor-critic algorithms, and applications. In arXiv preprint, 2018.

John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In ICML,2015.

Timothy PLillicrap , Jonathan J Hunt, Alexander Pritzel , Nicolas Heess, Tom Erez , Yuval Tassa , David Silver , and Daan Wierstra . Continuous control with deep reinforcement learning. In arXiv preprint, 2015.

Wilson Yan, Ashwin V angipuram, Pieter Abbeel, and Lerrel Pinto, Learning Predictive Representations for Deformable Objects Using Contrastive Estimation 2020.

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geof-frey Hinton. A simple framework for contrastive learning of visual representations. arXiv preprint, 2020.

He K ,Zhang X ,Ren S. Deep Residual Learning for Image Recognition[J].IEEE, 2016.

Fouad F Khalil and Pierre Payeur. Dexterous robotic manipulation of deformable objects with the multi-sensory feedback-a review. In Robot Manipulators Trends and Development. 2010. Infogan. In NeurIPS, 2018.

P Jiménez. Survey on model-based manipulation planning of deformable objects. Robotics and computer-integrated manufacturing, 2012.

Mitul Saha and Pekka Isto. Manipulation planning for deformable linear objects. In T-RO, 2007.

Hidefumi Wakamatsu, Eiji Arai, and Shinichi Hirai. Knot-ting/unknotting manipulation of deformable linear objects. IJRR, 2006.

Mark Moll and Lydia E Kavraki. Path planning for deformable linear objects. T-RO, 2006.

Samuel Rodriguez, Xinyu Tang, Jyh-Ming Lien, and Nancy M Amato. An obstacle-based rapidly-exploring random tree. In ICRA, 2006.

Barbara Frank, Cyrill Stachniss, Nichola Abdo, and Wolfram Burgard. Efficient motion planning for manipulation robots in environments with deformable objects. In IROS, 2011.

Grady Williams, Nolan Wagener, Brian Goldfain, Paul Drews, James M Rehg, Byron Boots, and Evangelos A Theodorou. Information-theoretic Mpc for model-based reinforcement learning. In ICRA, 2017.

Liangpeng Zhang, Ke Tang, Xin Yao.Explicit Planning for Efficient Exploration in Reinforcement Learning-NeurIPS 2019.

Dale McConachie, Mengyao Ruan, and Dmitry Berenson. Interleaving Planning and control or deformable object manipulation. In International Symposium on Robotics Research (ISRR), 2017.

Dmitry Berenson. Manipulation of deformable objects without modeling and simulating deformation. In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems pages 4525–4532. IEEE, 2013.

Dale McConachie and Dmitry Berenson. Estimating model utility for deformable object manipulation using multiarmed bandit methods. IEEE Transactions on Automation Science and Engineering, 15(3):967–979, 2018.

David Navarro-Alarcon, Y un-hui Liu, Jose Guadalupe Romero, and Peng Li. On the visual deformation serving of compliant objects:Uncalibrated control methods and experiments. The International Journal of Robotics Research, 33(11):1462– 1480, 2014.

Nabil Essahbi, Belhassen Chedli Bouzgarrou, and Grigore Gogu. Soft material modeling for robotic manipulation. In Applied Mechanics and Materials, Volume 162, pages 184– 193. Trans Tech Publ, 2012.

Chelsea Finn and Sergey Levine. Deep visual foresight for planning robotmotion. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 2786–2793. IEEE, 2017.

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. In arXiv preprint, 2018.

Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. arXiv preprint, 2019.

Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, Pieter Abbeel Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World In IROS 2017.

Kaiser, L. , Babaeizadeh, M. , Milos, P. , Osinski, B. , Campbell, R.H. , & Czechowski, K.. Model-based reinforcement learning for atari.Yuval Tassa, Yotam Doron, Alistair Muldal, Tom Erez, YazheLi, Diego de Las Casas, David Budden, Abbas Abdolmaleki, Josh Merel, Andrew Lefrancq. Deepmind control suite. In arXiv preprint, 2018.

Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In IROS, 2012.

Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. arXiv preprint, 2018.

Sascha Lange and Martin Riedmiller. Deep auto-encoder neural networks in reinforcement learning. In IJCNN, 2010

Lukasz Kaiser, Mohammad Babaeizadeh, Piotr Milos, Blazej Osinski,Roy H Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn, Piotr Kozakowski, Sergey Levine. Model-based reinforcement learning for atari. In arXiv preprint, 2019.







How to Cite

Gao, H., & Teng, Y. (2024). Contrastive Prediction and Estimation of Deformable Objects based on Improved Resnet. Frontiers in Computing and Intelligent Systems, 8(3), 37-43.