Document Type : Research Note


1 Assistant professor, Aerospace Research Institute, Ministry of Science Research and Technology, Tehran, Iran.

2 M.Sc. Student.Department of Aerospace Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran.

3 Ph.D. Student. Department of Aerospace Engineering, Science and Research Branch, Islamic Azad University, Tehran. Iran.

4 M.Sc. StudentGhaemshahr Branch, Islamic Azad University, Ghaemshahr, Iran.


This paper proposes a method based on the reinforcement Q-learning to solve the problem of fully autonomous optimal path planning of a space robot by increasing the number of available satellites in earth orbits, designation, and implementation of satellite orbital servicing stations considered by researchers. Nowadays, by considering the advancement in robotics science, space robots could be chosen as a part of solution for maintaining the damaged satellites in earth’s orbits. Guidance, control, and navigation of space robots throughout docking and joint maneuvers need a high degree of precision. In this paper, reinforcement Q-learning algorithm functionality in path planning is analyzed through various computational simulations. The finding results from computational simulations demonstrate the usefulness of the mentioned approach. 


Main Subjects

[1]  Satellite Database | Union of Concerned Scientists Available: https://www. ucsusa. org/resources/satellite-database
[2]  G. A. Landis, S. G. Bailey, and R. Tischler, "Causes of power-related satellite failures," in 2006 IEEE 4th World Conference on Photovoltaic Energy Conference, 2006, pp. 1943-1945.
[3]  A. Ellery, J. Kreisel, and B. Sommer, "The case for robotic on-orbit servicing of spacecraft: Spacecraft reliability is a myth," Acta Astronautica, vol. 63, pp. 632-648, 2008.
[4]  M. Tafazoli, "A study of on-orbit spacecraft failures," Acta Astronautica, vol. 64, pp. 195-205, 2009.
[5]  F. Sellmaier, T. Boge, J. Spurmann, S. Gully, T. Rupp, and F. Huber, "On-orbit servicing missions: Challenges and solutions for spacecraft operations," in SpaceOps 2010 Conference Delivering on the Dream Hosted by NASA Marshall Space Flight Center and Organized by AIAA, 2010, p. 2159.
[6]  E. Stoll, U. Walter, J. Artigas, C. Preusche, P. Kremer, G. Hirzinger, et al. , "Ground verification of the feasibility of telepresent on‐orbit servicing," Journal of Field Robotics, vol. 26, pp. 287-307, 2009.
[7]  Y. Wang, Z. Ma, Y. Yang, Z. Wang, and L. Tang, "A new spacecraft attitude stabilization mechanism using deep reinforcement learning method," in 8th European Conference for Aeronautics and Space Sciences (EUCASS), 2019.
[8]  S. Willis, D. Izzo, and D. Hennes, "Reinforcement learning for spacecraft maneuvering near small bodies," in AAS/AIAA Space Flight Mechanics Meeting, 2016, pp. 14-18.
[9]  M. Hatem and F. Abdessemed, "Simulation of the Navigation of a Mobile Robot by the QLearning using Artificial Neuron Networks," in CIIA, 2009.
[10] W. Adiprawita, A. S. Ahmad, J. Sembiring, and B. R. Trilaksono, "Simplified Q-learning for holonomic mobile robot navigation," in 2011 2nd International Conference on Instrumentation, Communications, Information Technology, and Biomedical Engineering, 2011, pp. 64-68.
[11] H. Wicaksono, K. Anam, P. Prihastono, I. A. Sulistijono, and S. Kuswadi, "COMPACT FUZZY Q LEARNING FOR AUTONOMOUS MOBILE ROBOT NAVIGATION," 2014.
[12] L. Khriji, F. Touati, K. Benhmed, and A. Al-Yahmedi, "Mobile robot navigation based on Q-learning technique," International Journal of Advanced Robotic Systems, vol. 8, p. 4, 2011.
[13] Y. Duan, "Fuzzy reinforcement learning and its application in robot navigation," in 2005 International Conference on Machine Learning and Cybernetics, 2005, pp. 899-904.
[14] J. R. Wertz and R. Bell, "Autonomous rendezvous and docking technologies: status and prospects," Space Systems Technology and Operations, vol. 5088, pp. 20-30, 2003.
[15] C. J. Dennehy and J. R. Carpenter, "A summary of the rendezvous, proximity operations, docking, and undocking (rpodu) lessons learned from the defense advanced research project agency (darpa) orbital express (oe) demonstration system mission," 2011.
[16] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction: MIT press, 2018.
[17]  G. Pollock, J. Gangestad, and J. Longuski, "Analysis of Lorentz spacecraft motion about Earth using the Hill-Clohessy-Wiltshire equations," in AIAA/AAS astrodynamics specialist conference and exhibit, 2008, p. 6762.