Progressive application of multidisciplinary research and development pushes the evolution of automation in many subsectors of agriculture to increase productivity, economic growth and environmental preservation with the help of robotics and artificial intelligence. Fruit harvesting robots have been developed mainly to provide support in the field for limited labour resources, to enable selective harvesting, to improve the efficiency and to preserve the quality of fruits. Even a small delay in harvesting can cause a maximum impact to the quality of the fruit. Selective fruit harvesting is an integration of different subcomponents. This paper provides a brief analysis of the techniques in selective fruit harvesting for the past 6 years starting from 2017 to 2022, associated principles, limitations and directions for future challenges. The first subcomponent is the vision system, it captures the information about the fruit in a tree canopy to perform efficient 2D and 3D localization. Hence getting accurate information from the vision system is more essential even in a complex agricultural environment. The detection of fruit from the vision system is normally done with two major methods such as traditional image processing and deep learning approaches. The result shows that the traditional methods provide high efficiency but the colour similarity, complex backgrounds and lightning conditions often makes failure in detection. Shortage of standard dataset and high-powered processing devices hinders the development of deep learning algorithms. Also, the usage of large data sets reduces the training speed even in pre-trained networks. For fruit grasping and detachment, the detection of plucking point is more essential to preserve the quality and for further storage. The elaborate description about the framework, limitations in current sensing, fruit and picking point detection algorithms provides guidelines to the researchers in building a fully automated robotic system to increase the processing speed and production rate. © IMechE 2022.