In the agricultural sector, labor shortages are increasing the need for automated harvesting using robots. However, some fruits, like tomatoes, are tricky to harvest. Tomatoes typically bear fruit in clusters, requiring robots to pick the ripe ones while leaving the rest on the vine, demanding advanced decision-making and control capabilities.
How robots learn to pick tomatoes
To teach robots how to become tomato pickers, Osaka Metropolitan University Assistant Professor Takuya Fujinaga, Graduate School of Engineering, programmed them to evaluate the ease of harvesting for each tomato before attempting to pick it.
Fujinaga’s new model uses image recognition paired with statistical analysis to evaluate the optimal approach direction for each fruit. The system involves image processing/vision of the fruit, its stems, and whether it is concealed behind another part of the plant. These factors inform robot control decisions and help it choose the best approach. The findings are published in Smart Agricultural Technology.
Shifting from recognition to harvest-ease
The model represents a shift in focus from the traditional ‘detection/recognition’ model to what Fujinaga calls a ‘harvest‑ease estimation’. “This moves beyond simply asking ‘can a robot pick a tomato?’ to thinking about ‘how likely is a successful pick?’, which is more meaningful for real‑world farming,” he explained.
When tested, Fujinaga’s new model demonstrated an 81% success rate, far above predictions. Notably, about a quarter of the successes were tomatoes that were successfully harvested from the right or left side that had previously failed to be harvested by a front approach. This suggested that the robot changed its approach direction when it initially struggled to pick the fruit.
Implications for the future of farming
Ultimately, Fujinaga’s research highlights the nuance involved in fruit-picking for robots with factors including fruit clustering, stem geometry, background leaves, and occlusion all being important. “This research establishes ‘ease of harvesting’ as a quantitatively evaluable metric, bringing us one step closer to the realization of agricultural robots that can make informed decisions and act intelligently,” he said.
Fujinaga sees a future where robots will be able to independently determine whether crops are ready for harvest. “This is expected to usher in a new form of agriculture where robots and humans collaborate,” he explained. “Robots will automatically harvest tomatoes that are easy to pick, while humans will handle the more challenging fruits.”