autonomous agricultural robot - collision avoidance methods overview

PROCEEDINGS OF THE INSTITUTE OF VEHICLES 2(106)/2016
Marcin Jasiński1, Jędrzej Mączak2, Przemysław Szulim3, Stanisław Radkowski4
AUTONOMOUS AGRICULTURAL ROBOT - COLLISION AVOIDANCE
METHODS OVERVIEW
1. Introduction
Syndicate of Industrial Institute of Agricultural Engineering in Poznan, with the
Institute of Vehicles of Warsaw University of Technology and PROMAR company
from Poznan was started a design of autonomous farm robot for sowing and cultivation
of wide row planting.
The aim of the project is to develop the structure and operation procedures of an
autonomous robot for sowing and wide row planting and conduct laboratory and
exploitation tests on an experimental model. Autonomous work of the robot (traction and
agronomic processes) will be implemented on the basis of data from different kinds of
sensors (cameras, position, distance and others). Positive test results will allow
utilisation of the robot in organic crops requiring mechanical removal of weeds or in
crops with application of selective liquid agrochemicals limited to the minimum The use
of the vision system, supported by the maps and known coordinates of the sown seeds,
will allow for their care on an early stage of plant development. The applicability of the
robot to onerous work in organic farming may encourage farmers to discontinue the use
of herbicides in crops include sugar beet, corn, etc. According to this, achieving the
objectives of the project will require the achievement of the following specific objectives
of the practice [1]:
• development of robot platform,
• selection of precision drill and develop methods of position sown seeds,
• develop tools to allow the processing of weeding both the surface and the
surface spacing between crops in a row,
• development of tools to selectively spray the surface and the surface of the soil,
• develop of the control system and control algorithms autonomous robot field in
terms of traction and agronomic processes.
2. Initial assumptions for robot
The autonomous farm robot should work in following working conditions:
• terrain: empowered field, field roads, mud, sand, grassy ground, rocky ground
or other hardened,
• work in the open 24 hours / day,
• work in areas with varying degrees of lighting and visibility,
• temperature: 5 to 40 ° C,
M.Sc. Eng. Marcin Jasiński, PhD., Institute of Vehicles, Warsaw University of Technology
Prof. M.Sc. Eng. Jędrzej Mączak, PhD., Assoc. Prof., Institute of Vehicles, Warsaw University
of Technology
3
M.Sc. Eng. Przemysław Szulim, Institute of Vehicles, Warsaw University of Technology
4
Prof. M.Sc. Eng. Stanisław Radkowski, PhD., Dean of Faculty Automotives and Heavy Machinery Design,
Warsaw University of Technology
1
2
37
• weather: average rainfall, moderate wind, fog,
• typical obstacles in the open area;
Projected robot will enable complex care of field crops including: red beet, sugar
beet, sweet corn, cabbage, lettuce, forest nurseries, orchard, production of vegetables and
ornamental plants. Additionally it should enable the mechanical destruction of weeds
and, if necessary, precise application of crop protection formulations and fertilizers.
Robot constructed by us will be have a smart weeders, equipped in LPS system, which
uses digital image analysis for steering working tools, for mechanical weed control.
3. Conception of navigation system
Main sensor system is based on a specialized GPS receiver providing position
information with an accuracy of less than 100 mm. This system will be used to: control
speed of the robot, guidance and maintenance robot on the designated path, precision
seeding - the exact information on where sowing the seeds will be used to build maps of
seeds, which will be used as supporting information for precision weeding, and to
control the position of and operation of key components. For tests a typical GPS (10 Hz)
will be used.
The front camera view will be used to increase positioning accuracy of the robot. It
will allow corrections of the robot path regarding the rows of plants. The vision system
will also be used for detection of non-moving objects. Simultaneously second vision unit
will also be used for acquiring camera images immediately before active hoes and
sprayer. Additionally information from the acceleration sensors and encoders built-in
wheels will be used in navigation purposes. To determine the angular acceleration the
IMU (Inertial Measurement Unit) will be required. This will enable a:
• trajectory correction of the robot,
• precise work of active hoe,
• position adjustment and precise dosing of liquid fertilizer plant health products.
The exact position of the robot is obtained from the fusion of signals from precision
GPS (in the test version standard GPS receiver Ublox NEO7) and integrated system of
inertial and magnetic sensors VN-100 [2] . VN-100 is a complete AHRS (attitude
heading reference system) system integrating measurements from three axial sensors:
acceleration, angular velocity and Earth magnetic field. All of the sensors have
temperature compensated sensitivity and common values. Moreover, all the skewness of
axes were calibrated. VN-100 device provides accurate information from all the sensors
and estimates of the spatial orientation angles, DCM (direct cosine matrix)
transformation matrix, and estimated values of linear accelerations and angular velocities
in the absolute coordinates (NED – north/east/ down) independent from the angular
orientation of the sensor. It is planned that two AHRS systems will be placed on one
machine: first related to the vehicle and the second with a connection to the tools (e.g.
seeder). AHRS integrated with the vehicle will provide the detailed momentary angular
orientation and acceleration of the body while the second set allows better estimation of
the momentary position of the tool thus allowing precise localisation of the seeds.
Detailed information about their positions, stored in the internal database, will be used in
further fieldwork related to the care of plants.
During the preliminary phase of the project Authors [3] are planning to test
possibility of usage of several low cost sensors for collision avoidance system (moving
objects detection):
38
•
•
360 Degree Laser Scanner Development Kit. RPLIDAR is a low laser scanner
(LIDAR) solution developed by RoboPeak [4]. The system can perform 360
degree scan within 6 meter range. The produced 2D point cloud data can be
used in mapping, localization and object/environment modelling.
HC-SR04 (Cytron Technologies) ultrasonic distance sensor with range of 5m.
The HC-SR04 ultrasonic sensor uses sonar to determine distance to an object
like bats or dolphins do. It offers excellent range accuracy and stable readings
in an easy-to-use package. Its operation is not affected by sunlight or black
material.
3. Collision avoidance methods
Automatic steering systems are commonly used in agriculture nowadays. Different
navigation methods have also been widely researched in the past years. The focuses in
the researches have usually been on either path tracking or path planning methods. With
these methods, the tractor is able to realize complete agricultural operations on the field.
However, the driver or operator is still needed to monitor the system and to ensure
that the tractor does not collide with anything.
Fig. 1. GOTO and FOLLOW algorithms for a master–slave robot system. The GOTO
algorithm can be adopted for harvesting hay. On the other hand, the FOLLOW algorithm
results in a more common, cooperative style of work. The slave follows the master from
a given relative distance, d, and relative angle, γ [5]
Noguchi et al. [5] have developed a concept of a masterslave system for farm
operations. In the concept, there are two different algorithms for the slave tractor or
robot: GOTO algorithm and Follow algorithm (Fig. 1). In the first one the tractor moves
to predefined location and in the latter one follows the master that is operated by the
human. The slave monitors the master position constantly through a radio link. The risk
index is calculated based on the current position of the master and the slave. The risk
39
index indicates a potential risk of collision. Two actions are used to prevent collisions if
the risk is too high: speed reduction and pathway correction. The simulations were used
to prove the functionality of the system.
Vougioukas [6] has used the Nonlinear Model Predictive Control (NMPC) method
to control the position of the vehicle. Moreover, the collision avoidance was included
into the controller by using additional cost from distance sensor readings. The controller
was able to follow a predefined path as well as avoid collisions with static obstacles. The
functionality was again proven with simulations.
Generally, the collision avoidance methodology is a widely studied research area.
The methods can be divided roughly into two categories, although some of the methods
can be used in both situations. The first category is collision free path planning
algorithms. The second one is more reactive real time obstacle avoidance methods.
The Potential Field Method creates an artificial repulsive force field around the
obstacle and an artificial attractive force at the goal. The direction of the movement is
chosen based on the artificial potential field [7].
In the Vector Field Histogram a set of candidate directions are created and the best
one that is closest to the goal is chosen. The candidate directions are created according to
the probability of obstacle density in every direction (Fig. 2). If the density is below a
certain threshold, the movement to that direction is allowed [8].
Fig. 2. Mapping of active cells onto the polar histogram [8]
The Velocity Objects uses candidate velocities rather than only directions. The
selection of the new velocity is done in the velocity space, where also obstacles
velocities are added [9].
40
Furthermore, also kinematic and dynamic constraints can be taken into account in
the set of candidate velocities in velocity space.
In the context of the agriculture, the field is usually quite static; the electricity poles,
wells, bugholes and large rocks are more or less stationary. If all obstacles are known
beforehand, the route could be designed beforehand with a suitable coverage path
planning method. One such method is for example in [10]. However, there might be a
situation where an obstacle is known to be in the field but the position is not mapped yet.
For example there is some moving object (animal, human or another machine) or the
original map was imperfect. In these situations, there has to be some device to recognize
these obstacles and a method to recalculate the route or simply to stop the navigation
before the collision. In Finnish fields the most common obstacle is an electricity pole
and therefore the attention is on pole type obstacles.
The collision avoidance problem can be divided into two different subproblems:
detecting the obstacle and bypass the obstacle.
The path tracking method is based on the Nonlinear Model Predictive Control.
Vougioukas [6] has used the Nonlinear Model Predictive Control (NMPC) method to
control the position of the vehicle. Moreover, the collision avoidance was included into
the controller by using additional cost from distance sensor readings. The controller was
able to follow a predefined path as well as avoid collisions with static obstacles. The
functionality was proven with simulations.
In the NMPC, the control values are calculated, so that the given cost function is
minimized. The constraints of the optimization problem are obtained from the system
model and the constraints of the states and control values. There are different ways to
include the object avoidance into the NMPC. One way is to add additional constraints to
the state values. Another way is to add an additional cost from the obstacles or simply to
modify the reference trajectory to go past the obstacle.
In this way, the modification of the cost function was chosen. The underlying path
tracking cost function is not changed nor the reference trajectory, but the cost from state
is modified. This is because of the calculation capacity and the possibility that the
obstacles could move.
When the reference trajectory is near an obstacle, it cannot be followed without
colliding to the obstacle. Therefore it is irrelevant to keep the cost from the reference
trajectory. Instead a cost that makes the vehicle drive past the obstacle should be added.
The obstacle is allowed to be closer on the side of the vehicle.
The calculated distance to the edge of the avoided area is used in the cost function,
when the obstacle is inside the avoided area or the obstacle is closer to the avoided area
than the vehicle is to the original reference trajectory. the cost is calculated only from
one obstacle. If there are multiple obstacles inside the avoided area, the one with the
largest value of the the distance from the obstacle to the edge of the avoided area is
chosen. The same methods are also used for the cost from the trailer position.
The raw measurements from Laser scanner 2D are at first transformed into Cartesian
coordinates [11]. The obstacles are detected from the transformed measurements using
clustering method. The whole clustering process is illustrated in Fig. 3. The initial
positions for the clusters are gained from the set of known obstacles. The cluster initial
position is added if known obstacle is in sight of the scanner.
41
Fig. 3. The clustering algorithm [11]
4. Summary
Currently, several autonomous vehicles development programs are underway
targeting various agricultural production sectors. Until control systems can be perfected
and development cost recouped, the growth in autonomous field production systems will
come in fits to starts.
42
In this paper, few collision avoidance method, with Nonlinear Model Predictive
Control, was presented. The collision avoidance was divided into two different
subproblems: detecting the obstacle and avoiding the obstacle.
The obstacles were detected from the 2D laser scanner measurements with the help
of a clustering algorithm. There was also a list of recognized obstacles, which reduced
false positive and false negative recognitions.
The obstacle avoidance method was built on top of the existing experimental
navigation system. Because the computational capacity was already exhausted, the form
of the original NMPC was left unchanged. The solution was to modify the cost function
near an obstacle. An artificial avoided area was created, where the obstacles are not
allowed to be. If there is an obstacle inside the avoided area, the cost from the path is
changed to cost from the obstacle.
Work financed from NCBiR
years 2015-2018 funds.
References:
[1]
Marcin Jasiński, Jędrzej Mączak, Stanisław Radkowski, Roman Rogacki,
Jarosław Mac, Jan Szczepaniak, Tadeusz Pawłowski: Autonomous agricultural
robot - initial assumptions of project. Zeszyty Naukowe IP, 102(2), 2015,
pp. 49-56.
[2]
VN-100
Rugged
VectorNav
Technologies.
http://www.vectornav.com/products/vn100-rugged <29.10.2015>
[3]
Jasiński Marcin, Mączak Jędrzej, Radkowski Stanisław, Korczak Sebastian,
Rogacki Roman, Mac Jarosław, Szczepaniak Jan: Autonomous Agricultural
Robot — Conception of Inertial Navigation System. Edited by: Szewczyk R.,
Zieliński C., Kaliczyńska M.: Challenges in Automation, Robotics and
Measurement Techniques. Advances in Intelligent Systems and Computing Vol.
440, pp. 669-679, 2016.
[4]
http://www.robopeak.com <29.10.2015>
[5]
Noguchi, N., Will, J., Reid, J. and Zhang Q. (2004). Development of a masterslave robot system for farm operations. Computers and Electronics in Agriculture.
44, pp. 1-19.
[6]
Vougioukas, S.G. (2007). Reactive trajectory tracking for mobile robots based on
non linear model predictive control. In: 2007 IEEE International Conference on
Robotics and Automation, ICRA, Barcelona, Spain. pp. 3074-3079.
[7]
Tilove, R. (1990). Local Obstacle Avoidance for Mobile Robots Based on the
Method of Artificial Potentials. In: 1990 IEEE International Conference on
Robotics and Automation, ICRA, Cincinnati, OH USA. pp. 566-571.
[8]
Borenstein, J. (1991). The Vector Field Histogram – Fast Obstacle Avoidance for
Mobile Robots. IEEE Transactions on Robotics and Automation. 7 (3), pp. 278288.
[9]
Fiorini, P. and Shiller, Z. (1998). Motion Planning in Dynamic Environments
Using Velocity Obstacles. The International Journal of Robotics Research. 17 (7),
pp. 760-772.
43
[10]
[11]
Oksanen, T. and Visala, A. (2009). Coverage Path Planning Algorithms for
Agricultural Field Machines. Journal of field robotics. 26 (8),
pp. 651-668.
Backman J., Oksanen T., Visala A.: Collision Avoidance Method with Nonlinear
Model Predictive Trajectory Control. Proc. of 4th IFAC Conference on Modelling
and Control in Agriculture, Horticulture and Post Harvest Industry August 27-30,
2013. Espoo, Finland. pp. 35-40.
Abstract
The aim of the paper is to make collision avoidance methods overview, used in
navigation system of autonomous robot for sowing and wide row planting. Autonomous
work of the robot in range of traction and agronomic processes will be implemented on
the basis of data from a many sensors (cameras, sensors position, sensors distance, and
others). Positive test results will allow for the use of the robot in organic crops requiring
mechanical removal of weeds or in crops with application of selective liquid
agrochemicals limited to the minimum.
The collision avoidance was divided into two different subproblems: detecting the
obstacle and avoiding the obstacle. The obstacles will be detected from the 2D laser
scanner measurements with the help of a clustering algorithm. It's was the path tracking
method based on the Nonlinear Model Predictive Control (NMPC).
Keywords: agriculture robot, care of plants, autonomous work, collision avoidance
methods.
AUTONOMICZNY ROBOT ROLNICZY - PRZEGLĄD METOD UNIKANIA
PRZESZKÓD
Streszczenie
Celem referatu jest przedstawienie metod unikania przeszkód wykorzystanych w
systemie nawigacji autonomicznego robota polowego przeznaczonego do siewu i
pielęgnacji upraw szerokorzędowych. Autonomiczna praca robota w zakresie trakcji i
realizacji procesów agrotechnicznych realizowana będzie na podstawie danych z szeregu
czujników (kamery, czujniki położenia, odległości i inne). Pozytywne wyniki badań
pozwolą na zastosowanie robota w ekologicznych uprawach wymagających
mechanicznego usuwania chwastów, lub w uprawach z ograniczonym do minimum
selektywnym stosowaniem ciekłych agrochemikaliów.
W metodach unikania przeszkód ważne są dwa cele: wykrycie przeszkody i
ominięcie przeszkody. W tym przypadku przeszkoda będzie wykrywana na podstawie
pomiarów ze skanera laserowego 2D, z wykorzystaniem algorytmu klasteryzacji. Jest to
metoda śledzenia ścieżki na podstawie nieliniowego modelu sterowania predykcyjnego
NMPC (Nonlinear Model Predictive Control).
Słowa kluczowe: robot rolniczy, ochrona roślin, praca autonomiczna, metody unikania
przeszkód.
44