Evolutionary Computation Based Real-time Robot Arm Path-planning Using Beetle Antennae Search

This paper presents a model-free real-time kinematic tracking controller for a redundant manipulator. Redundant manipulators are common in industrial applications because of the flexibility and dexterity they get from redundant joints. However, at the same time, the modeling of these systems becomes quite challenging, even for simple tasks like trajectory tracking. Some classical approaches are being used to tackle the issue, including a numerical approximation of the Jacobian and pseudo-inverse of the Jacobian matrix. These approaches have their limitations as they require exact parameters for the modeling of the manipulator; they are not immune to position error accumulation with time and put the manipulator way off the target position. Swarm-based meta-heuristic algorithms have given a new direction to the solution of the redundancy resolution problem. However, they are computationally intensive, formulated in discrete-time, and better suited for offline computation rather than real-time. We proposed a novel continuous-time Zeroing Neural Network with Beetle Antennae Search (ZNNBAS). The ZNNBAS algorithm can solve the quadratic optimization problem for redundancy resolution in real-time. To test its performance, we applied it on 7-DOF redundant manipulator with two trajectories to follow: character “M" and hypotrochoid. The manipulator was able to trace the reference trajectories with minimal tracking errors. Received on 22 November 2021; accepted on 07 January 2022; published on 18 January 2022


Introduction
The path planning of the robotics systems has always been an immense challenge for the researchers, especially when it involves multi-Degree Of Freedom (DOF) robots to perform daily chores like; picking, dropping, assembling, carrying, and placing objects [1][2][3]. Most of them are redundant manipulators as they possess more DOF than required for specific tasks. For example, it is widely recognized that six degrees of freedom are needed for end-effector motion trajectory. A robot with seven or more joints is known as a redundant manipulator for this task [4][5][6][7]. This socalled redundancy gives manipulator dexterity and flexibility in performing tasks [8][9][10]. The three major Ameer Tamoor Khan et al. In kinematic control, researchers have investigated different methods to come around the problem of redundancy resolution for the manipulators with known kinematic model [17][18][19][20]. The classical approach is the use of Jacobian matrix pseudo-inverse (JMPI) for the solution of redundancy resolution problem [21,22]. JMPI has its limitation as it is shown in [23] that it does not have the potential to produce repeatable results and is not immune to generating undesirable configurations. A later technique to solve the redundancy resolution problem of a robotic manipulator is to approach them as a constrained optimization problem [24][25][26]. These methods involve a function known as a fitness or objective function. Its purpose is to refine the angles of joints continuously; the solution to redundancy resolution is to maximize the objective value [27]. In this case, the objective function is not limited to kinematic control and can generate different random configurations within and outside the kinematic model; this optimization-driven approach changes the paradigm to solve the redundancy resolution of robotic manipulators. Inspired by this approach, [28] improved optimization technique and, instead of relying alone upon fitness function, introduced another penalty function to constrain the angles of joints within a specific range based on their mechanics. Similarly, another approach inspired by the neural network is introduced in [29,30] to solve the redundancy resolution problem where the dual neural network is used to solve the optimization problem in real-time. To be noted, all these methods require prior knowledge of the kinematic model of the robotic manipulator, and as they operate in an openloop, so they are not immune to Position error Accumulation (PEA). Uncertainties in manipulators like; length of the robotic arms, angles of the joints, human error, and Denavit-Hartenberg (DH) parameters can affect the performance of open-loop control of the redundant manipulator. If not, the manipulator's long-term use makes it vulnerable to several uncertainties because of friction and wearing away, which can change the initial parameters of the system.
In this paper, we have introduced a kinematicmodel-free method to solve the redundancy resolution problem of the manipulator. The classical approach is a numerical approximation to compute the inverse kinematic model of the robots [31][32][33], but they can fail to converge to a correct solution, and sometimes out of infinite many feasible solutions, they are limited to compute one [34]. This can be disadvantageous for the manipulators in trajectory planning as a single configuration may not be feasible given the limitations of manipulator mechanics [35]. Advanced techniques include training and data-driven approach, which uses a neural network to estimate the kinematic model of redundant manipulator [36]. Based on the estimated model, the robot is being controlled, but these techniques are computationally intensive and can not work in real-time as they require an extensive data set for estimation. Some approaches focus on the approximation of the Jacobian of a manipulator and design the kinematic controller of the velocity space. This technique requires the computation of Jacobian and its pseudo-inverse along the path in real-time, which is computationally expensive.
In this paper, we introduced time-invariant Zeroing Neural Network (ZNN) [37] with Beetle Antennae Search(BAS) [38] in continuous time (ZNNBAS) to design a model-free control for a redundant robotic manipulator. ZNN has different variants, but in the case of non-linear problems, they are designed to search for optimum solutions or zeros of the problem using a gradient in a closed-loop. However, meta-heuristic algorithms such as BAS do the random search in the search space to find the optimum solution to the problem. They are known to solve the non-convex problems, which include several local-minima along with global-minima [39]. Most of the meta-heuristic algorithms work in swarm-based approach where the group of searching particles, [40,41] they used discrete Particle Swarm Optimization (PSO) to model the kinematic control of 7-DOF robotic manipulator by employing several particles in space to search for the best joint configuration possible to move the endeffector in Cartesian space. There are mainly two problems with these systems.
• As the number of particles increases, the algorithm becomes computationally expensive.
• In discrete domain, particles jump from point to point, but they have continuous motion in reality.
Here we employed an algorithm in continuous time that mimics the beetle's nature. It is known as Beetle Antennae Search (BAS). It is widely used in several real-world applications [42][43][44][45][46][47][48][49][50][51]. This algorithm involves a single particle and overcomes the problems that discrete swarm particle optimization faces. We implemented time-invariant ZNN where instead of the gradient, we used a meta-heuristic BAS algorithm for a random search of an optimum solution in a closed loop to achieve the model-free design of redundant robotic manipulator. It is worth mentioning here that this method neither requires the kinematic model nor needs to compute Jacobian or inverse Jacobian on every iteration. It feedbacks the fitness value of the end-effector in Cartesian space to the joints in joint space to minimize their angle error. The objective of this paper is: • Proposed a model-free kinematic controller for redundant robotic manipulator using Zeroing neural network with meta-heuristic algorithm inspired from beetles in continuous time (ZNNBAS) which does not require the calculation of a jacobian or inverse jacobian matrix of manipulator making them computationally intensive.
• The proposed algorithm is immune to the limitations of discrete-time swarm-based metaheuristic algorithms.
• Theoretical analysis was done to prove the stability and convergence of the algorithm.
• Simulation results were achieved on KUKA LBR IIWA-14, a 7-DOF robotic manipulator. Two path tracking scenarios were tested to prove the efficiency of the algorithm.
The remainder of the paper is as follows: Section II discusses the problem formulation of a redundant robotic manipulator. Section III gives a detailed overview of ZNBAS in continuous time and its mathematical insight. In Section IV, we discuss the simulation scenarios, results, and discussion. In section V, we concluded aper with final remarks.

Problem Formulation
In this section, we will discuss in detail the kinematics of redundant manipulators and the kinematic control and trajectory tracking.

Kinematics of Redundant Manipulator
Consider a robotic manipulator; the position and orientation of its end-effector depend on the configuration of its joints. To demonstrate it consider m-DOF robotic manipulator i.e., m numbers of joints, and has an n-dimensional workspace or Cartesian space. The position of the end-effector in space corresponds to the unique configuration of the joint-space. The mapping from joint-space to Cartesian space is given as where x(t) corresponds to the position of end-effector in Cartesian space and x(t) ϵ ℜ n , similarly, θ(t) corresponds to the configuration of arms in joint-space and θ(t) ϵ ℜ m , in case of redundant manipulator m > n. Here, f (.) is a non-linear function, which is a non-linear transformation between joint and Cartesian space. In robotics, this is known as forward kinematics, where given the configuration of joints, one can evaluate the position of the end-effector in a workspace. However, in most of the real-world applications, forward kinematics is scarcely used as tasks are carried out in Cartesian space instead of jointspace, which is known as inverse kinematics and is given as where f −1 (.) Shows the transformation from Cartesian space to the joint-space. The above equation is the inverse of (1), for a known trajectory in Cartesian space, we can compute the trajectory for the arms in joint-space. This is possible only if we know the exact kinematics of the system, so the accuracy of any trajectory followed depends on the perfection of the model. In the case of redundant manipulators, there is no unique solution in the joint-space for a given configuration in a workspace, which means that for any configuration of end-effector in Cartesian space, there can exist an infinite number of configurations in the joint space. Thus, f −1 (.) is not uniquely invertible, and as f (.) is non-linear, so most of the time, the solution to its inverse is not possible. An approach to compute the kinematic model of a redundant manipulator is to use the Jacobian matrix. Take the time derivative of (1) whereẋ(t) corresponds to the velocity of end-effector in Cartesian space andẋ(t) ϵ ℜ n , similarly,θ(t) corresponds to the velocity of arms in joint-space and θ(t) ϵ ℜ m . J(θ(t)) ϵ ℜ nxm is a Jacobian matrix and is calculated as J(θ(t)) = ∂f (θ(t))/∂θ(t). The (3) is again a mapping from joint-space to Cartesian space in velocity domain. As mentioned earlier in real-world tasks include mapping from workspace to joint-space so take the inverse of the above equatioṅ In the case of a redundant manipulator, it would be a rectangular matrix, and there is no easy way to compute the inverse of a rectangular matrix. Researchers have come around this problem by computing the pseudoinverse of the Jacobian matrix, but it is computationally intensive. In addition, the Jacobian is valid only in the vicinity of θ(t) e.g., J(θ o ) is valid in a given range let us say ϵ from the past configuration i.e., |θ − θ o | < ϵ, where ϵ is very small. For the whole trajectory, the algorithm needs to compute it repeatedly, which makes it computationally very expensive.
Here we assumed that the correct kinematic mapping of the redundant manipulator is known and the calculated Jacobian generates a non-singular solution, so the pseudo inverse is computable. All these strict hardcore assumptions, which involve complex computation, make the system vulnerable to the uncertainties of the environment and system itself, calculation error, and it requires much off-line modeling of the system throughout the trajectory being followed. However, our algorithm is model-free; it does not need to know whether the manipulator is redundant for a particular task or not. It does not involve computationally expensive calculations like Jacobian and inverse Jacobian of the system. Our algorithm computes the objective function value based on the difference between end-effector coordinates in Cartesian space and the goal position and feeds it back to joints of the manipulator to tune their angles such that they follow the trajectory.

Kinematic control and trajectory Following
Consider a robotic arm manipulator, and its task is to grab a payload with the end-effector from a given location in Cartesian space, move it through the same space, and drop it to a goal position by following a given trajectory. The kinematic control in the task is as mentioned in an (1) to evaluate the states of links in joint-space and map it in the Cartesian space such that it follows the trajectory to the end-goal. To understand this, consider a reference trajectory i.e., x r (t), objective is given the states of links in joint-space i.e., θ r (t), the kinematic controller should generate the position and orientation of end-effector in Cartesian space such that it should generate a reference trajectory which is given as where θ r (t) is an angle of the links in joint-space which corresponds to the position of end-effector in Cartesian space i.e., x r (t). As mentioned earlier, in the case of the redundant manipulator, the controller can generate many infinite solutions in joint-space for the same position and orientation of the end-effector in Cartesian space, and it is possible that many of those solutions are not feasible with the mechanical constraints of the system, for example, angle of the joint going beyond its mechanical limit or prismatic joint extended out of its limit. Our objective is to follow the goal trajectory (5) such that the generated solutions lie within the mechanical constraints of the redundant manipulator. As we are proposing an optimization technique to solve the redundancy resolution problem so we will make an optimization problem out of this, which is subject to: is an optimization function that is to minimize or zero finding, and m in θ m represents the number of joints along with the angle constraints where θ + and θ − shows the maximum and minimum limits of the joint. The problem states that the controller needs to minimize the difference between the reference trajectory x r (t) and the runtime evaluated orientation and position of the end-effector f (θ(t)), all in Cartesian space. Optimization problem also includes the mechanical constraints of the joints, as in our case, we only have angular joints, so we included angle constraints only. The tuning parameter in this problem is the angles of the links in the joint space θ(t). The optimization problem is nothing more than an error function which in simple form is given as after this the optimization problem becomes subject to: The solution to this optimization problem will generate a trajectory in joint-space θ * (t) that will follow the reference trajectory in the same space θ r (t) which is given as. θ r (t) = θ * (t) As we mentioned earlier, our algorithm does not rely on the kinematic model of the manipulator. Instead, it is a model-free approach to solve the redundancy resolution problem of a redundant robotic manipulator.
Instead of kinematic control f (.), the algorithm will only rely on the state of orientation and position of endeffector coming from sensorsx(.), so now the error (7) becomes It is worth mentioning here that mostly the algorithms depends on the kinematic model to compute the coordinates of the end-effector. However, our algorithm does not rely on that and directly takes the coordinateŝ x(θ(t)) from the manipulator based on the fed angles θ(t). Now our optimization problem becomes Our algorithm ZNNBAS, with its zero-finding ability (ZNN) of non-linear function through random search (BAS), can solve the optimization problem in (7) with the objective function (9).

Control and Algorithm Design
In this session, we will discuss the algorithm formulation of ZNNBAS and then detail theoretical analysis on the stability and convergence of ZNNBAS.

Algorithm Formulation
Consider a 7-DOF redundant manipulator required to track a given trajectory, which means that it should minimize the objective value function given in (9), in such a way that all the constraints of the redundant manipulator angles meet as shown in (7). Our proposed algorithm involves a meta-heuristic algorithm which mimics the searching nature of Beetle insect (BAS) [38], as it senses the presence of food through its antennae. The closed-loop zero-finding nature of ZNN along with the random search of a single beetle in continuous time makes it a perfect combination to optimize the 7-DOF redundant robotic manipulator such that it follows the given reference trajectory. Consider the manipulator at time t is θ(t), ZNNBAS will start with a random direction based on its antennae sensation towards the optimum solution. The randomly generated direction vector by ZNNBAS is ⃗ b, so now the joint configuration of the 7-DOF redundant manipulator becomes where β(.) is a projection of joint-angles within the constraint space of joint-angles, as we mentioned earlier that we have to respect the angle limitations the manipulator as well (7). Any angle which transgress this constraint will be brought within its limit, the limiting mechanism or the formulation of β(.) projection is given as This projection function limits the joint-angles with the mechanical constraints of the manipulator. The λ in euq.10 is a scaling factor of the direction vector, it could be a scalar number λ const or a vector λ = [λ 1 , λ 2 , ...λ m ] t depends on application of how a particular joint should proceeds in joint-space. The robotic manipulator will move the seven joints based on their respective angles and eventually the end-effector will move to its new positionx(t). Here ZNNBAS algorithm will compute the error of reference trajectory with the newly computed coordinates of the end-effector as shown below This is the objective function value we mentioned in detail in (9). As the algorithm started from a random point in search space, the value of the objective function (12) will be high, which ZNNBAS will minimize with a shorter period and will bring close to zero.
ZNNBAS not only minimizes the objective function but its difference with previous values as well. ZNNBAS stores a few previous states of the objective function and then later t check the difference of the current objective function value with the α old value of the objective function as it is shown below To limit the difference of objective function ∆h, ZNNBAS used sign(h(θ ′ (t)) − h(θ ′ (t) − α)), it will limit the difference between two limits, in our case, we keep it sign min = −1 and sign max = 1. The (13) is a scalar value, but to update the angles of the joints we need m x 1 vector, equivalent to the number of joints of the manipulator. For that we will multiple it with input angles and the new updated angles for all the joints will becomeθ The bit of mathematical manipulation ended-up as a differential equation, which means that we will integrate all the previous values of the closed-loop system and the resultant will give us the new-updated angles for all the joints To further understand its working let's see the schematic of ZNNBAS to implement the control of redundant manipulator is shown in Fig.1  be seen that Random_Generator generates m random signals based on the number of links of the redundant manipulator. Those generated angles are then added to all the integrated states of the closed-loop system. The resultant is then split into two, one for the input delay box and the other for the manipulator. In the case of a manipulator, the joints of the robotic system rotate as per those input angles. As a result, the end-effector moves in the workspace. The manipulator output the xyz coordinates of the robotic manipulator to the object function h(.), which computes the error between input coordinates and the reference trajectory. The output is then added to delayed signals of h(.). The resultant is then limited using sign function and then multiplied with an input signal to generate m x 1 vector. The product is then integrated back with the previous state, and this closed-loop system keeps running, and eventually, the objective function error narrows down to zero (almost), and a redundant manipulator starts chasing the reference trajectory. We implemented it in MATLAB-SIMULINK. Remark 1: In the implementation of ZNNBAS, although we kept the λ given in (10) constant, but a modification could be introduced by scaling its value as per the amount of error generated by the objective function value. As in the initial stages, the manipulator is far from the reference trajectory, producing a larger error, so the manipulator should take big steps, and when it gets closer to trajectory and start chasing it, it should take smaller steps, which is given as where c 1 is a constant and h(x r (t)) is an optimization function in (9).

Theoretical Analysis
Here we will do the theoretical analysis of the ZNNBAS to prove that it is stable and converges to the optimal solution. For that, there are mainly two theorems to analyze one is related to the convergence of the joint angles, and the second is related to the convergence of the coordinates of the end-effector. Definition 1: For the chasing of the reference trajectory, the controller of the manipulator should show the convergence in the join-space of the manipulator. The objective function h(.) should decrease monotonically in time t. The ZNNBAS is said to be stable if h(x r (t 2 ), θ(t 2 )) < h(x r (t 1 ), θ(t 1 )) t 1 < t 2 (17) where x r (t) is a reference trajectory, the above equation means that objective function is a monotonically decreasing function of time.  (17). Consider the redundant manipulator at time t 1 has a joint-state θ(t 1 ) with the objective function h(x r (t 1 ), θ(t 1 )), and later in time instance t 2 the objective function becomes h(x r (t 2 ), θ(t 2 )) where t 1 < t 2 . The ZNNBAS will only update the joint-state of redundant manipulator if the objective function value h(.) decreases monotonically which is given as h(x r (t 2 ), θ(t 2 )) < h(x r (t 1 ), θ(t 1 )) t 1 < t 2 otherwise, it will bring the manipulator back to its previous joint-state, and will wait until the objective function value further converge. Definition 2: For the chasing of the reference trajectory, the controller of the manipulator should converge in the Cartesian space as well. It means that as time approaches to infinity t → ∞, the end-effector trajectory must trace the reference trajectory.

Simulation Methodology, Result, and Discussion
In this section, we presented a simulation methodology, which includes IIWA-14 redundant robotic manipulator tracking the character "M" and hypotrochoid (star) using ZNNBAS.

Simulation Methodology
In order to test the performance of our proposed ZNNBAS algorithm, we used a redundant manipulator IIWA-14 provided by the MATLAB Robotics Toolbox. The robotic model is very close to the real-world IIWA-14 manipulator and includes 7-DOF; we used it as a testbed to measure the efficiency of our algorithm. We used two-tracking paths as a benchmark to prove the validity of ZNNBAS. In our simulation we tested the algorithm by tracking two paths using 7-DOF redundant robotic manipulator; the first one is Character "M" and the second one is Hypotrochoid or star. For the character "M" trajectory the equation is given below Ameer Tamoor Khan et al. manipulator's trajectory is large, which narrows down as the manipulator starts chasing it, and this is all done by a single beetle particle working in continuous time. The Fig.2b shows the XY Z coordinates of the endeffector. It can be seen that character "M" is in y − z axes as the y and z coordinates of the end-effector move in the plane, and the x-coordinate remains constant.
Similarly, Fig.2c shows the angles of the redundant manipulator throughout the tracking. The Fig.2d shows the objective function value, and as it is seen that simulation lasts for 50sec, but even less than 10sec the error approaches zero, and during the time from 40sec to 50sec the error went down to 10 −5 , which shows how efficiently and precisely ZNNBAS optimizes the problem without showing any spikes of noise or undesirable behavior of manipulator. Fig.2e shows the error in each coordinate, and as it can be seen, that error approached zero in no time.
For the hypotrochoid (star) trajectory, the results are shown in Fig.3. The Fig.3a shows the 3D tracking of the redundant manipulator, which starts from the base or home position, which is erect and then moves through space randomly to trace the reference track, and eventually, it did track the hypotrochoid (star). This shows the power of a single beetle in continuous time and how efficiently and accurately it tracks down complex trajectories like a star. Fig.3b and Fig.3c show the coordinates of the end-effector and the angles of all the seven links of the IIWA-14 redundant manipulator. Similarly, Fig.3d shows the objective function value, and in a simulation of 50sec, it approaches 0 in less than 3sec. Likewise, Fig.3e shows the error in xyz coordinates of the end-effector, and it can be seen that they approach zero in no time.
The major drawback or limitation of ZNNBAS is the correct determination of α, which is the core hyper-parameter to determine the performance of the algorithm. Likewise, the other hyper-parameters are manually set. In future, we plan to work on these limitations to make ZNNBAS more robust and efficient.

Conclusion
The objective of this paper was to address the redundancy resolution problem of the redundant manipulator used in numerous industrial applications and their kinematic control design. We proposed a model-free control for redundant manipulators using Zeroing Neural Network with Beetle Antennae Search (ZNN) in continuous time. ZNN alone used a gradient-based method to search for an optimal solution that may limit it to local-minima, but with the random search of BAS, it can avoid localminima and converge to a global solution. The redundancy resolution problem is tackled through Particle Swarm Optimization (PSO) algorithm, but they are computationally intensive and do not mimic the nature of the insect army in general as they are discrete. ZNNBAS has overcome these challenges as it involves a single particle to solve the redundancy resolution problem of a redundant robotic manipulator in continuous time. We theoretically prove the stability and convergence of ZNNBAS, and a test case, we used 7-DOF IIWA-4 redundant manipulator provided by MATLAB and successfully tracked two trajectories i.e., character "M", and Hypothyroid (star).