Deep ocular tumor classification model using cuckoo search algorithm and Caputo fractional gradient descent

Cuckoo search algorithm

The cuckoo search algorithm (CSA) is a meta-heuristic algorithm inspired by the brood parasitism behavior of cuckoo species. It incorporates lévy flights in order to improve its performance. In CSA, each cuckoo lays just one egg in a randomly selected nest, and nests with high-quality eggs are retained for future generations. The host bird can choose to either discard the egg or build a new nest, and it discovers cuckoo eggs with a probability p_a ∈ [0, 1]. Unlike other meta-heuristic algorithms, the CSA method generates two populations of potential solutions using lévy flight and random walk. The population resulting from lévy flight exhibits exploration in the search space, with individuals being notably diverse due to the lévy function’s stochastic nature. CSA employs the switching parameter p_a to combine global and local random walks. The global random walk uses lévy flight operation Levy(λ) ∼ u = k^−λ, 1 < λ ≤ 3, to explore the search space, i.e.,  (3)${\displaystyle {\mathbf{x}}_j^{\left(k+1\right)}={\mathbf{x}}_j^{\left(k\right)}+\beta \otimes Levy\left(\lambda \right).}$

In the local random walk, two solutions, ${\mathbf{x}}_p^k$ and ${\mathbf{x}}_q^k$, are randomly selected through permutation. The new position of the jth nest at the kth iteration is calculated using Eq. (4), which involves the current position ${\mathbf{x}}_i^{\left(k\right)}$, step size β, step scaling factor s, Heaviside function H(u), probability of discovering a cuckoo egg p_a, element-wise product of vectors ⊗, and a random number drawn from a uniform distribution v. (4)${\displaystyle {\mathbf{x}}_j^{\left(k+1\right)}={\mathbf{x}}_j^{\left(k\right)}+\beta s\otimes H\left(p_a-v\right)\otimes \left({\mathbf{x}}_p^{\left(k\right)}-{\mathbf{x}}_q^{\left(k\right)}\right).}$

The algorithm’s basic steps, including cuckoo selection, solution generation, evaluation, nest replacement, abandonment, as well as update, are summarized in Algorithm 1 . The stochastic Eq. (4) represents a random walk, where the next location/state depends on the current location and the transition probability. However, in CS, a significant portion of new solutions should be generated through far-field randomization to ensure sufficient exploration of the search space and to avoid getting trapped in local optima.

 
_______________________ 
Algorithm 1 Cuckoo Search via Lvy Flights____________________________________________ 
  begin 
  Objective function f(x),x = (x1,...,xd)T 
    Generate initial population of n host nests xj (j = 0, 1,⋅⋅⋅,n) 
    while (k <MaxGeneration) or (stop criterion) do 
      Get a cuckoo randomly by Lvy flights using (3) 
        Evaluate its solution quality or objective value f(xj) 
        Choose a nest among n (say, l) randomly 
        if (f(xj) < f(xl)) then 
          Replace l by the new solution j 
      end if 
      A fraction (pa) of worse nests are abandoned 
        New nests/solutions are built using (4) 
        Keep best solutions (or nests with quality solutions) 
        Rank the solutions and find the current best 
        Update k ← k + 1 
    end while 
  Postprocess results and visualization 
    end__________________________________________________________________________________________    

In CSA, the step size plays a pivotal role and necessitates vigilant monitoring to discern the search area relevant to the practical problem. In our approach, we predefine these step size values, and they remain constant across generations. However, this fixed nature can pose challenges, potentially causing the algorithm to become entrenched in local optima, thereby complicating the task of discovering optimal solutions.

CSA utilizes the lévy flight strategy, enabling cuckoo nests to traverse via a combination of short and sporadic long-distance cooperative random searches. It is due to this particular mode that CSA exhibits a significant and unpredictable leap during the search process. Consequently, the search in the surrounding area of each cuckoo nest is not sufficiently robust, leading to a slow convergence speed and inadequate convergence accuracy for CSA. The gradient direction, which represents the direction of maximum value change, is indicated by a vector that comes from the x_worst point and terminates at the x_best point.

For i = 1, …, d, we define the functions f_i,x:ℝ → ℝ by f_i,x(y) = f(x + (y − x_i)e_i), where e_i represents the vector in ℝ^d with a 1 in the i-th coordinate and 0’s elsewhere. For a vector c_k = [c₀, c₁, c₂, …, c_d] ∈ ℝ^d, we define the Caputo fractional gradient of f by (5)${\displaystyle {c^k}^C{\nabla }_x^{\alpha }f\left(x^{\left(k\right)}\right)=\left({c_1}^{\text{Caputo}}{D}_x^{\alpha }{f}_{1,x}\left(x_1\right),\dots {,}_{c_d}^{\text{Caputo}}{D}_x^{\alpha }{f}_{d,x}\left(x_d\right)\right)\in {\mathbb{R}}^d.}$

We can now introduce a Caputo fractional gradient descent method (CFGD) in the following manner: Starting at an initial point x⁽⁰⁾, the k-th iterated solution is updated by (6)${\displaystyle {\mathbf{x}}^{\left(k+1\right)}={\mathbf{x}}^{\left(k\right)}-{\eta }_k\cdot {{\mathbf{c}}^k}^C{\nabla }_{\mathbf{x}}^{\alpha }f\left({\mathbf{x}}^{\left(k\right)}\right),k=0,1,\dots ,}$

where α ∈ (0, 1), $x^{\left(k\right)}=\left(x_i^k\right)$, $c^k=\left(x_i^k\right)$, γ_k ∈ ℝ and ${}_{c^k}^C{\nabla }_x^{\alpha }f\left(x^{\left(k\right)}\right)$ is expressed as:

$\text{diag}{\left({c_i^k}^{\text{Caputo}}{D}_x^{\alpha }I\left(x_i^k\right)\right)}^{-1}\left[{c^k}^C{\nabla }_x^{\alpha }f\left(x^{\left(k\right)}\right)+{\gamma }_k\text{diag}\left(|x_i^k-c_i^k|\right){c^k}^C{\nabla }_x^{\alpha +1}f\left(x^{\left(k\right)}\right)\right].$

Let f(x) be a real-valued sufficiently smooth function defined on ℝ^d, where c_α = (1 − α)2^−(1−α), we have (7)${\displaystyle {\left({c^k}^C{\nabla }_x^{\alpha }f\left(x^{\left(k\right)}\right)\right)}_i=c_{{\alpha }_k}{\int }_{-1}^1{f}_{i,x}^{\prime }\left({\Delta }_i^k\left(1+u\right)+c_i^k\right){\left(1-u\right)}^{-{\alpha }_k}du+c_{{\alpha }_k}{\gamma }_k|x_i^k-c_i^k|{\int }_{-1}^1{f}_{i,x}^{\prime \prime }\left({\Delta }_i^k\left(1+u\right)+c_i^k\right){\left(1-u\right)}^{-{\alpha }_k}du,}$

We observe that Eq. (7) involves integrals that can be accurately evaluated by the Gauss-Jacobi quadrature. Let ${\left\{\left({\lambda }_m,u_m\right)\right\}}_{m=1}^v$ be the Gauss-Jacobi quadrature rule of v points. Then, ${{\mathbf{c}}^k}^C{\nabla }_{\mathbf{x}}^{\alpha }f\left({\mathbf{x}}^{\left(k\right)}\right)$ can be approximated using the Gauss-Jacobi quadrature formula as: (8)${\displaystyle {\left({{\mathbf{c}}^k}^C\nabla {\mathbf{x}}^{\alpha }f\left({\mathbf{x}}^{\left(k\right)}\right)\right)}_i=C_{{\alpha }_k}\sum _{m=0}^M{\lambda }_m{f}_i^{\prime }\left({\Delta }_i^k\left(1+u_m\right)+c_i^k\right)+C_{{\alpha }_k}{\gamma }_k\left|x_i^k-c_i^k\right|\sum _{m=0}^M{\lambda }_m{f}_i^{\prime \prime }\left({\Delta }_i^k\left(1+u_m\right)+c_i^k\right).}$

The steepest descent direction of a locally smoothed original objective function is the generic CFGD. Incorporating information about the local gradient and curvature of the objective function, this formula provides a local approximation of the Caputo fractional gradient. The parameters α, γ, and the weights and nodes of the quadrature formula (λ_m, u_m) collectively control the method’s behavior, thus achieving a balance between exploration and exploitation in the optimization process. To enhance exploration in feasible regions, we incorporate the Caputo fractional gradient given in Eq. (8) into Eq. (3), which yields (9)${\displaystyle {\mathbf{x}}_j^{\left(k+1\right)}={\mathbf{x}}_j^{\left(k\right)}+{\beta }_k\otimes Levy{\left(\lambda \right)}_{{\mathbf{c}}^k}^C\nabla {\mathbf{x}}^{\alpha }f\left({\mathbf{x}}_{best}^{\left(k\right)}\right),}$ where c^k is computed as the mean of the three solutions x₁, x₂ and x₃, i.e., (10)${\displaystyle {\mathbf{c}}^k=\frac{{\mathbf{x}}_1+{\mathbf{x}}_2+{\mathbf{x}}_3}{3},}$ where ${\mathbf{x}}_{1,2,3}={\mathbf{x}}_j-r_{1,2,3}\left({\mathbf{x}}_{best}-{\mathbf{x}}_j^k\right)$, and r_1,2,3 denote random numbers that uniformly distributed over [0, 2]. Such random numbers augment the exploration of the new regions and help avoid local minima stagnation. Similarly, we incorporate the Caputo fractional gradient descent given in Eq. (8) into Eq. (4) to improve the exploitation around the best solution obtained so far, which yields (11)${\displaystyle {\mathbf{x}}_j^{\left(k+1\right)}={\mathbf{x}}_j^{\left(k\right)}+\beta s\otimes H\left(p_a-v\right){\otimes }_{{\mathbf{c}}^k}^C{\nabla }_{\mathbf{x}}^{\alpha }f\left({\mathbf{x}}_{best}^{\left(k\right)}\right),}$ where c^k is assigned to x_worst.

The integration of CFGD into the cuckoo search algorithm (CSA) necessitates the incorporation of CFGD components, particularly the Caputo fractional gradient, at strategic points within the CSA algorithm. This integration aims to boost the overall performance of the optimization process by leveraging the distinctive advantages offered by both CSA and CFGD. Through meticulous design, the integration is tailored to capitalize on these specific strengths. The primary objective is to achieve higher convergence and more efficient exploration of the solution space by combining CSA’s global search capabilities with CFGD’s gradient-based method. The introduction of additional components, such as random numbers and CFGD-specific rules, introduces new computational tasks that could potentially impact the overall efficiency of the algorithm. While this integration is designed to be modular and controllable, it is crucial to recognize that this diversity may lead to additional computational expenses. The pseudocode for CSA-CFGD is provided below in Algorithm 2 .

 
_________________________________________________________________________________________________ 
Algorithm 2 CSA-CFGD____________________________________________________________________ 
  begin 
  Objective function f(x),x = (x1,...,xd)T 
    Generate initial population of n host nests xj (j = 0, 1,⋅⋅⋅,n) 
    while (k ¡MaxGeneration) or (stop criterion) do 
      Get a cuckoo randomly by Lvy flights using (9) 
        Evaluate its solution quality or objective value f(xj) 
        Choose a nest among n (say, l) randomly 
        if (f(xj) < f(xl)) then 
          Replace l by the new solution j 
      end if 
      A fraction (pa) of worse nests are abandoned 
        New nests/solutions are built using (11) 
        Keep the best solutions (or nests with quality solutions) 
        Rank the solutions and find the current best 
        Update k ← k + 1 
    end while 
  Postprocess results and visualization 
    end__________________________________________________________________________________________