A Unified Framework for Combinatorial Optimization Based on Graph Neural Networks

Path and routing problems are common types of COPs that involve finding the most efficient ways to navigate through a graph, considering various constraints and objectives. We introduce some classical path and routing problems, including the Shortest Path Problem (SP), the Minimum Spanning Tree Problem (MST), and the Traveling Salesman Problem (TSP).

Definition 1 (Shortest Path Problems (SP)). Given a graph $G=(V,E)$, find an optimal path between two vertices such that the sum of the weights of its constituent edges is minimized, where a path in an graph is a sequence of vertices $P=(v_{1},v_{2},...,v_{n})\in V\times V\times...\times V$, such that $v_{i}$ is adjacent to $v_{i+1}$ for $1\leq i<n$.

Some variations of SP, such as the Single-Source Shortest Path Problem (SSP) [17] and the All-Pairs Shortest Path Problem (PSP) [18], are also presented. These variations help optimize routes and aid in decision-making for real-world applications. Specifically, SP has practical significance across multiple domains. For instance, it is used in road networks for navigation [19], path planning for autonomous robots [20], efficient data transmission in computer networks [21], and optimization of supply chain and logistics operations [22].

Definition 2 (Minimum Spanning Tree Problem (MST)). Given an edge-weighted undirected graph, span a tree that connects all vertices while not having any cycles and has a minimum total edge weight.

One representative example [23] involves communication networks striving for efficient data transmission and network design, where the MST is utilized to create a network with the minimum total communication cost. A related problem is the k-minimum spanning tree (k-MST) [24], which entails finding an MST that spans a specified subset of $k$ vertices in the graph, while minimizing the total weight of the tree. Additionally, the dynamic MST problem (DMST) [25] addresses the challenge of updating a previously computed MST in response to changes in the underlying graph. These changes can include modifications to edge weights, the insertion of new vertices, or the deletion of existing vertices.

Definition 3 (Traveling Salesman Problem (TSP)). Given a list of cities and the distances between each pair of cities in a complete weighted graph $G=(V,E)$, find a tour of minimum total weight that visits each city exactly once and returns to the origin city.

TSP can be modeled as an undirected, weighted graph. It is a minimization problem where the goal is to start and finish at a fixed vertex, visiting each other vertex exactly once. The Traveling Purchaser Problem (TPP) [26] and the Vehicle Routing Problem (VRP) are generalizations of the TSP, both of which belong to the class of NP-hard problems. The TPP, in particular, is a procurement/routing problem that involves the strategic selection of a purchasing plan for a specified set of products from a subset of suppliers, coupled with the determination of an optimal visiting tour for the purchaser. This is done to satisfy a predefined demand for products efficiently. The VRP pertains to the operations of a delivery company, where goods are distributed from one or more depots. Each depot has a set of home vehicles operated by drivers who navigate a given road network to serve a set of customers. Several variations and specializations of VRP exist, addressing different operational constraints and objectives. These include the Vehicle Routing Problem with Pickup and Delivery (VRPPD) [27], the Capacitated Vehicle Routing Problem (CVRP) [28], the Multi-Depot Vehicle Routing Problem (MDVRP) [29], the Electric Vehicle Routing Problem (EVRP) [30], and the Multi-echelon Vehicle Routing Problem (ME-VRP) [31, 32].

Network Flow Problems are a class of COPs that deal with the flow of resources through a network. The input of these problems is a flow network, which is a graph with numerical capacities assigned to its edges. The objective of Network Flow Problems is to establish a flow by assigning numerical values to each edge, ensuring adherence to the capacity constraints. Additionally, the flow must satisfy the condition of having incoming flow equal to outgoing flow at all vertices, except for specifically designated terminals such as sources and sinks. Two notable types of Network Flow Problems are the Maximum Flow Problem (MFP), which focuses on finding the maximum flow from the source to the sink, and the Minimum Cut Problem (MCP), which involves identifying the minimum capacity cut that separates the source and sink.

Definition 4 (Maximum Flow Problem (MFP)) Let $G=(V,E)$ be a network with $s$, $t\in V$ the source and the sink of $G$, respectively. The value of flow is denoted by $f$, representing the amount of flow passing from the source to the sink. The MFP is to determine the maximum flow value, denoted as $f_{max}$, that can be routed from the source to the sink within the network.

As a fundamental problem in network flow optimization, the MFP [33] has been extended to address more complex flow-related challenges, such as the Multi-Source Multi-Sink Maximum Flow Problem (MSMS-MFP) [34], and Maximum Cardinality Bipartite Matching (MCBM) [35]. Unlike the MFP, MSMS-MFP involves multiple source nodes and multiple sink nodes within the network [34]. For example, when applied to Internet routing, MSMS-MFP optimizes the flow of data packets between multiple sources and sinks to enhance network performance, reduce latency, and ensure efficient resource utilization [36]. As a classical flow problem, MCBM deals with finding the largest possible matching in a bipartite graph. In such graphs, vertices can be divided into two disjoint sets, and a matching consists of edges that do not share any common vertices [37]. MCBM provides a versatile framework for modeling and solving problems involving optimal pairings or assignments between two distinct groups.

Definition 5 (Minimum Cut Prolem (MCP)) Given a network $G=(V,E)$ with a source node $s$ and a sink node $t$, and a cut $C$ is a partition of the vertices $V$ into two disjoint sets, $S$ and $T$ (i.e., $V=S\cup T$), such that $\in S$ and $t\in T$. The MCP aims to find a cut with the minimum possible capacity among all cuts in the network.

The variations of the MCP encompass a wide range of scenarios, including weighted graphs, directed graphs, scenarios with designated terminals, and the partitioning of vertices into more than two sets. For instance, the graph partition problem [38], a recognized NP-hard challenge, involves partitioning a given graph into two or more parts. This process adheres to additional constraints, such as achieving a balance in the sizes of the parts created by the cut. Additionally, spectral clustering [39, 40, 41], a graph-based clustering method, leverages the spectral properties of matrices derived from the data. This method aims to find a partition that minimizes criteria related to the cut in the graph, where the cut refers to the sum of the weights of the edges connecting different partitions.

Coloring and Matching Problems involve constraints related to the relationships between vertices or edges. Graph coloring [42] involves assigning colors to vertices with the constraint that adjacent vertices must not share the same color. The optimization goal in graph coloring is to minimize the number of colors used. In contrast, the Matching problem [43] revolves around the selection of edges in such a way that no two chosen edges share a common vertex. The objective can be to maximize or minimize the number of edges in the matching, depending on the specific requirements of the problem.

Definition 6 (Graph Coloring) Given an undirected graph $G=(V,E)$, where $V$ is a set of vertices and $E$ is a set of edges, a graph coloring is an assignment of colors to the vertices of $G$ such that no two adjacent vertices share the same color and the number of colors required to achieve a valid coloring is minimized.

There are various graph coloring variants, each based on different constraints and objectives, such as Clique Coloring [44], Circular Coloring [45], and Hamiltonian Coloring [46]. These variants offer solutions to complex problems across different fields by capturing specific relationships and arrangements within graphs [47]. Clique Coloring aims to assign colors to vertices such that cohesive groups, or cliques, within a graph are represented by the same color. This approach has applications in network design [48], resource allocation [49], and conflict resolution [50]. Circular Coloring, which involves assigning colors to vertices arranged in a circular layout, is commonly applied in frequency assignment problems [51] and scheduling scenarios [52]. Hamiltonian Coloring, on the other hand, focuses on ensuring that no two adjacent vertices along a Hamiltonian cycle share the same color, which can find applications in circuit design [53] and routing problems [54].

Definition 7 (Matching Problem) Given an undirected graph $G=(V,E)$, where $V$ represents the set of vertices and $E$ denotes the set of edges, a matching $M$ in $G$ is a subset of edges such that no two edges in $M$ share a common vertex.

Commonly referred to as the bipartite matching problem (BMP) [55], this problem seeks to determine the largest possible set of non-overlapping edges in a graph, where each edge connects vertices from the two distinct partitions of the bipartite graph. In weighted bipartite graphs, the optimization problem aims to find a maximum-weight matching, also known as the assignment problem [56]. It is noteworthy that while the goal of the assignment problem is to maximize weights, it does not inherently guarantee stability. However, it has a wide range of applications, from project management [57] to logistics [58]. On the other hand, the stable marriage problem (SMP) seeks a stable matching between two equally sized sets of elements, each set having an ordering of preferences [59] In addition, the Chinese Postman Problem (CPP) [60] aims to finding a minimum-weight perfect matching in general graphs as a subproblem. When a graph has an Eulerian circuit, it inherently solves the CPP for that graph. However, in case an Eulerian circuit is absent, the optimization objective is to identify the minimum number of edges that need to be duplicated (or the subset of edges with the minimum total weight) to transform the graph into a multigraph with an Eulerian circuit [61]. Indeed, the CPP has proven to be a versatile and powerful tool in optimization, with several combinatorial problems being effectively reduced to it [62].

Social network problems within the class of graph-based COPs leverage the powerful framework of graph theory to model and address various challenges and optimization goals in social network contexts. Typical problems in this area include the Minimum Vertex Cover, Maximum Independent Set, Minimum Dominating Set, and Maximum Coverage Problems. Each of these problems presents unique challenges that help in understanding and optimizing different aspects of social networks.

Definition 8 (Minimum Vertex Cover (MVC)) Given an undirected graph $G=(V,E)$, find a set $S\subseteq V$ such that for every edge $(u,v)\in E$, at least one of $u$ or $v$ is in $S$. The MVC aims to minimize the cardinality of the vertex cover set, denoted as $|S|$.

The Maximum Independent Set (MIS) [63] and Maximum Clique (MC) [64] are closely related variants of the MVC problem in graph theory. Vertex cover optimization serves as a model for a wide range of real-world and theoretical problems. For example, the distribution of facilities in municipal services, which ensures that a set of monitor stations adequately covers the connections between other stations, may be modeled as a vertex cover minimization problem [65].

Definition 9 (Minimum Dominating Set Problem (MDS)) Given an undirected graph $G=(V,E)$, find a set $D\subseteq V$ such that every vertex in $V$ is either in $D$ or adjacent to a vertex in $D$. The MDS aims to minimize the cardinality of the dominating set, denoted as $|D|$.

Both the Total Dominating Set (TDS) [66] and the Connected Dominating Set (CDS) [67] are common variants of MDS that are extensively studied in the context of algorithmic problem-solving, especially in the design and optimization of networks [68]. In a TDS, every vertex in the graph must either be included in the set or be adjacent to a vertex that is part of the set. The goal is to find the smallest possible total dominating set for the graph. In contrast, a CDS requires that the set forms a connected subgraph, ensuring that every vertex in the set is either adjacent to or can be reached by some other vertex in the set. CDS are particularly relevant in the fields of network design and communication protocols [69], where connectivity is crucial.

Definition 10 (Maximum Coverage Problems (MCP)) Given a weight graph $G=(V,E,w)$ where $w:E\to\mathbb{R}^{+}$, find a set $S\subseteq V$ that maximizes the total weight of edges connected to vertices in $V$.

The Maximum $k$-Coverage Problem (MKCP) [70] is a typical extension of MCP where an additional constraint limits the number of subsets (sets) that can be chosen. Instead of aiming to cover as many elements as possible, the goal is to select a subset of $k$ sets to maximize coverage. The key constraint here is the fixed number $k$, which acts as a budget or limitation on the number of subsets that can be selected. On the other hand, by introducing an additional constraint related to the budget associated with each group (subset), the MCP can transition to the Maximum Coverage Problem with Group Budget Constraints (MCG) [71]. In MCG, each subset (group) is associated with a budget, and the goal is to maximize the coverage of elements subject to the constraint that the total budget spent on selected groups does not exceed a specified limit.

Scheduling and allocation problems are concerned with the effective allocation of resources, tasks, and time and are commonly modeled as COPs, with graph models being a prevalent representation method. However, not all problems require the use of graph models, and sometimes employing other mathematical models or algorithms is more convenient or efficient [72]. In this context, we focus on graph-based COPs, such as Resource Allocation Problems (RAP) [73] and Job Shop Scheduling Problems (JSSP) [74].

Definition 11 (Resource Allocation Problems (RAP)) Given a directed graph $G=(V,E)$, where $V$ is a set of nodes representing tasks, projects, or entities and $E$ is a set of directed edges representing relationships or dependencies between nodes. RAP is to minimize or maximize a performance metric related to the efficiency or satisfaction of the resource allocation when the resource demand for each node is satisfied and resource allocations are non-negative.

The optimization of RAP [75, 76] is typically formulated as a mixed-integer non-linear programming problem, which is non-convex and NP-hard. The Facility Location Problem (FLP) is a classic resource allocation problem that involves determining the optimal locations for facilities to meet demand in the most cost-effective way [77]. In addition, a series of extensions of RAP, such as the Network Resource Allocation Problem (NRAP) [78], the Human Resource Allocation Problem (HSAP) [73], and the Cloud Computing Resource Allocation Problem (CCRAP) [79], demonstrate the adaptability of resource allocation problems in different application domains.

Definition 12 (Job Shop Scheduling Problems (JSSP)) Given $n$ independent jobs: $J_{1},J_{2},\ldots,J_{n}$, and processing time for each job: $p_{i}$ for $i=1,2,\ldots,n$. JSSP is about the scheduling of a set of independent tasks to optimize a certain performance metric. A mathematical definition of the JSSP is given as follows:

	Minimize: A performance metric (e.g., total completion time or makespan)
	Subject to:
	$\displaystyle\quad S_{i}\geq\max(C_{j})\quad\forall i=1,2,\ldots,n$
	$\displaystyle\quad S_{i}+C_{i}=T_{i}\quad\forall i=1,2,\ldots,n$
	$\displaystyle\quad T_{i}\leq T_{j}\quad\forall i,j=1,2,\ldots,n\quad\text{and}\quad i\neq j,$

where each job can only be processed on one machine, and the start time of each job cannot precede the completion time of its predecessors.

JSSP can be usually formulated as a graph optimization problem, depending on the nature of the scheduling problem. Nodes in the graph often represent tasks or events, and edges between nodes may represent dependencies or constraints between tasks. The complexity of scheduling problems arises from the need to efficiently allocate limited resources to meet specific objectives. As technology evolves and societal priorities shift, scheduling problems are adapting to incorporate new considerations and objectives. These are evolving towards greater flexibility, human-centricity, and sustainability [80]. For example, Flexible Job Shop Scheduling Problems (FJSSP) [81] introduce a set of jobs organized into operations that need to be processed on a sequence of machines, unlike the single-machine focus of classical, non-flexible environments. Often, an FJSSP that incorporates the human element is referred to as a Dual Resource Constrained Flexible Job Shop Scheduling Problem (DRCFJSSP), indicating the simultaneous consideration of both machines and workers as critical resources in the scheduling process [82].

Packing problems [83] attempt to pack objects into containers with the goal of either maximizing occupancy within a single container or minimizing the total number of containers required to accommodate all objects. Two typical examples of the packing problems are the Bin Packing Problem [84] and the Knapsack Problem [85].

Definition 13 (Bin Packing Problem (BPP)) Given a finite set of items $I={1,2,...,n}$, each with a positive size $s_{i}$, where $i$ represents the item, a set of identical bins or containers, each with a fixed positive capacity $B$, the objective of BPP is to minimize the number of bins used to pack all items while adhering to the capacity constraint of each bin.

BPP is known to be NP-hard and involves packing a finite set of items, each with a weight, into a finite number of bins [86]. Various practical variations of the BPP exist, mainly depending on the dimensions of the bins, placement constraints, and priorities. For example, by incorporating an additional dimension, such as width or height, evolved versions of the BPP include the Two-Dimensional Bin Packing Problem (2BP) [87], the Three-Dimensional Bin Packing Problem (3BP) [88], and the Variable Sized Bin Packing Problem (VSBPP) [89]. In addition to the traditional BPP, a notable variant is the Online Bin Packing (OBP), where items with varying volumes arrive sequentially, and decisions about each item must be made in real time. Here, the decision-maker faces the choice of either selecting and packing the current item into the available bins or letting it pass [90].

Definition 14 (Knapsack Problem) Given a set of $n$ items, indexed as $i=1,2,...,n$, each item $i$ as an associated weight $w_{i}$ and a value $v_{i}$, and a knapsack with a fixed weight capacity $W$, the objective is to maximize the total value of items placed in the knapsack without exceeding its weight capacity.

The knapsack problem is similar to a scenario where someone has a fixed-size knapsack and needs to decide which items should be placed into it to maximize the total value. Each item has a weight and a value, and decisions are based on the knapsack’s capacity constraints [91]. The 0-1 Knapsack Problem [92] is one of the most common and well-known variants of this problem. Besides the 0-1 Knapsack Problem, two other prevalent variants are the Bounded Knapsack Problem (BKP) [93] and the Unbounded Knapsack Problem (UKP) [94]. In the BKP, multiple copies of each item are available, but there is a limit on how many copies of each item can be selected. The decision variable now represents the number of copies of item $i$ to include, with integer values ranging from 0 to a specified upper bound. Conversely, the UKP allows an unlimited number of copies for each item, simplifying the decision-making process. These variants illustrate the adaptability of the knapsack problem to model diverse real-world scenarios with varying constraints on the number of item copies that can be selected [95].

Cutting problems, specifically Cutting Stock Problems (CSPs), present optimization challenges that focus on efficiently cutting large raw material sheets or stock into smaller pieces to meet specific demands or requirements.

Definition 15 (Cutting Stock Problems (CSPs)) Given a set of $n$ items or raw materials, each with a specific length, a set of raw material rolls or stock lengths, each with a fixed length, along with the demand for each item, indicating the number of pieces needed. The CSP is to minimize the number of raw material rolls used to satisfy the demand for the items.

CSP is an NP-hard optimization challenge that arises in industrial applications and can be related to the knapsack problem [96]. CSP can be classified based on the dimensionality of the cuts: including the One-Dimensional Cutting Stock Problem (1DCSP) [97], the Two-Dimensional Cutting Stock Problem (2DCSP) [98], and the Three-Dimensional Cutting Stock Problem (3DCSP) [99]. The challenge of CSP is prevalent in numerous industrial processes and involves a diverse array of materials such as steel bars, rolls of paper or aluminum, wooden boards, metal sheets, printed circuit boards, glass or fiberglass sheets, and leather, among many others. Effective cutting strategies are crucial for reducing costs and enhancing efficiency. Consequently, CSP holds significant economic importance due to its direct impact on cost savings through optimized material utilization [100].

Covering Problems aim to find a subset of elements that cover or satisfy a certain criterion. The Set Cover Problem is a well-known combinatorial optimization problem with a variety of applications.

Definition 16 (Set Covering Problem) Given a finite set of elements, denoted as $U$, where $U={1,2,...,n}$, a collection of subsets of $U$, denoted as $S={S_{1},S_{2},...,S_{m}}$, where each $S_{i}$ represents a subset of $U$, and a cost associated with selecting each subset $S_{i}$, the objective of set covering problem minimizes the total cost by selecting a minimum number of subsets such that the union of the selected subsets covers all elements in $U$.

SCP is known to be NP-hard in the strong sense and poses challenges from the perspective of theoretical approximation [101]. Typically, SCP is characterized as the challenge of efficiently covering the rows of a $m$-row, $n$-column, zero-one matrix $(a_{ij})$ using a subset of the columns while minimizing costs. This versatility can be extended to two notable variants, namely Weighted Set Cover (WSC) [102] and Fractional Set Cover (FSC) [103]. In WSC, each set is associated with a positive weight, symbolizing its cost, which aims to identify a set cover with the minimum total weight, while the FSC permits the selection of fractions of sets, rather than whole sets. SCP holds practical significance, as it finds applications in modeling a diverse array of real-world problems, including but not limited to scheduling [104], manufacturing [105], service planning [106], and information retrieval [107].

The satisfiability problem, often referred to as SAT, aim to determine the satisfiability of a logical formula, i.e., whether there exists an assignment of truth values to variables that makes the formula true. When each clause contains at most two literals (binary clauses), the problem is known as a Boolean satisfiability problem (2-SAT).

Definition 17 (Boolean satisfiability (2-SAT)) Given a Boolean formula $\phi$ in conjunctive normal form (CNF), which is a conjunction (AND) of clauses, each clause is a disjunction (OR) of literals. A literal is either a variable $x_{i}$ or its negation $\neg x_{i}$, where $i$ is an index. The aim of 2-SAT is to determine whether there exists an assignment of truth values (true or false) to the variables in $\phi$ that makes the entire formula true.

SAT is an NP-complete decision problem that involves determining whether a propositional logic formula can be satisfied with appropriate value assignments to its variables [108]. SAT has impacted various related decision and optimization problems, termed as SAT extensions. One notable extension of SAT is Satisfiability Modulo Theories (SMT) [109]. Other extensions include maximum satisfiability (MaxSAT) [110] and Quantified-Boolean Formulas (QBF) [111]. Specifically, these extensions represent paradigmatic constraint-satisfaction problems with broad practical applications, including hardware and software design [112], bioinformatics [113], among others.

In the past decade, various graph embedding (or called network embedding) methods have been proposed to convert nodes in a graph (network) into dense and low-dimensional vectors while preserving the structure [119]. The basic idea of obtaining vector representations in low-dimensional space is that the original topology information, for example, the often-used adjacency matrix, is sparse and discrete, which is not most effective for subsequent processing. One natural idea is to apply matrix factorization, such as singular value decomposition, to reduce the dimension of the adjacency matrix. Alternatively, random-walk-based methods have been explored [120], inspired by the idea from the natural language processing community. In DeepWalk [121], one sequence, referred as one sentence in Word2Vec [122], is derived by implementing a random walk on nodes, referred as words in Word2Vec. Subsequently, the Skip-Gram model [122] proposed in Word2Vec is employed to learn the embedding vectors of nodes, which considers context in one sentence and co-occurrence rate among sentences to describe the local topology information of a node. Node2Vec  [123] employs a biased random walk strategy, allowing for a choice between breadth-first sampling and depth-first sampling. In addition, various methods incorporating additional information, such as node labels or attributes, are proposed to enhance the representation ability  [124, 125]. Apart from using random walk strategies to extract structure information, LINE  [126] generates embedding vectors of nodes by focusing on the connection between two nodes. The obtained embedding vectors of nodes are further used for downstream applications, including node classification  [127], link prediction  [128], recommendation  [129], and visualization  [130], to name a few.

Graph embedding methods aim to provide dense, continuous, and low-dimensional embeddings as the input to other off-the-shelf machine-learning methods to handle downstream tasks. These methods excel in specific scenarios but encounter significant limitations when it comes to inductive learning, which requires the model to generalize effectively to unseen data. Moreover, traditional graph embedding techniques often face challenges with exacting high-level representations, since they typically rely on random walk methods that may not capture the complexity inherent to graph data.

Recently, GNNs have attracted increased attention in machine learning community due to their ability to handle complex graph data more effectively than traditional graph embedding methods. Specifically, GNNs can update a node embedding by updating information from its neighbors iteratively. This method of capturing graph structures allows for learning more precise and meaningful representations that better reflect the relational properties of the data.

Figure 2 provides a timeline of some milestones in the GNN research from 2009. The term graph neural network was first introduced in 2009 [131]. During the same period, a related concept, neural networks for graphs (NN4G), was proposed in [132]. Both frameworks provide a general idea for updating node embeddings through the exchange of information with neighboring nodes. After that, GNNs failed to attract widespread attention until around 2015 and 2016 [133, 134], when two fundamental GNNs, the graph convolutional network GCN [135] and ChebNet [136], were published. In 2017, research on GNNs has entered a new era with numerous GNNs being proposed, including GraphSAGE [137], message passing neural networks MPNN [138], and graph attention networks GAT [139]. After that, numerous studies have built on classic GNNs mentioned above [140], [141], [142], [143], and integrated these networks with other machine learning techniques [144], [145], [146], [147], such as large language models [148]. Since 2020, research on GNNs has increasingly aimed to address their limitations, such as neighbor explosion problems [149], [150], [151], generalization problems [152], [153], [154], homophily assumptions [155], [156], [157, 158], limited discrimination power [159], [160], and hardware limitations and efficiency [161]. In the meantime, there has been significant efforts to apply GNNs to a wide range of practical, real-world challenges, such as dynamic graph-based problems [162, 163], [164], [165], attack and defense issues [166], biologically-based problems [167], [168], [169], [170], and recommender systems with billion-scale [171].

Without the loss of generality, GNNs can be classified into two categories: spectral-based approaches and spatial-based approaches  [16, 172]. With the help of the graph signal procession theory, new spectral-based GNNs are proposed by designing graph filters, while spatial-based GNNs aim at exploring the local topology by introducing novel aggregation or message-passing methods. Below, we give a brief review of representative GNNs according to this taxonomy.

Spectral-based GNNs are based on the graph signal processing theory [173], which mainly relies on the graph Fourier transformation, inverse graph Fourier transformation, and matrix transformation. The basic idea of spectral-based GNNs is to conduct convolution operations to the input data, so that the frequency within a certain range in the frequency domain is enhanced by graph filters.

Given a graph, its adjacency matrix is A and its node degree diagonal matrix is D, then the normalized graph Laplacian matrix is $\textbf{L}=\textbf{I}_{n}-\textbf{D}^{-1/2}\textbf{A}\textbf{D}^{-1/2}=\textbf{U}\mathbf{\Lambda}\textbf{U}^{T}$, where $\mathbf{\Lambda}$ and U are the diagonal matrix of eigenvalues and its corresponding matrix of eigenvectors. Since L is symmetric positive semi-definite, columns of U are orthogonal to each other and $\textbf{U}^{T}\textbf{U}=\textbf{I}$. We assume that there is a graph filter $\mathbf{g}$, then the graph convolution operation can be expressed as follows:

$$\textbf{x}*_{G}\textbf{g}=\mathcal{F}^{-1}(\mathcal{F}(\textbf{x})\odot\mathcal{F}(\textbf{g})),$$

(1)

where $*_{G}$ denotes the operator on graph $G$, $\odot$ represents element-wise product, the graph Fourier transformation $\mathcal{F}(\textbf{x})=\textbf{U}^{T}\textbf{x}$, and the inverse graph Fourier transformation $\mathcal{F}^{-1}(\mathbf{\hat{x}})=\textbf{U}^{T}\mathbf{\hat{x}}$. Therefore, the convolution operator can be rewritten as

	$\displaystyle\textbf{x}*_{G}\textbf{g}_{\theta}$	$\displaystyle=\textbf{U}(\textbf{U}^{T}\textbf{x}\odot\textbf{U}^{T}\textbf{g})$		(2)
		$\displaystyle=\textbf{U}\textbf{g}_{\theta}\textbf{U}^{T}\textbf{x},$		(3)

where $\textbf{g}_{\theta}=\text{diag}(\textbf{U}^{T}\textbf{g})$, and $\theta$ in $\textbf{g}_{\theta}$ represents learnable parameters in the graph filter $\mathbf{g}$. Note that the structure information A of a graph is contained in the matrix U.

In ChebNet  [136], the authors define a graph filter as $\sum_{k=0}^{K}\theta_{k}T_{k}(\mathbf{\tilde{\Lambda}})$, where $\mathbf{\tilde{\Lambda}}=2\mathbf{\Lambda}/\lambda_{max}-\textbf{I}_{n}$ and $T_{k+1}(\mathbf{\tilde{\Lambda}})$ is the Chebyshev polynomials, which is obtained by $T_{k+1}(\mathbf{\tilde{\Lambda}})=2\mathbf{\tilde{\Lambda}}T_{k}(\mathbf{\tilde{\Lambda}})-T_{k-1}(\mathbf{\tilde{\Lambda}})$. Therefore, the convolution operator in ChebNet is

	$\displaystyle\textbf{x}*_{G}\textbf{g}_{\theta}$	$\displaystyle=\textbf{U}(\sum_{k=0}^{K}\theta_{k}T_{k}(\mathbf{\tilde{\Lambda}}))\textbf{U}^{T}\textbf{x}$		(4)
		$\displaystyle=\sum_{k=0}^{K}\theta_{k}T_{k}(\mathbf{\tilde{L}}))\textbf{x},$		(5)

where $T_{k}(\mathbf{\tilde{L}}))=2T_{k}(\mathbf{L})/\lambda_{max}-\textbf{I}_{n}$. By applying the Chebyshev polynomial with different $K$ values, the model can capture information at different ranges of neighbors of a node in a graph.

Later, the graph convolutional network (GCN)  [135] was proposed to consider a special case of ChebNet when $K=1$ and $\lambda_{max}=2$. Thus, the graph filter is written as

$\displaystyle\textbf{x}*_{G}\textbf{g}_{\theta}=\theta(\textbf{I}_{n}+\textbf{D}^{-1/2}\textbf{A}\textbf{D}^{-1/2})\textbf{x}.$

(6)

However, the authors found out the GCN will be more stable by replacing $\textbf{I}_{n}+\textbf{D}^{-1/2}\textbf{A}\textbf{D}^{-1/2}$ with $\mathbf{\tilde{D}}^{-1/2}\mathbf{\tilde{A}}\mathbf{\tilde{D}}^{-1/2}$, where $\mathbf{\tilde{A}}=\textbf{A}+\textbf{I}_{n}$ and $\mathbf{\tilde{D}}$ is the diagonal degree matrix of $\mathbf{\tilde{A}}$. If we take a closer look at Eqn. (6), we can treat the graph filter as an aggregation of the mean information from one node’s neighbor ($\textbf{D}^{-1/2}\textbf{A}\textbf{D}^{-1/2}$) together with its own embedding ($\textbf{I}_{n}$), thus GCN establishes a bridge between spectral-based GNNs and spatial-based GNNs.

Recently, a large body of research has been proposed based on the idea of designing effective graph filters. In  [174], the authors proposed simple graph convolution (SGC) that contains a simple fixed low-pass filter, resulting in competitive experimental results, better scalability, and improved computational efficiency. Later, a novel frequency adaptive graph convolutional network (FAGCN)  [175] was proposed to learn both low-frequency and high-frequency signals.

Spatial-based GNNs aim to directly aggregate the neighbors’ information of each node to one node through edges, which can be considered as a message-passing strategy. Due to the intuitive characteristics of passing messages through edges, the spatial-based GNNs are more interpretable. Additionally, spatial-based GNNs do not need to solve the products and inverses of large matrices, making them efficient in handling large-scale graphs. Therefore, spatial-based GNNs have attracted more and more attention  [173].

In the message-passing neural network (MPNN)  [138], the authors propose a general framework of message-passing strategies, which is

$\displaystyle\textbf{m}_{v}^{t+1}=\sum_{u\in N(v)}M_{t}(\textbf{h}_{v}^{t},\textbf{h}_{u}^{t},\textbf{e}_{vu}),$

(7)

$\displaystyle\textbf{h}_{v}^{t+1}=U_{t}(\textbf{h}_{v}^{t},\textbf{m}_{v}^{t+1}),$

(8)

where $h_{v}^{t}$, $h_{u}^{t}$, $e_{vu}$ is the embedding of node $v$, node $u$ in the $t$-th hidden layer, and edge features, respectively. $N(v)$ means a set of neighbors of node $v$. $m_{v}^{t+1}$ represents the aggregated embedding, encompassing the embeddings of neighbors and their corresponding edges. $M_{t}$ and $U_{t}$ are two learnable matrices in the $t$-th hidden layer, which are optimized through the back-propagation. In general, operators in Eqn. (7) are considered as aggregation by the propagation of information among neighbors, while Eqn. (8) combines information aggregated from neighbors and target nodes by generating a new embedding vector. Various GNNs are proposed with different aggregation or combination methods. The spatial-based GNNs aggregate information from neighbors in an iterative way, that is, $T$-hop information could be obtained by a GNN with $T$ hidden layers. However, research has found that over-smoothing may occur when the number of hidden layers is large, caused by the similarity of embeddings of all nodes in a graph after long-distance neighbor information exchange. Multiple methods have been introduced to address this issue, such as skip connections  [176], neighborhood re-definition  [177], and adaptive aggregation  [178], just to name a few.

Later, GNNs based on attention methods emerged as highly promising approaches, drawing considerable interest within the research community. Different from other aggregation methods, attention mechanisms assume that the importance of neighbor nodes to a target node is different, which should be decided by both the target node and its corresponding neighbor nodes. By learning the importance of nodes, attention-based methods gain promising results in many fields  [179, 180]. Here, we introduce a piece of pioneer work, graph attention network (GAT)  [139], which has served as a baseline of several follow-up work. The embedding of node $v$ in the $(t+1)$ hidden layer is updated as

$$\textbf{h}_{v}^{(t+1)}=\sigma\left(\sum_{u\in N(v)\cup v}\alpha_{vu}^{(t+1)}{W}^{(t+1)}\textbf{h}_{u}^{(t)}\right),$$

(9)

where $\textbf{h}_{u}^{(t)}$ and $\textbf{h}_{v}^{(t+1)}$ are the embedding vectors of the neighbors in the $t$-th hidden layer and the target node in the $(t+1)$-th hidden layer, respectively. $\sigma$ is the activation function, ${W}^{(t+1)}$ is a learnable matrix in the $(t+1)$-th hidden layer, and $\alpha_{vu}^{(t+1)}$ is the attention parameter of node $u$ to node $v$, which can be obtained as follows,

$$\alpha_{vu}^{(t+1)}=\frac{\exp(\text{LeakyReLU}(\textbf{a}^{T}[{W}^{(t+1)}\textbf{h}_{v}^{(t)}||{W}^{(t+1)}\textbf{h}_{u}^{(t)}]))}{\sum_{w\in N(v)\cup v}\exp(\text{LeakyReLU}(\textbf{a}^{T}[{W}^{(t+1)}\textbf{h}_{v}^{(t)}||{W}^{(t+1)}\textbf{h}_{w}^{(t)}]))},$$

(10)

where a is a learnable vector. To generate several $\alpha_{vu}^{(t+1)}$ at one time by learning different ${W}^{(t+1)}$, GAT can be expanded to multi-head GAT, enhancing the expression ability of original one. Following this work, many other definitions and calculation methods have been proposed to express different importance degrees of nodes’ neighbors  [181, 182, 183].

Supervised learning stands out from other learning algorithms primarily due to the presence of labeled samples, which is used to enable GNNs to learn a mapping from input data to corresponding output labels. The loss function is usually designed to minimize the discrepancy between the predicted output generated from GNN models and true labels. For COPs with known solutions, supervised learning can be employed as a training method. The most commonly used loss functions include the Mean Squared Error (MSE) for graph regression tasks, logistic loss for binary classification tasks, and cross-entropy loss for multilabel classification tasks. Additionally, custom loss functions may be developed to consider specific characteristics of COPs, thereby further enhancing the solution’s effectiveness and efficiency.

When a GNN is adopted for combinatorial optimization, a supervised learning algorithm can be used to achieve an end-to-end GNN model for one-shot solution prediction  [184], often in combination with a local search method or a heuristic search method. In [185, 184], the authors apply GNNs with supervised learning to assess the existence of solutions under specific constraints for TSP and GCP, respectively. Authors in [186, 187, 188] use supervised learning to solve graph-matching problems. Since the solutions generated by GNNs in an end-to-end fashion may not be feasible or may be sub-optimal, some local or heuristic search methods are usually used to fine-tune or promote the quality of the obtained solutions. For example, in TSP, the output of GNNs will be a probability matrix instead of a solution, and therefore, a beam search can be conducted [189].

Different from supervised learning, unsupervised training of GNNs aims to uncover the structure of data without pre-defined labels, making it suitable for multimodel COPs with multiply optimal solutions and those COPs with no available ground-truth labels. The choice and design of loss function is crucial since it serves as a guiding metric, influencing the learning direction of GNNs. Therefore, various loss functions have been proposed to reveal the structure of a given graph and generate representations of nodes. Most recently, self-supervised learning, a specific extension of unsupervised learning, can generate labels from the input data itself, enabling GNNs to be trained on pretext tasks to obtain node representations or conduct transfer learning [190].

When GNNs are employed for combinatorial optimization, unsupervised GNNs are often conducted in an end-to-end fashion to generate solutions by minimizing a loss functions, reflecting the characteristics of a given problem. For instance, many ideas of solving GCPs apply unsupervised learning to obtain color assignments. In [191], the authors utilized a margin loss to minimize the difference between a threshold and node-pair distance. In [192], a loss function inspired by the Potts model in statistical mechanics is proposed, aiming to maximize the probability difference between two connected nodes. Furthermore, the authors promote the performance of GNNs by introducing self-information to the loss function [193]. Besides, unsupervised GNNs are also exploited to solve other COPs, such as constraint satisfaction problems [194, 195, 196] and non-periodic 2D tiling problems [197].

Reinforcement learning (RL) has gained significant popularity recently in the machine learning community, due to its ability in handling dynamic environments. In RLs, agents continually refine their policies through interactions with the environment, learning to adapt based on the outcomes of their actions. When integrated with GNNs, with the current state represented by existing solutions or their segments and GNNs modeling the agents. Reward functions in reinforcement learning are crucial as they guide models to optimize the policy. They define ’good’ behavior within the context of a given problem, directing the learning process by specifying the desirable outcomes for which the agent should strive.

GNNs, when combined with RL, are particularly well suited for solving COPs that require sequential decision-making. Therefore, this makes them apt for addressing path and routing problems, as presented in many work [198, 199]. For instance [200], S2V-DQN learns the policy of selecting each city iteratively when solving TSPs, thus constructing a feasible solution sequentially. On the other hand, an RL-based GNN model, ANYCSP [201], is applied to solve constraint satisfaction. Unlike constructing a complete solution through multiple steps, this model generates one solution in each action and outputs the best result among all steps.

Traditional methods for solving COPs can be roughly categorized into two classes: exact solvers [203, 204, 205] and approximate solvers [206, 207, 208]. For exact solvers, it is guaranteed that the global optimal solutions to an input problem will be found by the optimization algorithm. Branch-and-bound [209], branch-cut-and-price [210] and their variants are usually adopted by exact solvers as the general framework for solving COPs, which can achieve a good performance in small-scale problems with an integer programming formulation. Exact solvers branches the original problem into subproblems and bounds the objective values in order to explore feasible solutions in the search space. Specifically, the algorithm first initializes the lower bound and the upper bound of the objective function. Then based on a set of selected variables, multiple subproblems are generated by splitting the search space into distinct regions. Concorde ¹¹1https://www.math.uwaterloo.ca/tsp/concorde.html is an representative solver customized for routing problems. It has been adopted to calculate the optimal solutions to all 110 benchmark problems in TSPLIB [211] where the largest instance has 85,900 cities. Concorde combines both branch-and-bound and cutting-plane methods to reduce the search space and solve linear programming relaxations of the original problem iteratively. Different from Concorde which is specifically designed for TSP-like routing problems, Gurobi ²²2https://www.gurobi.com/ is a generic solver for any forms of COPs including integrality constraints, linear constraints, bound constraints and quadratic constraints. It has interfaces with different programming languages and platforms such as Python and MATLAB. The Gurobi interface allows the user to pass the customized optimization model into the solver and obtain the result without changing the environment.

However, the exponential time complexity hinders the application of exact solvers under certain circumstances. Since exact algorithms typically lead to an extremely large computing overhead, it is non-trivial to use exact solvers for combinatorial optimization as the scale of problems continues growing, even though the algorithm is guaranteed to produce an optimal solution in a finite period of time. For example, it may take an exact solver several months or even years to find the ideal solution to some difficult problems, whereas real-time decision makers typically demand answers within a few minutes. By contrast, approximate solvers are often significantly faster than exact solvers since they do not require an exhaustive search over all feasible solutions, especially for large-scale problems. Approximate solvers tackle COPs by using metaheuristic and heuristic algorithms such as genetic algorithms [212], simulated annealing [213], greedy algorithms [214] and local search methods [215]. Although such kind of methods cannot guarantee to find the global optimum, they are able to explore the search space efficiently and obtain near-optimal solutions within a reasonable amount of time, which are preferable in real-time decision-making tasks. Lin-Kernighan Heuristic (LKH) [216, 207, 206] is a representative approximate solver designed for solving the TSP and its variants. Since the objective of the TSP is to determine a shortest route for a travelling salesman to visit each city once and return to the starting city, the search space of possible solutions grows exponentially with the number of cities. Approximate solvers based on LKH solve the TSP by iteratively updating an initial solution with local search. During each optimization step, small modifications based on the previous solution are made to improve its quality. OR-Tools ³³3https://developers.google.com/optimization is an open-source project developed by Google with the primary aim of providing efficient tools and algorithms to solve operational research problems that arise in a wide range of applications of the company. Over years of development and expansion, OR-Tools integrates a variety of algorithms including both exact and approximate solvers for solving different types of COPs in vehicle routing, scheduling, flows and bin packing. It supports researchers and developers to model a problem in different programming languages and solve it with the provided commercial or open-source solvers such as CPLEX, Gurobi, SCIP and GLPK.

Traditional methods, including exact solvers and heuristic algorithms, used to be the foundation of optimization methods in practical applications. Nevertheless, with the continuous development of data science and the remarkable surge in computational power, machine learning techniques have recently emerged as a compelling paradigm for efficiently addressing optimization problems, in particular COPs. Compared to the traditional approaches mentioned above, machine learning methods show significant advantages in several key factors that contribute to the popularity and effectiveness of machine learning in solving COPs.

Learning from data. Firstly, machine learning methods have the ability to learning from historical data, while traditional methods solve each problem independently. It should be noted that many COP instances share the same problem structure and differ only in the parameter values. Learning from historical data enables the machine learning methods to extract generic paradigms from various problem instances, leading to solutions that are more likely to be optimal for unseen instances. On the contrary, traditional methods often rely on predefined rules based on the given case, which may not be universally optimal for all problems.

Handling large-scale problems. Secondly, machine learning performs efficiently when dealing with large-scale optimization tasks. Handling large-scale problems is a challenging task that often requires efficient algorithms and scalable approaches. Exact solvers may face computational limitations when dealing with large-scale problems due to their exponential time complexity. On the other hand, population-based heuristics always struggle with parallelization and a huge computational cost, since a huge amount of candidate solutions need to be evaluated to find an optimal one. Machine learning methods based on deep neural networks can leverage the power of parallel computing, leading to a faster training process and less inference time. The advantage is crucial when dealing with massive amount of data in large-scale problems.

Adaptation to dynamic environments. Thirdly, machine learning methods can flexibly adapt to dynamic environments. Machine learning models can be continuously retrained as new data becomes available, allowing them to adapt to dynamic environments. As new data is continuously integrated into the learning process, machine learning models can not only maintain relevance but also enhance their performance over time. This adaptability is particularly useful in scenarios where the optimization problem changes in a dynamic environment, since the model is able to remain effective and capable of providing optimal or near-optimal solutions across various states of the optimization environment. By contrast, exact solvers and heuristic algorithms with fixed rules and predefined strategies may lack adaptability in dynamic environments.

Providing uncertainty with solutions. Finally, machine learning models can provide different solutions with probabilities, while traditional solvers only generate deterministic solutions. In certain optimization scenarios where uncertainty and variability are inherent, the deterministic solutions may not be suitable and prove less effective. On the contrary, machine learning models that can provide uncertainty measures along with solutions offer insights into the likelihood of different outcomes to the decision-maker. As a result, the probabilistic solutions accommodate the complexities of uncertain environments by providing a more informed and adaptive decision-making framework.

Early work on solving graph-based COPs via machine learning techniques can be traced back to the pointer network [217], where an encoder-decoder structured model based on recurrent neural networks (RNNs) is proposed to generate feasible permutations for Euclidean TSP instances. An RNN model, known as the encoder, is employed to process all nodes in the input graph of a TSP instance. The encoder generates vector representations as the encoding for each node. Subsequently, the decoder, which is also implemented as an RNN model, takes the encoded node vectors as the inputs and performs an attention mechanism to generate probability distributions over the input nodes via a softmax layer. The point network can output a feasible permutation for the input TSP instance through an iterative decoding process. The encoder-decoder model is trained in a supervised learning manner with pre-calculated instance-solution pairs as labeled data. Similarly, Bello et al. [218] train an RNN model via reinforcement learning to make predictions on the distribution of city permutations in the TSP. The negative value of the tour length is adopted as a reward signal, and the parameters of the RNN model are optimized using the active search as a policy gradient method. In contrast to the pointer network, this work adopts reinforcement learning to avoid the need for optimal solutions during the training process. However, early work on graph-based COPs is built upon the seq2seq [219] framework and uses sequential data as the input, which fails to effectively leverage the inherent graph structure characteristics of the original problems. Additionally, the sensitivity to the input order usually has a negative influence on the model performance, especially when the problem has multiple optimal solutions such as the symmetric TSP.

Non-autoregressive methods have attracted increasing attention in solving graph-based COPs, given their ability to generate solutions simultaneously without depending on sequential processing. In contrast to autoregressive methods that generate each solution step-by-step, non-autoregressive methods show significant advantages in terms of the inference speed and search efficiency. Li et al. [225] combined graph convolution networks with classic heuristics to predict solutions for several NP-hard problems including maximal independent set, minimum vertex cover, maximal clique and satisfiability problems. A GCN model is trained to make predictions on the likelihood of each vertex belonging to the optimal solution to the input graph. To avoid generating a diffuse likelihood map when there are multiple optimal solutions to an input graph, a hindsight loss [240] is adopted in the training process to fit a diverse collection of solutions. Nonetheless, it also suffers from the limitation of poor generalisation, as local search accounts for a large part of the model performance. Nowak et al. [226] investigated how a GNN model can make predictions on unseen quadratic assignment problems by learning from a set of solved cases. Liu et al. [77] proposed a learning-based method based on residual gated graph convolutional networks [241] to predict Pareto optimal solutions for multi-objective facility location problems (MO-FLPs) in an end-to-end manner. The MO-FLP is transformed into a bipartite graph representation with two distinct sets of nodes denoting the candidate facilities and customers, respectively. Two GNN models are trained cooperatively with labeled data to learn the latent space embeddings for the nodes and edges of the input graph. Finally, a set of non-dominated solutions are sampled from the output probability distribution to approximate the ground-truth Pareto set of the problem.

Another branch of work focuses on solving routing problems such as the TSP and its variants in a non-autoregressive fashion. Joshi et al. [189] trained a residual gated graph convolutional network that takes a TSP graph as the input and outputs a heat map of probabilities at which each edge belongs to the shortest path (the optional solution). The algorithm introduces additional edge features as part of the input graph information, and adds residual connections for each graph convolutional layer to alleviate the over-smoothing issue occurred in the vanilla GCN. Finally, the edge representations of the last message-passing layer are passed through a multi-layer classifier to output the probability of each edge occurring in the final solution. During the prediction, beam search is adopted to convert the adjacency matrix into a set of feasible TSP tours in parallel. Despite the effectiveness, the model in [189] can only be trained and evaluated on TSP instances with a fixed number of cities. To generalize a trained model to larger cases than it has seen before, Fu et al. [228] expanded the work in [189] and proposed to reuse a model pre-trained on small-scale cases to predict heat maps on arbitrary large cases. During the preliminary stage, a large number of TSP instances with a fixed number of $m$ cities are randomly generated together with the optimal solutions computed by an exact solver Concorde. The case-solution pairs serve as the labeled dataset to train a graph convolutional residual neural network with attention mechanisms (Att-GCRN) via supervised learning. Once trained, the Att-GCRN model can predict the heat map for any given TSP instances with $m$ cities. During the prediction stage, an arbitrary large instance is first divided into a set of sub-graphs, each of which has exactly $m$ cities. The pre-trained Att-GCRN model generates heat maps for all sub-graphs. Then all heat maps are merged into a complete heat map via graph fusion. Finally, Monte Carlo tree search (MCTS) is adopted to search optimal solutions based on the merged heat map. Different from other MCTS-based approaches where the solution is built step-by-step, the model in [228] generates the solution non-autoregressively in which each state in MCTS is a complete tour of the TSP.

Many highly-structured combinatorial optimization tasks can be considered as a sequential decision-making process on graphs [189], where each node in a graph represents distinct decision points related to the original task, and the edges encapsulate the relationships between these decisions. The sequential nature of the optimization process becomes evident in this scenario. The selection on each node is no longer independent of each other as in the non-autoregressive methods. By contrast, the decision on one node will directly impact the feasible choices of all subsequent nodes, implying the interdependence and constraints inherent in the original problem. An intuitive example of such kind of sequential decision constraints could be found in the classic TSP. In a single-objective TSP task where the target is to minimize the total travelling distance by visiting each city once, the decision of the salesman at each step on which city to visit next will definitely affect all subsequent decisions as well as the final result. Actually, the choice of the departure city does not matter in the TSP, since the salesman will return to the starting city after traversing all cities. What really matters is the decision made on each step within the construction process of the entire route. When deciding on which city to visit next, an intuitive method is to simply select the nearest neighbouring city to the current location among all unvisited cities at each step. Such a greedy selection strategy can construct a feasible solution efficiently, and each choice appears to be optimal for the current state. Taking the global optimization target into consideration, however, such a greedy strategy may not be preferable for the TSP. Locally optimal decisions are made based on the current state at each step, without considering the subsequent implications. As a result, for TSP instances with specific city distributions, the partial solution constructed in the early stage may have a short travelling distance. But it may also lead to a situation where the travelling cost between each pair of the unvisited cities remains very large, resulting in an overall travelling distance that is far from the global optimal solution. A similar situation also occurs in non-routing problems such as the graph colouring problem. Since the aim of GCPs is to assign colors to the nodes of a graph in such a way that no two adjacent nodes share the same color, the decision made at each node regarding the assignment of colors will significantly influence all subsequent decisions, creating a ripple effect throughout the entire graph. Therefore, it is crucial to consider the common scenario of sequential decision restrictions when designing autoregressive methods for COP tasks.

This sequential decision-making process fits seamlessly with machine learning models that have advantages in capturing temporal dependencies and complicated relationships within sequential data. Dai et al. [200] encoded TSP instances as graph-structured data by using GNNs, which can capture the graph topology information of the original problem. The usage of GNN models can also better reflect the order-independent properties of TSP routes compared to the previous sequence-to-sequence models [217, 218]. Motivated by the situation in which most optimization instances share the same problem structure and differ in the data only, the authors in [200] proposed a machine learning framework which can automatically learn heuristic algorithms for graph-based combinatorial optimization tasks by combining reinforcement learning and graph embedding techniques. A graph embedding model is trained to determine actions at each step according to the current state of the input graph, and the learned policy serves as a meta-algorithm to construct solutions to the input problem incrementally. The proposed framework can generate feasible solutions in an autoregressive manner based on the graph structures, by adding nodes to the partial solution step by step. The structure2vec model [242] is adopted as the graph embedding network, and the model is trained to capture the neighborhood information of each node in the context of graphs. A greedy policy is parameterized by the structure2vec model and trained via Q-learning strategies. Deudon et al. [229] extended the neural combinatorial optimization framework in [218] for an efficient conjunction with traditional heuristics to solve COPs. Instead of using the Long Short-Term Memory (LSTM) architecture, attention mechanisms are adopted to design the critic for the model. The proposed framework in [229] follows an encoder-decoder structure, where the encoder maps the inputs to a set of continuous representations, and the decoder generates an output sequence in an autoregressive manner. At each step, the previously generated outputs are taken as additional inputs to the model, enabling the consideration of past information in the decision on the next city to visit. Similar to the work in [243], the action vectors are encoded as a set based on neural attention mechanisms instead of a sequence as in [218]. An algorithm similar to the autoregressive methods was proposed by Kool et al. [198], where graph attention networks [139] are adopted instead of the structure2vec model. The encoder-decoder model is built upon attention mechanisms and trained via REINFORCE with a greedy rollout baseline. A TSP instance is formulated as a fully connected graph with $n$ nodes, where $n$ is the number of cities. A feasible solution is defined as a permutation of all nodes, and a stochastic policy is defined by the attention-based encoder-decoder model to select solutions for any given TSP instances. Specifically, the encoder model is trained to generate the embeddings of all nodes simultaneously, while the decoder produces a sequence by outputting one node at a time. For each step, the decoder model takes not only the node embeddings produced by the encoder, but also a problem-specific mask and the corresponding context of the current state. The mask implies the set of cities that have been visited, and the context consists of information about the first and last node existing in the partial route. With a single set of hyperparameters, the model can adapt to different optimization problems with flexibility, advocating the possibility of learning strong heuristics for a wide range of COP tasks with GNNs.

Many other machine learning technologies are always combined with GNNs to further improve the model performance on various graph-based tasks.

GNNs with unsupervised learning. Generally, the reliance on labeled data in supervised learning-based methods may hinder their ability to generalize to large-scale problems [230, 244]. Training a deep neural network model with good performance often needs a large amount of optimal solutions, which can indeed be calculated by well-established solvers for conventional optimization problems. But for many real-world optimization tasks that have no exact mathematical formulations, it is non-trivial to collect a large number of optimal solutions for training a GNN model. To solve this problem, Wang et al. [245] proposed an unsupervised learning framework based on the relaxation-plus-rounding strategy. The relaxed solutions are parameterized by a GNN model, which can be trained in an end-to-end manner via the back-propagation process. Since the direct evaluation of many practical combinatorial problems could be expensive and time-consuming, an alternative way is to learn a proxy of the original objective function first, and then optimize the proxy instead of the original objective in order to reduce the evaluation overhead.

GNNs with reinforcement learning. Since training a GNN model with a sufficient amount of labeled data could be non-trivial in some scenarios where the objective functions of COPs are hard to be optimized, another solution is to train the model by trials and errors without the assistance of any labeled data. Actually, reinforcement learning is about training an agent to make a sequence of decisions in an environment to maximize the cumulative reward it receives over time. The agent learns from its experiences by updating a policy or value function based on the received rewards. Due to the inherent problem structures, many COPs can be considered as graph-based sequential decision problems. Therefore, it is a natural choice to train GNN models with reinforcement learning techniques while solving COPs. For example, Dai et al. [200] trained a graph embedding model with the DQN method in reinforcement learning to determine the order of nodes while constructing a sequential solution. Deudon et al. [229] used policy gradient to train an attention-based model in order to predict the distribution over city permutations for TSP tasks. Kool et al. [198] trained the encoder-decoder model by using the REINFORCE algorithm to build TSP solutions incrementally. Their performance on different COP tasks achieves state-of-the-art results with reinforcement learning technologies.

GNNs with contrastive learning. Due to the advantages of data efficiency, generalization ability and versatility, contrastive learning has emerged as a new paradigm in dealing with various machine learning-based tasks [246, 247]. Generally, contrastive learning is a training method designed to enable machine learning models to learn principal features from data without the distraction of other irrelevant information. The aim is to help the model better understand the differences between samples by contrasting them, in order to learn more meaningful representations. The loss function of contrastive learning is designed to minimize the distance between similar data pairs while maximizing the distance between different data pairs. This property of not relying on labelled data has proven effective in various domains including graph representation learning. Duan et al. [196] introduced contrastive learning into combinatorial problems and investigated the effect of augmentation design of pre-training for SAT problems. The authors advocated that label-preserving augmentations can lead to better performance on combinatorial problems than the label-agnostic augmentations [248, 249].

COPs typically involve interactions and constraints among multiple decision variables, where graph structures provide an intuitive and effective means to depict these relationships. In such graphs, nodes represent variables or components of the problem, while edges indicate the relationships or constraints between these variables.

GNNs are good at leveraging the inherent structures within graphs to enhance the resolution of COPs. By aggregating features from neighboring nodes, GNNs are capable of dynamically updating node states, which is fundamental in solving problems such as the shortest path, where the goal is to find the most efficient route between nodes [250]. Similarly, for constructing minimum spanning trees, GNNs simulate edge weights to create optimal connectivity with minimal costs [251, 252]. Additionally, in scenarios where understanding node importance is crucial, such as in network analysis, GNNs provide estimations of node centrality, helping identify key nodes based on their structural roles in the graph [253].

Moreover, GNNs provide advanced data representation capabilities that facilitate efficient problem-solving in areas requiring high accuracy in node and edge representations, such as coloring and matching problems. By using machine learning architectures like Sinkhorn, CMPNN, and SGMNet, GNNs enhance image coloring tasks and semantic matching challenges by learning and applying precise node embeddings, which significantly improve the quality and efficiency of the solutions [184, 192, 191, 254].

By modeling COPs as graph structures of nodes and edges, GNNs can leverage their profound capabilities in various applications, such as pattern recognition, network dynamics, and predictive analytics. We give some examples on how GNNs integrate complex relational data with advanced algorithms to optimize network operations, resource management, decision-making processes, and so on. These examples showcase the versatility and power of GNNs in extracting meaningful solutions from graph structures, contributing significantly to advances in various fields.

Complex network optimization. In the realm of complex network optimization, GNNs excel by integrating relational information among nodes with sophisticated algorithms. For the TSP, an NP-hard problem that seeks the shortest possible route visiting a set of locations and returning to the origin, GNNs combine node relational data with heuristic algorithms or reinforcement learning to find optimal solutions [200, 185]. Similarly, in network flow problems such as maximum flow and minimum cut, GNNs deploy flow classification and node deployment techniques to manage and optimize the flow of resources through a network, ensuring optimal operation and resource utilization [255, 256, 257].

Resource management and scheduling. Resource management and scheduling are critical in many industrial and technological environments where efficient resource allocation and timing are key to operational success. In resource allocation scenarios, GNNs model each resource as a node and optimize allocation by analyzing dependencies and exclusions represented as edges [258, 259]. For job shop scheduling, where tasks must be efficiently assigned to production units, GNNs use Markov decision processes to determine optimal task arrangements and scheduling, thereby enhancing productivity and reducing operational delays [260, 261].

Social network analysis and decision-making. In social network analysis, GNNs are particularly effective in solving problems like the minimum vertex cover and the minimum dominating set by transforming these challenges into node classification tasks. This method involves labeling each node based on whether it belongs to a particular set and using GNNs to predict these classifications based on the nodes’ features and their relationships. This approach allows for efficient and accurate decision-making, leveraging the structural information within the network to solve these graph-based decision problems [262].

GNNs effectively utilize the intrinsic structure of graphs (nodes and edges) to capture the topological and relational features necessary for solving various types of COPs. By designing appropriate graph representations that reflect the inherent features and constraints of each problem, GNNs can deliver robust and efficient optimization solutions across a variety of domains, from routing and scheduling to resource allocation and beyond. Additionally, by tailoring graph representations to reflect the specific features and constraints of each problem, GNNs optimize performance and enhance decision-making processes.

The use of graph representation in conjunction with GNNs marks a significant advance in tackling COPs. By structuring problems as graphs, GNNs leverage their inherent strengths to capture and analyze complex relational data effectively. This graph representation of COPs ensures that GNNs can adaptively respond to changes in problem data and structure, offering dynamic solutions that traditional methods cannot easily provide. As a result, the application of graph representation not only enhances the accuracy and efficiency of solving COPs but also expands the possibilities for innovation in optimization strategies across diverse industries. By capitalizing on the capabilities of GNNs through graph representations, we unlock new potential in computational optimization, setting the stage for solving some of the most challenging problems across various applications.

In many areas of combinatorial optimization, problems inherently lack graph structures, which means traditional relationships and interactions among elements are not depicted as nodes and edges [263, 264]. To leverage the capabilities of GNNs for these non-graph structured COPs, it is crucial to first convert these problems into equivalent graph structures. This transformation is essential for effective feature learning, problem modeling, and solution derivation by GNNs.

The conversion process involves a detailed mapping where each entity, resource, or constraint within the problem is represented as a node, and all relationships, including dependencies, exclusions, and priorities, are represented as edges. This reconfiguration translates the abstract elements of the problem into a graph format, laying down a solid foundation for further analysis of GNNs. Through this transformation, GNNs can effectively capture the complexities and correlations inherent in the problem, offering novel insights and resolution methods.

The process of transforming non-graph structured COPs into graph formats significantly enhances the application of GNNs. This equivalent conversion allows GNNs to exploit their graph-based analytical capabilities effectively, leading to more robust and scalable solutions. The following examples demonstrate how different types of COPs can be converted into graph structures, each benefiting from the unique insights and optimizations that GNNs provide.

Two-tier knapsack problem. This problem is structured as a tripartite graph that includes leader nodes, follower nodes, and constraint nodes, which represent different categories of items with attributes like profit and weight. This detailed graph-based configuration facilitates the management of hierarchical relationships and simultaneous constraints, making it easier to optimize decisions that affect multiple layers of the problem simultaneously. The graph structure helps in visualizing and tackling the dependencies and interactions between different groups, enhancing the efficiency of resource allocation and optimization [265, 266].

Set covering problem (SCP). In converting the SCP to a graph, universe nodes represent all possible elements that need coverage, element nodes correspond to specific items requiring coverage, and subset nodes represent the groups or sets that can provide this coverage. This transformation into a graph simplifies the visualization and management of the complex relationships between sets and their elements, thereby facilitating more effective coverage strategies and cost management. It enables the GNN to assess and optimize the coverage dynamically, ensuring that the most cost-effective and comprehensive solutions are identified [267, 268].

Cutting stock problem. Here, the transformation involves modeling the variables and constraints of the problem as nodes and edges in a graph, respectively. This setup allows GNNs to optimize the column generation method by predicting and evaluating potential solutions in the form of subgraphs. The graph-based approach enhances the algorithm’s efficiency by structuring the data in a way that is inherently suitable for the iterative and combinatorial nature of GNNs, thus improving the precision and speed of finding optimal cutting patterns [269].

Boolean satisfiability problem (SAT). The SAT can be transformed into a directed acyclic graph that represents the logic circuit of the problem, with nodes designated for inputs and various logic gates. This graph structure allows GNNs to apply sophisticated algorithms tailored to logic processing, effectively solving the SAT by analyzing the circuit’s logical flow and gate interactions. The graph-based approach not only simplifies the representation of complex logical relationships but also enhances the ability of GNNs to perform logic simplification and optimization tasks [270, 271, 272].

These examples underscore the adaptability and effectiveness of GNNs when dealing with various complex COPs transformed into graph formats. Then, by leveraging graph representations, GNNs can unlock new potentials in computational optimization, offering solutions that are both innovative and directly aligned with the inherent structure of each problem. This capability highlights the critical role of graph Conversion in broadening the applicability of GNNs across diverse industries and complex optimization challenges.

Converting non-graph structured problems into graph formats significantly enhances the capabilities of GNNs, extending their applicability and increasing their problem-solving effectiveness. This equivalent conversion allows GNNs to utilize their advanced graph-based learning algorithms, which excel in capturing and analyzing intricate relationships and structures within the data. Consequently, GNNs can perform detailed feature extraction, delve into the deeper dynamics of the problem, and generate insights that inform more effective and often more innovative solutions.

These GNN-based approaches are particularly advantageous in complex optimization scenarios where traditional algorithms may struggle to process and interpret the underlying relationships and constraints effectively. By visualizing and manipulating problems as graphs, GNNs can better analyze the impact of various elements and their interconnections, leading to solutions that are not only optimal but also more comprehensible in terms of strategic decision-making. Additionally, this conversion enhances computational efficiency, enabling faster processing and analysis, crucial in scenarios involving large datasets or requiring real-time decision-making.