Traveling salesman problem

This web page is a duplicate of https://optimization.mccormick.northwestern.edu/index.php/Traveling_salesman_problems

Author: Jessica Yu (ChE 345 Spring 2014)

Steward: Dajun Yue, Fengqi You

The traveling salesman problem (TSP) is a widely studied combinatorial optimization problem, which, given a set of cities and a cost to travel from one city to another, seeks to identify the tour that will allow a salesman to visit each city only once, starting and ending in the same city, at the minimum cost.1

History

Solution to 48 States Traveling Salesman Problem

The origins of the traveling salesman problem are obscure; it is mentioned in an 1832 manual for traveling salesman, which included example tours of 45 German cities but gave no mathematical consideration.2 W. R. Hamilton and Thomas Kirkman devised mathematical formulations of the problem in the 1800s.2

It is believed that the general form was first studied by Karl Menger in Vienna and Harvard in the 1930s.2,3

Hassler Whitney, who was working on his Ph.D. research at Harvard when Menger was a visiting lecturer, is believed to have posed the problem of finding the shortest route between the 48 states of the United States during either his 1931-1932 or 1934 seminar talks.2 There is also uncertainty surrounding the individual who coined the name “traveling salesman problem” for Whitney’s problem.2

The problem became increasingly popular in the 1950s and 1960s. Notably, George Dantzig, Delber R. Fulkerson, and Selmer M. Johnson at the RAND Corporation in Santa Monica, California solved the 48 state problem by formulating it as a linear programming problem.2 The methods described in the paper set the foundation for future work in combinatorial optimization, especially highlighting the importance of cutting planes.2,4

In the early 1970s, the concept of P vs. NP problems created buzz in the theoretical computer science community. In 1972, Richard Karp demonstrated that the Hamiltonian cycle problem was NP-complete, implying that the traveling salesman problem was NP-hard.4

Increasingly sophisticated codes led to rapid increases in the sizes of the traveling salesman problems solved. Dantzig, Fulkerson, and Johnson had solved a 48 city instance of the problem in 1954.5 Martin Grötechel more than doubled this 23 years later, solving a 120 city instance in 1977.5 Enoch Crowder and Manfred W. Padberg again more than doubled this in just 3 years, with a 318 city solution.5

In 1987, rapid improvements were made, culminating in a 2,392 city solution by Padberg and Giovanni Rinaldi. In the following two decades, David L. Appelgate, Robert E. Bixby, Vasek Chvátal, & William J. Cook led the cutting edge, solving a 7,397 city instance in 1994 up to the current largest solved problem of 24,978 cities in 2004.5

Description

Graph Theory

Let $G=(V,E)$ be a directed or undirected graph with set of vertices $V$ and set of edges $E=\{(x,y)|x,y\in V\}$ .3,6 Each edge $e\in E$ is assigned a cost $c_{e}$ . Let $\mathbb {H}$ be the set of all Hamiltonian cycles, a cycle that visits each vertex exactly once, in $G$ .6 The traveling salesman problem is to find the tour $h\in \mathbb {H}$ such that the sum of the costs $c_{e}$ in the tour is minimized.

Suppose graph $G$ is a complete graph, where every pair of distinct vertices is connected by a unique edge.6 Let the set of vertices be $V=\{v_{1},v_{2},...,v_{n}\}$ . The cost matrix is given by $C=(c_{ij})_{n\times n}$ where the cost of the edge joining node $v_{i}$ to node $v_{j}$ , denoted $c_{ij}$ , is given in entry $(i,j)$ .

In the context of the traveling salesman problem, the verticies correspond to cities and the edges correspond to the path between those cities. When modeled as a complete graph, paths that do not exist between cities can be modeled as edges of very large cost without loss of generality.6 Minimizing the sum of the costs for Hamiltonian cycle is equivalent to identifying the shortest path in which each city is visiting only once.

Classifications of the TSP

The TRP can be divided into two classes depending on the nature of the cost matrix.3,6

Symmetric traveling salesman problem (sTSP) - $C$ $C$ is symmetric ( $\forall ~i,j:c_{ij}=c_{ji}$ $\forall ~i,j:c_{ij}=c_{ji}$ )
- $G$ is undirected
- Applies when the distance between cities is the same in both directions
Asymmetric traveling salesman problem (aTSP) - $C$ $C$ is asymmetric ( $\exists ~i,j:c_{ij}\neq c_{ji}$ $\exists ~i,j:c_{ij}\neq c_{ji}$ )
- $G$ is directed
- Applies when there are differences in distances (e.g. one-way streets)

An ATSP can be formulated as an STSP by doubling the number of nodes.6

Variations of the TSP

MAX TSP: Rather than minimizing the tour cost, the MAX TSP seeks to identify a tour in which the total cost of edges is maximimum.6 The MAX TSP can be refomulated as a TSP by replacing each edge cost with its addivitive inverse.6
Bottleneck TSP: Objective is to locate a tour with the minimum cost for the edge with the largest cost, which can be reformulated as a TSP with exponentially large edge costs.6
TSP with mutiple visits (TSPM): A relaxation of the TSP problem, the TSPM attempts to route a salesman such that each node is visited at least once and the total travel distance is minimized.6 Reformulations into a TSP involves replacing edge costs with the shortest path distance.6
Messenger problem: Also known as the wandering salesman problem, this problem seeeks to find the Hamiltonian path of least cost from specified node $u$ to node $v$ in $G$ .6 The TSP reformulations incorporates a large negative cost for the edge $(v,u)$ .
Clustered TSP: The set of vertices $V$ in $G$ are parititioned into clusters $V_{1},V_{2},...,V_{k}$ .6 The objective is to identify the least cost tour such that cities within the same cluster must all be visited before moving to the next cluster.6 This can be reformualted by adding a large cost to each edge between clusters.6
Generalized TSP (GTSP): Given a set of clusters $V_{1},V_{2},...,V_{k}$ , the GTSP seeks to find the shortest cycle that passes through exactly one node in each cluster $V_{i}$ .6 The GTSP reduces to a TSP if $|V_{i}|=1~\forall ~i$ .6 The GTSP can be solved as a TSP by modifying the cost matrix.6
$m$ -salesmen TSP: Given $m$ salesmen located at a node $1$ , minimize the total distance traveled by each salesman $i$ where each visits every node in a subset $V_{i}$ of $G$ exactly once before returning to node $1$ .6

Formulation

Given a set of $n$ cities enumerated $0,1,...,n-1$ to be visited with the distance between each pair of cities $i$ and $j$ is given by $c_{ij}$ .1 Introduce decision variables $y_{ij}$ for each $(i,j)$ such that

$y_{ij}={\begin{cases}1~~{\text{if city }}j{\text{ is visited immediately after city }}i\\0~~{\text{otherwise}}\end{cases}}$

The objective function is then given by

${\text{min}}\sum _{i}\sum _{j}c_{ij}y_{ij}$

To ensure that the result is a valid tour, several contraints must be added.1,3

Go-to constraints: After visiting a city $i$ , the salesman must visit only one city next:; $\sum _{j}y_{ij}=1,~~\forall i=0,1,...,n-1$
Come-from constraints: When visiting a city, the salesman must have come from only one city:; $\sum _{i}y_{ij}=1,~~\forall j=0,1,...,n-1$
Subtour elimination: Ensure that a tour is fully connected, i.e. no subtours; $\sum _{i}\sum _{j}y_{ij}\leq |S|-1,~~\forall S\subset V,2\leq |S|\leq n-2$; where $S$ is the set of all tours of $G$

There are several other formulations for the subtour elimnation contraint, including circuit packing contraints, MTZ constraints, and network flow constraints.

aTSP ILP Formulation

The integer linear programming formulation for an aTSP is given by

${\begin{aligned}{\text{min}}&~~\sum _{i}\sum _{j}c_{ij}y_{ij}\\{\text{s.t}}&~~\sum _{j}y_{ij}=1,~~i=0,1,...,n-1\\&~~\sum _{i}y_{ij}=1,~~j=0,1,...,n-1\\&~~\sum _{i}\sum _{j}y_{ij}\leq |S|-1~~S\subset V,2\leq |S|\leq n-2\\&~~y_{ij}\in \{0,1\},~\forall i,j\in E\\\end{aligned}}$

sTSP ILP Formulation

The symmetric case is a special case of the asymmetric case and the above formulation is valid.3, 6 The integer linear programming formulation for an sTSP is given by

${\begin{aligned}{\text{min}}&~~\sum _{i}\sum _{j}c_{ij}y_{ij}\\{\text{s.t}}&~~\sum _{i<k}y_{ik}+\sum _{j>k}y_{kj}=2,~~k\in V\\&~~\sum _{i}\sum _{j}y_{ij}\leq |S|-1~~S\subset V,3\leq |S|\leq n-3\\&~~y_{ij}\in \{0,1\}~\forall i,j\in E\\\end{aligned}}$

Solutions

Exact algorithms

The most direct solution algorithm is a complete enumeration of all possible path to determine the path of least cost. However, for $n$ cities, the problem is $O(n!)$ time, and this method is practical only for extremely small values of $n$ .

Branch-and-bound algorithms are commonly used to find solutions for TSPs.7 The ILP is first relaxed and solved as an LP using the Simplex method, then feasibility is regained by enumeration of the integer variables.7

Other exact solution methods include the cutting plane method and branch-and-cut.8

Heuristic algorithms

Given that the TSP is an NP-hard problem, heuristic algorithms are commonly used to give a approximate solutions that are good, though not necessarily optimal. The algorithms do not guarantee an optimal solution, but gives near-optimal solutions in reasonable computational time.3 The Held-Karp lower bound can be calculated and used to judge the performance of a heuristic algorithm.3

There are two general heuristic classifications7:

Tour construction procedures where a solution is gradually built by adding a new vertex at each step
Tour improvement procedures where a feasbile solution is improved upon by performing various exchanges

The best methods tend to be composite algorithms that combine these features.7

Tour construction procedures

Nearest neighbor: The nearest neighbor algorithm follows a simple greedy procedure where the next city on a tour is simply the nearest city that has not yet been visited.8 This approach generally gives a tour within 25% of the Held-Karp lower bound.3
Greedy: A tour is constructed by repeatedly selecting the shortest edge that does not create a subtour, resulting in tours within 15-20% of the Held-Harp lower bound.3
Insertion: Insertion algorithms start with an arbitrary subset of cities and then select a new city $k$ that is not yet on the tour.3, 8 This city is inserted into the tour between two consecutive cities $i$ and $j$ such that the increase to the length of the tour (insertion cost) given by $d(i,k)+d(k,j)=d(i,j)$ is minimized.8

Tour improvement procedures

2-opt and 3-opt: Either two (2-opt) or three (3-opt) edges are randomly removed from a tour and reconnected in such a way that the tour is still feasible.3 This continues until no further improvements can be made. The 2-opt and 3-opt moves usually result in tours that are about 5% and 3% above the Held-Karp bound, respectively.3
k-opt: The k-opt move is the generalized case of the 2-opt and 3-opt methods that requires more computatinal time.3 A local optimum called the k-optimal is found by moving a tour to its best neighbor until no further improvements can be made, where a neighbor of a tour $t$ is one that can be obtained by deleting $k$ edges in $t$ and replacing them with another set of feasible edges.8
Tabu search: A 2-opt exchange mechanism that keeps a tabu list in order to prevent cycling and getting stuck in local minima.3,7
Simulated annealing: Analogous to material annealing, where at high temperature many possible states are reached by the system but as the system cools, the number of possibilities decreases.7 In the beginning, a high value of $T$ allows for a large number of moves. The number of moves decreasess with $T$ until a local minima is reached.

Applications

The importance of the traveling salesman problem is two fold. First its ubiquity as a platform for the study of general methods than can then be applied to a variety of other discrete optimization problems.5 Second is its diverse range of applications, in fields including mathematics, computer science, genetics, and engineering.5,6

Drilling of printed circuit boards: Consider a printed circuit board, where holes must be drilled in order to connect a conductor on one layer to that on another layer.3 These holes may be different sizes and, in order to drill two holes of different sizes consecutively, the head of the machine must perform a time-consuming drill equipment change.3 The process is then select a diameter, drill all the holes of that diameter, change to a different diameter, and repeat until complete.3 This problem can be seen as a series of TSPs, one for each diameter, where the nodes are the locations of the holes, the edges are the time required to move the drill head from one hole to the next, and the objective is to minimize machine head travel time.3
Job sequencing: Consider $n$ jobs that must be completed in order on a single machine and a setup time for executing job $j$ immediately after job $i$ .7 This problem can be formulated as a TSP where the nodes are jobs, the edges are the setup times, and the objective is to minimize the time required to complete all the jobs.7
Computer wiring: Consider a computer board on which a set of pins must be connected such that no more than two wires are attached to each pin.3 This simplifies to solving the TSP with the pins as nodes, the distance between the pins as edges, and the minimum length of wire as the objective.3
Crystallography: Consider an experiment in which a large number of X-ray crystallography measurements must be taken, where each requires the sample to be mounted and the detector to be positioned approprirately.7 Describing the time it takes to move the detector to a certain position as the cost, solving a TSP with the positions as nodes can identify the order to measurements needed to complete the experiment in the shortest time.7

Example

Suppose a Northwestern student, who lives in Foster-Walker, has to accomplish the following tasks:

Drop off a homework set at Tech
Work out a SPAC
Complete a group project at Annenberg

Distances between buildings can be found using Google Maps. Note that there is particularly strong western wind and walking east takes 1.5 times as long.

It is the middle of winter and the student wants to spend the least possible time walking. Determine the path the student should take in order to minimize walking time, starting and ending at Foster-Walker.

Start with the cost matrix (with altered distances taken into account):

	Ending Building
Starting Building	Tech	SPAC	Annenberg	Foster-Walker
Tech	0	3831	2079	2644
SPAC	2554	0	2028	4418
Annenberg	1384	3042	0	2519
Foster-Walker	3966	6627	3778.5	0

Method 1: Complete Enumeration

All possible paths are considered and the path of least cost is the optimal solution. Note that this method is only feasible given the small size of the problem.

Path	Start	Building 1	Building 2	Building 3	End	Cost
1	Foster-Walker	Tech	Annenberg	SPAC	Foster-Walker	13502 ft (2.56 mi)
2	Foster-Walker	Tech	SPAC	Annenberg	Foster-Walker	12344 ft (2.34 mi)
3	Foster-Walker	Annenberg	Tech	SPAC	Foster-Walker	13411.5 ft (2.54mi)
4	Foster-Walker	Annenberg	SPAC	Tech	Foster-Walker	12018.5 ft (2.28 mi)
5	Foster-Walker	SPAC	Annenberg	Tech	Foster-Walker	12683 ft (2.40 mi)
6	Foster-Walker	SPAC	Tech	Annenberg	Foster-Walker	13776 ft (2.61 mi)

From inspection, we see that Path 4 is the shortest. So, the student should walk 2.28 miles in the following order: Foster-Walker → Annenberg → SPAC → Tech → Foster-Walker

Method 2: Nearest neighbor

	Ending Building
Starting Building	Tech	SPAC	Annenberg	Foster-Walker
Tech	0	(5) 3831	(3) 2079	(4) 2644
SPAC	(7) 2554	0	(6) 2028	(8) 4418
Annenberg	(2) 1384	3042	0	2519
Foster-Walker	3966	6627	(1) 3778.5	0

Starting from Foster-Walker, the next building is simply the closest building that has not yet been visited. With only four nodes, this can be done by inspection:

Smallest distance is from Foster-Walker is to Annenberg
Smallest distance from Annenberg is to Tech
Smallest distance from Tech is to Annenberg (creates a subtour, therefore skip)
Next smallest distance from Tech is to Foster-Walker (creates a subtour, therefore skip)
Next smallest distance from Tech is to SPAC
Smallest distance from SPAC is to Annenberg (creates a subtour, therefore skip)
Next smallest distance from SPAC is to Tech (creates a subtour, therefore skip)
Next smallest distance from SPAC is to Foster-Walker

So, the student would walk 2.54 miles in the following order: Foster-Walker → Annenberg → Tech → SPAC → Foster-Walker

Method 3: Greedy

	Ending Building
Starting Building	Tech	SPAC	Annenberg	Foster-Walker
Tech	0	(9) 3831	(3) 2079	(6) 2644
SPAC	(5) 2554	0	(2) 2028	(11) 4418
Annenberg	(1) 1384	(7) 3042	0	(4) 2519
Foster-Walker	(10) 3966	(12) 6627	(8) 3778.5	0

With this method, the shortest paths that do not create a subtour are selected until a complete tour is created.

Smallest distance is Annenberg → Tech
Next smallest is SPAC → Annenberg
Next smallest is Tech → Annenberg (creates a subtour, therefore skip)
Next smallest is Anneberg → Foster-Walker (creates a subtour, therefore skip)
Next smallest is SPAC → Tech (creates a subtour, therefore skip)
Next smallest is Tech → Foster-Walker
Next smallest is Annenberg → SPAC (creates a subtour, therefore skip)
Next smallest is Foster-Walker → Annenberg (creates a subtour, therefore skip)
Next smallest is Tech → SPAC (creates a subtour, therefore skip)
Next smallest is Foster-Walker → Tech (creates a subtour, therefore skip)
Next smallest is SPAC → Foster-Walker (creates a subtour, therefore skip)
Next smallest is Foster-Walker → SPAC

So, the student would walk 2.40 miles in the following order: Foster-Walker → SPAC → Annenberg → Tech → Foster-Walker

Results

As we can see in the figure to the right, the heuristic methods did not give the optimal solution. That is not to say that heuristics can never give the optimal solution, just that it is not guaranteed.

Both the optimal and the nearest neighbor algorithms suggest that Annenberg is the optimal first building to visit. However, the optimal solution then goes to SPAC, while both heuristic methods suggest Tech. This is in part due to the large cost of SPAC → Foster-Walker. The heuristic algorithms cannot take this future cost into account, and therefore fall into that local optimum.

We note that the nearest neighbor and greedy algorithms give solutions that are 11.4% and 5.3%, respectively, above the optimal solution. In the scale of this problem, this corresponds to fractions of a mile. We also note that neither heuristic gave the worst case result, Foster-Walker → SPAC → Tech → Annenberg → Foster-Walker.

Only tour building heuristics were used. Combined with a tour improvement algorithm (such as 2-opt or simulated annealing), we imagine that we may be able to locate solutions that are closer to the optimum.

The exact algorithm used was complete enumeration, but we note that this is impractical even for 7 nodes (6! or 720 different possibilities). Commonly, the problem would be formulated and solved as an ILP to obtain exact solutions.

References

Vanderbei, R. J. (2001). Linear programming: Foundations and extensions (2nd ed.). Boston: Kluwer Academic.
Schrijver, A. (n.d.). On the history of combinatorial optimization (till 1960).
Matai, R., Singh, S., & Lal, M. (2010). Traveling salesman problem: An overview of applications, formulations, and solution approaches. In D. Davendra (Ed.), Traveling Salesman Problem, Theory and Applications. InTech.
Junger, M., Liebling, T., Naddef, D., Nemhauser, G., Pulleyblank, W., Reinelt, G., Rinaldi, G., & Wolsey, L. (Eds.). (2009). 50 years of integer programming, 1958-2008: The early years and state-of-the-art surveys. Heidelberg: Springer.
Cook, W. (2007). History of the TSP. The Traveling Salesman Problem. Retrieved from http://www.math.uwaterloo.ca/tsp/history/index.htm
Punnen, A. P. (2002). The traveling salesman problem: Applications, formulations and variations. In G. Gutin & A. P. Punnen (Eds.), The Traveling Salesman Problem and its Variations. Netherlands: Kluwer Academic Publishers.
Laporte, G. (1992). The traveling salesman problem: An overview of exact and approximate algorithms. European Journal of Operational Research, 59(2), 231–247.
Goyal, S. (n.d.). A suvey on travlling salesman problem.