Study on solution techniques of combinatorial optimization problem with large language models

Abstract

A conventional approach to solving combinatorial problems can be divided into three steps: formulating the problem, implementing the codes and improving algorithms to get solutions smarter. Using large language models (LLM), we might solve the problems without the formulation and the implementation. In this paper, we investigated solving performance when using LLM as a solver for the traveling salesman problem (TSP). The method for providing prompts is based on OPRO [Yang 2023], where the LLM generates new solutions from the prompt which contains previously generated solutions, iteratively. Not only natural language representations of the problem, but also directed graphs representation are utilized as prompts. Approximation ratio, which is the ratio of a minimal distance of the TSP to distance of obtained solution from LLM is investigated as a solution performance. We found that LLM, Gemini-1.5-flash, can generates solutions with approximation ratio 1.55 of TSP called “att48”.