Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Quantum Algorithms for Solving Dynamic Programming Problems

Pooya Ronagh

Published Jun 5, 2019• 13 citations
OpenAlex

Abstract

We present a general quantum algorithm for solving finite-horizon dynamic programming problems. Up to polylogarithmic factors, our algorithm provides a quadratic quantum advantage in terms of the number of states of a given dynamic programming problem. This speedup comes at the expense of the appearance of other polynomial factors representative of the number of actions of the dynamic programming problem, the maximum value of the instantaneous reward, and the time horizon of the problem. Our algorithm can be applied to combinatorial optimization problems solved classically using dynamic programming techniques. As one application, we show that the travelling salesperson problem can be solved in $O^*(\lceil c \rceil^4 \sqrt{2^n})$ on a quantum computer, where $n$ is the number of vertices of the underlying graph and $\lceil c \rceil$ is its maximum edge-weight. As another example, we show that the minimum set-cover problem can be solved in $O(\sqrt{2^n} \operatorname{poly}(m, n))$, where $m$ is the number of sets used to cover a universe of size $n$. Finally, we prove lower bounds for the query complexity of quantum algorithms and classical randomized algorithms for solving dynamic programming problems, and show that no greater-than-quadratic speedup in either the number of states or number of actions can be achieved for solving dynamic programming problems using quantum algorithms.

Related papers

The Problem of Dynamic Programming on a Quantum Computer

Pooya Ronagh

We discuss the problem of finite-horizon dynamic programming (DP) on a quantum computer. We introduce a query model for studying quantum and classical algorithms for solving DP problems, and provide example oracle constructions for the travelling salesperson problem, the minimum set-cover problem, and the edit distance problem. We formulate open questions regarding quadratic quantum speedups for DP and discuss their implications. We then prove lower bounds for the query complexity of quantum algorithms and classical randomized algorithms for DP, and show that no greater-than-quadratic speedup can be achieved for solving DP problems.

arXiv:1906.02229•Jun 5, 2019•5 citations

Quantum speedups for convex dynamic programming

David Sutter, Giacomo Nannicini, Tobias Sutter +1 more

We present a quantum algorithm to solve dynamic programming problems with convex value functions. For linear discrete-time systems with a $d$-dimensional state space of size $N$, the proposed algorithm outputs a quantum-mechanical representation of the value function in time $O(T γ^{dT}\mathrm{polylog}(N,(T/\varepsilon)^{d}))$, where $\varepsilon$ is the accuracy of the solution, $T$ is the time horizon, and $γ$ is a problem-specific parameter depending on the condition numbers of the cost functions. This allows us to evaluate the value function at any fixed state in time $O(T γ^{dT}\sqrt{N}\,\mathrm{polylog}(N,(T/\varepsilon)^{d}))$, and the corresponding optimal action can be recovered by solving a convex program. The class of optimization problems to which our algorithm can be applied includes provably hard stochastic dynamic programs. Finally, we show that the algorithm obtains a quadratic speedup (up to polylogarithmic factors) compared to the classical Bellman approach on some dynamic programs with continuous state space that have $γ=1$.

arXiv:2011.11654•Nov 23, 2020•4 citations

Quantum Speedups for Polynomial-Time Dynamic Programming Algorithms

Susanna Caroppo, Giordano Da Lozzo, Giuseppe Di Battista +2 more

We introduce a quantum dynamic programming framework that allows us to directly extend to the quantum realm a large body of classical dynamic programming algorithms. The corresponding quantum dynamic programming algorithms retain the same space complexity as their classical counterpart, while achieving a computational speedup. For a combinatorial (search or optimization) problem $\mathcal P$ and an instance $I$ of $\mathcal P$, such a speedup can be expressed in terms of the average degree $δ$ of the dependency digraph $G_{\mathcal{P}}(I)$ of $I$, determined by a recursive formulation of $\mathcal P$. The nodes of this graph are the subproblems of $\mathcal P$ induced by $I$ and its arcs are directed from each subproblem to those on whose solution it relies. In particular, our framework allows us to solve the considered problems in $\tilde{O}(|V(G_{\mathcal{P}}(I))| \sqrtδ)$ time. As an example, we obtain a quantum version of the Bellman-Ford algorithm for computing shortest paths from a single source vertex to all the other vertices in a weighted $n$-vertex digraph with $m$ edges that runs in $\tilde{O}(n\sqrt{nm})$ time, which improves the best known classical upper bound when $m \in Ω(n^{1.4})$.

arXiv:2507.00823•Jul 1, 2025•0 citations

Quantum Speedups for Polynomial-Time Dynamic Programming Algorithms

Caroppo, Susanna, Da Lozzo, Giordano, Giuseppe Di Battista +2 more

We introduce a quantum dynamic programming framework that allows us to directly extend to the quantum realm a large body of classical dynamic programming algorithms. The corresponding quantum dynamic programming algorithms retain the same space complexity as their classical counterpart, while achieving a computational speedup. For a combinatorial (search or optimization) problem P and an instance I of P, such a speedup can be expressed in terms of the average degree δ of the dependency digraph GP(I) of I, determined by a recursive formulation of P. The nodes of this graph are the subproblems of P induced by I and its arcs are directed from each subproblem to those on whose solution it relies. In particular, our framework allows us to solve the considered problems in Õ(|V (GP(I))|√δ) time. As an example, we obtain a quantum version of the Bellman-Ford algorithm for computing shortest paths from a single source vertex to all the other vertices in a weighted n-vertex digraph with m edges that runs in Õ(n√nm) time, which improves the best known classical upper bound when m ∈ Ω(n1.4).

•Jan 1, 2025•0 citations

Quantum Algorithms for Finite-horizon Markov Decision Processes

Luo, Bin, Yu‐Wen Huang, Jonathan Allcock +3 more

In this work, we design quantum algorithms that are more efficient than classical algorithms to solve time-dependent and finite-horizon Markov Decision Processes (MDPs) in two distinct settings: (1) In the exact dynamics setting, where the agent has full knowledge of the environment's dynamics (i.e., transition probabilities), we prove that our $\textbf{Quantum Value Iteration (QVI)}$ algorithm $\textbf{QVI-1}$ achieves a quadratic speedup in the size of the action space $(A)$ compared with the classical value iteration algorithm for computing the optimal policy ($π^{*}$) and the optimal V-value function ($V_{0}^{*}$). Furthermore, our algorithm $\textbf{QVI-2}$ provides an additional speedup in the size of the state space $(S)$ when obtaining near-optimal policies and V-value functions. Both $\textbf{QVI-1}$ and $\textbf{QVI-2}$ achieve quantum query complexities that provably improve upon classical lower bounds, particularly in their dependences on $S$ and $A$. (2) In the generative model setting, where samples from the environment are accessible in quantum superposition, we prove that our algorithms $\textbf{QVI-3}$ and $\textbf{QVI-4}$ achieve improvements in sample complexity over the state-of-the-art (SOTA) classical algorithm in terms of $A$, estimation error $(ε)$, and time horizon $(H)$. More importantly, we prove quantum lower bounds to show that $\textbf{QVI-3}$ and $\textbf{QVI-4}$ are asymptotically optimal, up to logarithmic factors, assuming a constant time horizon.

arXiv:2508.05712•Aug 7, 2025•0 citations

Quantum Speedups for Exponential-Time Dynamic Programming Algorithms

Andris Ambainis, Kaspars Balodis, Jānis Iraids +3 more

In this paper we study quantum algorithms for NP-complete problems whose best classical algorithm is an exponential time application of dynamic programming. We introduce the path in the hypercube problem that models many of these dynamic programming algorithms. In this problem we are asked whether there is a path from $0^n$ to $1^n$ in a given subgraph of the Boolean hypercube, where the edges are all directed from smaller to larger Hamming weight. We give a quantum algorithm that solves path in the hypercube in time $O^*(1.817^n)$. The technique combines Grover's search with computing a partial dynamic programming table. We use this approach to solve a variety of vertex ordering problems on graphs in the same time $O^*(1.817^n)$, and graph bandwidth in time $O^*(2.946^n)$. Then we use similar ideas to solve the travelling salesman problem and minimum set cover in time $O^*(1.728^n)$.

arXiv:1807.05209•Jul 13, 2018•1 citation
We present a general quantum algorithm for solving finite-horizon dynamic programming problems. Up to polylogarithmic factors, our algorithm provides a quadratic quantum advantage in terms of the number of states of a given dynamic programming problem. This speedup comes at the expense of the appearance of other polynomial factors representative of the number of actions of the dynamic programming problem, the maximum value of the instantaneous reward, and the time horizon of the problem. Our algorithm can be applied to combinatorial optimization problems solved classically using dynamic programming techniques. As one application, we show that the travelling salesperson problem can be solved in O∗(⌈c⌉42n)O^*(\lceil c \rceil^4 \sqrt{2^n})O∗(⌈c⌉42n​) on a quantum computer, where nnn is the number of vertices of the underlying graph and ⌈c⌉\lceil c \rceil⌈c⌉ is its maximum edge-weight. As another example, we show that the minimum set-cover problem can be solved in O(2npoly⁡(m,n))O(\sqrt{2^n} \operatorname{poly}(m, n))O(2n​poly(m,n)), where mmm is the number of sets used to cover a universe of size nnn. Finally, we prove lower bounds for the query complexity of quantum algorithms and classical randomized algorithms for solving dynamic programming problems, and show that no greater-than-quadratic speedup in either the number of states or number of actions can be achieved for solving dynamic programming problems using quantum algorithms.
We discuss the problem of finite-horizon dynamic programming (DP) on a quantum computer. We introduce a query model for studying quantum and classical algorithms for solving DP problems, and provide example oracle constructions for the travelling salesperson problem, the minimum set-cover problem, and the edit distance problem. We formulate open questions regarding quadratic quantum speedups for DP and discuss their implications. We then prove lower bounds for the query complexity of quantum algorithms and classical randomized algorithms for DP, and show that no greater-than-quadratic speedup can be achieved for solving DP problems.
We present a quantum algorithm to solve dynamic programming problems with convex value functions. For linear discrete-time systems with a ddd-dimensional state space of size NNN, the proposed algorithm outputs a quantum-mechanical representation of the value function in time O(TγdTpolylog(N,(T/ε)d))O(T γ^{dT}\mathrm{polylog}(N,(T/\varepsilon)^{d}))O(TγdTpolylog(N,(T/ε)d)), where ε\varepsilonε is the accuracy of the solution, TTT is the time horizon, and γγγ is a problem-specific parameter depending on the condition numbers of the cost functions. This allows us to evaluate the value function at any fixed state in time O(TγdTN polylog(N,(T/ε)d))O(T γ^{dT}\sqrt{N}\,\mathrm{polylog}(N,(T/\varepsilon)^{d}))O(TγdTN​polylog(N,(T/ε)d)), and the corresponding optimal action can be recovered by solving a convex program. The class of optimization problems to which our algorithm can be applied includes provably hard stochastic dynamic programs. Finally, we show that the algorithm obtains a quadratic speedup (up to polylogarithmic factors) compared to the classical Bellman approach on some dynamic programs with continuous state space that have γ=1γ=1γ=1.
We introduce a quantum dynamic programming framework that allows us to directly extend to the quantum realm a large body of classical dynamic programming algorithms. The corresponding quantum dynamic programming algorithms retain the same space complexity as their classical counterpart, while achieving a computational speedup. For a combinatorial (search or optimization) problem P\mathcal PP and an instance III of P\mathcal PP, such a speedup can be expressed in terms of the average degree δδδ of the dependency digraph GP(I)G_{\mathcal{P}}(I)GP​(I) of III, determined by a recursive formulation of P\mathcal PP. The nodes of this graph are the subproblems of P\mathcal PP induced by III and its arcs are directed from each subproblem to those on whose solution it relies. In particular, our framework allows us to solve the considered problems in O~(∣V(GP(I))∣δ)\tilde{O}(|V(G_{\mathcal{P}}(I))| \sqrtδ)O~(∣V(GP​(I))∣δ​) time. As an example, we obtain a quantum version of the Bellman-Ford algorithm for computing shortest paths from a single source vertex to all the other vertices in a weighted nnn-vertex digraph with mmm edges that runs in O~(nnm)\tilde{O}(n\sqrt{nm})O~(nnm​) time, which improves the best known classical upper bound when m∈Ω(n1.4)m \in Ω(n^{1.4})m∈Ω(n1.4).
We introduce a quantum dynamic programming framework that allows us to directly extend to the quantum realm a large body of classical dynamic programming algorithms. The corresponding quantum dynamic programming algorithms retain the same space complexity as their classical counterpart, while achieving a computational speedup. For a combinatorial (search or optimization) problem P and an instance I of P, such a speedup can be expressed in terms of the average degree δ of the dependency digraph GP(I) of I, determined by a recursive formulation of P. The nodes of this graph are the subproblems of P induced by I and its arcs are directed from each subproblem to those on whose solution it relies. In particular, our framework allows us to solve the considered problems in Õ(|V (GP(I))|√δ) time. As an example, we obtain a quantum version of the Bellman-Ford algorithm for computing shortest paths from a single source vertex to all the other vertices in a weighted n-vertex digraph with m edges that runs in Õ(n√nm) time, which improves the best known classical upper bound when m ∈ Ω(n1.4).
In this work, we design quantum algorithms that are more efficient than classical algorithms to solve time-dependent and finite-horizon Markov Decision Processes (MDPs) in two distinct settings: (1) In the exact dynamics setting, where the agent has full knowledge of the environment's dynamics (i.e., transition probabilities), we prove that our Quantum Value Iteration (QVI)\textbf{Quantum Value Iteration (QVI)}Quantum Value Iteration (QVI) algorithm QVI-1\textbf{QVI-1}QVI-1 achieves a quadratic speedup in the size of the action space (A)(A)(A) compared with the classical value iteration algorithm for computing the optimal policy (π∗π^{*}π∗) and the optimal V-value function (V0∗V_{0}^{*}V0∗​). Furthermore, our algorithm QVI-2\textbf{QVI-2}QVI-2 provides an additional speedup in the size of the state space (S)(S)(S) when obtaining near-optimal policies and V-value functions. Both QVI-1\textbf{QVI-1}QVI-1 and QVI-2\textbf{QVI-2}QVI-2 achieve quantum query complexities that provably improve upon classical lower bounds, particularly in their dependences on SSS and AAA. (2) In the generative model setting, where samples from the environment are accessible in quantum superposition, we prove that our algorithms QVI-3\textbf{QVI-3}QVI-3 and QVI-4\textbf{QVI-4}QVI-4 achieve improvements in sample complexity over the state-of-the-art (SOTA) classical algorithm in terms of AAA, estimation error (ε)(ε)(ε), and time horizon (H)(H)(H). More importantly, we prove quantum lower bounds to show that QVI-3\textbf{QVI-3}QVI-3 and QVI-4\textbf{QVI-4}QVI-4 are asymptotically optimal, up to logarithmic factors, assuming a constant time horizon.
In this paper we study quantum algorithms for NP-complete problems whose best classical algorithm is an exponential time application of dynamic programming. We introduce the path in the hypercube problem that models many of these dynamic programming algorithms. In this problem we are asked whether there is a path from 0n0^n0n to 1n1^n1n in a given subgraph of the Boolean hypercube, where the edges are all directed from smaller to larger Hamming weight. We give a quantum algorithm that solves path in the hypercube in time O∗(1.817n)O^*(1.817^n)O∗(1.817n). The technique combines Grover's search with computing a partial dynamic programming table. We use this approach to solve a variety of vertex ordering problems on graphs in the same time O∗(1.817n)O^*(1.817^n)O∗(1.817n), and graph bandwidth in time O∗(2.946n)O^*(2.946^n)O∗(2.946n). Then we use similar ideas to solve the travelling salesman problem and minimum set cover in time O∗(1.728n)O^*(1.728^n)O∗(1.728n).