Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2008.07454 · Quantum Science and Technology

Higher order derivatives of quantum neural networks with barren plateaus

Marco Cerezo, Patrick J. Coles

Published Apr 6, 2021• 118 citations
arXivHTMLPDF

Abstract

Abstract Quantum neural networks (QNNs) offer a powerful paradigm for programming near-term quantum computers and have the potential to speed up applications ranging from data science to chemistry to materials science. However, a possible obstacle to realizing that speed-up is the barren plateau (BP) phenomenon, whereby the gradient vanishes exponentially in the system size n for certain QNN architectures. The question of whether high-order derivative information such as the Hessian could help escape a BP was recently posed in the literature. Here we show that the elements of the Hessian are exponentially suppressed in a BP, so estimating the Hessian in this situation would require a precision that scales exponentially with n . Hence, Hessian-based approaches do not circumvent the exponential scaling associated with BPs. We also show the exponential suppression of higher order derivatives. Hence, BPs will impact optimization strategies that go beyond (first-order) gradient descent. In deriving our results, we prove novel, general formulas that can be used to analytically evaluate any high-order partial derivative on quantum hardware. These formulas will likely have independent interest and use for training QNNs (outside of the context of BPs).

DOI
10.1088/2058-9565/abf51a

Related papers

Impact of Barren Plateaus on the Hessian and Higher Order Derivatives

Marco Cerezo, Patrick J. Coles

The Barren Plateau (BP) phenomenon is an issue for certain quantum neural networks and variational quantum algorithms, whereby the gradient vanishes exponentially in the system size $n$. The question of whether high-order derivative information such as the Hessian could help escape a BP was recently posed in the literature. Here we show that the elements of the Hessian are exponentially suppressed in a BP, so estimating the Hessian in this situation would require a precision that scales exponentially with $n$. Hence, Hessian-based approaches do not circumvent the exponential scaling associated with BPs. We also show the exponential suppression of higher order derivatives. Hence, BPs will impact optimization strategies that go beyond (first-order) gradient descent.

•Aug 17, 2020•15 citations

Absence of Barren Plateaus in Quantum Convolutional Neural Networks

Arthur Chalom Pesah, Marco Cerezo, Samson Wang +3 more

Quantum neural networks (QNNs) have generated excitement around the possibility of efficiently analyzing quantum data.But this excitement has been tempered by the existence of exponentially vanishing gradients, known as barren plateau landscapes, for many QNN architectures.Recently, quantum convolutional neural networks (QCNNs) have been proposed, involving a sequence of convolutional and pooling layers that reduce the number of qubits while preserving information about relevant data features.In this work, we rigorously analyze the gradient scaling for the parameters in the QCNN architecture.We find that the variance of the gradient vanishes no faster than polynomially, implying that QCNNs do not exhibit barren plateaus.This result provides an analytical guarantee for the trainability of randomly initialized QCNNs, which highlights QCNNs as being trainable under random initialization unlike many other QNN architectures.To derive our results, we introduce a novel graph-based method to analyze expectation values over Haar-distributed unitaries, which will likely be useful in other contexts.Finally, we perform numerical simulations to verify our analytical results.

arXiv:2011.02966•Physical Review X•Oct 15, 2021•343 citations

Effect of barren plateaus on gradient-free optimization

Andrew Arrasmith, M. Cerezo, Piotr Czarnik +2 more

Barren plateau landscapes correspond to gradients that vanish exponentially in the number of qubits. Such landscapes have been demonstrated for variational quantum algorithms and quantum neural networks with either deep circuits or global cost functions. For obvious reasons, it is expected that gradient-based optimizers will be significantly affected by barren plateaus. However, whether or not gradient-free optimizers are impacted is a topic of debate, with some arguing that gradient-free approaches are unaffected by barren plateaus. Here we show that, indeed, gradient-free optimizers do not solve the barren plateau problem. Our main result proves that cost function differences, which are the basis for making decisions in a gradient-free optimization, are exponentially suppressed in a barren plateau. Hence, without exponential precision, gradient-free optimizers will not make progress in the optimization. We numerically confirm this by training in a barren plateau with several gradient-free optimizers (Nelder-Mead, Powell, and COBYLA algorithms), and show that the numbers of shots required in the optimization grows exponentially with the number of qubits.

arXiv:2011.12245•Quantum•Oct 5, 2021•248 citations

Quantum Generative Training Using Rényi Divergences

Mária Kieferová, Ortiz Marrero Carlos, Nathan Wiebe

Quantum neural networks (QNNs) are a framework for creating quantum algorithms that promises to combine the speedups of quantum computation with the widespread successes of machine learning. A major challenge in QNN development is a concentration of measure phenomenon known as a barren plateau that leads to exponentially small gradients for a range of QNNs models. In this work, we examine the assumptions that give rise to barren plateaus and show that an unbounded loss function can circumvent the existing no-go results. We propose a training algorithm that minimizes the maximal Rényi divergence of order two and present techniques for gradient computation. We compute the closed form of the gradients for Unitary QNNs and Quantum Boltzmann Machines and provide sufficient conditions for the absence of barren plateaus in these models. We demonstrate our approach in two use cases: thermal state learning and Hamiltonian learning. In our numerical experiments, we observed rapid convergence of our training loss function and frequently archived a $99\%$ average fidelity in fewer than $100$ epochs.

arXiv:2106.09567•Jun 17, 2021•11 citations

The Goldilocks Regime: Quantum-Aware Activation Functions for Barren Plateau Mitigation in Hybrid QNNs

R. Gökhan Türeci

Abstract The barren plateau phenomenon --- the exponential vanishing of gradient norms with increasing qubit number --- remains a fundamental obstacle in training hybrid classical-quantum neural networks. We propose that the choice of activation function at the classical-quantum interface constitutes an underexplored yet effective lever for barren plateau mitigation. Specifically, we introduce the concept of the Goldilocks Regime : the range of maximum activation derivative magnitudes \((\max|f'| \in (1.0,\, \pi/2])\) within which quantum-aware activation functions achieve optimal gradient preservation in parameterized quantum circuits.Through systematic experiments on hardware-efficient parameterized quantum circuits with up to 18 qubits using Qiskit 2.3.1 and the parameter shift rule, we evaluate three quantum-aware activation functions --- PiTanh (\((\pi\cdot\tanh(x))\)), SmoothAngle (\((\pi\cdot(2\cdot\text{sigmoid}(x)-1))\)), and LearnableAngle (\((\pi\cdot\tanh(\alpha x))\)) --- alongside random uniform initialization as a baseline.LearnableAngle with \((\alpha = 0.4)\) (\((\max|f'| \approx 1.257)\)), lying within the Goldilocks Regime, consistently achieves the highest gradient norm across all tested qubit counts, maintaining \((0.404)\) at \((n = 18)\) compared to \((0.013)\) for random initialization --- a 31-fold improvement. An ablation study incorporating the Physics-Constrained Surrogate Modelling (PCSM) framework reveals that physical derivative constraints provide the greatest benefit when the activation function operates outside the optimal Goldilocks Regime, establishing activation design and PCSM regularization as complementary strategies for gradient preservation.These results identify the Goldilocks Regime as a principled design criterion for quantum-aware activation functions and demonstrate that structured classical pre-networks can substantially delay the onset of barren plateaus without modifying the quantum circuit architecture.

•Apr 20, 2026•0 citations

Barren Plateaus and Trainability of Hybrid Quantum-Classical Algorithms

Celepci-Uludag, Dilara

Realizing successful quantum algorithms on current generation of quantum hardware depends on developing appropriate parametrized quantum circuits, which are the quantum computing counterpart to classical neural networks. A key consideration in this context is the algorithm’s trainability that can be hindered by the existence of so-called barren plateaus. In this case, the algorithm suffers from exponentially vanishing gradients as the number of qubits increases, resulting in a flat optimization landscape and making an efficient training intractable. Due to its serious impact on the success of such algorithms, this problem has gained substantial attention in recent years of quantum computing research. Theoretical and heuristic approaches have been proposed to investigate, avoid and mitigate this issue. In this work, we explore this phenomenon in detail and study the underlying mathematical concepts in depth, investigating the factors that influence the structure of the optimization landscape. Using corresponding mathematical tools, we diagnose whether a given variational circuit will exhibit barren plateaus when going beyond tens of qubits. Additionally, a review of mitigation strategies will be provided. The theory will then be applied to a use case that involves image classification to evaluate whether the model is affected by this phenomenon and which hyperparameters improve or degrade the overall performance.

•Jan 1, 2025•0 citations
Abstract Quantum neural networks (QNNs) offer a powerful paradigm for programming near-term quantum computers and have the potential to speed up applications ranging from data science to chemistry to materials science. However, a possible obstacle to realizing that speed-up is the barren plateau (BP) phenomenon, whereby the gradient vanishes exponentially in the system size n for certain QNN architectures. The question of whether high-order derivative information such as the Hessian could help escape a BP was recently posed in the literature. Here we show that the elements of the Hessian are exponentially suppressed in a BP, so estimating the Hessian in this situation would require a precision that scales exponentially with n . Hence, Hessian-based approaches do not circumvent the exponential scaling associated with BPs. We also show the exponential suppression of higher order derivatives. Hence, BPs will impact optimization strategies that go beyond (first-order) gradient descent. In deriving our results, we prove novel, general formulas that can be used to analytically evaluate any high-order partial derivative on quantum hardware. These formulas will likely have independent interest and use for training QNNs (outside of the context of BPs).
The Barren Plateau (BP) phenomenon is an issue for certain quantum neural networks and variational quantum algorithms, whereby the gradient vanishes exponentially in the system size nnn. The question of whether high-order derivative information such as the Hessian could help escape a BP was recently posed in the literature. Here we show that the elements of the Hessian are exponentially suppressed in a BP, so estimating the Hessian in this situation would require a precision that scales exponentially with nnn. Hence, Hessian-based approaches do not circumvent the exponential scaling associated with BPs. We also show the exponential suppression of higher order derivatives. Hence, BPs will impact optimization strategies that go beyond (first-order) gradient descent.
Quantum neural networks (QNNs) have generated excitement around the possibility of efficiently analyzing quantum data.But this excitement has been tempered by the existence of exponentially vanishing gradients, known as barren plateau landscapes, for many QNN architectures.Recently, quantum convolutional neural networks (QCNNs) have been proposed, involving a sequence of convolutional and pooling layers that reduce the number of qubits while preserving information about relevant data features.In this work, we rigorously analyze the gradient scaling for the parameters in the QCNN architecture.We find that the variance of the gradient vanishes no faster than polynomially, implying that QCNNs do not exhibit barren plateaus.This result provides an analytical guarantee for the trainability of randomly initialized QCNNs, which highlights QCNNs as being trainable under random initialization unlike many other QNN architectures.To derive our results, we introduce a novel graph-based method to analyze expectation values over Haar-distributed unitaries, which will likely be useful in other contexts.Finally, we perform numerical simulations to verify our analytical results.
Barren plateau landscapes correspond to gradients that vanish exponentially in the number of qubits. Such landscapes have been demonstrated for variational quantum algorithms and quantum neural networks with either deep circuits or global cost functions. For obvious reasons, it is expected that gradient-based optimizers will be significantly affected by barren plateaus. However, whether or not gradient-free optimizers are impacted is a topic of debate, with some arguing that gradient-free approaches are unaffected by barren plateaus. Here we show that, indeed, gradient-free optimizers do not solve the barren plateau problem. Our main result proves that cost function differences, which are the basis for making decisions in a gradient-free optimization, are exponentially suppressed in a barren plateau. Hence, without exponential precision, gradient-free optimizers will not make progress in the optimization. We numerically confirm this by training in a barren plateau with several gradient-free optimizers (Nelder-Mead, Powell, and COBYLA algorithms), and show that the numbers of shots required in the optimization grows exponentially with the number of qubits.
Quantum neural networks (QNNs) are a framework for creating quantum algorithms that promises to combine the speedups of quantum computation with the widespread successes of machine learning. A major challenge in QNN development is a concentration of measure phenomenon known as a barren plateau that leads to exponentially small gradients for a range of QNNs models. In this work, we examine the assumptions that give rise to barren plateaus and show that an unbounded loss function can circumvent the existing no-go results. We propose a training algorithm that minimizes the maximal Rényi divergence of order two and present techniques for gradient computation. We compute the closed form of the gradients for Unitary QNNs and Quantum Boltzmann Machines and provide sufficient conditions for the absence of barren plateaus in these models. We demonstrate our approach in two use cases: thermal state learning and Hamiltonian learning. In our numerical experiments, we observed rapid convergence of our training loss function and frequently archived a 99%99\%99% average fidelity in fewer than 100100100 epochs.
Abstract The barren plateau phenomenon --- the exponential vanishing of gradient norms with increasing qubit number --- remains a fundamental obstacle in training hybrid classical-quantum neural networks. We propose that the choice of activation function at the classical-quantum interface constitutes an underexplored yet effective lever for barren plateau mitigation. Specifically, we introduce the concept of the Goldilocks Regime : the range of maximum activation derivative magnitudes (max⁡∣f′∣∈(1.0, π/2])(\max|f'| \in (1.0,\, \pi/2])(max∣f′∣∈(1.0,π/2]) within which quantum-aware activation functions achieve optimal gradient preservation in parameterized quantum circuits.Through systematic experiments on hardware-efficient parameterized quantum circuits with up to 18 qubits using Qiskit 2.3.1 and the parameter shift rule, we evaluate three quantum-aware activation functions --- PiTanh ((π⋅tanh⁡(x))(\pi\cdot\tanh(x))(π⋅tanh(x))), SmoothAngle ((π⋅(2⋅sigmoid(x)−1))(\pi\cdot(2\cdot\text{sigmoid}(x)-1))(π⋅(2⋅sigmoid(x)−1))), and LearnableAngle ((π⋅tanh⁡(αx))(\pi\cdot\tanh(\alpha x))(π⋅tanh(αx))) --- alongside random uniform initialization as a baseline.LearnableAngle with (α=0.4)(\alpha = 0.4)(α=0.4) ((max⁡∣f′∣≈1.257)(\max|f'| \approx 1.257)(max∣f′∣≈1.257)), lying within the Goldilocks Regime, consistently achieves the highest gradient norm across all tested qubit counts, maintaining (0.404)(0.404)(0.404) at (n=18)(n = 18)(n=18) compared to (0.013)(0.013)(0.013) for random initialization --- a 31-fold improvement. An ablation study incorporating the Physics-Constrained Surrogate Modelling (PCSM) framework reveals that physical derivative constraints provide the greatest benefit when the activation function operates outside the optimal Goldilocks Regime, establishing activation design and PCSM regularization as complementary strategies for gradient preservation.These results identify the Goldilocks Regime as a principled design criterion for quantum-aware activation functions and demonstrate that structured classical pre-networks can substantially delay the onset of barren plateaus without modifying the quantum circuit architecture.
Realizing successful quantum algorithms on current generation of quantum hardware depends on developing appropriate parametrized quantum circuits, which are the quantum computing counterpart to classical neural networks. A key consideration in this context is the algorithm’s trainability that can be hindered by the existence of so-called barren plateaus. In this case, the algorithm suffers from exponentially vanishing gradients as the number of qubits increases, resulting in a flat optimization landscape and making an efficient training intractable. Due to its serious impact on the success of such algorithms, this problem has gained substantial attention in recent years of quantum computing research. Theoretical and heuristic approaches have been proposed to investigate, avoid and mitigate this issue. In this work, we explore this phenomenon in detail and study the underlying mathematical concepts in depth, investigating the factors that influence the structure of the optimization landscape. Using corresponding mathematical tools, we diagnose whether a given variational circuit will exhibit barren plateaus when going beyond tens of qubits. Additionally, a review of mitigation strategies will be provided. The theory will then be applied to a use case that involves image classification to evaluate whether the model is affected by this phenomenon and which hyperparameters improve or degrade the overall performance.