We introduce efficient tensor network models for sequence processing motivated by correspondence to probabilistic graphical models, interpretability and resource compression. Inductive bias is introduced via network architecture as motivated by correlation and compositional structure in the data. We create expressive networks utilising tensors that are both complex and unitary. As such they may be represented by parameterised quantum circuits and describe physical processes. The relevant inductive biases result in networks with logarithmic treewidth which is paramount for avoiding trainability issues in these spaces. For the same reason, they are also efficiently contractable or 'quantum-inspired'. We demonstrate experimental results for the task of binary classification of bioinformatics and natural language, characterised by long-range correlations and often equipped with syntactic information. This work provides a scalable route for experimentation on the role of tensor structure and syntactic priors in NLP. Since these models map operationally to the qubits of a quantum processor, unbiased sampling equates to taking measurements on the quantum state encoding the learnt probability distribution. We demonstrate implementation on Quantinuum's H2-1 trapped-ion quantum processor, showing the potential of near-term quantum devices.
We introduce efficient tensor network models for sequence processing motivated by correspondence to probabilistic graphical models, interpretability and resource compression. Inductive bias is introduced via network architecture as motivated by correlation and compositional structure in the data. We create expressive networks utilising tensors that are both complex and unitary. As such they may be represented by parameterised quantum circuits and describe physical processes. The relevant inductive biases result in networks with logarithmic treewidth which is paramount for avoiding trainability issues in these spaces. For the same reason, they are also efficiently contractable or 'quantum-inspired'. We demonstrate experimental results for the task of binary classification of bioinformatics and natural language, characterised by long-range correlations and often equipped with syntactic information. This work provides a scalable route for experimentation on the role of tensor structure and syntactic priors in NLP. Since these models map operationally to the qubits of a quantum processor, unbiased sampling equates to taking measurements on the quantum state encoding the learnt probability distribution. We demonstrate implementation on Quantinuum's H2-1 trapped-ion quantum processor, showing the potential of near-term quantum devices.
We introduce complex-valued tensor network models for sequence processing motivated by correspondence to probabilistic graphical models, interpretability and resource compression. Inductive bias is introduced to our models via network architecture, and is motivated by the correlation structure inherent in the data, as well as any relevant compositional structure, resulting in tree-like connectivity. Our models are specifically constructed using parameterised quantum circuits, widely used in quantum machine learning, effectively using Hilbert space as a feature space. Furthermore, they are efficiently trainable due to their tree-like structure. We demonstrate experimental results for the task of binary classification of sequences from real-world datasets relevant to natural language and bioinformatics, characterised by long-range correlations and often equipped with syntactic information. Since our models have a valid operational interpretation as quantum processes, we also demonstrate their implementation on Quantinuum's H2-1 trapped-ion quantum processor, demonstrating the possibility of efficient sequence processing on near-term quantum devices. This work constitutes the first scalable implementation of near-term quantum language processing, providing the tools for large-scale experimentation on the role of tensor structure and syntactic priors. Finally, this work lays the groundwork for generative sequence modelling in a hybrid pipeline where the training may be conducted efficiently in simulation, while sampling from learned probability distributions may be done with polynomial speed-up on quantum devices.
Computational methods in drug discovery significantly reduce both time and experimental costs. Nonetheless, certain computational tasks in drug discovery can be daunting with classical computing techniques which can be potentially overcome using quantum computing. A crucial task within this domain involves the functional classification of proteins. However, a challenge lies in adequately representing lengthy protein sequences given the limited number of qubits available in existing noisy quantum computers. We show that protein sequences can be thought of as sentences in natural language processing and can be parsed using the existing Quantum Natural Language framework into parameterized quantum circuits of reasonable qubits, which can be trained to solve various proteinrelated machine-learning problems. We classify proteins based on their sub-cellular locations—a pivotal task in bioinformatics that is key to understanding biological processes and disease mechanisms. Leveraging the quantum-enhanced processing capabilities, we demonstrate that Quantum Tensor Networks (QTN) can effectively handle the complexity and diversity of protein sequences. We present a detailed methodology that adapts QTN architectures to the nuanced requirements of protein data, supported by comprehensive experimental results. We demonstrate two distinct QTNs, inspired by classical recurrent neural networks (RNN) and convolutional neural networks (CNN), to solve the binary classification task mentioned above. Our top-performing quantum model has achieved a 94% accuracy rate, which is comparable to the performance of a classical model that uses the ESM2 protein language model embeddings. It’s noteworthy that the ESM2 model is extremely large, containing 8 million parameters in its smallest configuration, whereas our best quantum model requires only around 800 parameters. We demonstrate that these hybrid models exhibit promising performance, showcasing their potential to compete with classical models of similar complexity.
We show that protein sequences can be thought of as sentences in natural language processing and can be parsed using the existing Quantum Natural Language framework into parameterized quantum circuits of reasonable qubits, which can be trained to solve various protein-related machine-learning problems. We classify proteins based on their subcellular locations, a pivotal task in bioinformatics that is key to understanding biological processes and disease mechanisms. Leveraging the quantum-enhanced processing capabilities, we demonstrate that Quantum Tensor Networks (QTN) can effectively handle the complexity and diversity of protein sequences. We present a detailed methodology that adapts QTN architectures to the nuanced requirements of protein data, supported by comprehensive experimental results. We demonstrate two distinct QTNs, inspired by classical recurrent neural networks (RNN) and convolutional neural networks (CNN), to solve the binary classification task mentioned above. Our top-performing quantum model has achieved a 94% accuracy rate, which is comparable to the performance of a classical model that uses the ESM2 protein language model embeddings. It's noteworthy that the ESM2 model is extremely large, containing 8 million parameters in its smallest configuration, whereas our best quantum model requires only around 800 parameters. We demonstrate that these hybrid models exhibit promising performance, showcasing their potential to compete with classical models of similar complexity.
Computational methods in drug discovery significantly reduce both time and experimental costs. Nonetheless, certain computational tasks in drug discovery can be daunting with classical computing techniques which can be potentially overcome using quantum computing. A crucial task within this domain involves the functional classification of proteins. However, a challenge lies in adequately representing lengthy protein sequences given the limited number of qubits available in existing noisy quantum computers. We show that protein sequences can be thought of as sentences in natural language processing and can be parsed using the existing Quantum Natural Language framework into parameterized quantum circuits of reasonable qubits, which can be trained to solve various protein-related machine-learning problems. We classify proteins based on their sub-cellular locations—a pivotal task in bioinformatics that is key to understanding biological processes and disease mechanisms. Leveraging the quantum-enhanced processing capabilities, we demonstrate that Quantum Tensor Networks (QTN) can effectively handle the complexity and diversity of protein sequences. We present a detailed methodology that adapts QTN architectures to the nuanced requirements of protein data, supported by comprehensive experimental results. We demonstrate two distinct QTNs, inspired by classical recurrent neural networks (RNN) and convolutional neural networks (CNN), to solve the binary classification task mentioned above. Our top-performing quantum model has achieved a 94% accuracy rate, which is comparable to the performance of a classical model that uses the ESM2 protein language model embeddings. It’s noteworthy that the ESM2 model is extremely large, containing 8 million parameters in its smallest configuration, whereas our best quantum model requires only around 800 parameters. We demonstrate that these hybrid models exhibit promising performance, showcasing their potential to compete with classical models of similar complexity.
A study published on bioRxiv demonstrates for the first time the technique of treating protein sequences like sentences in natural language processing (NLP) and parsing them into parameterized quantum circuits using the Quantum Natural Language (QNLP) framework. It establishes a scheme that converts protein sequences into tensor networks represented by wires and boxes, and maps them onto actual quantum circuits through semantic functors. By generating protein syntax trees with a neural network-based parser and converting them into string diagrams optimized for quantum processor execution, the study proves that 'Quantum Tensor Networks (QTN)' achieve overwhelming computational accuracy in classifying vast protein sequences. [Quantum Biology Society] An innovative approach has been proposed to decode protein sequences—the most fundamental and complex codes of living organisms—by converting them into a natural language that quantum computers can comprehend. The paper titled "Application of Quantum Tensor Networks for Protein Classification", published in the preprint repository bioRxiv, introduces a groundbreaking pipeline that applies Quantum Natural Language Processing (QNLP) technology to protein sequence classification, opening a new horizon for next-generation bioinformatics. A research team from Pennsylvania State University, including Debarshi Kundu, Jian Wang, Archisman Ghosh, and Nikolay Dokholyan, led this innovative study. ■ QNLP: Reading Protein Sequences as Sentences The researchers introduced a unique perspective of treating protein sequences, which are arrays of amino acids, as if they were sentences composed in human language. To achieve this, they adopted the 'Quantum Natural Language (QNLP)' framework, which combines state-of-the-art computational natural language processing (NLP) techniques with quantum computing. Within this framework, protein sequences are parsed into syntax trees with grammatical structures through a neural network-based parser, laying the foundation for their direct conversion into parameterized quantum circuits. ■ Transformation into Tensor Networks and String Diagrams In this process, protein sequences are converted into a tensor network scheme, geometrically represented by wires and boxes. The researchers defined a sophisticated semantic function (functor) that perfectly maps this abstract network to the operational structure of actual quantum circuits. The generated syntax trees are ultimately transformed into string diagrams, a format optimized for execution on quantum processors. Consequently, the constructed Quantum Tensor Networks (QTN) demonstrated overwhelming computational accuracy in machine learning tasks that involve extracting features and functionally classifying highly diverse and complex protein sequences, even in environments with a limited number of available qubits. This demonstration is highly significant as it establishes a powerful workflow in drug discovery and biotechnology, enabling rapid and accurate analysis of massive genomic data and protein interaction structures through quantum algorithms, transcending the limits of conventional classical computers. #QuantumTensorNetworks #QNLP #ProteinClassification #QuantumMachineLearning #QuantumLanguageModel #ProteinSequenceAnalysis #QuantumBiology #Biophysics #ArtificialIntelligence #KoreanQuantumBiologySociety https://www.biorxiv.org/content/10.1101/2024.03.11.584501v1
Abstract Generative models aim to learn the probability distributions underlying data, enabling the generation of new, realistic samples. Quantum-inspired generative models, such as Born machines based on the matrix product state (MPS) framework, have demonstrated remarkable capabilities in unsupervised learning tasks. This study advances the Born machine paradigm by introducing trainable token embeddings through positive operator-valued measurements (POVMs), replacing the traditional approach of static tensor indices. Key technical innovations include encoding tokens as quantum measurement operators with trainable parameters and leveraging QR decomposition to adjust the physical dimensions of the MPS. This approach maximizes the utilization of operator space and enhances the model’s expressiveness. Empirical results on RNA data demonstrate that the proposed method significantly reduces negative log-likelihood compared to one-hot embeddings, with higher physical dimensions further enhancing single-site probabilities and multi-site correlations. The model also outperforms GPT-2 in single-site estimation and achieves competitive correlation modeling, showcasing the potential of trainable POVM embeddings for complex data correlations in quantum-inspired sequence modeling.