Quantum self-attention neural networks for text classification

Abstract

An emerging direction of quantum computing is to establish meaningful quantum applications in various fields of artificial intelligence, including natural language processing (NLP). Although some efforts based on syntactic analysis have opened the door to research in quantum NLP (QNLP), limitations such as heavy syntactic preprocessing and syntax-dependent network architecture make them impracticable on larger and real-world data sets. In this paper, we propose a new simple network architecture, called the quantum self-attention neural network (QSANN), which can compensate for these limitations. Specifically, we introduce the self-attention mechanism into quantum neural networks and then utilize a Gaussian projected quantum self-attention serving as a sensible quantum version of self-attention. As a result, QSANN is effective and scalable on larger data sets and has the desirable property of being implementable on near-term quantum devices. In particular, our QSANN outperforms the best existing QNLP model based on syntactic analysis as well as a simple classical self-attention neural network in numerical experiments of text classification tasks on public data sets. We further show that our method exhibits robustness to low-level quantum noises and showcases resilience to quantum neural network architectures.

Guangxi Li

Visiting Scholar

I obtained my BS and MS in Computer Science from University of Electronic Science and Technology of China. I obtained my PhD degree in Computer Science from University of Technology Sydney. My research interests include quantum computing and quantum machine learning.

Xin Wang

Associate Professor

Prof. Xin Wang founded the QuAIR lab at HKUST(Guangzhou) in June 2023. His research primarily focuses on better understanding the limits of information processing with quantum systems and the power of quantum artificial intelligence. Prior to establishing the QuAIR lab, Prof. Wang was a Staff Researcher at the Institute for Quantum Computing at Baidu Research, where he concentrated on quantum computing research and the development of the Baidu Quantum Platform. Notably, he spearheaded the development of Paddle Quantum, a Python library designed for quantum machine learning. From 2018 to 2019, Prof. Wang held the position of Hartree Postdoctoral Fellow at the Joint Center for Quantum Information and Computer Science (QuICS) at the University of Maryland, College Park. He earned his doctorate in quantum information from the University of Technology Sydney in 2018, under the guidance of Prof. Runyao Duan and Prof. Andreas Winter. In 2014, Prof. Wang obtained his B.S. in mathematics (with Wu Yuzhang Honor) from Sichuan University.