Improving NLP Accuracy with Stack-Propagation and Knowledge Distillation: A Joint Model for Intent Detection and Slot Filling

Abstract

 Intent detection and slot filling are fundamental tasks in Natural Language Understanding (NLU), constituting important components of intelligent question-answering systems. These tasks are closely interrelated, forming a core aspect of semantic understanding in natural language processing. In this paper, we propose an architecture using Stack-Propagation to improve the accuracy of NLU tasks. In stack propagation, we use a joint model that mainly incorporates token-level intent detection data with sentence word vectors as input for slot filling to capture intent semantic knowledge. Additionally, we use knowledge distillation (KD) to improve model efficiency and enhance the correlation between the two tasks. Our proposed framework significantly differs from existing joint models as it directly leverages intent information in the joint model and adopts token-level intent information for slot filling to ease error propagation. Furthermore, our model can explicitly incorporate intent information for slot filling with Stack-Propagation, making the interaction procedure more interpretable, while other models only interact with hidden states implicitly between the two tasks. We experimentally evaluated our model on two publicly available datasets, and the results demonstrate that it achieves state-of-the-art performance and outperforms previous methods by a significant margin, indicating its superiority in addressing the slot-filling and intent detection tasks. In future research, we will combine the stack-propagation frame with the KD module in the Transform model, which can boost our model performance to move forward a single step in the NLU task.