Uyghur Keyword Spotting and Speech Representation Learning Based on an E-Branchformer Encoder-Decoder Architecture

Abstract

Keyword spotting for Uyghur remains challenging because of limited labeled resources, agglutinative morphology, speaker diversity, and unstable boundary cues under partial observation. This paper presents a non-streaming E-Branchformer encoder-decoder framework that unifies Uyghur keyword spotting and speech representation analysis. Beyond a standard keyword spotting pipeline, the study explicitly investigates how hidden representations evolve when only a prefix of an utterance is available. To support this goal, the corpus is subjected to systematic data cleaning, including duplicate removal, damaged-file filtering, language-mix exclusion, and low-quality-sample screening. After unified preprocessing and normalization, a prefix-stage dataset is built by extracting the first 25%, 50%, 75%, and 100% of each utterance, which enables controlled analysis of completeness and discriminability across scanning stages. The proposed model employs an E-Branchformer encoder, an attention-based decoder, and joint CTC/attention training. A representation-oriented multi-task objective combines keyword classification with completeness prediction, while encoded features from different prefix stages are used for discriminability analysis. Experiments on a 134.1 h Uyghur speech corpus demonstrate that the proposed method improves keyword spotting performance over competitive baselines and yields more stable hidden representations under incomplete input. The model reaches an EER of 4.9% and an ATWV of 0.901, while the prefix-stage representation study shows consistent gains in 5-NN discrimination and decreasing completeness error as the observed speech grows. These results indicate that representation-oriented training is beneficial for both keyword spotting accuracy and interpretability.