Explicit State Tracking with Semi-Supervisionfor Neural Dialogue Generation

Xisen Jin, Wenqiang Lei, Zhaochun Ren, Hongshen Chen, Shangsong Liang, Yihong Zhao, Dawei Yin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

45 Scopus citations


The task of dialogue generation aims to automatically provide responses given previous utterances. Tracking dialogue states is an important ingredient in dialogue generation for estimating users' intention. However, the expensive nature of state labeling and the weak interpretability make the dialogue state tracking a challenging problem for both task-oriented and non-task-oriented dialogue generation: For generating responses in task-oriented dialogues, state tracking is usually learned from manually annotated corpora, where the human annotation is expensive for training; for generating responses in non-task-oriented dialogues, most of existing work neglects the explicit state tracking due to the unlimited number of dialogue states. In this paper, we propose the semi-supervised explicit dialogue state tracker (SEDST) for neural dialogue generation. To this end, our approach has two core ingredients: CopyFlowNet and posterior regularization. Specifically, we propose an encoder-decoder architecture, named CopyFlowNet, to represent an explicit dialogue state with a probabilistic distribution over the vocabulary space. To optimize the training procedure, we apply a posterior regularization strategy to integrate indirect supervision. Extensive experiments conducted on both task-oriented and non-task-oriented dialogue corpora demonstrate the effectiveness of our proposed model. Moreover, we find that our proposed semi-supervised dialogue state tracker achieves a comparable performance as state-of-the-art supervised learning baselines in state tracking procedure.
Original languageEnglish (US)
Title of host publicationProceedings of the 27th ACM International Conference on Information and Knowledge Management - CIKM '18
PublisherAssociation for Computing Machinery (ACM)
Number of pages10
ISBN (Print)9781450360142
StatePublished - Oct 22 2018


Dive into the research topics of 'Explicit State Tracking with Semi-Supervisionfor Neural Dialogue Generation'. Together they form a unique fingerprint.

Cite this