Guanyi Chen | publications

2025

On the Human-level Performance of Visual Question Answering Chenlian Zhou, Guanyi Chen*, Xin Bai, and Ming Dong In COLING 2025, Abu Dhabi
A Comparative Analysis of Metaphorical Cognition in ChatGPT and Human Minds Rui Mao, Guanyi Chen, Xiao Li, Mengshi Ge, and Erik Cambria Cognitive Computation

2024

Intrinsic Task-based Evaluation for Referring Expression Generation Guanyi Chen, Fahime Same, and Kees van Deemter In ACL 2024, Bangkok
Computational Modelling of Plurality and Definiteness in Chinese Noun Phrases Yuqi Liu, Guanyi Chen*, and Kees van Deemter In LREC-COLING 2024, Torino
GPTEval: A Survey on Assessments of ChatGPT and GPT-4 Rui Mao, Guanyi Chen, Xulang Zhang, Frank Guerin, and Erik Cambria In LREC-COLING 2024, Torino
A Survey on Semantic Processing Techniques Rui Mao*, Kai He*, Xulang Zhang*, Guanyi Chen*, JinJie Ni*, Zonglin Yang*, and Erik Cambria Information Fusion

2023

Varieties of specification: Redefining over- and under-specification Guanyi Chen, and Kees van Deemter Journal of Pragmatics
Models of reference production: How do they withstand the test of time? Fahime Same, Guanyi Chen, and Kees van Deemter In INLG 2023, Prague
A Multi-task Learning Model for Gold-two-mention Co-reference Resolution Ruicheng Liu, Guanyi Chen, Rui Mao, and Erik Cambria In IJCNN 2023, Queensland
Computational Modelling of Quantifier Use: Elicitation Experiments, Models, and Evaluation Guanyi Chen, and Kees van Deemter Journal of Artificial Intelligence Research
Neural Referential Form Selection: Generalisability and Interpretability Guanyi Chen, Fahime Same, and Kees van Deemter Computer Speech and Language

2022

Assessing Neural Referential Form Selectors on a Realistic Multilingual Dataset Guanyi Chen, Fahime Same, and Kees van Deemter In AACL 2022 on Eval4NLP Workshop, Online
Understanding the Use of Quantifiers in Mandarin Guanyi Chen, and Kees van Deemter In AACL-Findings 2022, Online [arXiv] [PDF]
DrivingBeacon: Driving Behaviour Change Support System Considering Mobile Use and Geo-information Jawwad Baig, Guanyi Chen, Chenghua Lin, and Ehud Reiter In INLG 2022 on NLG4Healthcare Workshop, Maine [PDF]
Computational Generation of Chinese Noun Phrases Guanyi Chen PhD Thesis, Utrecht University [PDF]
Non-neural Models can Matter: a Re-evaluation of Neural Referring Expression Generation Systems Fahime Same*, Guanyi Chen*, and Kees van Deemter In ACL 2022, Dublin [Abs] [arXiv] [PDF] [Data]
In recent years, neural models have often outperformed rule-based and classic Machine Learning approaches in NLG. These classic approaches are now often disregarded, for example when new neural models are evaluated. We argue that they should not be overlooked, since, for some tasks, well-designed non-neural approaches achieve better performance than neural ones. In this paper, the task of generating referring expressions in linguistic context is used as an example. We examined two very different English datasets (WEBNLG and WSJ), and evaluated each algorithm using both automatic and human evaluations. Overall, the results of these evaluations suggest that rule-based systems with simple rule sets achieve on-par or better performance on both datasets compared to state-of-the-art neural REG systems. In the case of the more realistic dataset, WSJ, a machine learning-based system with well-designed linguistic features performed best. We hope that our work can encourage researchers to consider non-neural models in future.
MMChat: Multi-Modal Chat Dataset on Social Media Yinhe Zheng*, Guanyi Chen*, Xin Liu, and Jian Sun In LREC 2022, Marseille [Abs] [arXiv] [PDF] [Data]
Incorporating multi-modal contexts in conversation is an important step for developing more engaging dialogue systems. In this work, we explore this direction by introducing MMChat: a large scale multi-modal dialogue corpus (32.4M raw dialogues and 120.84K filtered dialogues). Unlike previous corpora that are crowd-sourced or collected from fictitious movies, MMChat contains image-grounded dialogues collected from real conversations on social media, in which the sparsity issue is observed. Specifically, image-initiated dialogues in common communications may deviate to some non-image-grounded topics as the conversation proceeds. We develop a benchmark model to address this issue in dialogue generation tasks by adapting the attention routing mechanism on image features. Experiments demonstrate the usefulness of incorporating image features and the effectiveness in handling the sparsity of image features.

2021

What can Neural Referential Form Selectors Learn? Guanyi Chen*, Fahime Same*, and Kees van Deemter In INLG 2021, Online [Abs] [arXiv] [Code]
Despite achieving encouraging results, neural Referring Expression Generation models are often thought to lack transparency. We probed neural Referential Form Selection (RFS) models to find out to what extent the linguistic features influencing the RE form are learnt and captured by state-of-the-art RFS models. The results of 8 probing tasks show that all the defined features were learnt to some extent. The probing tasks pertaining to referential status and syntactic position exhibited the highest performance. The lowest performance was achieved by the probing models designed to predict discourse structure properties beyond the sentence level.
Affective Decoding for Empathetic Response Generation Chengkun Zeng, Guanyi Chen, Chenghua Lin, Ruizhe Li, and Zhigang Chen In INLG 2021, Online [Abs] [arXiv] [Code]
Understanding speaker’s feelings and producing appropriate responses with emotion connection is a key communicative skill for empathetic dialogue systems. In this paper, we propose a simple technique called Affective Decoding for empathetic response generation. Our method can effectively incorporate emotion signals during each decoding step, and can additionally be augmented with an auxiliary dual emotion encoder, which learns separate embeddings for the speaker and listener given the emotion base of the dialogue. Extensive empirical studies show that our models are perceived to be more empathetic by human evaluations, in comparison to several strong mainstream methods for empathetic responding.
Using BERT for choosing classifiers in Mandarin Jani Jarnfors, Guanyi Chen, Kees van Deemter, and Rint Sybesma In INLG 2021, Online
Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis Xutan Peng, Guanyi Chen, Chenghua Lin, and Mark Stevenson In NAACL 2021, Online [Abs] [arXiv] [Code]
Knowledge Graph Embeddings (KGEs) have been intensively explored in recent years due to their promise for a wide range of applications. However, existing studies focus on improving the final model performance without acknowledging the computational cost of the proposed approaches, in terms of execution time and environmental impact. This paper proposes a simple yet effective KGE framework which can reduce the training time and carbon footprint by orders of magnitudes compared with state-of-the-art approaches, while producing competitive performance. We highlight three technical innovations: full batch learning via relational matrices, closed-form Orthogonal Procrustes Analysis for KGEs, and non-negative-sampling training. In addition, as the first KGE method whose entity embeddings also store full relation information, our trained models encode rich semantics and are highly interpretable. Comprehensive experiments and ablation studies involving 13 strong baselines and two standard datasets verify the effectiveness and efficiency of our algorithm.

2020

Listener’s Social Identity Matters in Personalised Response Generation Guanyi Chen, Yinhe Zheng, and Yupei Du In INLG 2020, Online [Abs] [arXiv] [PDF]
Personalised response generation enables generating human-like responses by means of assigning the generator a social identity. However, pragmatics theory suggests that human beings adjust the way of speaking based on not only who they are but also whom they are talking to. In other words, when modelling personalised dialogues, it might be favourable if we also take the listener’s social identity into consideration. To validate this idea, we use gender as a typical example of a social variable to investigate how the listener’s identity influences the language used in Chinese dialogues on social media. Also, we build personalised generators. The experiment results demonstrate that the listener’s identity indeed matters in the language use of responses and that the response generator can capture such differences in language use. More interestingly, by additionally modelling the listener’s identity, the personalised response generator performs better in its own identity.
Lessons from Computational Modelling of Reference Production in Mandarin and English Guanyi Chen, and Kees van Deemter In INLG 2020, Online [Abs] [arXiv] [PDF] [Data]
Referring expression generation (REG) algorithms offer computational models of the production of referring expressions. In earlier work, a corpus of referring expressions (REs) in Mandarin was introduced. In the present paper, we annotate this corpus, evaluate classic REG algorithms on it, and compare the results with earlier results on the evaluation of REG for English referring expressions. Next, we offer an in-depth analysis of the corpus, focusing on issues that arise from the grammar of Mandarin. We discuss shortcomings of previous REG evaluations that came to light during our investigation and we highlight some surprising results. Perhaps most strikingly, we found a much higher proportion of under-specified expressions than previous studies had suggested, not just in Mandarin but in English as well.
Gradations of Error Severity in Automatic Image Descriptions Emiel van Miltenburg, Wei-Ting Lu, Emiel Krahmer, Albert Gatt, Guanyi Chen, Lin Li, and Kees van Deemter In INLG 2020, Online [Abs] [PDF]
Earlier research has shown that evaluation metrics based on textual similarity (e.g., BLEU, CIDEr, Meteor) do not correlate well with human evaluation scores for automatically generated text. We carried out an experiment with Chinese speakers, where we systematically manipulated image descriptions to contain different kinds of errors. Because our manipulated descriptions form minimal pairs with the reference descriptions, we are able to assess the impact of different kinds of errors on the perceived quality of the descriptions. Our results show that different kinds of errors elicit significantly different evaluation scores, even though all erroneous descriptions differ in only one character from the reference descriptions. Evaluation metrics based solely on textual similarity are unable to capture these differences, which (at least partially) explains their poor correlation with human judgments. Our work provides the foundations for future work, where we aim to understand why different errors are seen as more or less severe.
Improving Variational Autoencoder for Text Modelling with Timestep-Wise Regularisation Ruizhe Li, Xiao Li, Guanyi Chen, and Chenghua Lin In COLING 2020, Online [Abs] [arXiv] [PDF]
The Variational Autoencoder (VAE) is a popular and powerful model applied to text modelling to generate diverse sentences. However, an issue known as posterior collapse (or KL loss vanishing) happens when the VAE is used in text modelling, where the approximate posterior collapses to the prior, and the model will totally ignore the latent variables and be degraded to a plain language model during text generation. Such an issue is particularly prevalent when RNN-based VAE models are employed for text modelling. In this paper, we propose a simple, generic architecture called Timestep-Wise Regularisation VAE (TWR-VAE), which can effectively avoid posterior collapse and can be applied to any RNN-based VAE models. The effectiveness and versatility of our model are demonstrated in different tasks, including language modelling and dialogue response generation.
DGST: a Dual-Generator Network for Text Style Transfer Xiao Li, Guanyi Chen, Chenghua Lin, and Ruizhe Li In EMNLP 2020, Online [Abs] [arXiv] [PDF] [Code]
We propose DGST, a novel and simple Dual-Generator network architecture for text Style Transfer. Our model employs two generators only, and does not rely on any discriminators or parallel corpus for training. Both quantitative and qualitative experiments on the Yelp and IMDb datasets show that our model gives competitive performance compared to several strong baselines with more complicated architecture designs.
Out-of-domain Detection for Natural Language Understanding in Dialog Systems Yinhe Zheng, Guanyi Chen, and Minlie Huang IEEE/ACM Transactions Audio, Speech, and Language Processing [Abs] [arXiv]
Natural Language Understanding (NLU) is a vital component of dialogue systems, and its ability to detect Out-of-Domain (OOD) inputs is critical in practical applications, since the acceptance of the OOD input that is unsupported by the current system may lead to catastrophic failure. However, most existing OOD detection methods rely heavily on manually labeled OOD samples and cannot take full advantage of unlabeled data. This limits the feasibility of these models in practical applications. In this paper, we propose a novel model to generate high-quality pseudo OOD samples that are akin to IN-Domain (IND) input utterances and thereby improves the performance of OOD detection. To this end, an autoencoder is trained to map an input utterance into a latent code. Moreover, the codes of IND and OOD samples are trained to be indistinguishable by utilizing a generative adversarial network. To provide more supervision signals, an auxiliary classifier is introduced to regularize the generated OOD samples to have indistinguishable intent labels. Experiments show that these pseudo OOD samples generated by our model can be used to effectively improve OOD detection in NLU. Besides, we also demonstrate that the effectiveness of these pseudo OOD data can be further improved by efficiently utilizing unlabeled data.

2019

QTUNA: A Corpus for Understanding How Speakers Use Quantification Guanyi Chen, Kees van Deemter, Silvia Pagliaro, Louk Smalbil, and Chenghua Lin In INLG 2019, Tokyo [Abs] [PDF] [Supplemantary] [Data]
A prominent strand of work in formal semantics investigates the ways in which human languages quantify over the elements of a set, as when we say “All A are B”, “All except two A are B”, “Only a few of the A are B” and so on. Our aim is to build Natural Language Generation algorithms that mimic humans’ use of quantified expressions. To inform these algorithms, we conducted on a series of elicitation experiments in which human speakers were asked to perform a linguistic task that invites the use of quantified expressions. We discuss how these experiments were conducted and what corpora they gave rise to. We conduct an informal analysis of the corpora, and offer an initial assessment of the challenges that these corpora pose for Natural Language Generation.
Generating Quantified Descriptions of Abstract Visual Scenes Guanyi Chen, Kees van Deemter, and Chenghua Lin In INLG 2019, Tokyo [Abs] [PDF] [Supplemantary] [Code]
Quantified expressions have always taken up a central position in formal theories of meaning and language use. Yet quantified expressions have so far attracted far less attention from the Natural Language Generation community than, for example, referring expressions. In an attempt to start redressing the balance, we investigate a recently developed corpus in which quantified expressions play a crucial role; the corpus is the result of a carefully controlled elicitation experiment, in which human participants were asked to describe visually presented scenes. Informed by an analysis of this corpus, we propose algorithms that produce computer-generated descriptions of a wider class of visual scenes, and we evaluate the descriptions generated by these algorithms in terms of their correctness, completeness, and human-likeness. We discuss what this exercise can teach us about the nature of quantification and about the challenges posed by the generation of quantified expressions.
A Closer Look at Recent Results of Verb Selection for Data-to-Text NLG Guanyi Chen, and Jin-Ge Yao In INLG 2019, Tokyo [Abs] [PDF] [Supplemantary]
Automatic natural language generation systems need to use the contextually-appropriate verbs when describing different kinds of facts or events, which has triggered research interest on verb selection for data-to-text generation. In this paper, we discuss a few limitations of the current task settings and the evaluation metrics. We also provide two simple, efficient, interpretable baseline approaches for statistical selection of trend verbs, which give a strong performance on both previously used evaluation metrics and our new evaluation.
Persona-aware Dialogue Generation with Enriched Profile Yinhe Zheng, Guanyi Chen, Minlie Huang, Song Liu, and Xuan Zhu In NeurIPS 2019 on Conversational AI Workshop, Vancouver Nomination for Best Paper Award [Abs] [arXiv]
Endowing a dialogue system with particular persona is essential to deliver more human-like conversations. However, due to the difficulties of embodying personalities in natural language, this problem is still far from well studied. This paper proposes a novel task of generating dialogue responses conditioned on explicit personal profiles with rich attributes. A dataset is constructed to facilitate the proposed task and a persona-aware dialogue generation model is also introduced. In this model, a structured personal profile (in key-value pairs) is transformed and composed into a representation vector using a persona fusion module, and several different fusion methods are tested. Two techniques, namely \it persona-aware attention and persona-aware bias, are devised to capture and incorporate various persona attributes in the decoding process. Experiments and case studies demonstrate that our model is able to address proper attributes in different contexts.
A Dual-Attention Hierarchical Recurrent Neural Network for Dialogue Act Classification Ruizhe Li, Chenghua Lin, Matthew Collinson, Xiao Li, and Guanyi Chen In CoNLL 2019, Hong Kong, China [Abs] [PDF]
Recognising dialogue acts (DA) is important for many natural language processing tasks such as dialogue generation and intention recognition. In this paper, we propose a dual-attention hierarchical recurrent neural network for DA classification. Our model is partially inspired by the observation that conversational utterances are normally associated with both a DA and a topic, where the former captures the social act and the latter describes the subject matter. However, such a dependency between DAs and topics has not been utilised by most existing systems for DA classification. With a novel dual task-specific attention mechanism, our model is able, for utterances, to capture information about both DAs and topics, as well as information about the interactions between them. Experimental results show that by modelling topic as an auxiliary task, our model can significantly improve DA classification, yielding better or comparable performance to the state-of-the-art method on three public datasets.

2018

SimpleNLG-ZH: a Linguistic Realisation Engine for Mandarin Guanyi Chen, Kees van Deemter, and Chenghua Lin In INLG 2018, Tilburg [Abs] [PDF] [Code]
We introduce SimpleNLG-ZH, a realisation engine for Mandarin that follows the software design paradigm of SimpleNLG (Gatt and Reiter, 2009). We explain the core grammar (morphology and syntax) and the lexicon of SimpleNLG-ZH, which is very different from English and other languages for which SimpleNLG engines have been built. The system was evaluated by regenerating expressions from a body of test sentences and a corpus of human-authored expressions. Human evaluation was conducted to estimate the quality of regenerated sentences.
Modelling Pro-drop with the Rational Speech Acts Model Guanyi Chen, Kees van Deemter, and Chenghua Lin In INLG 2018, Tilburg [Abs] [PDF]
We extend the classic Referring Expressions Generation task by considering zero pronouns in “pro-drop” languages such as Chinese, modelling their use by means of the Bayesian Rational Speech Acts model (Frank and Goodman, 2012). By assuming that highly salient referents are most likely to be referred to by zero pronouns (i.e., pro-drop is more likely for salient referents than the less salient ones), the model offers an attractive explanation of a phenomenon not previously addressed probabilistically.
ABDN at SemEval-2018 Task 10: Recognising Discriminative Attributes using Context Embeddings and WordNet Rui Mao, Guanyi Chen, Ruizhe Li, and Chenghua Lin In NAACL 2018 on SemEval Workshop, New Orleans [Abs] [PDF]
This paper describes the system that we submitted for SemEval-2018 task 10: capturing discriminative attributes. Our system is built upon a simple idea of measuring the attribute word’s similarity with each of the two semantically similar words, based on an extended word embedding method and WordNet. Instead of computing the similarities between the attribute and semantically similar words by using standard word embeddings, we propose a novel method that combines word and context embeddings which can better measure similarities. Our model is simple and effective, which achieves an average F1 score of 0.62 on the test set.