Research
Publications
* below indicates equal contribution
2021
Underreporting of errors in NLG output, and what to do about it
Emiel van Miltenburg, Miruna Clinciu, Ondřej Dušek, Dimitra Gkatzia, Stephanie Inglis, Leo Leppänen, Saad Mahamood, Emma Manning, Stephanie Schoch, Craig Thomson, Luou Wen
In Proceedings of the 14th International Conference on Natural Language Generation, August, 2021.
pdf
I don't understand! Evaluation Methods for Natural Language Explanations
Miruna Clinciu, Arash Eshghi and Helen Hastie
In Proceedings of the SICSA eXplainable Artifical Intelligence Workshop 2021 (SICSA XAI 2021), Aberdeen, United Kingdom, June 1st, 2021.
pdf |
slides
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal, Pawan Sasanka Ammanamanchi, Aremu Anuoluwapo, Antoine Bosselut, Khyathi Raghavi Chandu, Miruna Clinciu, Dipanjan Das, Kaustubh D Dhole, Wanyu Du, Esin Durmus, Ondřej Dušek, Chris Emezue, Varun Gangal, Cristina Garbacea, Tatsunori Hashimoto, Yufang Hou, Yacine Jernite, Harsh Jhamtani, Yangfeng Ji, Shailza Jolly, Dhruv Kumar, Faisal Ladhak, Aman Madaan, Mounica Maddela, Khyati Mahajan, Saad Mahamood, Bodhisattwa Prasad Majumder, Pedro Henrique Martins, Angelina McMillan-Major, Simon Mille, Emiel van Miltenburg, Moin Nadeem, Shashi Narayan, Vitaly Nikolaev, Rubungo Andre Niyongabo, Salomey Osei, Ankur Parikh, Laura Perez-Beltrachini, Niranjan Ramesh Rao, Vikas Raunak, Juan Diego Rodriguez, Sashank Santhanam, João Sedoc, Thibault Sellam, Samira Shaikh, Anastasia Shimorina, Marco Antonio Sobrevilla Cabezudo, Hendrik Strobelt, Nishant Subramani, Wei Xu, Diyi Yang, Akhila Yerukola, Jiawei Zhou
pdf |
website
A Study of Automatic Metrics for the Evaluation of Natural Language Explanations
Miruna Adriana Clinciu, Arash Eshghi, H. Hastie
In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume (EACL), 2021.
pdf |
code |
poster |
slides
It’s Common Sense, isn’t it? Demystifying Human Evaluations in Commonsense-enhanced NLG systems
Miruna Clinciu*, Dimitra Gkatzia* and Saad Mahamood*
In Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval, EACL), 2021.
pdf |
code |
website |
poster |
presentation |
video
2020
Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definition
David Howcroft, Anya Belz, Miruna Clinciu, Dimitra Gkatzia, Sadid A. Hasan, Saad Mahamood, Simon Mille, Emiel van Miltenburg, Sashank Santhanam, and Verena Rieser
In Proceedings of the 13th International Conference on Natural Language Generation (INLG), 2020.
pdf |
code
Let's Evaluate Explanations!
Miruna-Adriana Clinciu and Helen Hastie
In Proceedings of HRI 2020 Workshop on Test Methods and Metrics for Effective HRI in Real World Human-Robot Teams (Extended Abstract), 2020.
pdf
2019
A Survey of Explainable AI Terminology
Miruna-Adriana Clinciu and Helen Hastie
In Proceedings of the 1st Workshop on Interactive Natural Language Technology for Explainable Artificial Intelligence (NL4XAI, INLG), 2019.
pdf |
slides