Automatic accuracy prediction for AMR parsing

• Verfasst von: Opitz, Juri [VerfasserIn]
• Verfasst von: Frank, Anette [VerfasserIn]
• Titel: Automatic accuracy prediction for AMR parsing
• Verf.angabe: Juri Opitz and Anette Frank
• E-Jahr: 2019
• Jahr: 17 Apr 2019
• Umfang: 12 S.
• Teil: year:2019
• Teil: elocationid:1904.08301
• Teil: pages:1-12
• Teil: extent:12
• Fussnoten: Gesehen am 15.07.2019
• Titel Quelle: Enthalten in: De.arxiv.org
• Ort Quelle: [S.l.] : Arxiv.org, 1991
• Jahr Quelle: 2019
• Band/Heft Quelle: (2019), Artikel-ID 1904.08301, Seite 1-12
• Datenträger: Online-Ressource
• Sprache: eng
• Bibliogr. Hinweis: Forschungsdaten: AMR parse quality prediction [Source code]
• Sach-SW: Computer Science - Computation and Language
• K10plus-PPN: 1669121089

Abstract

Abstract Meaning Representation (AMR) represents sentences as directed, acyclic and rooted graphs, aiming at capturing their meaning in a machine readable format. AMR parsing converts natural language sentences into such graphs. However, evaluating a parser on new data by means of comparison to manually created AMR graphs is very costly. Also, we would like to be able to detect parses of questionable quality, or preferring results of alternative systems by selecting the ones for which we can assess good quality. We propose AMR accuracy prediction as the task of predicting several metrics of correctness for an automatically generated AMR parse - in absence of the corresponding gold parse. We develop a neural end-to-end multi-output regression model and perform three case studies: firstly, we evaluate the model's capacity of predicting AMR parse accuracies and test whether it can reliably assign high scores to gold parses. Secondly, we perform parse selection based on predicted parse accuracies of candidate parses from alternative systems, with the aim of improving overall results. Finally, we predict system ranks for submissions from two AMR shared tasks on the basis of their predicted parse accuracy averages. All experiments are carried out across two different domains and show that our method is effective.