Paul Piwek

Title: Paul Piwek

Authors:

Abstract: This paper introduces the NECA MNLG, a Multi-modal Natural Language Generator. It has been developed in the context of the NECA system, which generates dialogue scripts for animated characters. The generator takes input that allows for the specification of syntactic, semantic, and pragmatic constraints on the output.

Introduction: The NECA MNLG is part of the NECA system, a project aimed at creating embodied emotional conversational agents. The system generates dialogue scripts for animated characters based on user input. This input includes information about the characters, their personalities, and their preferences, as well as a database with facts about the selected car. The generator processes this input in a pipeline that consists of several modules, each adding more concrete information to the dialogue plan. The pipeline includes a Dialogue Planner, a Multi-modal Natural Language Generator, a Speech Synthesis Module, a Gesture Assignment Module, and a Player.

Requirements: The NECA MNLG must support seamless integration of canned text, templates, and full grammar rules. This is necessary because the Dialogue Planner provides varying amounts of information for each dialogue act, including its type, speaker, addressees, semantic content, and emotions. Some dialogue acts have full semantic content, while others may only have domain-specific dialogue act types. This means the generator must be able to handle both types of input and map them to the appropriate output.

Methodology: The NECA MNLG uses a pipeline approach to generate dialogue scripts. It starts with the Dialogue Planner, which creates an abstract description of the dialogue. This is then passed to the Multi-modal Natural Language Generator, which specifies linguistic and non-linguistic realizations for the dialogue acts in the dialogue plan. The Speech Synthesis Module adds speech information, and the Gesture Assignment Module handles the temporal coordination of gestures and speech. Finally, the Player renders the dialogue script.

Results: The NECA MNLG successfully generates dialogue scripts for animated characters based on user input. It handles a variety of input types, including canned text, templates, and full grammar rules. The generator has been implemented in the context of the NECA system, which has been used to create a demonstrator in the car sales domain.

Implications: The NECA MNLG has implications for the field of natural language generation. It demonstrates how to create a generator that can handle a variety of input types and map them to the appropriate output. This could be useful for other natural language generation systems that need to process input with varying amounts of semantic content. Additionally, the NECA MNLG's use of a pipeline approach could provide a useful framework for other systems that need to generate dialogue scripts or other text-based output.

Link to Article: https://arxiv.org/abs/0312050v1 Authors: arXiv ID: 0312050v1