Document Type

Article

Publication Date

2023

DOI

10.3390/fi15120375

Publication Title

Future Internet

Volume

15

Issue

12

Pages

375 (1-36)

Abstract

Large language models (LLMs) excel in providing natural language responses that sound authoritative, reflect knowledge of the context area, and can present from a range of varied perspectives. Agent-based models and simulations consist of simulated agents that interact within a simulated environment to explore societal, social, and ethical, among other, problems. Simulated agents generate large volumes of data and discerning useful and relevant content is an onerous task. LLMs can help in communicating agents' perspectives on key life events by providing natural language narratives. However, these narratives should be factual, transparent, and reproducible. Therefore, we present a structured narrative prompt for sending queries to LLMs, we experiment with the narrative generation process using OpenAI's ChatGPT, and we assess statistically significant differences across 11 Positive and Negative Affect Schedule (PANAS) sentiment levels between the generated narratives and real tweets using chi-squared tests and Fisher's exact tests. The narrative prompt structure effectively yields narratives with the desired components from ChatGPT. In four out of forty-four categories, ChatGPT generated narratives which have sentiment scores that were not discernibly different, in terms of statistical significance (alpha level α = 0.05), from the sentiment expressed in real tweets. Three outcomes are provided: (1) a list of benefits and challenges for LLMs in narrative generation; (2) a structured prompt for requesting narratives of an LLM chatbot based on simulated agents' information; (3) an assessment of statistical significance in the sentiment prevalence of the generated narratives compared to real tweets. This indicates significant promise in the utilization of LLMs for helping to connect a simulated agent's experiences with real people.

Rights

© 2023 by the authors.

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution 4.0 International (CC BY 4.0) License.

Data Availability

Article states: "All the data, code, and generated narratives utilized in this article are freely accessible in an online repository [86]."

Original Publication Citation

Lynch, C. J., Jensen, E. J., Zamponi, V., O’Brien, K., Frydenlund, E., & Gore, R. (2023). A structured narrative prompt for prompting narratives from large language models: Sentiment assessment of ChatGPT-generated narratives and real tweets. Future Internet, 15(12), 1-36, Article 375. https://doi.org/10.3390/fi15120375

ORCID

0000-0002-4830-7488 (Lynch), 0000-0001-9079-0947 (Zamponi), 0000-0002-7694-7845 (Frydenlund)

Download

Share

COinS