Template Following of Generative Counterarguments for Large Language Models

Abstract

While debate-based education is effective in developing critical thinking skills, it is prohibitively costly for teachers to provide examples of counter-arguments from various perspectives. To assist in the feedback process, we explore the controllability of logical structures in generating counter-arguments. Specifically, we develop ten counter-argument templates and use them as an instruction for large language models to guide the structure of the generated counter-arguments. Our manual evaluation of 20 arguments shows that, despite the approach's simplicity, the generated counter-arguments generally adhere to the provided templates. However, it also reveals that the counter-arguments generated from templates with complex logical structures tend to lack self-consistency.