This research investigates the impact of integratingcontext-aware large language model-based non-player charac-ter (NPC) conversations on player engagement in role-playinggames. By utilizing the capabilities of context-aware dialoguesgenerated by language models, the study aims to enhance playerengagement with context-aware NPCs. Through an experimentalstudy (n=21), measuring the player engagement and perceivedconversation quality, the results demonstrate that context-awareLLM-based NPC conversations have high potential in increasingplayer engagement in RPGs. Design guidelines for implementingcontext-aware NPC interactions are identified, while acknowledg-ing the limitations of the study. Further research is recommendedto explore other aspects of the topic.