Abstract
CoHManD addresses the need for effective and safe communication between humans and machines on construction sites, where misunderstandings are costly and dangerous and cognitive resources are limited due to high task demands, time pressure, and dynamic conditions. The project aims to develop a cognitive, dialogue-based foundation model that enables intuitive natural-language communication while delivering only essential, task-critical information. Through predictive context modeling, CoHManD seeks to minimize cognitive load while maintaining operational clarity and decision confidence.
CoHManD combines a locally hosted multimodal Large Language Model with a domain-specific Linked Data model. Content quality is ensured using techniques such as contextual prompts grounded in the knowledge base, Chain-of-Thought reasoning, and tailored safeguards against misleading or nonsensical outputs. Stakeholder-centered methods, including focus groups, are used to identify task demands and communication requirements. System performance is evaluated in empirical studies examining information perception, mental load, and communication efficiency in various interaction scenarios.
CoHManD ensures robust, context-sensitive human–machine interaction even under incomplete or noisy data, substantially mitigating hallucinations in decision support. The system emphasizes mutual understanding by translating technical data into accessible formats and interpreting informal human input into precise machine-readable instructions. By mediating communication between humans and machines, CoHManD results in a validated framework for usable, cognitively efficient, and trustworthy human–machine communication on future construction sites.
