计算机与网络

针对多模态情绪分析在学习状态检测领域的应用，设计了一种名为递归多模态情绪系统(Recursive Multimodal Emotion Framework, RMEF)的递归层叠框架模型。该模型能够整合学习状态中的各类特征进行分类，包括研讨场合、交互属性以及时序连贯性等内容。在多模态特征提取研究中，通过OpenFace2.0从多人视频资料中获取了面部关键点位置、面部运动单元类型、目光聚焦方向和头部姿态信息。同时，提出了一种结合视觉与听觉的方法来探查实训对话场景，该方法将面部特别是嘴部信息与同一瞬间的听觉信息融合，以捕获当下讲话者的情况。实验结果表明，RMEF模型不仅提高了算法识别的精确程度，还显著增强了模型的适应能力，在企业从教育类软件开发迈向智能化进程中展现出重要价值，为多模态情绪分析在学习状态检测中的应用提供了新的技术路径和解决方案。

Aiming at the application of multimodal emotion analysis in the field of learning state detection, a recursive cascaded framework model named Recursive Multimodal Emotion Framework(RMEF) is designed. The framework can integrate various features in the learning state for classification purposes, including discussion occasions, interaction attributes, and temporal coherence. In the research of multimodal feature extraction, facial landmark positions, facial motion units, gaze direction, and head posture information are obtained from multi-person video data using OpenFace2.0. Additionally, a novel approach combining visual and auditory cues is developed to investigate practical training dialogue scenarios, which can fuse facial information, especially mouth related features, with concurrent auditory information to capture the situation of the current speaker. Experimental results demonstrate that the RMEF model not only improves the algorithm recognition accuracy but also significantly improves the model adaptability, which shows considerable value as enterprises transition from educational software development toward intelligent systems. The research provides new technical pathways and solutions for the application of multimodal emotion analysis in learning state detection.