A multi-purpose automatic editing system based on lecture semantics for remote education
This addresses the need for better information delivery in remote teaching, but it is incremental as it builds on existing multi-camera systems by incorporating semantic analysis.
The paper tackles the problem of poor viewer experience in remote education by proposing an automatic multi-camera editing system that selects the most relevant view based on lecture semantics, rather than just tracking the speaker, to guide student attention. It demonstrates effectiveness through qualitative and quantitative analyses, though no concrete numbers are provided.
Remote teaching has become popular recently due to its convenience and safety, especially under extreme circumstances like a pandemic. However, online students usually have a poor experience since the information acquired from the views provided by the broadcast platforms is limited. One potential solution is to show more camera views simultaneously, but it is technically challenging and distracting for the viewers. Therefore, an automatic multi-camera directing/editing system, which aims at selecting the most concerned view at each time instance to guide the attention of online students, is in urgent demand. However, existing systems mostly make simple assumptions and focus on tracking the position of the speaker instead of the real lecture semantics, and therefore have limited capacities to deliver optimal information flow. To this end, this paper proposes an automatic multi-purpose editing system based on the lecture semantics, which can both direct the multiple video streams for real-time broadcasting and edit the optimal video offline for review purposes. Our system directs the views by semantically analyzing the class events while following the professional directing rules, mimicking a human director to capture the regions of interest from the viewpoint of the onsite students. We conduct both qualitative and quantitative analyses to verify the effectiveness of the proposed system and its components.