2rd Place Solutions in the HC-STVG track of Person in Context Challenge 2021
This is an incremental improvement for video understanding tasks, specifically targeting person localization in videos using natural language queries.
The paper tackled the problem of localizing a spatio-temporal person in an untrimmed video based on a sentence, achieving second place with a vIOU score of 0.30025 in the HC-STVG track of the Person in Context Challenge 2021.
In this technical report, we present our solution to localize a spatio-temporal person in an untrimmed video based on a sentence. We achieve the second vIOU(0.30025) in the HC-STVG track of the 3rd Person in Context(PIC) Challenge. Our solution contains three parts: 1) human attributes information is extracted from the sentence, it is helpful to filter out tube proposals in the testing phase and supervise our classifier to learn appearance information in the training phase. 2) we detect humans with YoloV5 and track humans based on the DeepSort framework but replace the original ReID network with FastReID. 3) a visual transformer is used to extract cross-modal representations for localizing a spatio-temporal tube of the target person.