Main Menu (Mobile)- Block

Main Menu - Block

custom | custom

Search Results

filters_region_cap | custom

Filter

facetapi-Q2b17qCsTdECvJIqZJgYMaGsr8vANl1n | block

Associated Lab

facetapi-PV5lg7xuz68EAY8eakJzrcmwtdGEnxR0 | block

Publication Date

general_search_page-panel_pane_1 | views_panes

1 Janelia Publications

Showing 1-1 of 1 results
Your Criteria:
    Branson Lab
    05/14/20 | Detecting the Starting Frame of Actions in Video
    Kwak IS, Guo J, Hantman A, Branson K, Kriegman D
    2020 IEEE Winter Conference on Applications of Computer Vision (WACV). 2020 May 14:. doi: 10.1109/WACV45572.202010.1109/WACV45572.2020.9093405

    In this work, we address the problem of precisely localizing key frames of an action, for example, the precise time that a pitcher releases a baseball, or the precise time that a crowd begins to applaud. Key frame localization is a largely overlooked and important action-recognition problem, for example in the field of neuroscience, in which we would like to understand the neural activity that produces the start of a bout of an action. To address this problem, we introduce a novel structured loss function that properly weights the types of errors that matter in such applications: it more heavily penalizes extra and missed action start detections over small misalignments. Our structured loss is based on the best matching between predicted and labeled action starts. We train recurrent neural networks (RNNs) to minimize differentiable approximations of this loss. To evaluate these methods, we introduce the Mouse Reach Dataset, a large, annotated video dataset of mice performing a sequence of actions. The dataset was collected and labeled by experts for the purpose of neuroscience research. On this dataset, we demonstrate that our method outperforms related approaches and baseline methods using an unstructured loss.

    View Publication Page