Biomimetics, Vol. 10, Pages 566: A Novel Fish Pose Estimation Method Based on Semi-Supervised Temporal Context Network

Biomimetics doi: 10.3390/biomimetics10090566

Authors:
Yuanchang Wang
Ming Wang
Jianrong Cao
Chen Wang
Zhen Wu
He Gao

Underwater biomimetic robotic fish are emerging as vital platforms for ocean exploration tasks such as environmental monitoring, biological observation, and seabed investigation, particularly in areas inaccessible to humans. Central to their effectiveness is high-precision fish pose estimation, which enables detailed analysis of swimming patterns and ecological behavior, while informing the design of agile, efficient bio-inspired robots. To address the widespread scarcity of high-quality motion datasets in this domain, this study presents a custom-built dual-camera experimental platform that captures multi-view sequences of carp exhibiting three representative swimming behaviors—straight swimming, backward swimming, and turning—resulting in a richly annotated dataset. To overcome key limitations in existing pose estimation methods, including heavy reliance on labeled data and inadequate modeling of temporal dependencies, a novel Semi-supervised Temporal Context-Aware Network (STC-Net) is proposed. STC-Net incorporates two innovative unsupervised loss functions—temporal continuity loss and pose plausibility loss—to leverage both annotated and unannotated video frames, and integrates a Bi-directional Convolutional Recurrent Neural Network to model spatio-temporal correlations across adjacent frames. These enhancements are architecturally compatible and computationally efficient, preserving end-to-end trainability. Experimental results on the proposed dataset demonstrate that STC-Net achieves a keypoint detection RMSE of 9.71, providing a robust and scalable solution for biological pose estimation under complex motion scenarios.

Source link

Yuanchang Wang www.mdpi.com