Abstract
Studying intelligent virtual shooting in cloud performing arts scenes is of great significance to the sustainable development of the cloud performing arts industry. Difficulties in summarizing the language of shots and the need to consider the attributes of cameras and actor movements are key concerns in stage filming. We propose EVCPP: Example-driven Virtual Camera Pose Prediction for cloud performing arts that uses existing shooting videos to guide the shooting of virtual scenes. By using the camera behavior information from reference videos as external guidance weights, along with the camera intrinsic parameters and actor state information as covariates, we combine them with the historical pose of the camera in the virtual scene to predict the future camera pose. Meanwhile, we propose a combined loss function for our task. Our method has achieved promising results in virtual 3D cloud performing arts scenes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Wang, X., Courant, R., Shi, J., et al.: JAWS: just a wild shot for cinematic transfer in neural radiance fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16933–16942 (2023)
Lou, Z.: The Design and Application of Knowledge-based Intelligent Shot Planning System. Zhejiang University (2011)
Wang, M., Yang, G.W., Hu, S.M., et al.: Write-a-video: computational video montage from themed text. ACM Trans. Graph. 38(6), 1–13 (2019)
Xiong, Y., Heilbron, F.C., Lin, D.: Transcript to video: efficient clip sequencing from texts (2021)
Chen, J., Carr, P.: mimicking human camera operators In: 2015 IEEE Winter Conference on Applications of Computer Vision. IEEE (2015)
Chen, J., Le, H.M., Carr, P., et al.: Learning online smooth predictors for realtime camera planning using recurrent decision trees. In: Computer Vision & Pattern Recognition. IEEE (2016)
Huang, C., Lin, C.E., Yang, Z., et al.: Learning to film from professional human motion videos. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2019)
Jiang, H., Wang, B., Wang, X., et al.: Example-driven virtual cinematography by learning camera behaviors. ACM Trans. Graph. (TOG), 39(4), 45:1–45:14 (2020)
Jiang, H., Christie, M., Wang, X., et al.: Camera keyframing with style and control. ACM Trans. Graph. (TOG) 40(6), 1–13 (2021)
Lino, C., Christie, M.: Intuitive and efficient camera control with the toric space. ACM Trans. Graph. 34(4CD):82.1–82.12 (2015)
Yu, Z., Yu, C., Wang, H., et al.: Enabling automatic cinematography with reinforcement learning. In: 2022 IEEE 5th International Conference on Multimedia Information Processing and Retrieval (MIPR). IEEE, pp. 103–108 2022
Gschwindt, M., Camci, E., Bonatti, R., et al.: Can a robot become a movie director? learning artistic principles for aerial cinematography. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1107–1114 IEEE (2019)
Dang, Y., Huang, C., Chen, P., Liang, R., Yang, X., Cheng, K.T.: Path-analysis-based reinforcement learning algorithm for imitation filming. IEEE Trans. Multimedia (2022).https://doi.org/10.1109/TMM.2022.3151463
Dang, Y., Huang, C., Chen, P., Liang, R., Yang, X., Cheng, K.-T.: Imitation learning-based algorithm for drone cinematography system. IEEE Trans. Cogn. Dev. Syst. 14(2), 403–413 (2022). https://doi.org/10.1109/TCDS.2020.3043441
Geng, Z., Sun, K., Xiao, B., et al.: Bottom-up human pose estimation via disentangled keypoint regression (2021)
Seker, M., Mnnist, A., Losifidis, A., et al.: automatic main character recognition for photographic studies (2021). https://doi.org/10.48550/arXiv.2106.09064[P]
Zhou, H., Zhang, S., Peng, J., et al.: Informer: Beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35. no. 12, pp. 11106–11115 (2021)
Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Comput. 3(1), 79–87 (1991). https://doi.org/10.1162/neco.1991.3.1.79
Das, A., Kong, W., Leach, A., et al.: Long-term Forecasting with TiDE: Time-series Dense Encoder arXiv preprint arXiv:2304.08424 (2023)
Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need.In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Qiu, J., Wu, X., Jia, B. (2024). EVCPP:Example-Driven Virtual Camera Pose Prediction for Cloud Performing Arts Scenes. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14430. Springer, Singapore. https://doi.org/10.1007/978-981-99-8537-1_5
Download citation
DOI: https://doi.org/10.1007/978-981-99-8537-1_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8536-4
Online ISBN: 978-981-99-8537-1
eBook Packages: Computer ScienceComputer Science (R0)