Skip to main content
Springer Nature Link
Log in

CaR-PLive: Cloud-assisted reinforcement learning based P2P live video streaming: a hybrid approach

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In recent years, live video streaming has become one of the most popular and prevalent applications of the Internet. The Peer-to-Peer (P2P) and Content Delivery Network (CDN) are popular approaches to stream video contents. These approaches respectively have faced some drastic challenges such as obtaining the desired Quality of Service (QoS) level and minimizing economic cost. The cloud computing infrastructures can reveal proper solutions to these problems. The P2P systems can eliminate their bandwidth shortage by renting resources from the cloud environment. This paper depicts CaR-PLive as a hybrid cloud-assisted P2P live streaming system. CaR-PLive uses video servers such as Amazon EC2 from cloud to stream video contents and rents Cloud Storage Services (CSSs) such as Amazon S3 to assist P2P live streaming system to reach the desired playback continuity. In CaR-PLive, we proposed two stages (sub-windows) sliding window for buffer management that a sub-window belongs to the P2P system and another one belongs to CSS. The objective of CAR-PLive is to optimize the size of sub-windows to minimize the overall rental cost of CSS restricted to a desired QoS level. We formulate this problem as an optimization problem and model it with Markov Decision Process (MDP) and then propose a reinforcement learning based algorithm to solve this problem. Finally, we evaluate the performance of CaR-PLive by performing extensive simulations and experiments with realistic settings. Simulation results demonstrate that CaR-PLive efficiently mitigates overall CSS billing cost in different system configurations and provides desired playback continuity in different system settings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
€32.70 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  1. Adhikari VK, Yang G, Fang H, Varvello M, Hilt V, Steiner M, Zhang ZL (2012) Unreeling netflix: understanding and improving multi-CDN movie delivery. 2012 proceedings IEEE INFOCOM, 25–30 march 2012: 1620–1628. doi:https://doi.org/10.1109/INFCOM.2012.6195531

  2. Afergan MM, Leighton FT, Parikh JG (2012) Hybrid content delivery network (CDN) and peer-to-peer (P2P) network. Google Patents

  3. Aggarwal V, Xu C, Gopalakrishnan V, Jana R, Ramakrishnan KK, Vaishampayan VA (2011) Exploiting virtualization for delivering cloud-based IPTV services. 2011 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS): 637–641. doi:https://doi.org/10.1109/INFCOMW.2011.5928890

  4. Ahmad S, Bouras C, Buyukkaya E, Dawood M, Hamzaoui R, Kapoulas V, Papazois A, Simon G (2018) Peer-to-peer live video streaming with rateless codes for massively multiplayer online games. Peer-to-Peer Netw Appl 11(1):44–62

    Article  Google Scholar 

  5. Akamai Netsession (2018) http://www.akamai.com/client/. Accessed July 20 2018

  6. Amazon CloudFront (2019) https://aws.amazon.com/cloudfront/. Accessed July 10 2019

  7. Amazon simple storage service (Amazon S3) (2018) http://aws.amazon.com/s3/. Accessed July 5 2018

  8. Aslani R, Hakami V, Dehghan M (2018) A token-based incentive mechanism for video streaming applications in peer-to-peer networks. Multimed Tools Appl 77(12):14625–14653

    Article  Google Scholar 

  9. Bharambe AR, Herley C, Padmanabhan VN (2006) Analyzing and improving a BitTorrent networks performance mechanisms. In: Proceedings IEEE INFOCOM 2006. 25TH IEEE international conference on computer communications: 1–12. doi:https://doi.org/10.1109/INFOCOM.2006.328

  10. Bittorrent DNA (2018) http://www.bittorrent.com/dna. Accessed July 20 2018

  11. Budhkar S, Tamarapalli V (2018) Delay management in mesh-based P2P live streaming using a three-stage peer selection strategy. J Netw Syst Manag 26(2):401–425

    Article  Google Scholar 

  12. Castro M, Druschel P, Kermarrec A-M, Nandi A, Rowstron A, Singh A (2003) SplitStream: high-bandwidth multicast in cooperative environments. SIGOPS Oper Syst Rev 37(5):298–313. https://doi.org/10.1145/1165389.945474

    Article  Google Scholar 

  13. Chen Z, Yin H, Lin C, Liu X, Chen Y (2007) Towards a trustworthy and controllable peer-server-peer media streaming: an analytical study and an industrial perspective. IEEE GLOBECOM 2007 - IEEE global telecommunications conference: 2086–2090. doi:https://doi.org/10.1109/GLOCOM.2007.399

  14. Chih-Chiang W, Ying-Dar L (2016) CDNPatch: a cost-effective failover mechanism for hybrid CDN-P2P live streaming systems. Int J Commun Syst 29(17):2517–2533. https://doi.org/10.1002/dac.3193

    Article  Google Scholar 

  15. Cisco V (2018) Cisco visual networking index: forecast and trends, 2017–2022. White paper 1

  16. Gao G, Li R (2019) Collaborative caching in P2P streaming networks. J Netw Syst Manag 27(3):815–836

    Article  Google Scholar 

  17. Ghaderzadeh A, Kargahi M, Reshadi M (2018) ReDePoly: reducing delays in multi-channel P2P live streaming systems using distributed intelligence. Telecommun Syst 67(2):231–246

    Article  Google Scholar 

  18. Gheorghe G, Cigno RL, Montresor A (2011) Security and privacy issues in P2P streaming systems: a survey. Peer-to-Peer Netw Appl 4(2):75–91

    Article  Google Scholar 

  19. Gummadi KP, Saroiu S, Gribble SD (2002) King: estimating latency between arbitrary internet end hosts. Paper presented at the proceedings of the 2nd ACM SIGCOMM workshop on internet measurment, Marseille, France

  20. He Y, Guan L (2009) Improving the streaming capacity in P2P VoD systems with helpers. 2009 IEEE International Conference on Multimedia and Expo: 790–793. doi:https://doi.org/10.1109/ICME.2009.5202613

  21. He J, Wu D, Zeng Y, Hei X, Wen Y (2013) Toward optimal deployment of Cloud-assisted video distribution services. IEEE Trans Circ Syst Video Technol 23(10):1717–1728. https://doi.org/10.1109/TCSVT.2013.2255423

    Article  Google Scholar 

  22. He J, Wen Y, Huang J, Wu D (2014) On the cost–QoE tradeoff for cloud-based video streaming under amazon EC2's pricing models. IEEE Trans Circ Syst Video Technol 24(4):669–680

    Article  Google Scholar 

  23. Hei X, Liang C, Liang J, Liu Y, Ross KW (2007) A measurement study of a large-scale P2P IPTV system. IEEE Trans Multimed 9(8):1672–1687. https://doi.org/10.1109/TMM.2007.907451

    Article  Google Scholar 

  24. Huang Z, Mei C, Li LE, Woo T (2011) CloudStream: Delivering high-quality streaming videos through a cloud-based SVC proxy. 2011 proceedings IEEE INFOCOM: 201–205. doi:https://doi.org/10.1109/INFCOM.2011.5935009

  25. Ishakian V, Sweha R, Bestavros A (2017) AngelCast: Cloud-based peer-assisted live streaming using optimized multi-tree construction. Comput Commun 111:14–28

    Article  Google Scholar 

  26. Jin X, Kwok YK (2010) Cloud assisted P2P media streaming for bandwidth constrained Mobile subscribers. 2010 IEEE 16th international conference on parallel and distributed systems: 800–805. doi:https://doi.org/10.1109/ICPADS.2010.78

  27. Jin Y, Wen Y, Shi G, Wang G, Vasilakos AV (2012) CoDaaS: an experimental cloud-centric content delivery platform for user-generated contents. 2012 International Conference on Computing, Networking and Communications (ICNC): 934–938. doi:https://doi.org/10.1109/ICCNC.2012.6167562

  28. Li H, Zhong L, Liu J, Li B, Xu K (2011) Cost-effective partial migration of VoD services to content clouds. 2011 IEEE 4th international conference on Cloud computing: 203–210. doi:https://doi.org/10.1109/CLOUD.2011.41

  29. Liao X, Jin H, Liu Y, Ni LM, Deng D (2006) AnySee: Peer-to-peer live streaming. Proceedings IEEE INFOCOM 2006. 25TH IEEE international conference on computer communications: 1–10. doi:https://doi.org/10.1109/INFOCOM.2006.288

  30. Lin SH, Pal R, Wang BC, Golubchik L (2017) On market-driven hybrid-P2P video streaming. IEEE Trans Multimed 19(5):984–998. https://doi.org/10.1109/TMM.2016.2644868

    Article  Google Scholar 

  31. Liu Y, Guo Y, Liang C (2008) A survey on peer-to-peer video streaming systems. Peer-to-peer Netw Appl 1(1):18–28

    Article  Google Scholar 

  32. Lu Z, Wu J, Chen L, Huang S, Huang Y (2010) CPH-VoD: A Novel CDN–P2P-Hybrid Architecture Based VoD Scheme. In, Berlin, Heidelberg. Web information systems engineering – WISE 2010. Springer Berlin Heidelberg: 578–586

    Chapter  Google Scholar 

  33. Lu ZH, Gao XH, Huang SJ, Huang Y (2011) Scalable and Reliable live streaming service through coordinating CDN and P2P. In: 2011 IEEE 17th international conference on parallel and distributed systems: 581–588. doi:https://doi.org/10.1109/ICPADS.2011.113

  34. Magharei N, Rejaie R (2009) PRIME: peer-to-peer receiver-driven mesh-based streaming. IEEE/ACM Trans Netw 17(4):1052–1065. https://doi.org/10.1109/TNET.2008.2007434

    Article  Google Scholar 

  35. Mahini H, Dehghan M, Navidi H, Masoud Rahmani A (2016) GaMe-PLive: a new game theoretic mechanism for P2P live video streaming. Int J Commun Syst 29(6):1187–1203

    Article  Google Scholar 

  36. Montresor A, Abeni L (2011) Cloudy weather for P2P, with a chance of gossip. In: 2011 IEEE international conference on peer-to-peer computing: 250–259. doi:https://doi.org/10.1109/P2P.2011.6038743

  37. Mostafavi S, Dehghan M (2016) Game theoretic bandwidth procurement mechanisms in live P2P streaming systems. Multimed Tools Appl 75(14):8545–8568

    Article  Google Scholar 

  38. Mostafavi S, Dehghan M (2017) A stochastic approximation resource allocation approach for HD live streaming. Telecommun Syst 64(1):87–101

    Article  Google Scholar 

  39. Niu D, Hong X, Li B, Zhao S (2012) Quality-assured cloud bandwidth auto-scaling for video-on-demand applications. In: 2012 proceedings IEEE INFOCOM: 460–468. doi:https://doi.org/10.1109/INFCOM.2012.6195785

  40. Padmanabhan VN, Wang HJ, Chou PA (2003) Resilient peer-to-peer streaming. In: 11th IEEE international conference on network protocols, 2003. Proceedings: 16–27. doi:https://doi.org/10.1109/ICNP.2003.1249753

  41. Pal K, Govil MC, Ahmed M (2018) Priority-based scheduling scheme for live video streaming in peer-to-peer network. Multimed Tools Appl 77(18):24427–24457

    Article  Google Scholar 

  42. Pal K, Govil MC, Ahmed M (2018) Slack time–based scheduling scheme for live video streaming in P2P network. Int J Commun Syst 31(2):e3440

    Article  Google Scholar 

  43. Payberah AH (2013) Live streaming in P2P and hybrid P2P-cloud environments for the open internet. KTH Royal Institute of Technology

  44. Payberah AH, Kavalionak H, Kumaresan V, Montresor A, Haridi S (2012) Clive: Cloud-assisted P2P live streaming. 2012 IEEE 12th Int Conf Peer-to-Peer Comput (P2P) 2012:79–90. https://doi.org/10.1109/P2P.2012.6335820

    Article  Google Scholar 

  45. Pianese F, Perino D, Keller J, Biersack EW (2007) PULSE: an adaptive, incentive-based, unstructured P2P live streaming system. IEEE Trans Multimed 9(8):1645–1660. https://doi.org/10.1109/TMM.2007.907466

    Article  Google Scholar 

  46. PPTV (2019). https://www.pptv.com/. Accessed July 10 2019

  47. Qiu X, Li H, Wu C, Li Z, Lau FCM (2012) Dynamic scaling of VoD services into hybrid clouds with cost minimization and QoS guarantee. 19th Int Packet Video Workshop (PV) 2012:137–142. https://doi.org/10.1109/PV.2012.6229726

    Article  Google Scholar 

  48. Rocha V, Kon F, Cobe R, Wassermann R (2016) A hybrid cloud-P2P architecture for multimedia information retrieval on VoD services. Computing 98(1–2):73–92

    Article  MathSciNet  Google Scholar 

  49. Rodríguez-Silva DA, Adkinson-Orellana L, Gonz'lez-Castaño FJ, Armiño-Franco I, Gonz'lez-Martínez D (2012) Video surveillance based on Cloud storage. In: 2012 IEEE fifth international conference on Cloud computing: 991–992. doi:https://doi.org/10.1109/CLOUD.2012.44

  50. Rongfei M (2019) Super node selection algorithm combining reputation and capability model in P2P streaming media network. Pers Ubiquit Comput:1–8

  51. SopCast (2019). http://www.sopcast.com/. Accessed July 10 2019

  52. Sutton RS, Barto AG (1998) Introduction to reinforcement learning. MIT press

  53. Tian Y, Babcock R, Taylor C, Ji Y (2018) A new live video streaming approach based on Amazon S3 pricing model. IEEE 8th Ann Comput Commun Workshop Conf (CCWC) 2018:321–328. https://doi.org/10.1109/CCWC.2018.8301615

    Article  Google Scholar 

  54. Tran DA, Hua KA, Do T (2003) ZIGZAG: an efficient peer-to-peer scheme for media streaming. IEEE INFOCOM 2003. Twenty-second annual joint conference of the IEEE computer and communications societies (IEEE cat. No.03CH37428) 1282:1283–1292. https://doi.org/10.1109/INFCOM.2003.1208964

    Article  Google Scholar 

  55. Varga A, Hornig R (2008) An overview of the OMNeT++ simulation environment. In: Proceedings of the 1st international conference on simulation tools and techniques for communications, networks and systems & workshops, 2008. ICST (Institute for Computer Sciences, Social-Informatics and …: 60

  56. Wang J, Ramchandran K (2008) Enhancing peer-to-peer live multicast quality using helpers. 2008 15th IEEE Int Conf Image Process 2008:2300–2303. https://doi.org/10.1109/ICIP.2008.4712251

    Article  Google Scholar 

  57. Wang M, Xu L, Ramamurthy B (2011) Improving multi-view peer-to-peer live streaming systems with the divide-and-conquer strategy. Comput Netw 55(18):4069–4085

    Article  Google Scholar 

  58. Wang F, Liu J, Chen M CALMS: Cloud-assisted live media streaming for globalized demands with time/region diversities. In: 2012 Proc IEEE INFOCOM, 25–30 march 2012 2012. pp 199–207. doi:https://doi.org/10.1109/INFCOM.2012.6195578

  59. Wang M, Xu L, Ramamurthy B (2013) Exploring the design space of multichannel peer-to-peer live video Streaming systems. IEEE/ACM Trans Netw 21(1):162–175. https://doi.org/10.1109/TNET.2012.2194165

    Article  Google Scholar 

  60. Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8(3):279–292. https://doi.org/10.1007/bf00992698

    Article  MATH  Google Scholar 

  61. Wowza Streaming Cloud (2019) https://www.wowza.com/pricing/streaming-cloud-plans. Accessed July 10 2019

  62. Wu C, Li B, Zhao S (2008) Multi-Channel Live P2P Streaming: Refocusing on Servers. IEEE INFOCOM 2008-The 27th Conference on Computer Communications. IEEE: 1355–1363

  63. Wu D, Liu Y, Ross K (2009) Queuing network models for multi-channel P2P live streaming systems. In: IEEE INFOCOM 2009: 73–81. doi:10.1109/INFCOM.2009.5061908

  64. Wu Y, Wu C, Li B, Qiu X, Lau FCM (2011) CloudMedia: When Cloud on Demand Meets Video on Demand. 2011 31st international conference on distributed computing systems, 20–24 June 2011 2011: 268–277. doi:https://doi.org/10.1109/ICDCS.2011.50

  65. Xiao W, Bao W, Zhu X, Wang C, Chen L, Yang LT (2016) Dynamic request redirection and resource provisioning for Cloud-based video services under heterogeneous environment. IEEE Trans Parallel Distrib Syst 27(7):1954–1967. https://doi.org/10.1109/TPDS.2015.2470676

    Article  Google Scholar 

  66. Xinyan Z, Jiangchuan L, Bo L, Yum YSP CoolStreaming/DONet: a data-driven overlay network for peer-to-peer live media streaming. In: Proc IEEE 24th Ann Joint Conf IEEE Comput Commun Soc., 13–17 march 2005 2005. pp 2102–2111 vol. 2103. doi:https://doi.org/10.1109/INFCOM.2005.1498486

  67. Yin H, Liu X, Zhan T, Sekar V, Qiu F, Lin C, Zhang H, Li B (2010) LiveSky: enhancing CDN with P2P. ACM Trans Multimed Comput Commun Appl 6(3):1–19. https://doi.org/10.1145/1823746.1823750

    Article  Google Scholar 

  68. Zattoo (2019) https://zattoo.com. Accessed July 10 2019

  69. Zhang X, Liu J, Li B, Yum Y-S (2005) CoolStreaming/DONet: A data-driven overlay network for peer-to-peer live media streaming. Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies, IEEE: 2102–2111

  70. Zhang J, Xing W, Wang Y, Lu D (2014) Modeling and performance analysis of pull-based live streaming schemes in peer-to-peer network. Comput Commun 40:22–32

    Article  Google Scholar 

  71. Zhang J, Zhang X, Yang C (2018) Towards the multi-request mechanism in pull-based peer-to-peer live streaming systems. Comput Netw 138:77–89

    Article  Google Scholar 

  72. Zhengye L, Yanming S, Ross KW, Panwar SS, Yao W Substream trading: towards an open P2P live streaming system. In: 2008 IEEE international conference on network protocols, 19–22 Oct. 2008 2008. pp 94–103. doi:https://doi.org/10.1109/ICNP.2008.4697028

  73. Zhou Y, Chiu D, Lui JCS (2011) A simple model for chunk-scheduling strategies in P2P streaming. IEEE/ACM Trans Networking 19(1):42–54. https://doi.org/10.1109/TNET.2010.2065237

    Article  Google Scholar 

  74. Zhu W, Luo C, Wang J, Li S (2011) Multimedia Cloud Computing. IEEE Signal Process Mag 28(3):59–69. https://doi.org/10.1109/MSP.2011.940269

    Article  Google Scholar 

  75. Zhu Z, Li S, Chen X (2013) Design QoS-aware multi-path provisioning strategies for efficient Cloud-assisted SVC video streaming to heterogeneous clients. IEEE Trans Multimed 15(4):758–768. https://doi.org/10.1109/TMM.2013.2238908

    Article  Google Scholar 

Download references

Acknowledgements

The authors offer their gratitude to Dr. Amir H. Payberah at KTH Royal Institute of Technology, Stockholm, Sweden, Department of Software and Computer System for providing the simulation codes and experimental results of CLIVE.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

    Majid Sina & Amir Masoud Rahmani

  2. Department of Computer Engineering and Information Technology, Amirkabir University of Technology, Hafez Ave, Tehran, 15914, Iran

    Mehdi Dehghan

  3. Computer Science, University of Human Development, Sulaimanyah, Iraq

    Amir Masoud Rahmani

Authors
  1. Majid Sina
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mehdi Dehghan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amir Masoud Rahmani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mehdi Dehghan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sina, M., Dehghan, M. & Rahmani, A.M. CaR-PLive: Cloud-assisted reinforcement learning based P2P live video streaming: a hybrid approach. Multimed Tools Appl 78, 34095–34127 (2019). https://doi.org/10.1007/s11042-019-08102-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08102-1

Keywords

Search

Navigation