Skip to main content

Advertisement

Springer Nature Link
Log in

Bullfighting extreme scenarios in efficient hyper-scale cluster computing

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Data centres are quickly evolving to support new demands for Cloud-Computing services. Extreme workload peaks represent a challenge for the maintenance of the performance and service level agreements, even more when operation costs need to be minimised. In this paper, we first present an extensive analysis of the impact of extreme workloads in large-scale realistic Cloud-Computing data centres, including a comparison between the most relevant centralised resource-managing models. Moreover, we extend our previous works by proposing a new energy-efficiency policy called Bullfighter which is able to keep performance key performance indicators while reducing energy consumption in extreme scenarios. This policy employs queue-theory distributions to foresee workload demands and adapt automatically to workload fluctuations even in extreme environments, while avoiding the fine-tuning required for other energy policies. Finally, it is shown through extensive simulation that Bullfighter can save more than 40% of energy in the aforementioned scenarios without exerting any noticeable impact on data-centre performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
€32.70 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Beloglazov, A., Buyya, R.: Managing overloaded hosts for dynamic consolidation of virtual machines in cloud data centers under quality of service constraints. IEEE Trans. Parallel Distrib. Syst. 24(7), 1366–1379 (2013). https://doi.org/10.1109/TPDS.2012.240

    Article  Google Scholar 

  2. Beloglazov, A., Buyya, R.: Openstack neat: a framework for dynamic and energy-efficient consolidation of virtual machines in openstack clouds. Concur. Comput. Pract. Exp. 27(5), 1310–1333 (2015). https://doi.org/10.1002/cpe.3314

    Article  Google Scholar 

  3. Benstock, D., Cegla, F.: Extreme value analysis (EVA) of inspection data and its uncertainties. NDT & E Int. 87, 68–77 (2017)

    Article  Google Scholar 

  4. Burns, B., Grant, B., Oppenheimer, D., Brewer, E., Wilkes, J.: Borg, omega, and kubernetes. Commun. ACM 59(5), 50–57 (2016)

    Article  Google Scholar 

  5. Cheng, Y., Anwar, A., Duan, X.: Analyzing alibaba’s co-located datacenter workloads. In: 2018 IEEE International Conference on Big Data (Big Data), pp. 292–297 (2018). https://doi.org/10.1109/BigData.2018.8622518

  6. Coles, S.G., Tawn, J.A.: Statistical methods for multivariate extremes: an application to structural design. J. R. Stat. Soc. Ser. C (Appl. Stat.) 43(1), 1–31 (1994)

    MATH  Google Scholar 

  7. Duy, T.V.T., Sato, Y., Inoguchi, Y.: Performance evaluation of a green scheduling algorithm for energy savings in cloud computing. In: 2010 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW), pp. 1–8. IEEE, Piscataway (2010)

  8. Embrechts, P., Klüppelberg, C., Mikosch, T.: Modelling Extremal Events: For Insurance and Finance, vol. 33. Springer, Berlin (2013)

    MATH  Google Scholar 

  9. Fernández-Cerero, D., Fernández-Montes, A., Jakóbik, A., Kołodziej, J., Toro, M.: Score: simulator for cloud optimization of resources and energy consumption. Simul. Model. Pract. Theory 82, 160–173 (2018)

    Article  Google Scholar 

  10. Fernández-Cerero, D., Fernández-Montes, A., Ortega, J.A.: Energy policies for data-center monolithic schedulers. Expert Syst. Appl. 110, 170–181 (2018). https://doi.org/10.1016/j.eswa.2018.06.007

    Article  Google Scholar 

  11. Fernández-Cerero, D., Jakóbik, A., Fernández-Montes, A., Kołodziej, J.: Game-score: game-based energy-aware cloud scheduler and simulator for computational clouds. Simul. Model. Pract. Theory 93, 3–20 (2019)

    Article  Google Scholar 

  12. Gumbel, E.J.: Statistics of Extremes. Courier Corporation, North Chelmsford (2012)

    MATH  Google Scholar 

  13. Hindman, B., Konwinski, A., Zaharia, M., Ghodsi, A., Joseph, A.D., Katz, R.H., Shenker, S., Stoica, I.: Mesos: a platform for fine-grained resource sharing in the data center. NSDI 11, 22–22 (2011)

    Google Scholar 

  14. Hüsler, J., Li, D.: Statistical analysis of extreme values with applications to insurance, finance, hydrology and other fields. In: Reiss, R.D., Thomas, M. (eds.) Statistical Analysis of Extreme Values with Applications to Insurance, Finance, Hydrology and Other Fields, pp. 144–151. Birkhäuser, Boston (2007)

    Google Scholar 

  15. Juarez, F., Ejarque, J., Badia, R.M.: Dynamic energy-aware scheduling for parallel task-based application in cloud computing. Fut. Gen. Comput. Syst. 78, 257–271 (2016)

    Article  Google Scholar 

  16. Kalid, S., Syed, A., Mohammad, A., Halgamuge, M.N.: Big-data NOSQL databases: a comparison and analysis of “big-table”, “dynamodb”, and “cassandra”. In: 2017 IEEE 2nd International Conference on Big Data Analysis (ICBDA), pp. 89–93 (2017). https://doi.org/10.1109/ICBDA.2017.8078782

  17. Katz, R.W.: Statistics of extremes in climate change. Clim. Change 100(1), 71–76 (2010)

    Article  Google Scholar 

  18. Kotz, S., Balakrishnan, N., Johnson, N.L.: Continuous Multivariate Distributions: Models and Applications, vol. 1. Wiley, New York (2004)

    Book  Google Scholar 

  19. Lee, Y.C., Zomaya, A.Y.: Energy efficient utilization of resources in cloud computing systems. J. Supercomput. 60(2), 268–280 (2012)

    Article  Google Scholar 

  20. Reiss, C., Tumanov, A., Ganger, G.R., Katz, R.H., Kozuch, M.A.: Heterogeneity and dynamicity of clouds at scale: Google trace analysis. In: Proceedings of the Third ACM Symposium on Cloud Computing, p. 7. ACM, New York (2012)

  21. Resnick, S.I., et al.: Heavy tail modeling and teletraffic data: special invited paper. Ann. Stat. 25(5), 1805–1869 (1997)

    Article  Google Scholar 

  22. Ricciardi, S., Careglio, D., Sole-Pareta, J., Fiore, U., Palmieri, F., et al.: Saving energy in data center infrastructures. In: 2011 First International Conference on Data Compression, Communications and Processing (CCP), pp. 265–270. IEEE, Piscataway (2011)

  23. Schwarzkopf, M., Konwinski, A., Abd-El-Malek, M., Wilkes, J.: Omega: flexible, scalable schedulers for large compute clusters. In: Proceedings of the 8th ACM European Conference on Computer Systems, pp. 351–364. ACM, New York (2013)

  24. Shehabi, A., Smith, S.J., Sartor, D.A., Brown, R.E., Herrlin, M., Koomey, J.G., Masanet, E.R., Horner, N., Azevedo, I.L., Lintner, W.: United states data center energy usage report. Tech. Rep. (2016)

  25. Sohrabi, S., Tang, A., Moser, I., Aleti, A.: Adaptive virtual machine migration mechanism for energy efficiency. In: Proceedings of the 5th International Workshop on Green and Sustainable Software, pp. 8–14. ACM, New York (2016)

  26. Van Heddeghem, W., Lambert, S., Lannoo, B., Colle, D., Pickavet, M., Demeester, P.: Trends in worldwide ICT electricity consumption from 2007 to 2012. Comput. Commun. 50, 64–76 (2014)

    Article  Google Scholar 

  27. Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., Wilkes, J.: Large-scale cluster management at Google with Borg. In: Proceedings of the Tenth European Conference on Computer Systems, p. 18. ACM, New York (2015)

Download references

Acknowledgements

Project RTI2018-098062-A-I00, funded by FEDER/Spanish Ministry of Science and Innovation—National Research Agency.

Author information

Authors and Affiliations

  1. Department of Computer Languages and Systems, University of Seville, 41012, Seville, Spain

    Damián Fernández-Cerero & Alejandro Fernández-Montes

  2. Department of Applied Economy I, University of Seville, 41018, Seville, Spain

    Francisco J. Ortega-Irizo & Francisco Velasco-Morente

Authors
  1. Damián Fernández-Cerero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francisco J. Ortega-Irizo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alejandro Fernández-Montes
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francisco Velasco-Morente
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alejandro Fernández-Montes.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (ZIP 3576 kb)

Appendix: Bullfighter parameterisation raw data

Appendix: Bullfighter parameterisation raw data

Table 7 Performance and energy-efficiency results of various Bullfighter configurations
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fernández-Cerero, D., Ortega-Irizo, F.J., Fernández-Montes, A. et al. Bullfighting extreme scenarios in efficient hyper-scale cluster computing. Cluster Comput 23, 3387–3403 (2020). https://doi.org/10.1007/s10586-020-03094-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-020-03094-2

Keywords

Search

Navigation