A CCG virtual system for big data application communication costs analysis
Yu, Yongen, Zou, Hongbo, Tang, Wei, & Liu, Liwei (2014) A CCG virtual system for big data application communication costs analysis. In Lin, J., Pei, J., Hu, X., Chang, W., Nambiar, R., Aggarwal, C., et al. (Eds.) Proceedings of 2014 IEEE International Conference on Big Data (Big Data), IEEE, Washington, DC, pp. 54-60.
Network topology and routing are two important factors in determining the communication costs of big data applications at large scale. As for a given Cluster, Cloud, or Grid system, the network topology is fixed and static or dynamic routing protocols are preinstalled to direct the network traffic. Users cannot change them once the system is deployed. Hence, it is hard for application developers to identify the optimal network topology and routing algorithm for their applications with distinct communication patterns. In this study, we design a CCG virtual system (CCGVS), which first uses container-based virtualization to allow users to create a farm of lightweight virtual machines on a single host. Then, it uses software-defined networking (SDN) technique to control the network traffic among these virtual machines. Users can change the network topology and control the network traffic programmingly, thereby enabling application developers to evaluate their applications on the same system with different network topologies and routing algorithms. The preliminary experimental results through both synthetic big data programs and NPB benchmarks have shown that CCGVS can represent application performance variations caused by network topology and routing algorithm.
Impact and interest:
Citation counts are sourced monthly from and citation databases.
These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.
Citations counts from theindexing service can be viewed at the linked Google Scholar™ search.
|Item Type:||Conference Paper|
|Keywords:||CCG; Virtual System; SDN; network topology; routing; virtualization|
|Divisions:||Current > Schools > School of Information Systems
Current > QUT Faculties and Divisions > Science & Engineering Faculty
|Deposited On:||20 Oct 2015 01:21|
|Last Modified:||20 Oct 2015 01:22|
Repository Staff Only: item control page