A CCG virtual system for big data application communication costs analysis

Yu, Yongen, Zou, Hongbo, Tang, Wei, & Liu, Liwei (2014) A CCG virtual system for big data application communication costs analysis. In Lin, J., Pei, J., Hu, X., Chang, W., Nambiar, R., Aggarwal, C., et al. (Eds.) Proceedings of 2014 IEEE International Conference on Big Data (Big Data), IEEE, Washington, DC, pp. 54-60.

View at publisher


Network topology and routing are two important factors in determining the communication costs of big data applications at large scale. As for a given Cluster, Cloud, or Grid system, the network topology is fixed and static or dynamic routing protocols are preinstalled to direct the network traffic. Users cannot change them once the system is deployed. Hence, it is hard for application developers to identify the optimal network topology and routing algorithm for their applications with distinct communication patterns. In this study, we design a CCG virtual system (CCGVS), which first uses container-based virtualization to allow users to create a farm of lightweight virtual machines on a single host. Then, it uses software-defined networking (SDN) technique to control the network traffic among these virtual machines. Users can change the network topology and control the network traffic programmingly, thereby enabling application developers to evaluate their applications on the same system with different network topologies and routing algorithms. The preliminary experimental results through both synthetic big data programs and NPB benchmarks have shown that CCGVS can represent application performance variations caused by network topology and routing algorithm.

Impact and interest:

0 citations in Scopus
Search Google Scholar™

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

ID Code: 88811
Item Type: Conference Paper
Refereed: Yes
Keywords: CCG; Virtual System; SDN; network topology; routing; virtualization
DOI: 10.1109/BigData.2014.7004413
ISBN: 9781479956654
Divisions: Current > Schools > School of Information Systems
Current > QUT Faculties and Divisions > Science & Engineering Faculty
Deposited On: 20 Oct 2015 01:21
Last Modified: 20 Oct 2015 01:22

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page