Parallel data mining on cycle stealing networks

Robertson, Calum Stewart (2004) Parallel data mining on cycle stealing networks. Masters by Research thesis, Queensland University of Technology.

Abstract

In a world where electronic databases are used to store ever-increasing quantities of data it is becoming harder to mine useful information from them. Therefore there is a need for a highly scalable parallel architecture capable of handling the ever-increasing complexity of data mining problems.

A cycle stealing network is one possible scalable solution to this problem. A cycle stealing network allows users to donate their idle cycles to form a virtual supercomputer by connecting multiple machines via a network.

This research aims to establish whether cycle stealing networks, specifically the G2 system developed at the Queensland University of Technology, are viable for large scale data mining problems. The computationally intensive sequence mining, feature selection and functional dependency mining problems are deliberately chosen to test the usefulness and scalability of G2.

Tests have shown that G2 is highly scalable where the ratio of computation to communication is approximately known. However for combinatorial problems where computation times are difficult or impossible to predict, and communication costs can be unpredictable, G2 often provides little or no speedup.

This research demonstrates that existing sequence mining and functional dependency mining techniques are not suited to a client-server style cycle stealing network like G2. However the feature selection is well suited to G2, and a new sequence mining algorithm offers comparable performance to other existing, non-cycle stealing, parallel sequence mining algorithms. Furthermore new functional dependency mining algorithms offer substantial benefit over existing serial algorithms.

Impact and interest:

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

Full-text downloads:

348 since deposited on 03 Dec 2008
14 in the past twelve months

Full-text downloads displays the total number of times this work’s files (e.g., a PDF) have been downloaded from QUT ePrints as well as the number of downloads in the previous 365 days. The count includes downloads for all files if a work has more than one.

ID Code: 15970
Item Type: QUT Thesis (Masters by Research)
Supervisor: Geva, Shlomo & Kelly, Wayne
Keywords: data mining problems, parallel sequence algorithms, functional dependency
Divisions: Past > QUT Faculties & Divisions > Faculty of Science and Technology
Department: Faculty of Information Technology
Institution: Queensland University of Technology
Deposited On: 03 Dec 2008 03:54
Last Modified: 28 Oct 2011 19:41

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page