A critical evaluation and comparison of two formative measures of system quality using criterion variables

Ali, Azim, Tate, Mary, Rabaa'i, Ahmad A., & Zhang, Meng (2012) A critical evaluation and comparison of two formative measures of system quality using criterion variables. In Australasian Conference on Information Systems (ACIS), 3 - 5 December 2012, Geelong, NSW.

View at publisher


Evaluating the validity of formative variables has presented ongoing challenges for researchers. In this paper we use global criterion measures to compare and critically evaluate two alternative formative measures of System Quality. One model is based on the ISO-9126 software quality standard, and the other is based on a leading information systems research model. We find that despite both models having a strong provenance, many of the items appear to be non-significant in our study. We examine the implications of this by evaluating the quality of the criterion variables we used, and the performance of PLS when evaluating formative models with a large number of items. We find that our respondents had difficulty distinguishing between global criterion variables measuring different aspects of overall System Quality. Also, because formative indicators “compete with one another” in PLS, it may be difficult to develop a set of measures which are all significant for a complex formative construct with a broad scope and a large number of items. Overall, we suggest that there is cautious evidence that both sets of measures are valid and largely equivalent, although questions still remain about the measures, the use of criterion variables, and the use of PLS for this type of model evaluation.

Impact and interest:

3 citations in Scopus
Search Google Scholar™

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

Full-text downloads:

204 since deposited on 04 Feb 2013
10 in the past twelve months

Full-text downloads displays the total number of times this work’s files (e.g., a PDF) have been downloaded from QUT ePrints as well as the number of downloads in the previous 365 days. The count includes downloads for all files if a work has more than one.

ID Code: 56845
Item Type: Conference Paper
Refereed: Yes
Keywords: System quality, formative constructs, criterion variables, ISO 9126, IS-Impact
Divisions: Current > QUT Faculties and Divisions > Science & Engineering Faculty
Copyright Owner: Copyright 2012 [please consult the author]
Deposited On: 04 Feb 2013 23:17
Last Modified: 18 Jul 2017 05:10

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page