Epistemologically surveys provide one way of obtaining and
validating knowledge. Consequently, it is important that the context
for this type of IS research be considered. Hence, this section also
provides a summary that explains when survey research is appropriate
and the steps required to use the survey approach correctly.
First, one should understand what a survey is actually used for. A survey is a way of going from observations to theory validation. As part of a panel discussion (Newsted, Chin, Ngwenyama, and Lee, 1996), Chin presented an exhibit that nicely outlines how this is done:
These concepts are the constructs of interest to researchers. The usual objective for Information Systems researchers using this approach is to determine the relationship of these constructs as a way of making sense of behavior surrounding and involving IS.
The methodology used to study these relationships is also an important aspect of survey research. Techniques range from the reporting of simple scale means, the use of analysis of variance of results in different conditions, through regression analysis and the analysis of paths between constructs, to the use of "second generation" techniques such as LISREL and PLS, which analyze measurement models and structural models simultaneously.
Trochim's (1998) web pages show how some of the simpler of these approaches work. It has links dealing with:
Chin's (1995) website provides pointers toward more advanced approaches using structural equation modeling (SEM). Among other things it discusses:
In a panel discussion at the 1996 International Conference on Information Systems, Newsted, Chin, Ngwenyama, and Lee (1996) raised the question whether surveys had outlived their usefulness. The panelists concluded that surveys are appropriate in certain conditions and less useful in others. But this is more complicated than just saying surveys should be used only in objective or positivist research and not in more subjective or interpretivist research. Burrel and Morgan (1979) provide a detailed distinction between these approaches.
In the '96 panel, Allen Lee indicated that in positivist research, surveys are particularly useful in determining the actual values of variables under study, and the strengths of relationships among them. In an interpretivist context surveys are appropriate as a complement to other forms of data or observations. They can serve as a way to add to one's knowledge through "triangulation" as one of several methods. Thus it is important to realize that while surveys are typically used in quantitative research, they can also help qualitative researchers as well. The panel gives further details. In particular one should review the notes below the slides in Lee's presentation as they discuss the inter-relations between these types of research and describe situations where these approaches and orientations fit together. For those interested in the qualitative approach, Myers provides an excellent site offering considerable detail on this type of research.
Overall the survey approach can be seen to have the following strengths and weaknesses:
After having decided to employ survey research it is advisable to search the literature for measures of your constructs of interest. This process can be expedited by reviewing our list of constructs or using the search engine. If you are unable to find an appropriate instrument with an associated methodology, it may be necessary to develop one. To do so requires a theoretical foundation in the area of study and an understanding of the survey process. But, even when an existing instrument is being used, an appropriate theory and a correct process are necessary.
A carefully constructed theory is a precursor to the actual use of an instrument. Theories propose constructs and their expected relations. They guide the investigation of these relations as they attempt to help one understand behavior and identify regularities in it. However, this is also a two-way street: instrument development can often refine theories as well. As Straub (1989) indicates, "Attention to instrument issues ... brings greater clarity to the formulation and interpretation of research questions. In the process of validating an instrument, the researcher is engaged, in a very real sense, in a reality check. He or she finds out in relatively short order how well conceptualization of problems and solutions matches with actual experience of practitioners." (p. 148).
A number of exemplary survey-based studies describe instrument development within a theoretical context. In the following three studies researchers have taken particular care to consider theories and to identify the steps in instrument creation and validation:
With a theoretical foundation in place, the activities of the survey process can be considered. Grover (1997) provides a detailed check list to be followed in the development and use of an instrument. Significant among these steps are:
Dillman (1978) offers a number of very practical tips to facilitate the surveying process and increase response rates. Salant and Dillman (1994) provide a very good update to Dillman's initial classic.
The following references provide more detail on the theories and processes involved in surveying. These are divided into general references on surveying, sources specific to IS, and those dealing with a variety of methodology issues. A broad understanding of survey methodology as well as knowledge about specific IS issues are useful in using the survey approach.
Since surveying involves a number of detailed and sometimes tedious steps, it is reasonable to ask if parts of it can be automated (especially since much of the analysis of survey data has been automated). The answer is a tentative "yes." However, care must be taken to ensure that computerized surveying leads to the same results as traditional, noncomputerized approaches using paper and pencil.
Bratton and Newsted (1995) summarize much of the work in this area and show that differences in results can be due as much to different interfaces as to differences in techniques. Interestingly, fewer differences seem to be found in newer work (Liefeld, 1992; Potosky and Bobko, 1997), as compared to studies before 1990 (e.g. Liefeld, 1988; Kiesler and Sproull, 1986). Recently, however, Webster and Compeau (1996) have found that, while there may not be differences in means between computerized and noncomputerized surveying, there may be differences in correlations with other variables such as those involving training performance. While there appears to be a growing acceptance of automated survey methodologies, there is a need for caution in the use of computerized techniques.
Automated surveying can be done using the Internet. Mueller (1997) provides a good example of how surveys can be done on the Web. He uses an instrument that is part of his on-going research on computer experience.
There are a number of commercial companies that provide computerized surveying for a fee. While we have not tested these products, they appear to have useful capabilities -- perhaps, especially for those wishing to do commercial surveys in IS.
Survey Instruments in IS