Surveys are ubiquitous. All of us, almost daily, receive questionnaires in the mail and telephone calls at dinnertime by marketing surveyors. The high prevalence and incidence of surveys in modern life tends to trivialize the serious nature of survey research. Also, the low cost and the apparent accessibility of survey methods can make such approaches seem like low-hanging fruit to junior faculty seeking to produce scholarly publications for professional advancement.

Obviously, the concrete tools of survey research are available to anyone with access to a telephone or postal services. However, like psychotherapy, survey research may seem like an easy thing to do but actually requires a great deal of highly specific expertise to do it right. Also, although the technology of conducting surveys is more accessible and less expensive than, say, a positron emission tomography scanner and a cyclotron, surveys do expend real resources—by both researchers and respondents. A survey of 1,000 psychiatrists that takes each an hour to fill out "costs" $100,000 in respondents' time (at $100 per hour). Survey research is serious business.

Detailed knowledge of sampling, design, and analysis are critical to the proper conduct of studies involving survey research. It would be expected that individuals undertaking them either possess the necessary methodological knowledge and skills or have colleagues with the appropriate expertise as part of their team.

Perhaps even more important, though, are the fundamental approaches and values of science that underlie research across all fields, from molecular genetics to clinical trials to health services and educational research. The article by Sierles in this issue provides useful suggestions for conducting survey research on educational issues, but we are concerned that some readers may get the impression that survey research is a casual process, lacking the seriousness of purpose and need for scrupulous attention to scientific process of other, "harder" research.

There are four essential steps that individuals should consider before undertaking a survey project:

1. Explicitly define the question. As with any research study, the first steps are to define the independent and dependent variables. In survey research, choosing the right items to capture these variables is critically important. Experts in survey development use both qualitative and quantitative research methods to determine whether survey questions adequately capture the content of interest and to ensure that survey respondents and researchers have the same understanding of the questions. The careful development of tools for assessing consumer satisfaction with health plans is an example of extensive cognitive laboratory testing and piloting (+1). Whenever possible, it is preferable to search for questionnaire items that are well validated, because it saves the investigator time, makes the research comparable to previous studies, and helps guard against problems that arise from poorly worded questions.

2. Determine whether a survey is the best way to answer the research question. Survey research is well suited to research on knowledge, attitudes, and beliefs. How else but to ask them can you capture individuals' opinions or beliefs? Surveys may also be useful for capturing behaviors and practices, especially those that are rarely documented otherwise, as Sierles showed with his study of cheating by medical students. However, as Sierles pointed out, individuals may knowingly or unknowingly misrepresent their behaviors or actions to "look good" in the research. It is important to determine whether other, more reliable data sources might exist to capture the information of interest. For example, using medical records can be a more accurate way of assessing physicians' prescribing behavior than asking physicians about their usual practices.

3. Figure out whether your survey, as designed, will actually answer the research question. Every questionnaire item and instruction should be scrutinized for errors, from typographical mistakes to incorrect response categories. Make sure that each item is properly framed. If the questionnaire asks about experiences that happen over time, the time period should be specified and consistent throughout the survey. There is no substitute for sufficient piloting of the instrument and the process. It is also essential to plan the analysis necessary to answer the research question before the survey is administered. Statistical and data management consultation should occur as early as possible in the development of the survey, and certainly before the survey is conducted. Plans for data coding and entry as well as the database construction can identify questions that may need to allow respondents to choose multiple responses. (For example, many patients have multiple types of health insurance, so any "type of insurance" question should allow respondents to endorse multiple plans, such as Medicaid, Medicare, employer-sponsored, and so on.) Preparing the analysis plan and drafting dummy tables often identifies missing information that is critical in defining independent or dependent variables as well as important confounders or covariates.

4. Ruthlessly consider and reveal all potential threats to the validity of your study. In survey research, response bias and inadequate measurement of potential confounding variables are common threats to a study's internal and external validity. Since briefer surveys generally obtain higher response rates, these factors are related. Investigators should strive to create a questionnaire that adheres closely to the prespecified research questions while assessing important potential confounding factors. A thorough literature review and a clearly articulated conceptual model simplify the task of item selection and provide a rationale for the inclusion of each item. Similarly, this conceptual model and consideration of potential confounders should also help the investigator in assessing whether the planned survey method will likely yield a representative sample of the population of interest. An e-mail survey may be an excellent way to ask students about the quality of Internet educational materials. E-mail surveys may be less useful for gauging patients' views of educational materials if many patients do not have Internet access—particularly if the researcher is interested in reaching low-income populations that are less likely to have Internet access (+4). It is possible to get an adequate response rate but still have a study sample that is not representative of the population of interest; efforts to anticipate threats to generalizability and ways to reach populations that often do not participate in survey research should be considered.

We certainly do not wish to discourage junior faculty who have research questions they wish to answer within an evidence-based framework. However, such projects should not be undertaken lightly. As these principles suggest, researchers should think through all of the scientific and practical issues before collecting any data, and they should identify any potential flaws after the process is completed.

People usually like to be asked about their opinions and experiences. They are more likely to be interested in divulging personal information when a survey has a well-developed rationale and when the instrument shows that the investigator is knowledgeable about the topic, has planned the questionnaire carefully, and poses questions that make sense. Surveys that have a strong conceptual framework are a better value for the research dollar and also show respect for the participants' time.

The authors are supported by the Mental Health Intervention Research Center (MH 30915; David Kupfer, PI) at the University of Pittsburgh.