APPENDIX M Possibility of Interviewer Bias

Although we could withhold the predictions and rationales from the interviewers, we could not conceal which families contained the addicts and which the controls. With this knowledge, biased interviewers could slant the data in accordance with their own expectations and prejudices—expectations which could well correspond to our own. They might, for example, pursue certain lines of information for one type of family, but not for the other. Accordingly, we did some elaborate checks on our interviewers to deal with this difficult methodological problem

Before they started field work, each interviewer was asked to give her own predictions as to the differences that would be found between addicts and controls on the items covered in the questionnaire. Each interviewer was also asked to describe, by checking thirty-five rating scales, what she thought the mother of the addict would be like. From the rating scales, we obtained a measure of negative, hostile attitudes toward the mothers of the addicts.
After the data were collected, we compared the reports for families of addicts and families of controls of those interviewers whose expectations were similar to our predictions with those who had differing expectations. On the separate comparisons for each of the hypotheses and on a combined index of gross environmental pathology, there was no case of significant difference in the reports by the two classes of interviewers.

Similar comparisons were made of the results reported by interviewers who had differing initial attitudes toward the mother of the addict. An analysis of the ratings of mothers of addicts by the interviewers (prior to any actual contact with such mothers) showed that, as one might expect, none of our interviewers had favorable attitudes. There was, however, wide variation among the interviewers, ranging from a fairly neutral attitude to a very negative and hostile one. The interviewers were then divided into three groups: those with rather neutral attitudes, those with moderately negative ones, and those who pictured the mothers of addicts in strongly negative terms. A comparison of their reports on addicts and controls again showed no difference among the three types of interviewers.

We also used a combined design involving, at the same time, interviewer expectations and attitudes (four equal groups of interviewers—three above the median in attitude and above the median in agreement of their predictions with our own, three above the median in attitude and below the median in agreeing with us, etc.) with no change in result. There is good reason, therefore, to believe that the data were not influenced by the preconceptions or prejudicial attitudes of the interviewers.1

The statistical technique used was that of analysis of variance, which is one for testing the hypothesis that the average differences between two or more groups are no larger than can reasonably be expected by chance if these groups are random samples of the same population. The principles of the test are that scores can be divided into a number of components from which one can obtain independent estimates of the population variance and that the probability of discrepancies in these estimates can be assessed. The probabilities of ratios (referred to as F-ratios) of independent estimates of the population variance have been calculated (and tabled) on the basis of certain assumptions; but it is known that the test is quite rugged in that the probability estimates tend to remain accurate even when there is reason to believe that there are quite marked deviations from conditions assumed in the derivation of the probabilities of the F-ratios.

In the present application, we had, for any given index, an estimate of the population variance based on the average index differences for addict and control families (with the interviewer effects balanced out), another estimate based on the average index difference obtained by the various classes of interviewers (with the differences related to types of families balanced out); another estimate based on what are known as "interaction effects" (in the present instance, various types of interviewers reacting differently to the two types of families, i.e., an estimate based on the degree to which the average index differences between addict and control families differ for the types of interviewers); and, finally, an estimate based on the residual variance (i.e., on the index variance that remains after subtracting a component related to family-type differences, a component related to interviewer-type differences, and a component related to the interaction effect). The latter estimate was used as the denominator in computing three F-ratios--one to assess the chance probability of getting differences comparable to that obtained between the addict and control families, one to assess the chance probability of getting differences comparable to that obtained between the classes of interviewers, and one to similarly assess the interaction effect.

When all the analyses of variance for each of the indexes and for the combined index of over-all environmental deprivation were assembled, we found that in no instance was there either a significant interviewer-type effect or a significant interaction between interviewer type and addict—control difference.

1 Details of the checks on the possibility of interviewer bias are given in Robert S. Lee, "The Family of the Addict" (New York University Graduate School of Arts and Science, doctoral dissertation, 1957), pp. 133-149.