Deck 9: Collecting Research Data Through Observation and Content Analysis

Question 1

A major advantage of observation over questionnaires is that observation

A) is not susceptible to the self-report bias of questionnaires.
B) requires less time for data collection and analysis.
C) is not susceptible to theoretical saturation.
D) needs to be employed only at one point in time in order to obtain reliable data.

Answer 1

is not susceptible to the self-report bias of questionnaires.

Question 2

The number of students who enter a classroom after the lesson has started is an example of

A) an inferential observational variable.
B) an evaluative observational variable.
C) a descriptive observational variable.
D) a duration observational variable.

Answer 2

a descriptive observational variable.

Question 3

Which of the following is an example of an inferential observational variable?

A) The number of times the audience applauds during a lecture
B) The lecturer's empathy with the audience
C) The amount of time that the lecturer used an overhead projector
D) The quality of the examples provided by the lecturer

Answer 3

The lecturer's empathy with the audience

Question 4

A researcher records whether a particular student is on-task or off-task when the lesson begins, five minutes later, and every five minutes thereafter until the lesson is over. This procedure for recording observations is called

A) continuous recording.
B) frequency-count recording.
C) duration recording.
D) interval recording.

Question 5

Measuring the elapsed time that target behaviors occur is referred to as

A) duration recording.
B) time sampling.
C) interval recording.
D) continuous recording.

Question 6

One disadvantage of using standard observation forms to collect data is that

A) they tend to have low reliability.
B) typically it is not possible to find a schedule that includes all the variables that the researcher wishes to measure.
C) they generally require a high level of inference.
D) they are limited to frequency-count recording.

Question 7

Audio or video recording is especially useful in observational research when

A) teacher-student interaction occurs with low frequency.
B) descriptive variables are to be recorded.
C) inter-rater reliability must be calculated.
D) untrained observers are used.

Question 8

In the training of observers, it is desirable to

A) show video recordings of events similar to those that will be observed during actual data collection.
B) avoid the use of video recordings, because they do not provide the context needed for reliable observation.
C) train each observer independently so that the observers do not bias one another.
D) allow the observers to develop their own rules for resolving disagreements in recording events.

Question 9

The extent to which a trained observer's scores agree with those of an expert observer is a measure of

A) intra-observer reliability.
B) inter-observer reliability.
C) criterion-related reliability.
D) test-retest reliability.

Question 10

Intra-observer reliability is a measure of the extent to which

A) observers agree with each other during data collection.
B) each observer is consistent in her observational codings.
C) observers' codings agree with those of an expert observer.
D) observations of all relevant variables are recorded on the observation form.

Question 11

The term observer drift refers to the tendency for observers to

A) stop attending to the situation during long periods of recording observational data.
B) decline in inter-rater reliability over time.
C) use different standards for making ratings of groups toward whom they have a bias.
D) deviate from the standard definitions of observational variables they learned during training.

Question 12

Observers who tend to rate most individuals at the high end of a scale are exhibiting

A) observer drift.
B) the error of leniency.
C) acquiescence.
D) contamination.

Question 13

Which one of the following is not a characteristic of qualitative observation?

A) The observer looks at events from a holistic perspective.
B) The observer attempts to remain neutral during data collection.
C) The observer may change the focus of observation as data collection proceeds.
D) The observer is not constrained by prespecified hypotheses and variables.

Question 14

The main difference between reactive observation and nonreactive observation involves whether

A) individuals know they are being observed
B) efforts have been made to control observer bias.
C) data are analyzed with quantitative or qualitative methods.
D) observers use an observation form in coding data.

Question 15

A researcher who becomes a substitute teacher in order to collect qualitative data about the culture of substitute teaching has assumed the role of

A) an observer participant.
B) a participant observer.
C) a complete participant.
D) an unobtrusive participant.

Question 16

In qualitative observation, the point at which the researcher decides that new observational findings are replicating earlier findings is called

A) the focused stage.
B) the reflective stage.
C) grounded-theory replication.
D) theoretical saturation.

Question 17

In observation at thestage, observers seek to refine and deepen their understandingof the specific elements most essential to the study's purposes.

A) theoretical
B) descriptive
C) focused
D) selective

Question 18

As data collection proceeds in qualitative observation, the researcher's observations are likely to become more

A) focused.
B) objective.
C) reflective of the observer's role as complete participant.
D) oriented to the study of material culture.

Question 19

To prepare for making field observations, a qualitative researcher would find it most helpful to

A) use the same training procedures as those used in quantitative observation.
B) serve an apprenticeship with an experienced qualitative researcher.
C) attempt to reduce his level of observer personal bias.
D) focus on learning procedures to avoid reliability decay.

Question 20

Field notes in qualitative research should

A) focus on theoretical constructs rather than on concrete details of the events being observed.
B) focus on duration and frequency variables.
C) be made only after the observer leaves the research site.
D) include both descriptive details and the observer's reflections.

Question 21

In qualitative research, observer bias

A) is best avoided by making video recordings of the events being studied.
B) is best avoided by having someone other than the researcher who designed the study collect the data.
C) can be reduced by using both quantitative and qualitative methods of data collection.
D) is usually an issue at the descriptive stage of data collection.

Question 22

A limitation of unobtrusive measures is that they

A) alter the meaning of the variable that is being studied.
B) are susceptible to response sets.
C) are influenced by the observer's presence.
D) have uncertain reliability.

Question 23

Unobtrusive measures have the advantage of

A) requiring less time for data collection.
B) not being susceptible to observer effects.
C) permitting the researcher to study a total behavior pattern.
D) not requiring the use of sampling techniques.

Question 24

Qualitative researchers tend to study physical objects

A) when they want to understand how groups construct social reality.
B) very rarely, because they are decontextualized.
C) when they can interview the individuals who constructed or used them.
D) when the objects have a communication function.

Question 25

The categories used to code documents in a quantitative content analysis

A) should be based on a consistent unit of analysis, such as a sentence or a paragraph.
B) should be developed by the researcher during the content analysis.
C) should be checked to determine whether raters can use them to code documents reliably.
D) all of the above.

Question 26

As a criterion for confirming interpretations based on qualitative analysis of documents and records,external coherence refers to

A) the compatibility between different parts of the researcher's theoretical arguments and conclusions.
B) the match between the researcher's interpretations and accepted theory inside and outside the discipline.
C) how many new lines of inquiry are opened up by the researcher's theoretical position.
D) the degree of correspondence between the data and the researcher's coding categories.

Question 27

State one advantage and one disadvantage of observation as a data-collection method in research.

Question 28

An observer collects quantitative data about (a) the number of meetings that an administrator attends during one work week, (b) how effective the administrator is in each meeting, and (c) the extent to which the administrator employs participatory decision making. Classify each of these observational variables using the three classification categories presented in the textbook.

Question 29

In recording observational variables in quantitative research, the researcher must decide whether to make a continuous recording or to use some other type of recording procedure. Name and briefly describe two other possible types of recording procedure.

Question 30

State one advantage and one disadvantage of using a standard observation form in a research project.

Question 31

State two tasks that a computer can perform when a researcher includes observational data in a research study.

Question 32

If observers disagree with one another's ratings while being trained to use an observational schedule, what is one procedure that the researcher can use to resolve the disagreement?

Question 33

Name three steps recommended for training observers for a quantitative research study.

Question 34

Describe a procedure that a researcher could use to check the inter-rater reliability of three observers who will collect data on 20 meetings of a university faculty senate.

Question 35

Describe one type of rating error that occurs in quantitative observation.

Question 36

Describe two characteristics of research participants that have been found to be associated with biased observations by observers.

Question 37

Explain one major difference between observation in quantitative research and observation in qualitative research.

Question 38

A qualitative researcher who is studying a private Christian school participates in many of its activities as a volunteer, for example, by serving as a teacher aide, driving the school bus, and cleaning up after church services. The observer is of a different faith from those in the school community. What type of observational role is the researcher performing?

Question 39

Describe two advantages of using multiple observers who are diverse in factors relevant to the phenomenon being studied in a qualitative observation study.

Question 40

What typically occurs during the focused stage of observation in a qualitative research study?

Question 41

While a qualitative researcher is in the process of collecting observational data, a more experienced researcher advises him to change his data-collection procedures. Is it acceptable for the researcher to make the recommended changes, and why?

Question 42

A qualitative researcher wishes to make notes about what she is observing in a field situation but does not wish to be seen doing so. How can the researcher deal with this situation?

Question 43

Descibe two desirable characteristics that qualitative researchers should aim to achieve in preparing field notes of their observations.

Question 44

Suppose the individuals being observed appear to be affected by the observer's presence. Compare how a qualitative researcher and a quantitative researcher would handle this situation.

Question 45

Name and briefly explain four types of observer effects in qualitative research.

Question 46

How might computer printouts generated by trainees in an architectural drafting firm be studied (a) by a quantitative researcher as unobtrusive measures and (b) by a qualitative researcher as objects of material culture?

Question 47

How would a quantitative researcher and a qualitative researcher differ in their approach to analyzing the content of a textbook?

Question 48

Name three types of communication media that quantitative or qualitative researchers might study through nonreactive observational research methods.

Question 49

Explain the distinction typically made between documents and records in qualitative observation research.

Question 50

A researcher is interested in testing the theory that children learn by imitating the behavior of others. To test the theory, she asks one group of teachers to be exceptionally neat in their classrooms for a period of several weeks, for example, conspicuously taking time to arrange their desks (i.e., the teacher's own desk in the classroom) neatly. She asks another group of teachers to be sloppy for a similar period of time. The researcher's prediction is that children whose teachers are neat will themselves be neater by the end of the experiment than will children whose teachers are sloppy. Give examples of unobtrusive measures of the children's degree of neatness that could be used in this experiment.

Question 51

A researcher wants to determine whether the cognitive level of students' essays improves from the first to the fourth year of college. She has a collection of essays written by the same students over a four-year period of college attendance. What are three aspects of essay-writing style that could be content-analyzed to yield a measure of cognitive level?

Question 52

A quantitative researcher and a qualitative researcher want to use observational methods to study the phenomenon of collegiality among the counselors and other service staff in a high school. How are the two researchers likely to differ in (a) their approach to conceptualizing collegiality and (b) the types of data that they would collect?

Question 53

A quantitative researcher and a qualitative researcher are working as members of a team project whose goal is to study how journeyman contractors work with apprentice construction workers to guide their learning of their trades. While training observers, the quantitative researcher finds out that some of them bias their observational ratings in favor of contractors who were originally trained as carpenters. Upon hearing of this bias, the qualitative researcher realizes that she, too, shares their bias that contractors with a carpentry background are more effective than those with another construction specialization (e.g., electricians, cabinet makers, or painters). How would each researcher be likely to deal with this bias?