Deck 8: Doing Nonexperimental Research

Question 1

_____ include both the general and specific classes of behavior one is interested in observing.

A) Behavioral categories
B) Social interactions
C) Action classes
D) Frequency counts

Answer 1

Behavioral categories

Question 2

Which of the following is a method that is used to develop valid behavioral categories?

A) Making some preliminary observations and basing your categories on these observations
B) Conducting a literature search to get some ideas about relevant behavioral categories
C) Conducting your study and then adjusting behavioral categories when you analyze your data
D) Making some preliminary observations and conducting a literature search to get some ideas about relevant behavioral categories

Answer 2

Making some preliminary observations and conducting a literature search to get some ideas about relevant behavioral categories

Question 3

In an observational study of the behavior of monkeys in a zoo, you count the number of times a particular behavior occurs within a given time interval. This is the _____ method of quantifying behavior.

A) frequency
B) duration
C) behavioral
D) behavior counting

Answer 3

frequency

Question 4

In an observational study of the behavior of children in a day care center, you record how long a particular behavior lasts. This is the _____ method of quantifying behavior.

A) frequency
B) duration
C) latency
D) intervals

Question 5

Which of the following would be an example of the intervals method of quantifying behavior?

A) Counting the number of times that a behavior occurs during successive 2-minute intervals
B) Recording how long a behavior lasts across time intervals
C) Dividing observation time into discrete periods and recording whether or not a behavior occurs within each period
D) None of the answers is correct.

Question 6

In a study of children's play, you record not only the frequency of aggressive play but also how children make transitions from one type of play to another. This is an example of recording:

A) single events only.
B) unrepresentative behavior.
C) behavior sequences in addition to single events.
D) None of the answers is correct.

Question 7

According to the text, an advantage of recording behavior sequences is that:

A) they are easier to record than single events.
B) you get a more complete picture of complex social behaviors than when you record single events.
C) you need not worry about using multiple observers.
D) it is unnecessary to compute an interrater reliability score.

Question 8

In an observational study of patients in a psychiatric ward, you alternate 5-minute periods of observation with 5-minute periods of recording behavior. This is an example of _____ sampling.

A) time
B) individual
C) event
D) alternating interval

Question 9

Which of the following statements is true of time sampling as a technique for coping with the complexities of behavioral observation?

A) It is most useful when one can clearly define a behavior of interest and focus on that one behavior.
B) It involves simply coding predefined behaviors into categories.
C) It is most appropriate when a behavior occurs continuously rather than in short bursts spaced over time.
D) It involves repeating observations for various individuals in an observed group over successive time periods.

Question 10

In an observational study of children in a third-grade classroom, you select one participant at a time for observation. This is an example of _____ sampling.

A) unit
B) event
C) time
D) individual

Question 11

If you can identify one behavior as more important than others in an observational study and focus on that one behavior, then you can use _____ sampling.

A) time
B) individual
C) event
D) frequency

Question 12

Which of the following does the text list as an advantage of recording behavior (for example, videotaping)?

A) A behavior can be reviewed several times at your leisure.
B) Multiple observers can view a behavior without disrupting the subjects' natural behavior.
C) A camera often can be hidden more easily than a live observer.
D) All of the answers are correct.

Question 13

Recording behavior on video for later review:

A) eliminates the need for blind observers.
B) does not eliminate the need to classify and measure behavior.
C) reduces interrater reliability.
D) should be avoided because the camera interprets as well as records.

Question 14

_____ is defined as the degree to which two or more observers agree in their classification or quantification of behavior.

A) Intraclass correlation
B) Experimenter bias
C) Interrater reliability
D) Naturalistic observation

Question 15

When you have multiple observers, why should you check for interrater reliability?

A) Establishing interrater reliability helps ensure that your observers are accurate and that you can reproduce your procedures.
B) You can check to see that your observers meet some standard of reliability that you have established.
C) You can detect and correct any problems with additional observer training.
D) All of the answers are correct.

Question 16

Which of the following statements is true of percent agreement as a measure of interrater reliability?

A) It can be extended to cases involving more than two observers by computing the agreement for each possible pair of observers and averaging the result.
B) It understates the true level of interrater agreement in those cases where behaviors occur with very high or low frequency.
C) It makes provisions for estimating the extent to which an agreement observed may have occurred by chance.
D) It was designed primarily for ordinal data and is inapplicable to nominal-scale data.

Question 17

A drawback of using percent agreement to assess interrater reliability is that it:

A) may understate the degree of agreement, especially if an exact match is used to define agreement.
B) gives you only a raw estimate of agreement.
C) may overstate the degree of agreement when behaviors occur with very high or low frequency, as these have high levels of chance agreement.
D) All of the answers are correct.

Question 18

Cohen's Kappa (K) is used to:

A) evaluate the validity of your behavior categories.
B) determine how reliable a single observer is.
C) evaluate interrater reliability.
D) determine whether a single observer's observations are accurate.

Question 19

Identify a true statement about the method of Cohen's Kappa (K) to assess interrater reliability.

A) It uses an analysis-of-variance (ANOVA) approach to assess reliability.
B) It involves assessing the reliability if one's observations are scaled on an interval or ratio scale of measurement.
C) It involves constructing a two-way table in which the columns represent observers' ratings and the rows represent participants.
D) It assesses the observed level of agreement relative to the level of agreement that would be expected by chance.

Question 20

The text proposes that _____ can be used as an index of interrater reliability.

A) Pearson's product-moment correlation
B) the analysis of variance
C) the Mann-Whitney U test
D) phi

Question 21

According to the text, a drawback of using Pearson's product-moment correlation coefficient as an index of interrater reliability is that:

A) it cannot be computed for more than two observers.
B) two sets of numbers from two observers can be highly correlated even when they disagree markedly.
C) it tends to overstate the degree of correlation between observers' data.
D) All of the answers are correct.

Question 22

To compute _____, you begin by constructing a table in which the columns represent observers' ratings and the rows represent the participants observed.

A) percent agreement
B) an intraclass correlation coefficient (ICC)
C) Cohen's Kappa
D) Pearson's product-moment correlation

Question 23

When reporting data from multiple observers, you can average across observers when:

A) there is a high level of agreement.
B) there is a low level of agreement.
C) you use only percent agreement as your measure of interrater reliability.
D) only one observer disagrees with the rest.

Question 24

When multiple observers disagree and you have recorded the behavior electronically and can review it, a good solution is to:

A) select one observer's data as reliable and disregard the rest.
B) have the observers meet and resolve any discrepancies.
C) throw out the observers' data and score the behavior yourself.
D) threaten to punish your observers if they don't get their act together.

Question 25

Selecting one observer as a main observer is a way to deal with:

A) observer bias.
B) data from multiple observers.
C) low interrater reliability.
D) All of the answers are correct.

Question 26

Observer bias occurs when:

A) only one observer is used.
B) observers are blind to the purposes of a study.
C) observers know the goals of a study or the hypotheses being tested.
D) All of the answers are correct.

Question 27

A(n) _____ is one who is unaware of the hypotheses being tested in an observational study.

A) blind observer
B) unbiased observer
C) covert observer
D) reliable observer

Question 28

In the context of observational research, observer bias can be avoided by:

A) using a blind observer who is unaware of the hypotheses being tested.
B) designating one of two observers as the "main observer" and the other as the "secondary observer."
C) using multiple observers and measuring their level of agreement.
D) allowing observers to interpret what they see rather than simply recording behavior.

Question 29

The main advantage of collecting quantitative data is that:

A) you can enter them into a spreadsheet.
B) a wide range of statistical tests is available for analyzing them.
C) they render statistical analysis of the data unnecessary.
D) you don't have to worry about interrater reliability.

Question 30

Qualitative data consist of:

A) measured quantities represented as numbers.
B) numerical data shown to be of high quality.
C) written records of observed behavior that you analyze qualitatively.
D) All of the answers are correct.

Question 31

You are interested in studying the mating habits of chimpanzees. Consequently, you travel to Africa and observe a group of chimpanzees from afar. The method being used in this example is:

A) participant observation.
B) naturalistic observation.
C) field experimentation.
D) content analysis.

Question 32

In naturalistic observation, the mere act of observing may disturb your subjects. To combat this possibility, you as an observer may:

A) use special equipment, such as a blind, to prevent the subjects from detecting your presence.
B) habituate your subjects to your presence.
C) ask them to pretend that you aren't there.
D) use a blind to avoid detection or habituate your subjects to your presence.

Question 33

An advantage of naturalistic observation is that:

A) the observations you make are not tainted by an artificial laboratory environment, so you can be reasonably sure that your observations are representative of naturally occurring behavior.
B) it allows you to investigate the underlying causes of behavior.
C) it is less time-consuming than other types of observation.
D) All of the answers are correct.

Question 34

In _____, a researcher becomes immersed in the behavioral or social system being studied.

A) naturalistic observation
B) ethnography
C) sociometry
D) archival research

Question 35

When conducting an ethnographic study as a participant observer, _____.

A) you act as a functioning member of the group being observed
B) you observe as a nonmember
C) the possibility of subject reactivity is not of concern
D) you take charge of the group being observed and direct its activities

Question 36

The first step when conducting an ethnographic study is to:

A) gain access to a field setting.
B) gain entry into the group to be studied.
C) make observations and record data.
D) apply for membership in the group to be studied.

Question 37

In an ethnographic study, if you decide to conduct your observations as a functioning member of the group you are studying, you are using:

A) sociometry.
B) naturalistic observation.
C) participant observation.
D) nonparticipant observation.

Question 38

In ethnography, your role as a researcher is to:

A) study animal subjects in their natural environment.
B) determine the network of interpersonal relations within a group.
C) identify the factors in a person's past that may have contributed to current behavioral problems.
D) make careful observations and record the social structure of a group.

Question 39

To analyze qualitative ethnographic data, you:

A) read your field notes and identify any themes or hypotheses.
B) systematically extract any major topics, issues, or themes present in your field records.
C) code any systematic patterns in your notes and consider doing an in-depth content analysis.
D) All of the answers are correct.

Question 40

_____ involves identifying and measuring interpersonal relationships within a group.

A) The case history method
B) Sociometry
C) Ethnography
D) Content analysis

Question 41

A _____ is a graphical representation of the pattern of interpersonal relationships (such as friendships) within a group.

A) sociogram
B) frequency histogram
C) Venn diagram
D) social chart

Question 42

To study the workings of a right-wing political group, you join the group and make observations at the group's regular meetings. This is an example of:

A) participant observation.
B) naturalistic observation.
C) a field survey.
D) a field experiment.

Question 43

Observations that do not alter your participant's behavior are said to be _____ observations.

A) obtrusive
B) blind
C) noninvasive
D) unobtrusive

Question 44

Both the case history method and archival research are:

A) unethical.
B) experimental techniques used with observational research.
C) purely descriptive, and therefore, causal relationships cannot be established based on these.
D) suitable for use in the laboratory as well as in the field.

Question 45

Broca's famous study of Tan is an example of:

A) archival research.
B) a case history.
C) content analysis.
D) quasi-experimental research.

Question 46

You are a psychotherapist who has a client with a unique combination of symptoms. With your client's permission, you write an article summarizing and interpreting those symptoms. The method of research being used here is:

A) the case history method.
B) archival research.
C) content analysis.
D) a quasi-experiment.

Question 47

_____ is defined as a descriptive technique in which one observes and reports on a single case.

A) The case history
B) Naturalistic observation
C) Archival research
D) The casebook method

Question 48

The case history method can be used to:

A) describe a single case.
B) contrast two cases to point out their differences.
C) analyze a case that illustrates some empirical or theoretical point.
D) All of the answers are correct.

Question 49

In _____, you study existing records.

A) a case history
B) archival research
C) naturalistic observation
D) participant observation

Question 50

When doing archival research, you:

A) need not worry about developing hypotheses.
B) need not worry about developing clear categories of variables.
C) should have specific research questions in mind.
D) All of the answers are correct.

Question 51

Which of the following is a concern you might have if you were planning an archival study?

A) How to gain access to the records you need
B) Which specific aspects of the archived material to focus on
C) How complete the records you are studying will be
D) All of the answers are correct

Question 52

A disadvantage of archival research is that:

A) institutional review board (IRB) rules governing archival research are stricter than those governing other types of research.
B) existing records may be difficult to access and may be incomplete.
C) an IRB review is required.
D) All of the answers are correct.

Question 53

_____ is used when one wants to analyze a written or spoken record for the occurrence of specific categories or events, items, or behavior.

A) Ethnography
B) Naturalistic observation
C) Content analysis
D) Hermeneutics

Question 54

Which of the following is a defining characteristic of content analysis as a nonexperimental research design?

A) It should have generality.
B) It should be subjective.
C) It should be random and unstructured.
D) It should avoid archival sources for analysis.

Question 55

If you were to analyze the transcripts of congressional hearings for instances of certain types of behavior, you would be using:

A) naturalistic observation.
B) participant observation.
C) the case history method.
D) content analysis.

Question 56

In a content analysis of jury deliberations, you record the number of times that each of several phrases is used. These phrases constitute the _____ unit.

A) content
B) context
C) recording
D) analysis

Question 57

In a content analysis of children's literature, you record the number of times that either males or females are portrayed in dominant roles. You also record the type of situation in which those dominant roles are displayed. The information about the situations constitutes the _____ unit.

A) context
B) recording
C) analysis
D) content

Question 58

A requirement for performing content analysis is:

A) an institutional review board (IRB) review of proposed research.
B) a clear operational definition of terms.
C) that only one observer should be used.
D) to avoid developing hypotheses before beginning a content analysis.

Question 59

_____ can help in ensuring a valid content analysis.

A) Developing a clearly defined hypothesis
B) Reviewing the literature to aid in the development of categories
C) Using a "blind" rater (one who does not know the purpose of your study)
D) All of the answers are correct.

Question 60

Identify a true statement about the information extraction technique.

A) It makes collecting data from large databases cumbersome and unreliable.
B) It involves doing searches that lack specific rules.
C) It involves using specific words, expressions, and numeric values while analyzing archived records.
D) It transforms structured archival records to a more unstructured format that is easier to analyze.

Question 61

In a _____, you summarize published research and draw conclusions based on what you have read.

A) traditional literature review
B) meta-analysis
C) literature analysis
D) library research analysis

Question 62

A _____ is a method of reviewing and summarizing literature in which you statistically combine or compare the results from different studies.

A) traditional literature review
B) meta-analysis
C) literature analysis
D) library research analysis

Question 63

When doing a meta-analysis, you should:

A) avoid having a specific question in mind.
B) avoid relying on archival research.
C) begin with a clearly testable research idea in mind.
D) choose articles only from refereed journals.

Question 64

The unit of analysis in a meta-analysis should be:

A) the effect of one set of generally defined variables on another set.
B) the impact of variable X on variable Y.
C) the number of articles that can be found on a given topic.
D) None of the answers is correct.

Question 65

Which of the following is a factor that should be included when meta-analyzing literature?

A) A full reference citation
B) The use of any control groups and steps taken to reduce confounding
C) The results from statistical analyses
D) All of the answers are correct.

Question 66

In a meta-analysis, the file drawer phenomenon:

A) inflates the probability of making a Type I error.
B) deflates the probability of making a Type I error.
C) inflates the probability of making a Type II error.
D) has no effect on the probability of making a Type I error.

Question 67

In a meta-analysis, the file drawer phenomenon occurs because of:

A) observers who have strong feelings or characteristics that bias the results.
B) the unavailability of techniques to extract information from video streams.
C) studies that are unable to achieve statistically reliable findings and fail to reach publication.
D) the huge amount of time and effort required to review and collect data from large databases.

Question 68

You would compare two studies in a meta-analysis if you wanted to find out:

A) whether the studies produced significantly different results.
B) the average size of an effect reported.
C) whether the studies used similar methods.
D) whether the file drawer phenomenon exists.

Question 69

You would combine the results from the studies in a meta-analysis if you wanted to find out:

A) whether the studies produced significantly different results.
B) the potency of a variable across studies.
C) whether the studies used similar methods.
D) whether the file drawer phenomenon exists.

Question 70

When comparing studies, looking at _____ is the preferred technique.

A) p values
B) t or F values
C) effect sizes
D) None of the answers is correct.

Question 71

When assessing the quality of different studies to be meta-analyzed, _____.

A) the refereed-nonrefereed dimension is the best
B) the articles in nonrefereed journals should be given greater weight
C) you can have experts on methodology evaluate articles
D) None of the answers is correct.

Question 72

A frequent criticism of meta-analysis is that:

A) different studies use very different methodologies that cannot be compared directly.
B) not enough research exists in a given research area to allow for a valid meta-analysis.
C) meta-analysis does not produce results that differ from a more traditional literature review.
D) All of the answers are correct.

Question 73

Rosenthal (1984) has argued that studies using different methods should:

A) be meta-analyzed separately.
B) not be meta-analyzed at all.
C) be blocked according to the method used and analyzed accordingly.
D) be discarded from a meta-analysis.

Question 74

When doing a meta-analysis, you:

A) should do only a limited search of the literature.
B) may find information in many articles that is incomplete.
C) may encounter resistance from authors whom you contact to learn why they published their articles.
D) should avoid having in mind a specific hypothesis to test.

Question 75

When Cooper and Rosenthal (1980) compared meta-analysis with traditional literature review, they found that meta-analysts were:

A) less willing to reject the null hypothesis.
B) more willing to reject the null hypothesis.
C) as likely to reject the null hypothesis as those conducting the traditional literature review.
D) less willing to retain the null hypothesis.

Question 76

When Cooper and Rosenthal (1980) compared meta-analysis with traditional literature review, they found that:

A) there was no difference between the two techniques in evaluating methodology.
B) meta-analysis yields more methodological flaws.
C) traditional literature reviews have more methodological flaws.
D) meta-analysis was more reliable in detecting inappropriately applied statistics.

Question 77

Observational techniques are limited to nonexperimental studies.

Question 78

The first step when planning an observational study is to develop behavioral categories.

Question 79

Poorly conceptualized behavioral categories affect the quality of the results from an observational study.

Question 80

Developing sound behavioral categories is the only major concern you should have when doing observational research.