Deck 8: Assessing Measurement

Question 1

The product of measurement is considered a(n) ______ of what happened because researchers find it useful to admit the possibility that data include some error:

A) functional relationship
B) correlation
C) function
D) estimate

Answer 1

D

Question 2

A behavior analyst supervisor asks a therapist to record on a data sheet the number of math problems a client completes during treatment sessions. In an effort to evaluate the data reported on math completion, the supervisor collects the math sheets periodically and records the number of math problems completed. She then compares the math sheets to the number recorded by the therapist on the corresponding data sheet. The behavior analyst supervisor is recording:

A) observed values
B) trends
C) true values
D) treatment integrity

Answer 2

C

Question 3

What are the two requirements for evaluating the accuracy of a set of observations

A) the data must be observed values, and it must be possible to measure these events directly
B) the data must represent physical events, and it must be possible to measure these events directly
C) the data must represent physical events, and they must be true values
D) the data must be true values, and they must be able to be measured using direct assessments

Answer 3

B

Question 4

The only way to assess accuracy is:

A) through interobserver agreement scores
B) by calibrating the transducing device
C) through continuous observation
D) by comparing observed values with true values

Question 5

______ refers to the consistency of the relationship between observed values and the events that actually occurred

A) Reliability
B) Accuracy
C) Validity
D) Believability

Question 6

There are two ways to assess reliability. One way is to repeatedly measure the same sample of the subject's behavior. What is the second way?

A) Arrange for direct measurement of the target behavior on a periodic basis
B) Collecting corroborative data
C) Use an interobserver agreement procedure
D) Compare observed values to true values

Question 7

When it is not clear that the data represent what they are supposed to represent, the issue of ______ arises:

A) reliability
B) accuracy
C) validity
D) believability

Question 8

If the target behavior is directly measured, the data are automatically .

A) reliable
B) accurate
C) believable
D) valid

Question 9

When accuracy and reliability are found to be unacceptable, the investigator can either abandon the project or use ______ to evaluate the data and adjust the procedure so that its output meets desired standards:

A) absolute measurement
B) interobserver agreement
C) calibration
D) treatment integrity

Question 10

Interobserver agreement does not provide information about either ______ or ______.

A) believability; reliability
B) accuracy; believability
C) reliability; accuracy
D) agreement; believability

Question 11

Below are the results of a partial-interval recording procedure. What is the interobserver agreement score?
$\begin{array}{|c|l|l|l|l|l|l|l|l|l|l|}\hline \text { Observer A } & \mathrm{X} & \mathrm{X} & & \mathrm{X} & & \mathrm{X} & \mathrm{X} & \mathrm{X} & \mathrm{X} & \\\hline \text { Observer B } & \mathrm{X} & \mathrm{X} & & \mathrm{X} & \mathrm{X} & \mathrm{X} & & \mathrm{X} & \mathrm{X} & \mathrm{X} \\\hline\end{array}$

A) 50%
B) 60%
C) 70%
D) 80%

Question 12

Below are the results of a whole-interval recording procedure. What is the interobserver agreement score? $\begin{array}{|l|l|l|l|l|l|l|l|l|l|l|}\hline \text { Observer } \mathrm{A} & \mathrm{X} & \mathrm{X} & & \mathrm{X} & & \mathrm{X} & \mathrm{X} & \mathrm{X} & \mathrm{X} & \\\hline \text { Observer } \mathrm{B} & \mathrm{X} & \mathrm{X} & & \mathrm{X} & \mathrm{X} & \mathrm{X} & & \mathrm{X} & \mathrm{X} & \mathrm{X} \\\hline\end{array}$

A) 50%
B) 60%
C) 70%
D) 80%

Question 13

How frequently should data be evaluated?

A) The investigator should assess the data no less than 80% of the total number of sessions
B) The data should be evaluated as soon as the project/treatment begins and then at the very end of the project/treatment
C) The investigator should wait for the study or treatment to finish before analyzing the data
D) Ideally, the investigator should assess each session, but at least as often as they see necessary based on what the target behavior is and what the results are showing

Question 14

Regardless of whether data assessment focuses on accuracy, reliability, or believability, there are a number of procedural details that must be decided. All of the following should be considered when implementing assessment procedures, except:

A) when assessment efforts should be made
B) how frequently data should be evaluated
C) is this research basic or applied in nature?
D) sampling different circumstances throughout a study

Question 15

The guiding interest in evaluating the quality of data is to ensure that _______.

A) no time was wasted by the researcher or practitioner
B) data collected are 100% accurate for every session
C) they closely approximate what really happened with behavior
D) other people in the field are impressed with your results

Question 16

When reporting complex assessment information, it is sometimes easiest for readers to understand if it is described using ______ instead of described in text:

A) a table or graph
B) bullet points
C) a presentation
D) an example

Question 17

When assessing measurement procedures, what is the first priority?

A) Collecting IOA data
B) Determining accuracy of data
C) Evaluating reliability of data
D) Determining believability of overall results

Question 18

A value that results from observing and recording procedures is called a(n) ______ value.

A) true
B) false
C) observed
D) defined

Question 19

To say that a measure is precise means that it is ___________.

A) reliable
B) free from errors
C) valid
D) observable

Question 20

We assess the ______ of data to make decisions about validity, reliability, and accuracy of data.

A) quantity
B) quality
C) qualifications
D) quartiles

Question 21

Why does the quality of behavioral data matter?

Question 22

What is estimation and how does it relate to the goal of researches and practitioners?

Question 23

Explain why interobserver agreement data provide no information about accuracy or reliability.

Question 24

Why is it useful to view the results of measurement as only an estimation of what really happened?

Question 25

How can you be sure that true values contain no error?

Question 26

What is accuracy and how can it be determined?

Question 27

What is reliability? Describe two basic ways to assess reliability.

Question 28

What is validity? Why is it always an issue in the case of indirect measurement?

Question 29

Why is direct measurement automatically valid?

Question 30

Describe the sequence or priority of the experimenter's interest in assessing the data.

Question 31

What are the requirements for obtaining true values?

Question 32

Describe the general procedure for obtaining interobserver agreement (not a specific method for IOA).

Question 33

How often should measurement be evaluated?

Question 34

Complete the following sentence using the terms "accurate" and "reliable." If data are ______, they must be ______. Why is the converse not true?

Question 35

What is the difference between observed and true values?

Question 36

Describe the basic procedure of calibration.

Question 37

Define believability. How is it fundamentally different from accuracy, reliability, and validity?

Question 38

Describe how accuracy and reliability differ.

Question 39

Describe how validity differs from accuracy.

Question 40

When assessing measurement procedures and data, what is the first priority?