Deck 7: Standardized Measurement and Assessment

Question 1

Which of the following variables is measured on a ratio scale?

A) Mia's score on a personality test
B) Class valedictorian and salutatorian
C) Bill's score on an achievement test
D) The time it took Betsy to finish an assignment, measured in seconds

Answer 1

D

Question 2

Explain the meaning of validity and validity evidence.

Answer 2

not answered

Question 3

Which is the correct order of the scales of measurement?

A) Nominal, ordinal, interval, ratio
B) Ordinal, interval, ratio, nominal
C) Interval, ordinal, ratio, nominal
D) Ratio, nominal, ordinal, interval

Answer 3

A

Question 4

Explain the meaning of reliability.

Question 5

_____ is the process of assigning symbols or numbers to objects, events, people, or characteristics according to a specific set of rules.

A) Assessment
B) Evaluation
C) Measurement
D) Observation

Question 6

There are many assumptions underlying testing and measurement. Which of the following is one of them?

A) A major decision should always be made on the basis of a single test score
B) Multiple sources of error are always present in the measurement process
C) Fair and unbiased tests cannot be developed
D) Standardized testing and assessment do not benefit society

Question 7

Explain the different methods of collecting validity evidence.

Question 8

Which one of the following variables is measured on an ordinal scale?

A) Class rank in high school
B) Racial or ethnic group
C) State of residence
D) Personality test score

Question 9

Explain the meaning of measurement.

Question 10

Identify the different types of standardized tests and the sources of information on these tests

Question 11

The consistency of a set of test scores over time is known as:

A) Equivalent forms reliability
B) Split-half reliability
C) Test-retest reliability
D) Interscorer reliability

Question 12

Which one of the following variables is measured on an interval scale?

A) Paul's place in line
B) Gender of participants
C) Foreign language studied in high school
D) Score on an IQ test

Question 13

Explain the different scales of measurement, including the type of information communicated by each one.

Question 14

An advantage of ratio and interval scales of measurement is that:

A) More powerful mathematics can be done on them
B) Most variables used in educational research have ratio properties
C) They are used to make qualitative distinctions only
D) There are no mathematical advantages to ratio scales

Question 15

Explain the characteristics of each of the methods for computing reliability.

Question 16

The accuracy of inferences or interpretations made from test scores is known as:

A) Reliability
B) Validity
C) Stability
D) Consistency

Question 17

Which one of the following is measured on a nominal scale?

A) Scores on an IQ test.
B) Class rank in high school
C) Students' gender
D) Number of items correct on a spelling test

Question 18

Articulate the seven assumptions underlying testing and assessment.

Question 19

The reliability of a test refers to:

A) The consistency or stability of the test scores.
B) Whether a test measures what it is supposed to measure.
C) Whether a test is valid.
D) Its content sampling.

Question 20

Which of the following is true about testing and assessment?

A) Testing and assessment is usually error-free.
B) Test-related attitudes and behavior cannot be used to predict non-test-related attitudes and behavior.
C) There is only one source of error.
D) Traits and states can be quantified and measured.

Question 21

When choosing a test or assessment procedure, you should choose one that is:

A) Reliable only
B) Valid only
C) Neither reliable nor valid
D) Both reliable and valid

Question 22

Criterion-related validity evidence refers to whether a test:

A) Adequately samples the content it purports to measure
B) Is predictive of a future or concurrent behavior
C) Is internally consistent
D) Meets the criterion for reliability

Question 23

A test is homogeneous if:

A) The test measures a single construct
B) The test has high test-retest reliability
C) The items do not correlate with the total score
D) There is a reliable alternative form

Question 24

Jane needs to calculate interscorer reliability for her study. She will do this by:

A) Correlating the judgments of two raters
B) Calculating coefficient alpha
C) Calculating a test-retest correlation
D) Calculating KR-20

Question 25

Concurrent validity evidence refers to:

A) The relationship between test scores and criterion scores obtained at the same time
B) Whether scores on a test correlated with scores on the same test given a week later
C) Whether a test predicts scores on some criterion measure (e.g., success in training)
D) Whether a test has an adequate coefficient alpha

Question 26

Systematic (i.e., non-random) error is associated with:

A) Reliability problems
B) Validity problems
C) Homogeneous test items
D) None of the above

Question 27

Content-related validity evidence refers to:

A) Judgment that the test is reliable
B) Judgment that the test is internally consistent
C) Correlation between scores on the test and a criterion behavior
D) Judgment that the test adequately samples and represents the construct domain it purports to measure

Question 28

Coefficient alpha is an index of which of the following?

A) Test-retest reliability
B) Interscorer reliability
C) Equivalent-forms reliability
D) Internal consistency reliability

Question 29

Dr. Spock has developed a test to identify students who are gifted. He administered the test to students with very high IQs and students with very low IQs and compared the scores. He found that students with high IQs got very high scores on his giftedness test. These results show:

A) Concurrent evidence
B) Known groups evidence
C) Content-related evidence
D) Factor analytic evidence

Question 30

If a test measures a single construct then:

A) The items should correlate with the total score.
B) The items should not correlate with the total score.
C) The test should not correlate with other measures of the same construct.
D) There must be a reliable alternative form.

Question 31

_____ refers to the consistency or stability of test scores.

A) Reliability
B) Validity
C) Convergent evidence
D) Interpolation

Question 32

Test-retest reliability refers to:

A) Whether test items are answered consistently within a test
B) Whether scores on the first half of a test correlate with scores on the second half
C) Whether scores on a test given at one point in time correlate with scores on the same test at a second testing
D) Whether experts agree that a test measures what it is supposed to measure

Question 33

As the number of items on a test increases, coefficient alpha tends to _____.

A) Increase
B) Decrease
C) Stay the same
D) No longer be important.

Question 34

Which of the following is a type of criterion-related validity evidence?

A) Concurrent evidence
B) Content evidence
C) Expert judgments
D) Factor analysis

Question 35

A researcher has developed a test and is working on collecting psychometric data so it can be published. Currently she is collecting data about psychological and social consequences of using this test. She is doing this to:

A) Establish divergent validity
B) Collect what is known as known groups evidence
C) Establish consequential validity
D) Establish convergent validity

Question 36

A general rule of thumb states that coefficient alpha should be:

A) Greater than or equal to .05
B) Greater than or equal to .70:
C) Less than .70
D) Less than .05

Question 37

A test that is internally consistent is also said to be:

A) Homogeneous
B) Heterogeneous
C) Mutidimensional
D) Valid

Question 38

Error that is present every time an instrument is used is known as:

A) Assessment error
B) Testing error
C) Systematic error
D) Unregulated error

Question 39

The Spearman-Brown formula is used to:

A) Correct validity coefficients
B) Correct test-retest reliability coefficients
C) Correct equivalent forms reliability coefficients
D) Correct the split-half reliability coefficient

Question 40

Equivalent-forms reliability refers to:

A) The consistency of a group of individuals' scores on two different forms of a test measuring the same thing
B) Whether scores on the first half of a test correlate with scores on the second half
C) Whether experts agree that a test measures what it is supposed to measure
D) Whether a test is predictive of behavior

Question 41

A group of supervisors agrees that 16 items measure issues important to a particular kind of work. This is evidence of:

A) Criterion-related validity evidence
B) Content-related validity evidence
C) Convergent validity evidence
D) Divergent validity evidence

Question 42

What is the name of the scale of measurement that has a true zero point as well as the characteristics of the rank ordering and equal distances between scale points?

A) Nominal scale
B) Ordinal scale
C) Interval scale
D) Ratio scale

Question 43

A validity coefficient is a correlation between:

A) scores and a relevant criterion
B) scores on the same test given two weeks apart to the same person
C) the scores on even and odd items on a test
D) scores on two separate forms of the same test

Question 44

Professor X develops a test of emotional intelligence. Which of the following represent convergent and discriminant evidence?

A) The test correlates highly with another test of emotional intelligence and is not correlated with self-efficacy
B) The test correlates highly with another test of emotional intelligence and is highly correlated with self-efficacy
C) The test does not correlate with another test of emotional intelligence, but does correlate with self-efficacy
D) The test correlates with neither other tests of emotional intelligence nor tests of self-efficacy

Question 45

The difference between achievement and aptitude tests is that:

A) Achievement tests measure more the results of formal learning.
B) Aptitude test scores are unrelated to school performance.
C) Achievement test scores are influenced mostly be informal learning.
D) Achievement test scores do not predict school performance.

Question 46

Distinguishable, relatively enduring ways in which one individual differs from another are which of the following?

A) Reliability
B) Traits
C) States
D) Systematic Error

Question 47

Intelligence tests typically measure:

A) The ability to think abstractly and learn from experience
B) State variables
C) Learning styles
D) School achievement ability

Question 48

An employment test correlates .90 with work performance after one year. This is a type of:

A) Criterion-related validity evidence
B) Content-related validity evidence
C) Internal consistency reliability
D) Convergent/discriminant validity

Question 49

Factor analysis is used to determine:

A) The number of underlying dimensions or factors measured by a test
B) Test-retest reliability
C) Interrater reliability
D) Alternate forms reliability

Question 50

A scale of measurement that uses symbols, such as words or numbers to label, classify, or identify people or objects is which of the following?

A) Nominal scale
B) Ordinal scale
C) Interval scale
D) Ratio scale

Question 51

A test correlates .90 with a test of the same construct, but .10 with a test of a different construct. This is evidence of:

A) Criterion-related validity evidence
B) Content-related validity evidence
C) Internal consistency reliability
D) Convergent and discriminant validity
E) Alternate forms reliability

Question 52

Which of the following is an example of convergent validity evidence?

A) The test has a high coefficient alpha
B) The test has a high test-retest correlation
C) The test correlates highly with other tests of the same construct
D) The test has a high KR-20 coefficient

Question 53

A rank-order scale of measurement is:

A) Nominal scale
B) Ordinal scale
C) Interval scale
D) Ratio scale

Question 54

The gathering and integrating of data to make educational evaluations is specifically called:

A) Measurement
B) Testing
C) Assessment
D) Survey

Question 55

The specific group for which the test publisher or researcher provides evidence for test validity and reliability is known as the _____.

A) Test group
B) Benchmark group
C) Norming Group
D) Comparison Group

Question 56

A trait refers to:

A) How one acts when alone
B) An individual's state in one particular situation
C) How someone feels during times of stress
D) Enduring characteristics of people.

Question 57

Ms. Green and Mr. Potter conduct behavioral observations for their school. When comparing results on the same children, Ms. Green's ratings are always more negative than Mr. Potter's. This is an example of:

A) Random Error
B) Traits
C) States
D) Systematic Error

Question 58

The consistency with which the items on a test measure a single construct is called:

A) Reliability Coefficient
B) Test-Retest Reliability
C) Equivalent Forms Reliability
D) Internal Consistency

Question 59

A test has a coefficient alpha of .75. This is evidence of:

A) Interscorer reliability
B) Alternate forms reliability
C) Internal consistency reliability
D) Test-retest reliability

Question 60

A scale of measurement that has equal distances between adjacent scale points but does not have a true zero point is known as:

A) Nominal scale
B) Ordinal scale
C) Interval scale
D) Ratio scale

Question 61

A researcher wants to understand what helps some students from disadvantaged backgrounds to excel throughout their school years while others do not. This researcher is planning to examine what?

A) Traits
B) Reliability
C) States
D) Validity

Question 62

In order to improve counseling services, a researcher wants to understand what affects high school students' beliefs and feelings immediately after they have received low or high test scores. This researcher is planning to examine what?

A) Traits
B) Reliability
C) States
D) Validity

Question 63

A research team is developing a test of school readiness. They believe that there are four dimensions of school readiness. They completed a factor analysis on their data. If the test was developed well, how many factors should have been evident in the factor analysis?

A) One
B) Three
C) Four
D) There is not enough information to tell.

Question 64

Clyde's personality is being assessed in a laboratory setting. He is asked to interact with new people and his behavior is rated. This is an example of a(n):

A) Self-report measure
B) Projective measure
C) Performance measure
D) Achievement measure

Question 65

If a research project was designed to determine if a questionnaire given to newly married couples can determine the likelihood that they will eventually divorce, this would be an example of what type of evidence?

A) Concurrent
B) Content
C) Discriminant
D) Predictive

Question 66

The difference between true scores and observed scores on tests is called

A) Error
B) Homogeneity
C) Heterogeneity
D) Reliability

Question 67

Consistency of scores on a test over time is known as:

A) Reliability Coefficient
B) Test-Retest Reliability
C) Equivalent Forms Reliability
D) Internal Consistency

Question 68

If a researcher collects data using an instrument from a group of participants who vary greatly from the instrument's norming group, what does this mean?

A) The study is only moderately valid
B) You should downgrade your evaluation of the quality of the research study because the measurement was of highly questionable validity
C) You can assume that the instrument still worked well
D) The study had a high interscorer reliability correlation

Question 69

When constructing an instrument to measure attitudes toward exercise, a researcher had a panel of experts examine the items generated to make sure the items adequately represented the domain of interest. This would be an example of:

A) Reliability analysis
B) Factory Analysis
C) Content-related evidence validity
D) Validity coefficient

Question 70

Anna is participating in research project. She is asked to look at ambiguous drawings and tell the researcher what she sees in the drawings. This is an example of a(n):

A) Self-report measure
B) Projective measure
C) Performance measure
D) Achievement measure

Question 71

Which of the following is a form of internal consistency reliability?

A) Test-retest
B) Interscorer
C) Split-half
D) Equivalent forms

Question 72

A test developer was looking at the reliability and validity of the test she developed. She found that the coefficient alpha was -.88 and that the convergent validity coefficient was -.92. What can she say about the reliability and validity of her test?

A) The test is reliable and valid
B) The test is reliable but not valid
C) The test is not reliable but valid
D) The test is neither reliable nor valid

Question 73

Validity evidence based on the relationship between test scores and criterion scores obtained at the same time is known as:

A) Content-Related Evidence
B) Concurrent Evidence
C) Predictive Evidence
D) Alternate Forms Evidence

Question 74

Validity evidence based on the relationship between test scores collected at one point in time and criterion scores obtained at a later time is known as:

A) Norm-Related Evidence
B) Concurrent Evidence
C) Predictive Evidence
D) Discriminant Evidence

Question 75

The relatively permanent patterns that characterize and can be used to classify individuals is called:

A) Intelligence
B) Personality
C) Homogeneity
D) IQ

Question 76

For one of your tests in this class, the instructor is going to give you a calculus test. Which of the following outcomes of the use of this test in your class focused on educational research?

A) The test is a reliable and valid test for this class
B) The test is reliable but not valid for this class
C) The test is not reliable but valid for this class
D) The test is neither reliable nor valid

Question 77

It is not a good idea to assess test-retest reliability by two tests that are given a few days apart because:

A) The correlation will be larger due to remembering how items were answered
B) The correlation will be smaller due to learning new things
C) Test-retest reliability is not assessed by giving the same test twice
D) To calculate test-retest reliability the two tests must be given 6 hours apart

Question 78

Validity evidence based on the extent to which scores from a test can be used to predict or infer performance on some criterion such as a test or future performance is which of the following? a Test-retest evidence.
B) Concurrent evidence
C) Criterion-related evidence
D) Alternative forms evidence

Question 79

Validity evidence based on the relationship between the focal test scores and independent measures of the same construct is known as:

A) Criterion-Related Evidence
B) Predictive Evidence
C) Convergent Evidence
D) Discriminant Evidence

Question 80

Evidence that the scores on your focal test are not highly related to the scores from other tests that are designed to measure theoretically different constructs is known as:

A) Criterion-Related Evidence
B) Concurrent Evidence
C) Convergent Evidence
D) Discriminant Evidence