Question 1

If traditional measures of reliability are applied to a criterion-referenced test,the reliability estimate will likely be
A) spuriously low.
B) spuriously high.
C) exactly zero.
D) None of these

Accepted Answer

The answer of If traditional measures of reliability are applied...

Question 2

Which of the following would result in the LEAST appropriate estimate of reliability for a speed test?
A) test-retest
B) alternate-form
C) split-half from a single administration of the test
D) split-half from two independent testing sessions

Accepted Answer

Split-half from a single administration of the test would result in the least appropriate estimate of reliability for a speed test because it does not allow for any variability in performance due to testing on different occasions or with different test forms. This method only assesses internal consistency within a single administration, which may not accurately reflect the consistency of performance over time or across different forms of the test. Test-retest and alternate-form methods are generally preferred for assessing reliability in speed tests, as they better capture variability in performance over time and across different test conditions. Split-half from two independent testing sessions can also provide a more accurate estimate of reliability by allowing for variability in performance due to testing on different occasions.

Question 3

Which type(s)of reliability estimates would be most appropriate for a measure of heart rate?
A) test-retest
B) alternate-form
C) parallel form
D) internist consistency

Accepted Answer

Test-retest reliability estimate would be the most appropriate for a measure of heart rate as it involves measuring the same variable at two different points in time to determine the consistency of the measurements. This is suitable for heart rate measurements as it may vary due to factors such as physical activity or emotional state, and therefore, measuring it at two different times can help determine the reliability of the measurement.

Question 4

Classical reliability theory estimates the portion of a test score that is attributed to ________,and domain sampling theory estimates ________.
A) specific sources of variation; error
B) error; specific sources of variation
C) the skills being measured; variation
D) the skills being measured; content knowledge

Accepted Answer

Classical reliability theory estimates the portion of a test score that is attributed to error, whereas domain sampling theory estimates specific sources of variation.

Question 5

Interpretations of criterion-referenced tests are typically made with respect to
A) the total number of items the examinee responded to.
B) the material that the examinee evidenced mastery of.
C) a comparison of the examinee's performance with that of others who took the test.
D) a formula that takes into account the total number of items for which no response was scorable.

Accepted Answer

Criterion-referenced tests are designed to measure an individual's mastery of a particular set of knowledge or skills. Therefore, interpretations of these tests are typically made with respect to the material that the examinee evidenced mastery of.

Question 6

Which type(s)of reliability estimates would be appropriate for a speed test?
A) test-retest
B) alternate-form
C) split-half from two independent testing sessions
D) All of these

Accepted Answer

The answer of Which type(s)of reliability estimates would be appropriate...

Question 7

Traditional measures of reliability are inappropriate for criterion-referenced tests because variability
A) is maximized with criterion-referenced tests.
B) is minimized with criterion-referenced tests.
C) is variable with criterion-referenced tests.
D) cannot be determined with criterion-referenced tests

Accepted Answer

Traditional measures of reliability, such as test-retest and inter-rater reliability, are based on variability in scores among test-takers. However, criterion-referenced tests are designed to measure whether a test-taker has mastered a specific set of skills or knowledge, not to rank test-takers based on their scores. As a result, variability in scores is minimized in criterion-referenced tests, making traditional measures of reliability inappropriate.

Question 8

A measure of clerical speed is obtained by a test that has respondents alphabetize index cards.The manual for this test cites a split-half reliability coefficient for a single administration of the test of .95.What might you conclude?
A) The test is highly reliable.
B) The published reliability estimate is spuriously low and would have been higher had another estimate been used.
C) The split-half estimate should not have been used in this instance.
D) Clerical speed is too vague a construct to measure.

Accepted Answer

The answer of A measure of clerical speed is obtained...

Question 9

The standard deviation of a theoretically normal distribution of test scores obtained by one person on equivalent tests is
A) the standard error of the difference between means.
B) the standard error of measurement.
C) the standard deviation of the reliability coefficient.
D) the variance.

Accepted Answer

The standard deviation of a theoretically normal distribution of test scores obtained by one person on equivalent tests is the standard error of measurement. It represents the amount of measurement error in a test score, and indicates how much a person's test score will vary if they take the same test multiple times.

Question 10

Item response theory (IRT)focuses on the
A) circumstances that inspired the development of the test.
B) test administration variables.
C) individual items of a test.
D) "how and why" of the Interborough Rapid Transit line

Accepted Answer

Item response theory (IRT) focuses on the individual items of a test, specifically on how each item relates to the underlying construct being measured and how individuals are likely to respond to each item.

Question 11

Generalizability theory focuses on which of the following?
A) the circumstances under which a test was developed
B) the circumstances under which a test is administered
C) the circumstances under which a test is interpreted
D) All of these

Accepted Answer

Generalizability theory focuses on evaluating the influence of various factors on the reliability and validity of test scores, including the circumstances under which a test was developed, administered, and interpreted. Therefore, all of these choices are relevant to this theory.

Question 12

Which estimate of reliability is most consistent with the domain sampling theory?
A) test-retest
B) alternate-form
C) internal-consistency
D) interscorer

Accepted Answer

The domain sampling theory suggests that in order to have a reliable measure or test, it should cover all relevant aspects of the domain being tested. Internal-consistency measures the extent to which all items in a test or measure are measuring the same construct or domain. Therefore, internal-consistency is the most consistent with the domain sampling theory. Test-retest and alternate-form reliability measures are more focused on temporal stability and consistency over different forms or versions of the test, while inter-scorer reliability is focused on consistency between different raters or scorers.

Question 13

Typically,speed tests
A) contain items of a uniform difficulty level.
B) are completed by fewer than 1% of all test-takers.
C) have low validity coefficients.
D) yield high rates of false positives.

Accepted Answer

Speed tests typically contain items of a uniform difficulty level, meaning that each item is designed to be equally challenging for all test-takers. This allows for more accurate and reliable comparisons between individuals' scores. The other options (B, C, and D) do not accurately describe speed tests.

Question 14

If a test is homogeneous
A) it is functionally uniform throughout.
B) it will likely yield a high internal-consistency reliability estimate compared with a test-retest reliability estimate.
C) it would be reasonable to expect a high degree of internal consistency.
D) All of these

Accepted Answer

A homogeneous test is functionally uniform throughout, so choice A is correct. This uniformity in the test content and format makes it reasonable to expect a high degree of internal consistency, which is captured by the measure of the reliability coefficient. Therefore, choice C is also correct. Finally, since a homogeneous test has consistent item content and is free from practice effects, it is likely to yield a high internal-consistency reliability estimate compared with a test-retest reliability estimate. Therefore, choice B is also correct.

Question 15

The fact that the length of a test influences the size of the reliability coefficient is based on which theory of measurement?
A) classical test theory (CTT)
B) generalizability theory
C) domain sampling theory
D) item response theory (IRT)

Accepted Answer

The fact that the length of a test influences the size of the reliability coefficient is a fundamental property of classical test theory (CTT), which is concerned with the reliability and validity of test scores. Other theories of measurement, such as generalizability theory, domain sampling theory, and item response theory, may also address issues related to reliability and validity, but they do not predict how the length of a test affects its reliability under CTT.

Question 16

An estimate of the reliability of a speed test is a measure of
A) the stability of the test.
B) the consistency of the response speed.
C) the homogeneity of the test items.
D) All of these

Accepted Answer

The answer of An estimate of the reliability of a...

Question 17

The Spearman-Brown formula can be used for which types of tests?
A) speed and multiple-choice
B) true-false and multiple-choice
C) speed, true-false, and multiple-choice
D) trade school and driving tests

Accepted Answer

The Spearman-Brown formula can be used for tests that measure reliability, such as speed, true-false, and multiple-choice tests. It is not applicable to trade school or driving tests, as those types of tests do not measure reliability.

Question 18

Use of the Spearman-Brown formula would be INAPPROPRIATE to
A) estimate the effect on reliability of shortening a test.
B) determine the number of items needed in a test to obtain the desired level of reliability.
C) estimate the internal consistency of a speed test.
D) All of these

Accepted Answer

The Spearman-Brown formula is used to estimate the increase in internal consistency reliability that would result from lengthening a test or increasing the number of items. It is not appropriate for shortening a test or determining the number of items needed for a desired level of reliability. Additionally, it is not typically used for speed tests, as speed tests often have low internal consistency due to their nature of measuring speed rather than accuracy or consistency.

Question 19

If a time limit is long enough to allow test-takers to attempt all items,and if some items are so difficult that no test-taker is able to obtain a perfect score,then the test is referred to as a ________ test.
A) speed
B) power
C) reliable
D) valid

Accepted Answer

A test that is long enough for test-takers to attempt all items and contains some difficult items that no one can answer correctly is referred to as a power test. Power tests are used to assess a person's upper limit of ability or knowledge in a particular area. Speed tests, on the other hand, are designed to measure how quickly a person can complete a task with a set time limit. Reliable and valid are characteristics of all good tests, but they are not specific to a particular type of test.

Question 20

A Kuder-Richardson (KR)or split-half estimate of reliability for a speed test would provide an estimate that is
A) spuriously low.
B) spuriously high.
C) insignificant.
D) equal to a test-retest method.

Accepted Answer

The answer of A Kuder-Richardson (KR)or split-half estimate of reliability...

Quiz 5: Reliability

If traditional measures of reliability are applied to a criterion-referenced test,the reliability estimate will likely be

Which of the following would result in the LEAST appropriate estimate of reliability for a speed test?

Which type(s) of reliability estimates would be most appropriate for a measure of heart rate?

Classical reliability theory estimates the portion of a test score that is attributed to ,and domain sampling theory estimates .

Interpretations of criterion-referenced tests are typically made with respect to

Which type(s) of reliability estimates would be appropriate for a speed test?

Traditional measures of reliability are inappropriate for criterion-referenced tests because variability

A measure of clerical speed is obtained by a test that has respondents alphabetize index cards.The manual for this test cites a split-half reliability coefficient for a single administration of the test of .95.What might you conclude?

The standard deviation of a theoretically normal distribution of test scores obtained by one person on equivalent tests is

Item response theory (IRT) focuses on the

Generalizability theory focuses on which of the following?

Which estimate of reliability is most consistent with the domain sampling theory?

Typically,speed tests

If a test is homogeneous

The fact that the length of a test influences the size of the reliability coefficient is based on which theory of measurement?

An estimate of the reliability of a speed test is a measure of

The Spearman-Brown formula can be used for which types of tests?

Use of the Spearman-Brown formula would be INAPPROPRIATE to

If a time limit is long enough to allow test-takers to attempt all items,and if some items are so difficult that no test-taker is able to obtain a perfect score,then the test is referred to as a ________ test.

A Kuder-Richardson (KR) or split-half estimate of reliability for a speed test would provide an estimate that is