Question 1

The higher the item difficulty index , the ________ the item.

Accepted Answer

The item difficulty index measures how many test-takers answer an item correctly. A higher index indicates more people answered correctly, meaning the item is easier.

Question 2

If an assessment identifies a child as having ADHD when the child does not have ADHD, this would be an example of :

Accepted Answer

A false positive is when the assessment result identifies a condition that is not actually present. In this case, the child is identified as having ADHD when they do not actually have it.

Question 3

An important parameter in item response theory is the slope of the curve, which is an indicator of how well the item discriminates.If there is a ________ slope then the item is probably a good discriminator ,whereas a ________ slope probably indicates the item does not discriminate very well.

Accepted Answer

A steep slope indicates that the item discriminates well between high and low ability individuals, while a flat slope indicates that the item does not discriminate well and may be too easy or too hard for all individuals. Therefore, C is the correct choice as it correctly identifies the slopes that are indicative of good and poor discrimination in item response theory.

Question 4

An example of a false negative is when an instrument:

Accepted Answer

A false negative occurs when the instrument fails to identify a condition or characteristic that is actually present. In this case, the instrument is identifying a client as not being suicidal when they actually are, which is a false negative.

Question 5

The difference between the percentage of examinees in the upper group who answer an item correctly and the percentage of examinees in the lower group who answer the item correctly is called the ____________ index.

Accepted Answer

Item discrimination index measures the extent to which an item can distinguish between examinees who have high ability and those who have low ability. It is calculated by finding the difference between the percentage of high-ability examinees who answer correctly and the percentage of low-ability examinees who answer correctly.

Question 6

When considering evidence of validity and associated social consequences of instruments counselors should consider all of the following except for:

Accepted Answer

The other three options (use, interpretation, social implications) are important considerations when evaluating evidence of validity and associated social consequences of instruments. Cost effectiveness, while important in the decision-making process, is not directly related to validity or social consequences.

Question 7

Using decision theory, both expectancy tables and group separation help to determine:

Accepted Answer

Expectancy tables help in understanding the likelihood of a particular score being obtained by a test taker based on the group they belong to, thereby determining whether the test differentiates between groups. Group separation is used to determine whether there are significant score differences between different subgroups or categories within a sample, which helps with understanding whether the instrument differentiates between groups. Hence, both tools help in determining whether an instrument differentiates between groups.

Question 8

Standard error of measurement is to standard error of estimate as:

Accepted Answer

The standard error of measurement (SEM) and standard error of estimate (SEE) are both measures of the accuracy of a test or measurement. However, they are used in different contexts.

The SEM is a measure of how much error is inherent in a test or measurement. It reflects the fact that test scores are never perfectly accurate, but are subject to some degree of random error. The SEM is used primarily in the context of reliability analysis, where it is used to estimate the amount of true score variance in a set of test scores.

The SEE, on the other hand, is a measure of how accurately a statistical model (such as regression analysis) can predict an outcome variable from a set of predictor variables. It reflects the fact that there is often some degree of error in the predictions made by the model. The SEE is used primarily in the context of validity analysis, where it is used to evaluate the accuracy of the model's predictions.

In analogy to the other answer choices, concurrent validity and predictive validity both measure the validity of a test or measurement, but in different contexts (e.g., at the same time vs. in the future). Content validity and face validity both measure the validity of a test or measurement, but in different ways (e.g., by assessing the relevance of the test items vs. how well the test "looks" like it measures what it claims to measure). A normal distribution and a skewed distribution are both types of probability distributions, but with different shapes (e.g., symmetric vs. asymmetrical).

Thus, the closest analogy to the relationship between the SEM and SEE is the relationship between a reliability coefficient (used to measure internal consistency or test-retest reliability) and a validity coefficient (used to measure the relationship between a test score and an external criterion).

Question 9

In using regression to predict an individual's performance on the criterion based on his or her score on the instrument, the prediction is least accurate when the correlation is:

Accepted Answer

When the correlation is 0.00, there is no linear relationship between the predictor and criterion variables, making it impossible to accurately predict an individual's performance.

Question 10

A method used in order to examine whether an instrument differentiates between the groups using the approach of decision theory is:

Accepted Answer

Expectancy tables are used in decision theory to compare the expected outcomes of different actions. In the context of testing, expectancy tables can be used to compare the expected scores of different groups on an instrument, in order to determine whether the instrument is able to differentiate between the groups. Therefore, A is the best choice. Tables of specification are generally used in test development, not in evaluating an instrument's ability to differentiate between groups. Coefficient alpha is a measure of internal consistency reliability, not a method for evaluating group differences. Regression analysis can be used to evaluate group differences, but it is not specifically tied to the approach of decision theory. Therefore, D is not the best choice.

Question 11

If an instructor in an appraisal in counseling course was interested if an exam question was too easy or too difficult, she would want to examine the ________ of that question.

Accepted Answer

Examining the item difficulty of a question would allow the instructor to determine whether it was too easy or too difficult. Item difficulty refers to the proportion of test-takers who answer a specific question correctly, and is a measure of the question's difficulty level.

Question 12

An Expectancy Table is a useful device for determining the:

Accepted Answer

An Expectancy Table is used to determine the cutoff score of a selection test by showing the probability of success on a criterion measure for different test score ranges.

Question 13

When considering the criterion-related evidence of validity, one should be concerned with whether the criterion is:

Accepted Answer

The criterion-related evidence of validity involves determining if the test scores align with an external criterion, such as job performance or academic achievement. In order to ensure the validity of this relationship, the criterion must not only be reliable and unbiased, but also free from criterion contamination - meaning that it is not affected by any factors besides the construct being measured. Therefore, all of the choices (A, B, and C) are important considerations when evaluating criterion-related evidence of validity.

Question 14

A statistical procedure used to determine the generalizability of an instrument's validity is:

Accepted Answer

Meta-analysis is a statistical procedure that is used to determine the extent to which an instrument's validity can be generalized across different populations, settings, and measurement conditions. It involves analyzing the results of multiple studies that have used the instrument and synthesizing the findings in order to determine the overall validity of the instrument.

Question 15

The standard error of estimate is an index of the:

Accepted Answer

The standard error of estimate gives an estimate of the probable range of scores on the criterion variable for a given value of the predictor variable. It is a measure of how far scores on the criterion variable deviate from the predicted scores, and it indicates the precision with which the predictor variable can predict scores on the criterion variable. Therefore, the best answer is D.

Question 16

In the equation Y' = a + bX, the b represents the:

Accepted Answer

The b coefficient in the equation represents the slope or regression coefficient, which indicates the amount of change in the predicted Y score for each unit change in X. The intercept or intercept constant is represented by the a coefficient. The standard error of estimate is a measure of the variability of the predicted Y scores, and the predicted score on the criterion is calculated by plugging in a specific value for X.

Question 17

An item difficulty index provides:

Accepted Answer

The item difficulty index is calculated as the proportion of individuals who answered the item correctly.

Question 18

When we use Expectancy Tables, hit is to ________ as miss is to _________ .

Accepted Answer

In an expectancy table, "hit" refers to correctly predicting a specific outcome, while "miss" refers to incorrectly predicting the outcome. "Right" can be used synonymously with "hit," while "wrong" can be used synonymously with "miss."

Question 19

In appraisal, regression equations are most frequently used to:

Accepted Answer

Regression equations are mathematical formulas that allow us to predict an individual's score on a criterion variable based on their scores on one or more predictor variables. In appraisal, regression equations are most commonly used to predict test-takers' performance on the criterion, which could be anything from job performance to academic achievement.

Question 20

Validity evidence is to ________ as item analysis is to ________.

Accepted Answer

Validity evidence pertains to the overall quality of the instrument as a whole, while item analysis focuses on the individual characteristics of each item, such as difficulty level and discrimination power. Therefore, the best answer is A, as it accurately reflects the relationship between validity evidence and item analysis.

Quiz 4: Validity and Item Analysis