Overview of the Standard Setting Tool

The Standard Setting Tool consists of two modules, SSBM and SSAM. These modules allow users to organize and participate in standard setting sessions.

• In an SSBM standard setting session, subject matter experts determine the placement of proficiency standard cut values Cut values are the thresholds between two proficiency standards in the ordered item booklet (OIB). For example, if items 1-10 in a booklet require a different proficiency level than items 11-20, then the cut value is placed at item 11. for the ordered item booklet An ordered item booklet (OIB) is a set of test items arranged from easiest to most difficult, based on actual student performance data.. These proficiency standards identify the difficulty level of all the items in a booklet.

• In an SSAM standard setting session, subject matter experts map assertions to performance levels within the ordered scoring assertion booklet An ordered scoring assertion booklet (OSAB) is a set of test items, within each of which the assertions are ordered from easiest to most difficult, based on actual student performance data. The items themselves are not ordered from easiest to most difficult.. These mappings identify the difficulty level of all the assertions in each item.

Standard setting sessions are organized into various rooms, based on what needs to be reviewed in each room. A user with a Room Lead role manages the standard setting process for a given room, while the Panelist users in that room review the booklet’s items or assertions and make determinations.

After the Panelists perform the standard setting, Psychometricians review the data collected from the room and provide feedback. Each booklet goes through multiple rounds of bookmark placement (for SSBM) or assertion mapping (for SSAM), as well as multiple rounds of feedback, to ensure that there is a general consensus.

At the end of a session, the room sets the official proficiency standard cut values for the booklet. If the session includes a Moderation step, then a Panelist who has been designated as a Room Scribe may adjust the cut values, if necessary.

For information about user roles, see the section Overview of User Roles.