Exam It em Wri t i n g Gu i de

SALESFORCE ITEM WRITING

BASICS FOR ITEM WRITERS
Version 2.3

© 2019, 2018 Salesforce.com, Inc. All rights reserved.

SALESFORCE ITEM WRITING BASICS FOR ITEM WRITERS

CONTENTS
Purpose .............................................................................................................................. 2
Certification Exams and Target Audience ....................................................................... 2
Terminology ....................................................................................................................... 3
Item Types .......................................................................................................................... 5
Writing Rules and Guidelines ........................................................................................... 6
Targeting Cognitive Levels for Multiple Choice Item Writing ...................................... 10
Review – Item Writing Checklist .................................................................................... 11

©Copyright 2019, 2018 Salesforce.com, Inc. All rights reserved. i

SALESFORCE ITEM WRITING BASICS FOR ITEM WRITERS

PURPOSE
The purpose of this guide is to help you author items that come as close as possible to
simulating the integration and application of knowledge and problem solving required to
perform competently in the job role. It is based on industry best practices and experience.

CERTIFICATION EXAMS AND TARGET AUDIENCE

Salesforce Certification exams determine if a candidate meets the baseline standard to
perform the essential duties of a job role. Certification exams use pass/fail scoring. This
means that a test taker who barely passes a certification exam is equally as “certified” as
someone who scores 100%. Understanding the target audience and job roles is critical in
developing an exam that can accurately determine this outcome.

Certification exams are meant to measure a certain level of expertise based on job role.
Some test candidates will achieve this measure and others will not. The test taker who barely
achieves the determined level, is referred to as the Minimally Qualified Candidate (MQC). It
is important to remember that the MQC is not considered a subject matter expert in the
domain. The MQC is a conceptualization of the exam candidate who has the minimum
knowledge, skills, and experience necessary to perform the job role; but who will continue to
learn and grow their abilities.

©Copyright 2019, 2018 Salesforce.com, Inc. All rights reserved. 2

SALESFORCE ITEM WRITING BASICS FOR ITEM WRITERS

TERMINOLOGY
Exam development uses terminology that is often unfamiliar to those outside the field, such
as an MQC. Understanding these definitions will help subject matter experts (SMEs),
facilitators, and stakeholders clearly communicate during the exam development process.

Parts of an Item
 Item - the entire exam question including the question portion and answer choices, plus
any additional information such as exhibits or scenario text

 Stem – the portion of the item that contains the question as well as any additional text
that precedes or follows the question

 Options – all possible answer choices

 Key – the correct answer option(s) for the question posed in the stem

 Distractors – the incorrect answer options

©Copyright 2019, 2018 Salesforce.com, Inc. All rights reserved. 3

SALESFORCE ITEM WRITING BASICS FOR ITEM WRITERS

Other Terminology
 Congruence is the measure of how well an item conforms to the knowledge, skill, and
ability of an objective – both the domain topic and cognitive level.

 Cueing/Double-Cueing conveys how many answers a candidate should choose. Double-

cueing is the number of correct answers written twice; once within the stem and once as
a stand-alone sentence after the stem.

 Cut Score is the minimum score required to pass an exam.

 Exam Blueprint defines the content and number of items in an exam.

 Item Bank is the collection of all items for an exam.

 Objectives specify the domain content that is tested. Strong objectives contain three
parts: measurable task, condition/dependency, and performance standard.

 Reliability shows how consistently an exam performs as expected each time it is used.

 Test Form is the group of items that are delivered to a candidate who takes an exam.
Multiple forms are created for exam security.

 Validity show how well an exam measures what it is supposed to measure.

©Copyright 2019, 2018 Salesforce.com, Inc. All rights reserved. 4

SALESFORCE ITEM WRITING BASICS FOR ITEM WRITERS

ITEM TYPES
Most Salesforce exam items are multiple-choice format. They may have one or more keyed
answers. Items with more than one keyed answer, referred to as multiple select, explicitly
state the number of options that should be chosen through double-cueing in the stem.
Salesforce multiple select items are scored either correct or wrong. No partial credit is given
for choosing some, but not all, keyed answers.

The goal is to write items that give candidates the opportunity to show what they know and
how their knowledge might be applied to realistic situations. The items should not test
memorization or recall of facts, but rather the application of this knowledge. For example:

 Represent the ability to understand, describe and explain a concept, procedure or

process.

 Test important foundational grasp and require understanding and comprehension.

 Represent the ability to integrate and apply knowledge in new contexts, analyze and
troubleshoot, and solve problems.

 Present material in a context representative of real on-the-job situations.

 Evaluate and weigh the costs and benefits of various plausible solutions.

 Make decisions about what knowledge is relevant to the problem at hand.

 Select the appropriate solution from several that could accomplish the task.

Items should not aim to trick people or test on less common situations. Item writers should
also avoid asking about things that people would not be expected to know from memory.

©Copyright 2019, 2018 Salesforce.com, Inc. All rights reserved. 5

SALESFORCE ITEM WRITING BASICS FOR ITEM WRITERS

WRITING RULES AND GUIDELINES

When item writing, the goal is to present an item that is clear and relevant for a Minimally
Qualified Candidate (MCQ) to answer. To this goal, there are rules that must be followed.

Mandatory Rules for Items

1. Item must be congruent with the objective.

 Domain congruence indicates that an item maps to the skills and abilities described
in the objective.
 Cognitive congruence indicates that an item maps to the cognitive complexity level
identified in the objective. (See more in the Cognitive Complexity section below.)

2. Item must be relevant and appropriately difficult for the MQC.

 Always keep the MQC in mind.

 The item should be something meaningful that the MQC needs to know.
 Do not waste items on irrelevant content.
 Items should be at the correct difficulty level for the MQC; not too hard or too easy.

3. Items must be technically accurate.

 Subject matter expertise is required to write the items.

4. Items must include a reference.

 Include a link to a Salesforce publically available source for learning the specific item
content, such as Trailhead or product documentation.

©Copyright 2019, 2018 Salesforce.com, Inc. All rights reserved. 6

SALESFORCE ITEM WRITING BASICS FOR ITEM WRITERS

Writing Clear Items

Writing a clear item can be more challenging than it sounds. Candidates should be able to
understand an item without reading it several times. The rules and guidelines below
contribute to an item being clear and easily understood by all candidates.

Rules for the Stem

1. Use clear, concise, and simple language.
 Remove all unnecessary words.
 Use multiple sentences to break up complex phrasing or situations.
2. Include all pertinent information for the MQC test taker to be able to understand and
answer the question correctly.
 Focus on one central thought, problem, or idea per item.
 Items are presented in random order, so each must be independent.
 Leave out anything that is not needed to answer the question.
3. Phrase the final sentence of the stem as a question, starting with a question word and
ending with a question mark.
 Use “Which” before a noun and “What” before a verb.
 Do not use fill in the blank or complete the sentence formats
4. Do not write negatively worded stems.
 State questions in a positive form.
5. Provide cueing for multiple select items.

Rules for the Answer Options

1. Select the number of answer options.
 Multiple choice items have four options.
 Multiple select items with two correct answers have four options
 Multiple select items with three correct answers have five options.
2. The correct key is indicated.
3. All answer options must answer the question posed.
4. All distractors must be plausible.
 Avoid clues that give away the key or eliminate distractors.
 Avoid absolutes; such as always and never.
5. All answer options must be parallel (similar in length, construction, and grammar).
 Avoid excessive detail in the key and sparsely worded distractors.
6. All answer options must be mutually exclusive.
 An option should not include or eliminate another.
 Do not create an option that is the opposite of another option.
 Do not use All of the above, None of the above, A and B only, or similar combinations.
7. Avoid answer options that negate the premise of the stem.
 For example, avoid asking: “Which tool should be used to….” and then having options
such as: "There is no tool that does this." or “This cannot be done.”

©Copyright 2019, 2018 Salesforce.com, Inc. All rights reserved. 7

SALESFORCE ITEM WRITING BASICS FOR ITEM WRITERS

General Guidelines
This section has additional detail on some of the rules above, plus general guidelines for
writing clear and effective items.

1. Do not use trick items or distractors.

 Trick items are unfair to test takers and threaten the validity of the test.
 Trick items measure test-taking skills more than the skills and experience of the
candidate.
 Items should always be written as clear, accurate, and straightforward as possible.
 The question presented should lead a candidate to think of an answer without
reading the options. Example - Avoid “Which statement is true?” because it cannot
be answered without reading the options
 Do not try to make an item “harder” by using complex, overly-precise wording or
similar technique. Likely this is not a suitable item for the MQC and should be
rethought completely.
2. Beware of unintentional trickiness.
 Ambiguity in either the stem or options
 Overlapping options
 Discrimination between options is too fine
 Multiple options potentially correct, but only a single response allowed.
3. Focus each item on one central thought, problem, or idea.
4. Avoid excessive verbiage.
 Wordy items take longer to read, comprehend, and answer.
 Write items as clear and concise as possible without compromising details that are
needed.
 Get to the point quickly in the stem and present clean, clear options for the
examinee to choose.
5. Avoid unnecessary background information or teaching the test taker.
6. Avoid using “of the following” in the question to refer to the answer options.
 A simple “Which…” followed by a noun is more efficient and effective.
7. Avoid opinion-based items.
 Never ask "What would you...", This is subjective since the examinee's answer can
never be wrong.
 Instead ask “What should…” or “According to XXX, what is recommended…”.
8. Avoid absolute or vague modifiers and superlative adjectives.
 Avoid using absolutes such as always, all, never, only, and none.
 The use of absolute modifiers in options makes it easy to eliminate options and
increases guessing probability.
 Use caution with superlative adjectives such as best, most, fastest, cheapest, lowest,
etc. unless it is clearly documented and will not change.
9. Use consistent verbiage.
 Determine consistent wording and capitalization for common product-related
terminology and phrasing prior to item writing.
 Use this verbiage consistently throughout an exam.
10. Do not lift exact wording from training materials or documentation.
 Create novel item content.
 Step back from the training and think independently about the item that could be
constructed.

©Copyright 2019, 2018 Salesforce.com, Inc. All rights reserved. 8

SALESFORCE ITEM WRITING BASICS FOR ITEM WRITERS

11. Keep items independent.

 Items are presented in random sequence.
 Content should be independent from item to item.
 Do not give the answer away to one item in the stem of another. These are
considered enemy items.
12. Double cue the stem for multiple select items.
 Double-cueing consists of writing one cue within the stem question, such as “Which
two solutions…”. The second cue is added automatically by the Salesforce item
development tool, Webassessor, based on the number of correct options indicated
in the key.
13. Write to a world-wide audience.
 Avoid culture-specific examples or scenarios.
 Carefully consider usage of topics and formats that vary across geographies.
Examples include: numbers, addresses, time, and measurements.

©Copyright 2019, 2018 Salesforce.com, Inc. All rights reserved. 9

SALESFORCE ITEM WRITING BASICS FOR ITEM WRITERS

TARGETING COGNITIVE LEVELS FOR MULTIPLE CHOICE ITEM

WRITING
Certification exam objectives should be written to reflect the cognitive processes required by
the job at the level of the MQC. In addition each exam item should be authored to
complement the cognitive complexity or “depth” as indicated in the specific aligned
objective.

The three levels of cognitive complexity are represented in the figure. Most Salesforce
certification exams do not test at the Remember level.

Level 1: Remember (R) – Evaluates a testee’s ability to recall certain facts.

 Test questions focus on identification, memorization, and recall of

information.

Level 2: Understand/Apply (U/A) – Evaluates a testee’s ability to read test content and
extrapolate and interpret important information. Testees take new concepts and apply them
to another situation.
 Test questions focus on use of facts, rules and principles.
 Test questions focus on applying facts or principles.
 Test questions present multiple pieces of information that are needed to
answer the question.

Level 3: Analyze/Evaluate (A/E) – Testees have the ability to take new information and
break it down into parts to differentiate between them. Testees are able integrate and apply
knowledge in new contexts to analyze, troubleshoot, and solve problems.

 Test questions focus on separation of a whole into component parts

 Test questions focus on combining ideas to form a new whole
 Test questions focus on developing opinions, judgments or decisions

©Copyright 2019, 2018 Salesforce.com, Inc. All rights reserved. 10

SALESFORCE ITEM WRITING BASICS FOR ITEM WRITERS

REVIEW – ITEM WRITING CHECKLIST

When you complete item writing, use this checklist ensure all aspects are complete and the
item is ready for further peer review and editing.

Item Criteria

 Item is congruent with the objective (both cognitive and technical level)

 Item is relevant for the MQC.

 Item is appropriately difficult for the MQC.

 Item is technically accurate.

 Includes a reference.

Stem Criteria

 Stem uses clear and simple language.

 Stem includes all pertinent information to be presented independently from other

items.

 The stem is positively worded.

 The stem is phrased as a question.

 Cueing is provided for multiple select items.

Options Criteria

 There are four or five options with at least two distractors.

 The correct key is marked

 All distractors are incorrect.

 All options answer the question posed.

 All options are plausible.

 All options are clearly worded.

 All options are similar in length, construction, and grammar

 All options are mutually exclusive

©Copyright 2019, 2018 Salesforce.com, Inc. All rights reserved. 11