Designing Assessments

On this page you’ll find information on:

 

What are assessments?

Assessments are broken down into two types: formative and summative.

Formative assessment: Formal and informal processes teachers and students use to gather evidence for the purpose of improving learning. Teachers use assessment information during the learning to diagnose student needs, plan next steps in instruction, provide students with targeted practice, and offer effective feedback.

Summative assessment: Assessment information used to provide evidence of student achievement for the purpose of making a judgment about student competence or program effectiveness.

(source: From Classroom Assessment for Student Learning by Jan Chappuis, Rick Stiggins, Steve Chappuis and Judith Arter)

Summative Assessment: Usually given at the end of instruction to assess mastery of learning outcomes. Types: exams, presentations, creation of a product, portfolio, group presentation. Formative assessments are given frequently throughout the course to evaluate progress. *Feedback must be given to be effective. Types include learning logs/HW/activities, discussions, reflection, group presentations, practice quizzes.

Image from https://fsw.instructure.com/courses/1018292/pages/formative-and-summative-assessment

 

Authentic Assessment

Authentic Assessment means assessing learning by having students do something real, as opposed to a more traditional testing approach.

Sometimes it is not feasible to have completely authentic assessment for time, financial or safety reasons. For example, if your students are learning about putting out forest fires, you can’t very likely set a forest on fire to give them an authentic experience of trying to contain it. However, it is a good goal to try to be as authentic as possible in your learning experiences, including assessment.

 

Rubrics

A rubric is a scoring guide and feedback tool that outlines expectations for an assessment. It includes standards (how well the criteria are demonstrated) and criteria (the key elements of performance that are being assessed). Rubrics…

  • Set clear expectations for performance
  • Speed up marking.
  • Set objective criteria for performance.
  • Clearly connect to goals.
  • Are a tool for learning as well as marking.

Example: A Rubric for Creating Nachos

Learning Outcome: Create an authentic three-course Mexican meal.
Assessment Goal: Create an appealing nacho dish including cheese, toppings, and sides.
Criteria Strong Medium Weak (Not Yet)
Chips Crispy, flavourful, and hold toppings well (intact). Slightly soggy or lacking in flavour<, somewhat intact. Soggy, stale, crumbly/broken.
Cheese Plentiful, layered throughout, fully melted. Lacking in quantity or distribution, or partially melted. Not enough cheese, not well distributed, and not fully melted.
Toppings Creative, plentiful, well distributed, bite-sized. Basic, inappropriate size, or lacking in quantity or distribution. Underwhelming or missing, of inappropriate or inconsistent size.
Sides Fresh, plentiful, and varied. Not enough, or stale, or not varied.  Not enough, or missing.
Presentation  Arrives hot, visually appealing.  Arrives lukewarm or messy.  Arrives cold, or unappealing. 

Back to top

Common Assessment Types

Dropbox Assignments:

These assignments may come in several forms, like asking students to write a research paper, self-reflection document, PowerPoint presentation, or any other assignment that requires a student to draft or complete a file and submit it for subjective review and grading. To guide students in the completion of such assignments and provide a consistent format, consider using the Template for Assignments This template ensures students understand how the assessment is linked to course outcomes and the criteria for success.

Quizzes – Multiple-Choice Questions:

While multiple-choice questions are not always the most authentic assessment of students’ learning, sometimes they are necessary. But how do you write a good (defensible) multiple-choice question?

Each question must:

  • minimize the amount of reading in each item (aim for grade 10 reading level).
  • include a stem (statement or question to which learners respond).
  • avoid vague frequency terms (e.g. always, never, often, usually).
  • include the central idea of the question in the stem instead of the choices.
  • ensure the directions in the stem are very clear.
  • word the stem positively (i.e. avoid negatives such as NOT or EXCEPT, but capitalize if used).
  • not use “which of the following…”.
  • use “which” before a noun and “what” before a verb.
  • not include more than one sentence.
  • contain four answer options:
    • 1 key (best or most appropriate of available options)
    • 3 distractors (plausible yet incorrect options to the stem).
    • A single, specific issue (as concretely as possible).
    • Acronyms spelled out.
    • vary the location of the correct answer.
    • ensure distractors and key all follow grammatically from the stem, and are homogeneous in content and grammatical structure.
    • ensure all options are of the same length (count the numbers of words!).
    • place options in logical (e.g. alphabetical or numerical) order.
    • avoid “none of the above” and “all of the above”.
    • make all distractors plausible.
    • phrase choices positively (avoid negatives, such as NOT).
    • avoid clues to the right answer, such as:
      • always, never, completely, and absolutely.
      • options that are identical or similar to words in the stem.
      • grammatical inconsistencies that clue test-taker to the correct option.
      • a conspicuously correct option.
      • pairs or triplets of options that clue test-taker to the correct option
      • blatantly absurd, ridiculous options.

Click Multiple Choice – quiz questions for a printable PDF version of the source document. 

Classroom Assessment Techniques:

Back to top

Considering Assessment Accessibility:

Wherever possible, applying UDL guidelines to assessments supports all students’ success. This can be done by including additional time on assessments for students to read carefully, evaluate their answers, and potentially lower anxiety; all which ensures a more authentic assessment of learner skills and knowledge. Faculty can determine which of their assessments would be best suited to additional time, and this does not need to be applied across the board. Proactively adding additional time to assessments can reduce the effort of arranging time accommodations through Accessible Education Services (AES).

As the most common additional time accommodations are typically 50% to 100%, the recommendation is to add 100% additional time to assessments when possible or alternatively, unlimited time. It is essential that students be informed that additional time has been added.

Review the steps below:

    1. Determine where additional time can be proactively added to assessments in the course.
    2. Determine the appropriate length of time.
    3. Add the additional time to the assessment.
    4. Modify the bold text below and provide the following statement anywhere assessment information has been provided (for example, on the course learning plan, the online course overview page, etc.):

Additional Time for Online Course Quizzes/Tests/Exams

The quizzes/tests/exams in this course have been designed using Universal Design for Learning guidelines and (give the percentage) additional time has already been added to each quiz/test/exam in support of all students.”

In addition, Flexible Due Dates should also be considered and incorporated whenever possible in your courses. There are many ways you can add flexibility to your assignment policies, and doing so offers benefits to all students as well as faculty. For more information and ideas that you can tailor to your own courses, read Flexible Due Dates.

Back to top

Designing Assessments for Academic Integrity

There are several strategies to encourage academic integrity in both in-person and alternate delivery courses. These strategies range from providing resources that set students up for success, to designing assessments that make it difficult for students to plagiarize: 

    • If possible, use authentic assessments that emphasize problem-solving, links to personal experience, or focus on distinctive tasks. For example, ask students to “evaluate,” “create,” or “analyze” a given theory or problem.
    • Consider including a critical reflection component where students reflect on their own learning.
    • Include follow-up questions such as “Expand upon the ideas behind the information you referenced” or “Explain why you choose this description, example, phrase, reference, etc.”
    • Use scaffolded, multi-stage assignments, where students submit multiple drafts of their work
    • Use D2L’s Quizzes tool features such as randomized question order and response option order, to decrease the potential for academic misconduct.
    • Use timed assessments only when learning outcomes warrant a timed assessment.
    • Communicate the College’s academic integrity policy at the start of the course and reference it when performing assessments.>
    • Educate students about academic integrity and provide resources to help their understanding of academic integrity and the penalties for not following College policy.
    • Consider requiring student engagement with the academic integrity policy as a restriction in D2L, before providing access to the assessment.
    • Encourage student-instructor and student-student interaction throughout the course. Research shows that students who feel more connected to their instructor are less likely to engage in academic misconduct.
    • In remote courses, synchronous video conferencing, using platforms such as WebEx or Teams, can promote and maintain student-instructor connections online, where feasible.
    • Direct students to research and writing supports so that students understand how to complete the assignment or exam, including proper referencing practices in the discipline.
    • Consider using Respondus Lockdown Browser and Monitor for online tests or exams in D2L.
    • Make use of plagiarism detection software such as Turnitin.com, or use a Google search of unique phrases from a submitted paper.

Back to top

  • Addressing Artificial Intelligence

  • What is Generative AI (like Chat GPT)?  Generative Artificial Intelligence (generative AI) creates new content based on what it has learned from existing content. The process of learning from existing content is called training and results in the creation of a statistical model. When given a prompt, generative AI uses this statistical model to predict what an expected response might be and provides new content. (Source: https://youtu.be/G2fqAlgmoPo). Faculty should educate themselves on how generative AI functions if they are to use it in their courses.
  • What are the concerns? There are two initial concerns with the use of generative AI at Fleming: privacy and plagiarism.
  • Privacy: 
  • Privacy is the responsibility of everyone at Fleming. Generative AI tools are not search engines and “take” what you give them:
    • Anything submitted to generative AI tools and detection tools becomes the property of that tool, which can lead to privacy and intellectual property issues. Faculty and staff must consider college policies for privacy and intellectual property by not submitting others’ information to any generative AI tool, such as ChatGPT.
    • ChatGPT will take credit for works submitted as if it generated it itself. It does this by remembering everything that was submitted, and it is employed to evolve the AI model used for future results. ChatGPT may also falsely take credit for content it learns.

     

  • Plagiarism:
  • To help address concerns for misuse of generative AI tools for assessments, faculty should consider the following:
    • The use of generative AI tools can be adopted by faculty as an opt-in model (you decide for each course). The course outline is the best way to make clear your course expectations.  Where explicit adoption is not made clear, the assumption will be that use of generative AI is not permitted in the course.
    • There are three examples of course outline statements that faculty can adopt to help provide clarity to students around generative AI tool use in their classes:
    • NO USE: Use of generative AI tools (like ChatGPT) are not permitted in this course.
    • NO ASSESSMENTS: Generative AI tools (like ChatGPT) may only be used to assist exploratory learning and cannot directly contribute to any assessment as part of this course.
    • SOME USE: Generative AI tools (like ChatGPT) can only be used where directed in this course.
    • Further, to help faculty address questionable assessment submissions, it is also recommended students document their drafting process. This requirement can be communicated in the course outline as:
    • DOCUMENTATION REQUIREMENT: It is the responsibility of students to maintain a history of records and supporting documentation to demonstrate their efforts in all academic submissions, even if submission of these is not part of the final academic deliverable.
    • Students with accommodations registered with Accessible Education Services might have some permitted use, even if faculty decide to disallow generative AI tools within their courses. The Letter of Accommodation for students would address this. Faculty should discuss how best to use generative AI tools with the accommodated student and their counsellor, if needed.
    • The existing Fleming College Academic Integrity Policy 2-201Ais to be used where a student submits work that is not their own original work. Combined with the documentation requirement suggestion for course outlines (above), this should reduce the faculty time needed to investigate concerning assessments.
    • AI detection tools claiming to detect generative AI content may lead to “false-positive” detection, where the tool flags content as AI-generated when the student authored it themselves. These detection tools are currently not reliable and as such are not to be used to evaluate student work. The college will continue to evaluate the efficacy of tools that detect generative AI.

     

  • Best Practices:

Faculty should consider some best practices in their courses to help discourage non-permitted generative AI use:

  • Generative AI is not a search engine and each question asked will generate a different response, even if asked by the same person again. To learn how these tools works, consider the following two resources for good starting points.
  • This Twitter thread from Andrew Piper of McGill University, offers some practical considerations for how to address the use of AI in your courses, including the “AI Disclosure” wording he uses with his students.
  • Assessment and genAI
  • Faculty can connect with industry partners and professionals to learn how or if artificial intelligence is/will be used in their day-to-day work and activities. Give consideration to the appropriateness of introducing artificial intelligence as a learning strategy if it is leveraged in the industries and professions students are studying toward so that we are preparing graduates for the careers they plan on pursuing.
  • Faculty should provide examples and direction for students on appropriate use allowed in their course(s).
  • For assessments, use a multi-step process where students submit course work at multiple stages in the process. Faculty can provide formative feedback to students at early stages, which would make mis-using generative AI challenging for students.
  • Faculty could request an early baseline sample of student work as a comparison for assessments submitted during the course. If the style or tone of the original differs significantly with the comparison, a discussion with the student can be had to understand the process leading to those changes.
  • Class discussions can help students understand why AI tools are or are not allowed in classes.

Back to top

Resources