Spotlight Three: Assessments & Mapping Thinkspace | Faculty of Arts | Seneca POLYTECHNIC

Assessments & Mapping: Diversifying Your "How"

Earlier spotlights reviewed the contextual 'who, what, where and why' that inform postsecondary classroom assessment. The purpose of this spotlight is:

  • To provide perspectives, resources and tools for developing a course assessment plan using the values of Reconciliation, EDI and Sustainability.
  • To share examples of assessment approaches that inform an assessment plan.
  • To offer opportunities to reflect on the perspectives, resources, tools, and approaches that could be central to Reconciliation, EDI, and Sustainability.

WHAT IS ASSESSMENT?

Classroom assessment is a process that teachers and students use in collecting, evaluating, and using evidence of student learning for a variety of purposes, including diagnosing student strengths and weaknesses, monitoring student progress toward meeting desired levels of proficiency, assigning grades, providing feedback to students and parents, and enhancing student learning and motivation. Classroom assessment includes both qualitative understandings and expressions of student thinking, and quantitative measures of student learning as long as these are collected, interpreted, and used in the context of individual classroom learning communities. Classroom assessment instruments may be designed by the teacher or may be externally designed and selected by the teacher for a particular purpose (e.g., a unit test in a textbook, or a set of embedded questions in a computer-based learning program). However, assessments must be locally controlled by the teacher who sets the purpose, and not an external agent, as is the case for interim/benchmark assessments (Brookhart & McMillan, 2020, p 4-5).

REFLECTION:

  • What knowledge or skills do you hope students can demonstrate in your course and why?

Why "decolonize" assessments?

In colonial epistemologies, this may feel like a straightforward answer: you believe "they need to know X" because you know X and believe it is valuable, traditional for the field, or can be considered an employability skill. A decolonized approach to assignment design does not exclusively have predesignated knowledges or methods for students to mimic, but makes space for students to demonstrate and bring to bear indigenous, multicultural, or international knowledges to the course.

This approach reflects an awareness that "Western knowledge has been engaged in epistemicide, or the killing of other knowledge systems" (Hall and Tandon, 2017, p. 6).

By designing assignments for multiple knowledges or methods, you can engage in, what Shiv Visvanathan refers to as, "cognitive justice" and encourage development of a collective.

In the forthcoming section, we review assessment form and function, which become the grounding for five calls to action related to assessment.

ASSESSMENT PURPOSE

Traditionally, assessment plans are considered in light of form and function. Thinking about assessment function can still support our efforts towards Reconciliation, EDI, and Sustainability.

Cronbach (2000) synthesizes three main functions that assessments can have:

  1. Course improvement: deciding what instructional materials and methods are satisfactory and where change is needed.
  2. Decisions about individuals: identifying the needs of the pupil for the sake of planning instruction, judging pupil merit for purposes of selection and grouping, acquainting a pupil with their own progress and deficiencies.
  3. Administrative regulation: judging how good the school system is, how good individual teachers are, etc. (p. 236)

Reflection:

  • What do you see as the assessment purpose in your course?
  • Are items 1 and 3 within the scope of your classroom assessment purpose?
  • Does your assessment purpose align with your values, beliefs, educational philosophy, and assessment approach?

ASSESSMENT FORM

Consideration to assessment form is also a traditional approach to creating an assessment plan.

Assessment form is an opportunity for Reconciliation, EDI, and Sustainability:

For example: Diversifying assessment forms is generally understood as a best practice (Tkatchov et al., 2020). Assessment diversification is important because different forms can counter benefits and weaknesses and also complement individual students’ strengths and weaknesses (Tkatchov et al., 2020). Moreover, assessments can be chosen so that they showcase students’ theoretical skills and applied skills, as well as demonstrate competencies (Tkatchov et al., 2020).

REFLECTION:

  • Do you see assessment diversification in your assessment plan? Why or why not?

In the forthcoming sections, five 'Action Steps' are shared to inspire your assessment plans.

ACTION STEP #1: Make deeper connections between our epistemologies, our pedagogies, and our assessments.

Teacher beliefs play an important role in the classroom as they create the measurement culture of the classroom (Brookhart, 2003). Without a doubt, our individual values and beliefs inform our approach to teaching (Baird et al, 2017; Brookhart, 2003; Gerritsen-van Leeuwenkamp et al, 2017).

Reflect:

  • How do you believe humans learn?
  • What values and beliefs do you hold about teaching, learning, our students, and our learning context that support your efforts towards Reconciliation, EDI and Sustainability?
  • How are your values reflected in your teaching pedagogy or teaching philosophy?
  • In turn, how are these values reflected in your assessment plan?

ACTION STEP # 2: Develop assessment plans in community with our students.

Perhaps the simplest form of radical inclusivity we can offer is the co-creation of assessments in community with students (Ecclestone & Swann, 1999).

Reflect:

  • How would you characterize your assessment context, including your student demographics?
  • How can students inform your assessment plans?

ACTION STEP #3: Consider Cultural Validity

In recent years, there have been growing conversations about the centrality of cultural validity to assessment (Preston & Claypool, 2021).

There is a possible gap in reflecting student self-efficacy, cultural validity/social identity, and socio-emotional aspects to learning in classroom assessment.

Some of Ontario's postsecondary standards identify cultural validity as a necessity, in addition to characteristics that hold EDI in high esteem.

CQAAP Standard 4 requires all Ontario colleges to provide evidence of assessment quality, learning outcome to assessment mapping, authentic assessments, collaboration with faculty on authentic, consistent, valid and reliable assessment of students, approaches to assessment that consider Indigenous ways of knowing, “fair and equitable evaluation of student achievement through valid assessment methods” (Ontario College Quality Assurance Service, 2020, p. 8), formative feedback, and clear assessment guidelines for students (Ontario College Quality Assurance Service, 2020).

Reconciliation Canada is one of many supports to faculty who are trying to weave in Indigenous ways of knowing into their classroom:

How to imagine our position: A Middle Ground of Knowing

Encouraging non-Western knowledges in your classroom or assessments does not then require you to be an expert in all things. Canadian scholar and settler Celia Haig-Brown thinks about creating a "third space" for learning when different knowledge systems are welcomed into post-secondary institutions.

She was inspired by learning about the two-row wampum belt, Guswentha, from Anishnaabe professor John Burrows. Guswentha includes two bands of purple beads "representing two nations involved in the treaty, like two vessels traveling on a flowing river, will continue to exist in lasting peace and friendship as they maintain their separateness and in­tegrity following parallel trajectories" (Haig-Brown, 2008, p. 260).

She proposes educators think of themselves in a space between the two parallel bands of knowledges, "the one that engages competing knowledges ... [so that] there is a chance for people in them to see what happens across the differences, across the space between" (ibid).

REFLECT:

  • How can your assessments begin to incorporate cultural validity?

ACTION STEP #4: Actively consider ability

Overall, we have found a gap in the research regarding testing accommodations and assessment (Potter et al., 2016).

In Academic Ableism: A Conversation with Jay Dolmage, we find a starting place to reflect and act. Key ideas include understanding that "ableism is never alone," but intersects with other forms of discrimination (21:14); the better way to confront institutional ableism in PSEs is through consultation with students (and faculty and staff) with disabilities(27:12); and the problem with retrofit solutions are manifold (35:19).

With regards to assessment design and ability, there is no evidence that responses to time-limited tests and exams demonstrate or correlate understanding or cognitive ability (Kyllonen, P. C., & Zu, J., 2016). Likewise, there is no evidence that test grade correlates with study time (Hammonds, F., & Mariano, G., 2015). There is, however, evidence that timed tests and exams deepen inequity for women in STEM (Salehi, S., et al., 2019; Major, J. C., et al., 2020), and online surveillance software for timed tests and exams have discriminatory effects on students' mental health, non-white students, and students with disabilities (Barrett, L., 2021). These evidences suggest that avoiding timed tests and exams, where possible, can be a move toward equity.

Reflect:

  • How does our society's limited understanding about ability impact our assessments?
  • How might this impact student success in your classroom?

ACTION STEP #5: Consider assessment 'approach'

Historically, western approaches to assessment focus on cognitive knowing. In this section, we contrast cognitive, socio-cultural, and Indigenous approaches to assessment.

A cognitive theoretical approach has been and continues to be common since the 1960s (Baird et al., 2017). In this paradigm, content, memorization, and product-oriented learning goals are the focus.

Currently, many classroom assessment researchers take a sociocultural or socio-cognitive approach (Bonner, 2013) to classroom assessment, where learning is constructed by the learner in interaction with the environment and with other agents, such as teachers and peers (Baird et al., 2017; Brookhart & McMillan, 2020). In sociocultural theory, “learning is perceived through changing relationships among the learner, other human participants, and the tools (materials and symbolic) available in a given context” (Moss, 2003, p. 14) … “thus learning involves not only acquiring new knowledge and skills but taking on a new identity and social position within a particular discourse … or community of practice” (Moss, 2003, p. 14). The socio-constructivist approach acknowledges the construction of knowledge through agent, student, and context (Bonner, 2013).

More and more, assessment in the colonized, Western world, is returning to Indigenous ways of knowing. For example, the move to a socio-cognitive approach to assessment reflects the four areas of the medicine wheel (Native Voices, n.d.): cognitive, emotional, physical, and spiritual. Thus, positioning assessment through a sociocultural learning theory lens, may support the connection to cultural assessment validity (Preston & Claypool, 2021) and may also provide some consideration to reconciliation and equity.

Identifying the significance of noncognitive skills in any review of student achievement, Bonner (2013) notes that, “aggregated over multiple courses, the combination of academic achievement with so-called extraneous variables such as effort turns out to relate well to success at the postsecondary level” (p. 96).

Place-based Assessments

Decolonized education is rooted in connections to place. The Centre for Youth & Society clarifies that place-based assessments "restore cultural knowledge" and "re-establish links to the community ... [which are] essential for contextualizing knowledge, deepening understanding, encouraging community involvement, and reconnecting students with a vital support system."

REFLECT:

  • When you look at your course syllabus, can you identify the assessment approach? Memorization? Cognitive skills? Socio-cultural skills? Socio-cognitive skills? Indigenous? Cultural competency?

IN CONCLUSION

In this spotlight, we have reviewed assessment form, function, approach, as well as assessment action steps that inform our efforts towards Reconciliation, EDI, and Sustainability in curriculum.

The material provided here is meant to be inspiring, not overwhelming. However, the ideas are neither intended as an exhaustive list or "the right" way to conceptualize assessments.

Some Helpful Resources

Please click on the button below to access some of the resources that have helped us build this section. This Leganto page of resources is ongoing!

Citations

Baird, J.A., Andrich, D., Hopfenbeck, T. N., & Stobart, G. (2017). Assessment and learning: Fields apart? Assessment in Education: Principles, Policy & Practice, 24(3), 317–350. https://doi.org/10.1080/0969594X.2017.1319337

Bonner, S. (2013). Validity in classroom assessment: purposes, properties and principles. In J. McMillan, J. (Ed.), SAGE handbook of research on classroom assessment. (pp. 87-106). SAGE.

Brookhart, S. M. (2003). Developing Measurement Theory for Classroom Assessment Purposes and Uses. Educational Measurement: Issues and Practice, 22(4), 5–12. https://doi.org/10.1111/j.1745-3992.2003.tb00139.x

Brookhart, S., & McMillan, J. (2020) Eds. Classroom assessment and educational measurement. Routledge.

Cronbach L.J. (2000) Course Improvement through Evaluation. In: Stufflebeam D.L., Madaus G.F., Kellaghan T. (eds) Evaluation Models. Evaluation in Education and Human Services, 49. Springer, Dordrecht.

Ecclestone, K. & Swann, J. (1999). Litigation and learning: tensions in improving university lecturers’ assessment practices. Assessment in Education, 6(3), 377-389.

Gerritsen-van Leeuwenkamp, K. J., Joosten-ten Brinke, D., & Kester, L. (2017). Assessment quality in tertiary education: An integrative literature review. Studies in Educational Evaluation, 55, 94–116.

Hall, B.L. and Tandon, R. (2017) ‘Decolonization of knowledge, epistemicide, participatory research and higher education’. Research for All, 1 (1), 6–19. DOI 10.18546/RFA.01.1.02

https://www.uvic.ca/research/centres/youthsociety/assets/docs/briefs/decolonizing-education-research-brief.pdf

Haig-Brown. (2008). Working a Third Space: Indigenous Knowledge in the Post/Colonial University. Canadian Journal of Native Education, 31(1), 253–. Link: https://ojs.library.ubc.ca/index.php/CJNE/article/view/196454/191631

Moss, P. A. (2003). Reconceptualizing validity for classroom assessment. Educational Measurement: Issues and Practice, 22(4), 13–25. https://doi.org/10.1111/j.1745- 3992.2003.tb00140.x

Native Voices (n.d.). Medicine Ways: Traditional Healers and Healing; The Medicine Wheel and the Four Directions. https://www.nlm.nih.gov/nativevoices/exhibition/healing- ways/medicine-ways/medicine-wheel.html

Potter, K., Lewandowski, L., & Spenceley, L. (2016). The influence of a response format test accommodation for college students with and without disabilities. Assessment & Evaluation in Higher Education, 41(7), 996-1007.

Preston, J. P., & Claypool, T. R. (2021). Analyzing assessment practices for Indigenous students. Frontiers in Education, 6, 1-11.

Tkatchov, M., Hugus, E., & Barnes, R. (2020). Reconciling assessment quality standards and "double assessment" in competency-based higher education. Journal of Competency- Based Education, 5(3) doi:https://doi.org/10.1002/cbe2.1215