National Institute for Learning Outcomes Assessment | 1
knowledge accountability connection self-reflection educate action understand communicate listen learn access quality innovation success
ingenuity intellect curiosity challenge create achievement connection self-reflection educate action understand communicate listen learn access
quality innovation success ingenuity intellect curiosity challenge knowledge accountability connection understand communicate listen learn
access quality innovation success ingenuity self-reflection educate action understand intellect knowledge accountability connection self-reflection
educate action understand communicate curiosity challenge create achievement connection self-reflection curiosity challenge create achievement
connection self-reflection knowledge accountability connection self-reflection educate action understand communicate listen learn access quality
innovation success ingenuity intellect curiosity challenge educate innovation success ingenuity intellect curiosity challenge create achievement
knowledge accountability connection self-reflection educate action understand communicate curiosity challenge create achievement connection
self-reflection understand communicate listen learn access quality action educate action understand communicate listen learn action understand
communicate listen learn access quality innovation success ingenuity intellect curiosity challenge knowledge accountability connection access
quality self-reflection curiosity challenge create achievement learn access quality innovation success ingenuity self-reflection educate action
understand intellect knowledge accountability connection self-reflection educate action understand knowledge accountability connection self-
reflection educate action understand communicate listen learn access quality innovation success ingenuity intellect curiosity challenge connection
knowledge accountability connection self-reflection educate action understand communicate listen learn access quality innovation success ingenuity
challenge create achievement connection self-reflection educate action understand connection self-reflection understand communicate listen learn
access quality action create achievement connection self-reflection educate action understand communicate listen learn access quality innovation
success educate action communicate listen learn access quality action educate action understand communicate educate innovation success self-
reflection knowledge accountability communicate listen learn achievement connection self-reflection educate action understand communicate listen
learn access quality innovation success ingenuity intellect access quality innovation success self-reflection curiosity challenge create achievement
connection self-reflection understand educate action understand communicate listen learn action understand communicate listen learn access
quality innovation success ingenuity curiosity challenge create achievement connection self-reflection understand communicate listen learn
Equity and Assessment:
Moving Towards Culturally Responsive Assessment
Erick Montenegro and Natasha A. Jankowski
National Institute for Learning Outcomes Assessment
January 2017
Occasional Paper #29
www.learningoutcomesassessment.org
National Institute for Learning Outcomes Assessment | 2
NILOA Mission
e National Institute for Learning
Outcomes Assessment’s (NILOA) primary
objective is to discover and disseminate
the ways that academic programs and
institutions can productively use assessment
data internally to inform and strengthen
undergraduate education, and externally to
communicate with policy makers, families,
and other stakeholders.
Abstract....3
Equity and Assessment:
Moving Towards Culturally Responsive Assessment...4
Limiting Learning Demonstration...6
Culturally Responsive Assessment...8
Student Learning Outcomes Statements...11
Assessment Approaches...12
Use of Assessment Results...13
Final oughts...14
References...17
NILOA National Advisory Panel...21
About NILOA...22
Table of Contents
Please cite as:
Montenegro, E., & Jankowski, N. A. (2017, January). Equity and assessment: Moving towards culturally responsive assessment
(Occasional Paper No. 29). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning
Outcomes Assessment (NILOA).
National Institute for Learning Outcomes Assessment | 3
Abstract
As colleges educate a more diverse and global student population, there is increased need to ensure every student
succeeds regardless of their dierences. is paper explores the relationship between equity and assessment,
addressing the question: how consequential can assessment be to learning when assessment approaches may not be
inclusive of diverse learners? e paper argues that for assessment to meet the goal of improving student learning
and authentically document what students know and can do, a culturally responsive approach to assessment is
needed. In describing what culturally responsive assessment entails, this paper oers a rationale as to why change
is necessary, proposes a way to conceptualize the place of students and culture in assessment, and introduces three
ways to help make assessment culturally responsive.
National Institute for Learning Outcomes Assessment | 4
Equity and Assessment:
Moving Towards Culturally Responsive Assessment
Erick Montenegro and Natasha A. Jankowski
Introduction
College enrollment has become increasingly diverse in terms of students’
race, ethnicity, gender identity, socioeconomic status, sexual orientation,
age, ability, etc. is trend is only expected to continue as the United
States moves into a majority-minority nation by the year 2050, and college
enrollments continue to increase. Conducting assessment in a manner that
takes into consideration the various needs of dierent student populations is
a responsibility of higher education. For one, underrepresented students are
more likely to be low-income and rst-generation (Del Rios & Leegwater,
2008; Li & Carroll, 2007; Benitez, 1998), and there are vast dierences
between the needs of lesbian, gay, bisexual, transgender, queer, intersex, asexual
(LGBTQIA) (Check & Ballard, 2014; Mallory, 2009), undocumented (Kim
& Diaz, 2013; Perez, 2010), nontraditional (Macqueen, 2012), and special-
needs students attending higher education institutions (Froese-Germain &
McGahey, 2012). Further, students are increasingly mobile, with transfer
students coming from mostly traditionally underrepresented backgrounds,
attending multiple institutions (Backes & Velez, 2015; Shapiro et al, 2012)
and facing their own challenges in higher education (Tobolowski & Cox,
2012).
Various areas of higher education are aware of the need to accommodate
dierent student populations because “individual dierences are clearly
important to student success” (Strange & Banning, 2015, p. 61). For
example, approaches to teaching, student development, student services,
and campus programs have been analyzed and altered to improve outcomes
for specic student groups (Ladson-Billings, 1995a; Ladson-Billings, 1995b;
Schuh, Jones, Harper, & Associates, 2011; Kezar, 2011; Lara & Wood,
2015; Bowen, Chingos, & McPherson, 2009, Aronson & Laughter, 2016).
Within the eld of campus advising, the issue of microaggressions through
lack of cultural awareness has been raised (Chu, 2016) and the work of the
Association of American Colleges and Universities (AAC&U) in their Equity
Imperative outlines the need to understand who students are, disaggregate
data to look for inequities, and explore policy changes for unintended
impacts on student groups. Conversations in K-12 have addressed the notion
of equity from the standpoint of equity traps (McKenzie & Scheurich, 2004)
within schools and the need to prepare school leaders to not only expose but
address them through courageous conversations about inequities (Singleton,
2012). In a literature review of culturally responsive school leadership,
Khalifa, Gooden, and Davis (2016) argue that culturally responsive leaders
need to continuously support minoritized students through examination of
assumptions about race and culture. Further, they argue that as demographics
continue to shift, so should practice that responds to student needs, nding
that it is “deleterious for students to have their cultural identities rejected in
school and unacknowledged as integral to student learning” (p. 1285).
Conducting assessment in
a manner that takes into
consideration the various
needs of dierent student
populations is a responsibility
of higher education.
National Institute for Learning Outcomes Assessment | 5
Culturally relevant and culturally responsive pedagogies sought to outline
ways in which teachers could address unique learning needs of diverse student
populations. Gloria Ladson-Billings (1995a, 1995b) recommends using
culturally relevant pedagogy as a way to allow students in populations outside
of the majority to maintain their cultural integrity all-the-while succeeding
academically. Culturally relevant pedagogy aims to “produce students who
can achieve academically…demonstrate cultural competence, and develop
students who can both understand and critique the existing social order”
(Ladson-Billings, 1995a, p. 474). In culturally responsive pedagogy, teachers
use aspects of students’ cultures in an asset-based approach as opposed to
decit-based to make the course material relevant to them, and increase
their skill acquisition, engagement, and learning outcomes (Ladson-Billings,
1995a). Yet, Geneva Gay (2010) has argued that solely modifying teaching
practices cannot solve the challenges faced by ‘minoritized’ students.
In terms of assessing student learning, the eld has been largely quiet when
it comes to issues of equity. Assessment, if not done with equity in mind,
privileges and validates certain types of learning and evidence of learning
over others, can hinder the validation of multiple means of demonstration,
and can reinforce within students the false notion that they do not belong in
higher education. For equity gaps to be addressed, an entire institution needs
to explore the combination of solutions and supports needed for students
to be successful (Jones, 2015; Methvin & Markham, 2015), of which
assessment is one. However, little of the conversation thus far has focused on
the connection points between demonstration of student learning and issues
of equity. Instead, assessment has remained largely unchanged in regards to
inclusivity, and little urgency has been given to ensuring that students are
provided with just and equitable means to demonstrate their learning. ere
is a dierence between assessing all students in the same way in relation to a
specic outcome of interest and making sure assessments are appropriate and
inclusive of all students. Being attentive to how students may understand
questions, tasks, and assignments dierently, as well as feedback regarding
their learning, is not only benecial to students but to internal improvement
eorts as well. Intentionally choosing appropriate assessment tools or
approaches that oer the greatest chance for various types of students to
demonstrate their learning so that assessment results may benet students
from all backgrounds advances our collective interest in student success.
Without examining issues of equity the students who may stand the most to
gain from assessment eorts may have the least benet since their learning
is not accurately assessed and feedback may not be relevant to impact
learning. If assessments are to be holistic in their goal of improving student
learning, then incorporating a culturally responsive approach to assessment
is a priority. As C. Carney Strange and James Banning (2015) state, student
cultures “can play an important role, for good or otherwise, in introducing
students to and maintaining their engagement in the learning process” (p.
53). It also creates opportunities for students to experience deep learning
(Entwistle, 2001) by honoring students’ prior knowledge and experience.
However, before we present the concept of culturally responsive assessment,
it is useful to unpack an assumption that hinders consideration of diverse
learner needs within assessment—that while learners may take multiple
paths to and through learning, they must demonstrate their knowledge and
skills in the same way.
Assessment, if not done with
equity in mind, privileges
and validates certain types
of learning and evidence
of learning over others,
can hinder the validation
of multiple means of
demonstration, and can
reinforce within students the
false notion that they do not
belong in higher education.
National Institute for Learning Outcomes Assessment | 6
Limiting Learning Demonstration
ere is an assumption at play within the eld of assessment that while
there are multiple ways for students to learn, students need to demonstrate
learning in specic ways for it to count. For instance, in a specic course
dierent approaches may be used to engage students in the material, but
demonstration of a students’ knowledge, skills, and abilities are done
uniformly in the same assignment or approach—so while there may be
multiple approaches and methods used across a program or institution for
assessing student learning, at each instance of demonstration a single approach
is employed. Regardless of the literature on the multiple ways students acquire
knowledge, assessment asks students, at each instance of demonstration,
to show they have the knowledge and skills of interest through the same
means. William Sedlacek (1994) discusses the need for the development of
multicultural assessment standards within the Association for Assessment in
Counseling (AAC). While the focus is upon assessment within the context
of counseling support and services, the interest of addressing the needs of
those with “cultural experiences dierent from…White middle-class men
of European descent, those with less power to control their lives, and those
who experience discrimination in the United States” (p. 550), remains the
same for assessment of or for learning in higher education. Sedlacek (1994)
identies ve fallacies related to culture and assessment, stressing that most
measures were not designed with nontraditional or underserved populations
in mind, that few assessment specialists are trained in developing measures
for use with nontraditional populations, and that larger issues exist that
need to be explored and addressed when promoting diversity, equity, and
inclusivity through assessment. Of note is the fallacy referred to as the three
musketeers, which is the idea that in order to make a measure equally valid
for everyone, everyone completes the same measure—all for one and one
for all—as a means to ensure fairness instead of using dierent measures
for dierent groups. Yet, Sedlacek (1994) argues, “if dierent groups have
dierent experiences and dierent ways of presenting their attributes and
abilities…it is unlikely that we could develop a single measure or test item
that would be equally valid for all” (p. 550); further arguing that there is no
need to employ the same measure when what is desired is equity of results,
not process.
ere are institutions providing students with support and opportunity to
choose from a variety of approaches or even design how they will be assessed
in cooperation with faculty members, presenting students with agency and
choice in the assessment process (Singer-Freeman & Bastone, 2016) and
most institutions use a combination of assessment methods to gauge learning
(Kuh et al., 2014). In a study at the University of East London, students were
allowed to choose how they were assessed, signicantly improving attainment
among learners without an academic background (Grove, 2016). Instead of
completing exams based on coursework, students were given the option to do
a presentation, poster, or debate. Using the alternative assessment techniques
“helped mitigate the fact that many rst-year students had not been in
formal education for some time” allowing them space to demonstrate their
learning, not their exam-taking abilities (Grove, 2016). Further, a similar
approach was used at the University of Dublin where students were able to
make a poster instead of taking an exam. In both instances, students had to
ere is an assumption at play
within the eld of assessment
that while there are multiple
ways for students to learn,
students need to demonstrate
learning in specic ways for it
to count.
National Institute for Learning Outcomes Assessment | 7
e manner in which students
demonstrate learning is
irrelevent when student
demonstration is held to the
same learning outcomes and
evaluative criteria.
demonstrate their learning on the same learning outcomes and evaluative
criteria, but the manner in which they did so was irrelevant. Rubrics were
used such that the evaluation of the work was the same, thus quality ensured,
but the demonstration could be dierent. In Canada, a study was undertaken
within a large, third-year psychology class regarding dierentiated evaluation
to examine student engagement, quality of learning experience, and address
challenges associated with increased student diversity (Gosselin & Gagné,
2014). Dierentiated evaluation allowed students to choose how they would
be evaluated though all students were still required to take mid-term and
nal exams. Students had the option of adding a term project through
preparing a mini-class or participating in a community service learning
program. e study found positive impact on student achievement and on
the learning experience, with students performing below class average seeing
grade improvement when completing a term project. Further, students who
completed the project performed better on the nal exam in comparison to
those that did not, and the option helped to alleviate stress of sitting for an
exam. Qualitative responses from students that selected the project option
indicated that they saw the alternative as an opportunity to demonstrate
their learning through a format over which they felt more control. Gosselin
and Gagné (2014) argued that there are “methods of assessment that can
foster inclusiveness and academic success whilst upholding high standards
for the quality of student learning” yet interestingly “most innovations in
this context have focused on teaching rather than on student learning” (p. 6).
e dierentiated evaluation approach complemented the existing structure
and allowed the relationship between faculty and student to shift to one
of collaboration instead of power, regarding decisions about how students
demonstrate their learning.
e need to fold in culture and student experience into assessment is stressed
in the everyday expertise framework—a perspective of learning that takes
into account how students demonstrate knowledge and skills in their daily
life with the other people around them (Toomey Zimmerman & Bell,
2012). e framework allows for learning to have multiple dimensions
including individual, social, and cultural, requiring a broad consideration
of how people learn within and across learning environments, noting that
learners do not act with equal competency in all settings, even if the content
is the same. Toomey Zimmerman and Bell (2012) argue that the dierence
in performance indicates that learners competent in informal and everyday
settings may falter in more formalized learning settings, requiring alternative
means to demonstrate their knowledge outside of the traditional classroom.
Beyond the many benets from engaging students in co-curricular
experiences (Meents‐DeCaigny & Sanders, 2015; Schuh, Jones, Harper, &
Associates, 2011; Schuh, 2009; Pascarella & Terenzini, 2005; Del Rios &
Leegwater, 2008), co-curricular learning provides a means to address the
issues raised by the everyday expertise framework by widening our lens of
where learning happens to include experiences beyond the classroom. In
addition to conceptions shifting where learning happens, there has been a
rise in competency-based education (CBE) which releases the time structure
in which learning occurs in terms of credit hours. CBE programs stress that
authentic artifacts, or demonstrations of student learning, need to come
from a variety of sources to engage learners with curricula and assessment
National Institute for Learning Outcomes Assessment | 8
that reect not just multiple ways to learn but multiple ways to demonstrate
mastery of a competency (Jobs for the Future, 2016). However, there are
calls for signicant research to determine how best to design assessments
for underprepared learners that also elevate and validate their skills through
alternative measures (Person, Goble, & Bruch, 2014). While learning may
happen anywhere and learners may need dierent lengths of time in their
learning process, there is still the issue of who gets to validate that learning
has occurred, or that demonstrations of learning are of the ‘right type.’
While there is movement to more inclusive means of assessment and active
engagement with students as partners in learning, it is clear that the challenges
of various minority groups on campus dier from those of the majority
(Ellis & Chen, 2013; Rankin, Weber, Blumenfeld, & Frazer, 2010; Miller,
Bradbury, & Pedley, 1998), yet higher education still privileges certain types
of learners, certain ways of demonstrating knowledge, and certain learning
spaces by not consistently oering transparency, dierentiated assessments,
or empowering students in their own learning. Students need to develop
and apply their knowledge and skills across multiple contexts in dierent
courses through a range of methods (Newman, Carpenter, Grawe, & Jaret-
McKinstry, 2014, p. 14) with integrative liberal learning requiring students
to engage in “ongoing demonstration to themselves and to others, of the
gains made through curricula, programs, and the educational experience
as a whole” (Ferren & Paris, 2015, p. 5). Yet, the signals education sends
to students about what is validated or counts as demonstration of learning
can be detrimental and reinforce for marginalized students that they do not
belong because their learning ‘doesn’t count.’ What is needed is collaboration,
where students, faculty and sta “draw together their life experiences and
aspirations with classroom, co-curricular, and community opportunities”
(Ferren & Paris, 2015, p. 20).
Culturally Responsive Assessment
Dening “culture” and explaining what is meant by culturally responsive
assessment is complicated. e issue is that culture, whether speaking
about it in terms of an organization, a campus, or an individual, has been
historically dicult to dene. Higher education has a tendency to group
student dierences and issues around race under the term ‘diversity,’ which
is often discussed in relation to benets to White students as opposed to
African Americans, Latinx, Asian Americans, and Native Americans who
continue to be underrepresented in higher education (Dowd & Bensimon,
2015). While diversity eorts on college campuses have brought attention
to the vast dierences among students—including gender, religion, sexual
orientation, etc.—the term diversity fails to address issues surrounding race/
ethnicity and does not account for the dierent histories, needs, interests, and
issues aecting distinct groups of students on campus. With this in mind,
one can see why it would be benecial to use culture instead of diversity as
the imperative to refocus assessment into a more inclusive endeavor.
is paper draws from and expands on past denitions of culture to develop
an understanding that culture should be thought of as: (1) the explicit
elements that makes people identiable to a specic group(s) including
behaviors, practices, customs, roles, attitudes, appearance, expressions of
While learning may happen
anywhere and learners
may need dierent lenghts
of time in their learning
process, there is still the issue
of who gets to validate that
learning has occurred, or that
demonstrations of learning are
of the ‘right type.’
National Institute for Learning Outcomes Assessment | 9
identity, language, housing region, heritage, race/ethnicity, rituals, religion;
(2) the implicit elements that combine a group of people which include
their beliefs, values, ethics, gender identity, sexual orientation, common
experiences (e.g. military veterans and foster children), social identity;
and (3) cognitive elements or the ways that the lived experiences of a
group of people aect their acquisition of knowledge, behavior, cognition,
communication, expression of knowledge, perceptions of self and others,
work ethic, collaboration, and so on. e culturally relevant component
involves assuring that the assessment process—beginning with student
learning outcome statements and ending with improvements in student
learning—is mindful of student dierences and employs assessment methods
appropriate for dierent student groups. Underlying the culturally relevant
component is the focus on students—the importance of keeping students at
the center, which requires their involvement at every step in the assessment
process and builds upon their lived experience.
In addition, it is important to understand the concept of intersectionality and
its eect on culture. Traditionally, intersectionality is thought of in racial/
ethnic identity intersecting with class, gender, and sexual orientation to shape
how people of color experience oppression (Solorzano & Yosso, 2002; Huber,
2010; Cho, 1997). However, for purposes of this paper, intersectionality is
the way that aspects of a person’s identity cannot be fully separated from one
another, play a central role in peoples’ experiences and making meaning of
those experiences. is is related to Susan Jones and Marylu McEwen’s (2000)
multiple dimensions of identity which treats a student’s identity as dynamic
and changing depending on the relative contextual salience of other elements
of one’s identity (e.g. race/ethnicity, sexual orientation, gender, religion);
with no single aspect of one’s identity understood singularly, but only in
relation to the other dimensions. For example, a White male that identies
as a member of the lesbian, gay, bisexual, transgender, queer, intersex, and
asexual (LGBTQIA) community and practices Judaism is shaped by the
intersectionality of these four elements. A Latina that is a single-mother from
a low-socioeconomic background is shaped by the intersectionality of these
elements. An undocumented English as a second language, rst-generation
student will experience college, acquire knowledge, and demonstrate
knowledge dierently than an international English as a second language
rst-generation student. e culture—the explicit, implicit, and cognitive
elements—of the people in these examples shape their college experiences,
and while one aspect of their culture may manifest itself more than another
in specic contexts, they all aect the outcomes being assessed.
inking of culture in the way that it is dened here can serve as a reference
point for what to consider when engaging in assessment and developing/
choosing/implementing assessment tools and methods. Culture is by no
means simple, and it is by no means easily denable. It is dependent on
the context in which culture is discussed. Culture permeates the individual,
group, entire institutions, countries, and continents; and at the same time
the individuals that comprise cultural groups are multicultural through
intersectionality. Perhaps Lang (1997) stated it best when he said “attempts
at dening culture in a denite way are futile” (p. 389). However, developing
an inclusive understanding of culture, and making it explicit that culture is
much more than race/ethnicity and aects students’ lives on multiple levels,
e culturally relevant
component involves assuring
that the assessment process
—
beginning with student
learning outcome statements
and ending with improvements
in student learning
—is
mindful of student dierences
and employs assessment
methods appropriate for
dierent student groups.
National Institute for Learning Outcomes Assessment | 10
including learning and how they demonstrate learning, will help ensure
culturally responsive assessment and increase the eectiveness and impact of
learning outcomes assessment eorts.
In addition to the term of culture, it is important to note the use of responsive
to indicate “an action-based, urgent need to create contexts and curriculum
that responds to the social, political, cultural, and educational needs of
students; it is armative and seeks to identify and institutionalize practices
that arm indigenous and authentic cultural practices of students” (Khalifa,
Gooden, & Davis, 2016, p. 1278). Students who experience validation from
faculty and integrate academically and socially are more likely to persist and
be successful (Karp, Hughes, & O’Gara, 2011). Assessment approaches
and processes can help reinforce a sense of belonging or add to students’
belief that they do not belong because their learning or experiences are not
deemed as valid or important. Susan Headden and Sarah McKay (2015)
stress this point, arguing that student motivation is connected to student’s
beliefs that they are able to do the work and have a sense of control over
the work. For rst-generation college goers and African American students
“stereotypes about academic performance can turn into self-fullling
prophecies…even feedback on papers can reinforce or foster learning…that
students are cared about and respected as learners” (Headden & McKay,
2015, p. 15). An environment focused on students’ unique learning interests
and needs enables students to incorporate prior and everyday experiences in
meaning construction (Land, Hannan, & Oliver, 2012). Involvement with
culture is also important as Cathleen Spinelli (2008) argues that there are a
disproportionate number of students with cultural and linguistic dierences
that are misidentied as learning disabled. As a result, students are classied
incorrectly, not academically challenged, and do not receive appropriate
services. Spinelli (2008) further argues that when looking specically at the
case of English language learners, informal assessment provided a solution
to the need of assessment of learning, but in a manner adaptable to language
and cultural diversity, individual learning styles, and personal challenge
while also informing instruction.
Culturally responsive assessment is thus thought of as assessment that is
mindful of the student populations the institution serves, using language
that is appropriate for all students when developing learning outcomes,
acknowledging students’ dierences in the planning phases of an assessment
eort, developing and/or using assessment tools that are appropriate for
dierent students, and being intentional in using assessment results to improve
learning for all students. Culturally responsive assessment involves being
student-focused, which does not simply mean being mindful of students.
Instead, being student-focused calls for student involvement throughout the
entire assessment process including the development of learning outcome
statements, assessment tool selection/development process, data collection
and interpretation, and use of results. An essential aspect of maintaining
focus on students is truly understanding the student population at the
institution and/or level at which the assessment is being conducted. Once we
understand who our students are we can begin to tailor assessment processes
and materials to have the greatest impact for their learning. Institutions with
high enrollment of traditionally underrepresented students have already
begun tailoring their learning outcomes assessment approaches based on
Assessment approaches and
processes can help reinforce
a sense of belonging or add
to students’ belief that they
do not belong because their
learning or experiences are not
deemed as valid or important.
National Institute for Learning Outcomes Assessment | 11
the student populations that they serve (Montenegro & Jankowski, 2015;
Nunley, Bers, & Manning, 2011; Baker, Jankowski, Provezis, & Kinzie,
2012). Minority-Serving Institutions (MSIs) have been found to increase
self-esteem, engagement, critical thinking skills, leadership skills and
opportunities, and help the identity formation processes for traditionally
underrepresented students; which helps increase students’ persistence
through college (Conrad & Gasman, 2015; Del Rios & Leegwater, 2008;
Conrad et al, 2013). e work at these institutions can serve as guideposts
for the development of culturally responsive assessment practices.
Student Learning Outcomes Statements
Learning outcomes assessment as a process begins with developing learning
outcome statements that clearly state what students should know and be
able to demonstrate upon completion of a course, academic program,
college, making use of student services, etc. To develop student learning
outcomes statements using a cultural lens necessarily involves students in
the development process. Poorly constructed learning outcomes make it
dicult for students to demonstrate their learning for a myriad of reasons
(e.g. not understanding what is expected of them, not understanding how
the course/program is expected to contribute to their learning). In addition,
it is students that will directly benet from the feedback they receive as a
result of assessment. Clarity of outcomes and curricular structure matters
in general education (Gaston, 2015), assignment design (Winkelmes et al,
2016), co-construction of knowledge for deep learning (Juvova et al, 2015;
von Glasersfeld, 2005), and new course design models like competency-based
education (Jobs for the Future, 2016). Further, in the National Research
Council report, How People Learn, (2000) principles for designing learner-
centered environments emphasized the importance of individual social and
cultural contexts in learning. Such perspectives require dierent approaches
to curricular design, teaching, and assessment, and squarely place learner
preconceptions and experiences as an integral part of the learning process.
Assessment is a eld of alignment, and this also originates from learning
outcomes statements. Hutchings (2016) denes alignment as “the linking of
intended student learning outcomes with the processes and practices needed
to foster those outcomes” (p. 5). Similarly to how academic programs, student
services, and other institutional programs aim to align with and promote
the mission of the college or university, learning outcomes statements of
departments, programs, and courses should align with those of the institution.
Outcome statements need to be culturally responsive because they align with
assignments, evaluative criteria, and institutional and departmental goals. If
outcome statements are not culturally responsive, then there are implications
for various levels of the institution; not just for students. Learning outcome
statements which are written to inform educational policy and practice,
and are clear about expected prociencies make it possible for programs,
departments, institutions, and students to meet their goals (National
Institute for Learning Outcomes Assessment, 2016). e language and
operative verbs in learning outcome statements serve as a guide for students
to understand departmental/program expectations, as well as understand
how their educational experiences prepare them for their careers and lives
after college (National Institute for Learning Outcomes Assessment, 2016).
However, if learning outcomes statements are not written with attention to
To develop student learning
outcomes statements using
a cultural lens necessarily
involves students in the
development process.
National Institute for Learning Outcomes Assessment | 12
cultural relevancy, then it becomes dicult to accurately infer the learning
gains of dierent groups of students.
Cli Adelman (2015) speaks about the importance of being intentional and
mindful of language when writing learning outcomes statements as this can
lead to creating assignments that allow for genuine judgement of student
achievement. However, taking this a step further and being mindful of how
the language of learning outcomes statements might be appropriate for/
inclusive of certain student groups but not others can lead to more holistic
assessments. Flawed assessment designs may unintentionally skew scores
for certain student populations and ensuring this does not happen begins
with the writing of culturally responsive learning outcomes statements that
consider students, their dierent ways of learning, and the diverse ways
they demonstrate learning. One way to make statements more culturally
responsive is to explicitly dene terms and use scenarios or examples that
are relatable to various student groups. A sample tool that incorporates these
elements of being intentional and explicit in writing learning outcomes and
clearly dening learning is the Degree Qualications Prole (DQP) and
Tuning process. e DQP was developed by Lumina Foundation (2014),
and by coupling Tuning processes to it serves as a way to clearly outline
what students know and should be able to do after attaining a degree (Ewell,
2013).
Traditionally, learning outcomes statements are written by and for faculty
and administrators. As a result, faculty and administrators dene the
intended learning outcomes and what it looks like to demonstrate those
outcomes. If, instead, we write learning outcome statements for and with
students, then we increase the chances of students understanding what is
expected of them. In addition, instead of students’ knowledge conforming
to how we traditionally measure it, students would now have agency in how
to demonstrate learning. is would result in learning outcomes, as well as
the assessment process, becoming a more inclusive endeavor.
Assessment Approaches
ere is a need for assessments that allow students to demonstrate their
learning in various ways while also being transparent about the learning
that is taking place, help students reect on their learning experiences, and
allow students to actively participate in the learning and assessment process.
Course-level assessments such as culturally responsive rubrics, portfolios, and
capstone projects can lead to more valid, appropriate, holistic, and formative
assessment where results are more indicative of what all students can do
or lead to more targeted improvements in teaching and learning. Rubrics,
which help instructors gauge student learning, skills development, and
acquisition of learning outcomes, provide criteria by which to assess whether
or not the learning outcome was demonstrated. Rubrics, when they undergo
a culturally conscious development process and are shared with students,
can be a way to accurately assess learning for all students while allowing
variation in how the learning is demonstrated. While rubrics are at times
created by individual faculty members to t the context of specic courses or
programs, the Association of American Colleges and Universities (AAC&U)
Valid Assessment of Learning in Undergraduate Education (VALUE)
Rubrics serve as examples of rubrics for institutions to employ. In addition,
ere is a need for assessments
that allow students to
demonstrate their learning
in various ways while also
being transparent about the
learning that is taking place,
help students reect on their
learning experiences, and allow
students to actively participate
in the learning and assessment
process.
National Institute for Learning Outcomes Assessment | 13
capstones—which can be entire courses or student projects—can also be
an avenue through which the learning of students can be better gauged by
allowing students to design their own projects in partnership with faculty.
ey provide “tangible, visible, self-explanatory evidence of exactly what
students have and haven’t learned” (Suskie, 2004, p. 95). As comprehensive,
culminating experiences, capstones allow students to demonstrate a wide-
range of skills and knowledge that oftentimes draws from previous work,
experiences, and learning that occurred throughout their coursework.
Finally, portfolios oer a similar freedom for students to demonstrate
their learning and provide a more holistic representation of what students
know and can do. e use of portfolios provides students the option to
select demonstrations and add commentary and reection, furthering
their agency in the process and selection of assessment evidence. Portfolios
represent student work over time and demonstrate various forms of learning
(Kuh et al, 2015; Banta, Grin, Flateby, & Kahn, 2009) which may not
be easily captured by other forms of assessment. Portfolios are “authentic
assessment that draws on the work students do in regular course activities
and assignments” and “reconnect assessment to the ongoing work of
teaching and learning and to the work of faculty, raising the prospects for
productive use” (Kuh et al, 2015, p. 36). Portfolios provide the opportunity
to get students invested into the course beyond grade attainment, and help
to deepen students’ educational experiences through allowing them to
make connections between conceptual issues, theoretical knowledge, and
real world experiences (Singer-Freeman & Bastone, 2016). Additionally,
portfolios can be made available online. Eportfolios can be easily accessed
by potential employers, as well as other institutions, which provides students
in the job market or looking to transfer a means to easily demonstrate their
knowledge and skills. Kuh et al (2015) mention a few of the advantages
that portfolios have for assessment, including advancing student success,
catalyzing change, and making learning more visible for students. ese
impacts can be furthered by applying a cultural lens when assessing student
portfolios. By being mindful of how culture aects students’ meaning-
making processes, cognition, and demonstrations of learning, we can better
understand and appreciate the learning gains that students make. In fact,
at the program-level, assessment approaches such as rubrics and portfolios
are used more often than surveys and other approaches (Ewell, Paulson, &
Kinzie, 2011).
Use of Assessment Results
Implementing formative assessment methods means very little if assessment
data are not used to inform learning at various levels of the institution or if it
has no meaning to students to improve their own learning. e rst step in
creating change is analyzing the data by student populations. Disaggregating
the data is instrumental in informing changes to higher education. While
the data may tell a positive story about overall learning, disaggregation may
yield the observation that rst-generation students are struggling in a course,
female students are making use of resources aimed at supporting their
education at disproportionate rates, or Latinx students are not reaching the
same institutional learning outcomes as other racial/ethnic groups. In either
hypothetical case, disaggregating the data allows researchers, administrators,
and practitioners to see themes that they otherwise would have missed and
By being mindful of how culture
aects students’ meaning-
making processes, cognition, and
demonstrations of learning, we can
better understand and appreciate
the learning gains that students
make.
National Institute for Learning Outcomes Assessment | 14
could inform changes that would positively impact students’ education.
In addition, disaggregation of assessment data should not only be used to
uncover surface-level ndings such as Latinx students excel at “ABC” while
rst-generation students struggle with “XYZ.” As Aydin Bal & Audrey Trainor
(2016) state, “researchers must also include an examination of processes
(e.g. the racialization of disability… and the institutional acts of exclusion
based on ability dierences) and institutions…that reproduce, regardless of
intentionality, disparities” (p. 330-331). is means that disaggregating data
should explore why the condition exists in the rst place, and then be used
to inform/develop possible solutions.
In using assessment results, it is also useful to be mindful of our own
assumptions. Similar to how a researcher’s bias cannot be fully removed from
his/her/zer’s study and can either harm or enhance his/her/zer’s research, so
can the biases of faculty and sta aect assessment eorts and use of results.
It is unrealistic and counterproductive for assessment professionals to think
they are approaching their work from an impartial stance or to assume that
the students being assessed also operate from an impartial stance. Failing to
recognize how culture and our own experiences aect the assessment process
can limit the impact of assessment. In discussing the need for faculty to be
attentive to the changes in the institution’s student population, Goldrick-
Rab and Cook (2011) warn against comparing all students against the
researcher’s subconscious idea of what students do/should do. Failing to be
aware of our own biases or subconscious ideas and failing to disaggregate
assessment data in a culturally responsive manner may cause the assessment
endeavor to implement outdated norms as a means of comparison, which
can misclassify certain students as underachievers, confusing, or outliers; and
can also lead to the mistake of failing to connect the data to the actual lived
experiences and realities of the students the institution serves (Goldrick-Rab
& Cook, 2011). is can also lead to unintentionally reinforcing negative
assumptions about certain student groups. Treating dierent racial/ethnic
groups under an aggregate umbrella, as has been the recent case with the
term “underrepresented minorities,” minimizes the voice of various groups
and ignores their salient dierences (Dowd & Bensimon, 2015) which
impact their needs, experiences, learning, and demonstration of that
learning. Finally, it would be worthwhile to connect assessment results to
other campus assessment strategies. While certain data collection eorts
on campus may seem unrelated, occurrences on campus seldom happen in
isolation. Connecting dierent assessment eorts and resulting data sets can
better inform issues related to student attrition, success, campus climate,
pedagogy, and others (Hurtado & Halualani, 2014).
Final oughts
Students’ college experiences are inseparable from other daily experiences such
as those encountered at work, microaggressions endured on campus, family
life, and employment. More often than not, students’ college experiences are
aected by students’ own culture and cultural dierences with faculty, sta,
and peers. It has long been known that students of dierent backgrounds
experience college dierently and respond dierently to similar situations,
stimuli, experiences, requests, questions, etc. So, if we also know that students
from dierent cultures who have similar education backgrounds respond
Students’ college experiences
are inseparable from other
daily experiences such as
those encountered at work,
microaggressions endured
on campus, family life, and
employment.
National Institute for Learning Outcomes Assessment | 15
and perform signicantly dierent, why would we design assessments,
execute them, and then make changes based on assessment results without
considering the cultural relevance of the assessment eort and analyze how
the assessment might aect all students/benet certain population(s) and
hinder others? Why would we not include students in the assessment process
to improve our approaches?
e focus of assessment as a means to improve student learning is an agreed
upon purpose of the work. As Kuh, Ikenberry, Jankowski, Cain, Ewell,
Hutchings, and Kinzie (2015) state, “gathering information about collegiate
outcomes has a practical goal: using it to improve both student learning
and institutional performance” (p. 51); and “harnessing evidence of student
learning, making it consequential in the improvement of student success
and strengthened institutional performance is what matters” (p. 4). Yet,
how consequential can assessment truly be when assessment approaches are
minimally inclusive of our current student populations? Using assessment
tools and approaches that work for the majority of students but are less
mindful of students identifying with groups outside of the majority
population places a signicant portion of students at a disadvantage, leads
to a decrease in the quality of education, creates a disconnect between
students and the institution, and contributes to achievement gaps (Slee,
2010; Sullivan, 2010; Qualls, 1998). Assessment that overlooks issues of
diversity and equity contributes to inequalities in outcomes (Bal & Trainor,
2016). e same can be said for assessment approaches that do not take into
account students’ culture.
Students have dierent ways to demonstrate their knowledge and we need
to use assessment metrics that appropriately elicit demonstrations of what
students know. One example of the diverse ways students can demonstrate
learning comes from Nick Sousanis’ (2015) published dissertation exploring
how people construct knowledge. Instead of writing a typical manuscript,
Sousanis demonstrated his knowledge in a graphic novel format. At times,
the illustrations said more than the words on the page, and both pictures
and words united to tell a powerful academic story. is way of presenting
scholarly work, while unconventional in academia, is still a powerful
demonstration of learning. Sousanis’ chosen method of demonstrating his
knowledge on a specic topic is not wrong, it is just dierent. We undo
boundaries through the awareness that “it is our [own] vision, and not what
we are viewing, that is limited” (Sousanis, 2015, p. 42). How assessment is
often operationalized or experienced by students has not moved to a position
where it continuously regards students’ diverse methods of demonstrating
knowledge as appropriate. Instead, dierent can often be marked as wrong.
If assessment is about demonstrating learning, then we need to allow students
the space to show their knowledge. Students are highly varied in customs,
identity, and understanding, and it is all shaped by culture which aects
learning; and thus, should aect how we measure learning. If assessment
is done for improvement and with the goal of using the results to benet
student learning, then having outcome assessments that appropriately tell
the stories of what students know and can do is of imperative importance.
Our assessments approaches—how we assess and the process of assessment
itself—should align with the students we have, empowering them with
narratives to share and document their learning journey.
Assessment that overlooks
issues of diversity and equity
contributes to inequalities in
outcomes. e same can be
said for assessment approaches
that do not take into account
students’ culture.
National Institute for Learning Outcomes Assessment | 16
What is needed is not to help
learners conform to the ways
of higher education, thus
reinforcing inequities and
expectations based on ideologies
the students may ascribe to,
but to empower students for
success through intentional
eorts to address inequality
within our structures, create
clear transparent pathways,
and ensure that credits and
credentials are awarded by
demonstration of learning, in
whatever form that may take.
In summary, assessing students in the same way without paying attention
to their dierences works if students are all privy to the same educational
opportunities, are all at the same academic standing, have similar experiences
on campus, work through knowledge in similar fashion, understand questions
in similar ways, and benet from the same programs, pedagogical styles,
support services, and interactions. However, we know this is not the case.
“While absolute growth in the college-going population helped shape today’s
college milieu, compositional changes also impacted the college experience,
turning it into a set of highly diverse experiences that led to very dierent
outcomes” (Goldrick-Rab & Cook, 2011, p. 257). Sara Goldrick-Rab &
Marjorie Cook (2011) continue to say that “as the student body grew more
diverse, so did the kinds of colleges and universities serving them; at the same
time, opportunities both expanded in number and became more distinct
and disparate, reecting and preserving key aspects of the inequality of
opportunity and outcomes” (p. 255). Continuing to assess students as if there
are no dierences will only work to preserve key aspects of inequality and
widen the achievement gap. It is no secret that there is a disparity between the
academic attainment of students based on race/ethnicity (Bowen, Chingos,
& McPherson, 2009; Condron, Tope, Steidl, & Freeman, 2013; Santiago,
Galdeano, & Taylor, 2015; Carnevale & Strohl, 2013) and social class (Kezar,
2011; Bowen, Chingos, & McPherson, 2009). We need to ask ourselves, is it
that we want students to demonstrate their knowledge and skills or attainment
of learning outcomes in a particular way, or that they demonstrate their
learning? What is needed is not to help learners conform to the ways of higher
education, thus reinforcing inequities and expectations based on ideologies
the students may not ascribe to, but to empower students for success through
intentional eorts to address inequality within our structures, create clear
transparent pathways, and ensure that credits and credentials are awarded by
demonstration of learning, in whatever form that may take.
National Institute for Learning Outcomes Assessment | 17
References
Adelman, C. (2015). To imagine a verb: e language and syntax of learning outcomes statements. Urbana, IL: University of
Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).
Aronson, B., & Laughter, J. (2016). e theory and practice of culturally relevant education: A synthesis of research across
content areas. Review of Education Research, 86(1), 163-206.
Backes, B., & Velez, E. D. (2015). Who transfers and where do they go? Community college students in Florida. Washington,
DC: National Center for Analysis of Longitudinal Data in Education Research (CALDER).
Baker, G. R., Jankowski, N., Provezis, S. & Kinzie, J. (2012). Using assessment results: Promising practices of institutions
that do it well. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes
Assessment (NILOA).
Bal, A., & Trainor, A. A. (2016). Culturally responsive experimental intervention studies: e development of a rubric for
paradigm expansion. Review of Education Research, 86(2), 319-359.
Banta, T. W., Grin, M., Flateby, T. L., & Kahn, S. (2009, December). Three promising alternatives for assessing college
students’ knowledge and skills. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning
Outcomes Assessment.
Benitez, M. (1998). Hispanic-serving institutions: Challenges and opportunities. New Directions for Higher Education,
102(2), 57-68.
Bowen, W. G., Chingos, M. M. & McPherson, M. S. (2009). Crossing the nish line: Completing college at America’s public
universities. Princeton, NJ: Princeton University Press.
Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How people learn: Brain, mind, experience, and school.
Washington, DC: National Academy Press.
Carnevale, A. P., & Strohl, J. (2013). Separate and unequal: How higher education reinforces the intergenerational reproduction
of White racial privilege. Washington, DC; Georgetown University, Georgetown Public Policy Institute, Center on
Education and the Workforce.
Check, E., & Ballard, K. (2014). Navigating emotional, intellectual, and physical violence directed toward LGBTQQ
students and educators. Art Education, 67(3), 6-11.
Cho, S. K. (1997). Converging stereotypes in racialized sexual harassment: Where the model minority meets Suzie Wong.
In A. K. Wing, (Ed). Critical Race Feminism: A Reader (2nd. Ed). New York, NY: New York University Press.
Chu, J. Y. (2016, October 21). e need to be culturally aware when advising students (essay). Inside Higher Ed. Retrieved
from https://www.insidehighered.com/views/2016/10/21/need-be-culturally-aware-when-advising-students-essay.
Condron, D. J., Tope, D., Steidl, C. R., & Freeman, K. J. (2013). Racial segregation and the Black/White achievement
gap, 1992 to 2009. e Sociological Quarterly, 54, 130-157.
Conrad, C., Gasman, M. Lundberg, T., Nguyen, T., Commodore, F., & A. C. Samayoa (2013). Using educational data
to increase learning, retention, and degree attainment at Minority Serving Institutions. Philadelphia, PA: Penn Center for
Minority Serving Institutions. Retrieved from http://www.gse.upenn.edu/pdf/cmsi/using_educational_data.pdf.
Conrad, C., & Gasman, M. (2015). Educating a diverse nation: Lessons from Minority Serving Institutions. Cambridge,
MA: Harvard University Press.
Del Rios, M., & Leegwater, L. (2008). Increasing student success at minority serving institutions: Findings from the BEAMS
project. Washington, DC: Institute for Higher Education Policy (IHEP).
Dowd, A. G., & Bensimon, E. M. (2015). Engaging the “race question”: Accountability and equity in U.S. higher education.
New York, NY: Teachers College at Columbia University.
Ellis, L. M., & Chen, E. C. (2013). Negotiating identity development among undocumented immigrant college students:
A grounded theory study. Journal of Counseling Psychology, 60(2), 251-264.
Entwistle, N. (2001). Styles of learning and approaches to studying in higher education. Kybernetes, 30(5/6), 593-603.
Ewell, P. T. (2013). e Lumina Degree Qualications Prole: Implications for assessment. Urbana, IL: University of Illinois
and Indiana University, National Institute for Learning Outcomes Assessment.
Ewell, P. T., Paulson, K., & Kinzie, J. (2011). Down and in: Assessment practices at the program level. Urbana, IL: University
of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).
National Institute for Learning Outcomes Assessment | 18
Ferren, A. S., & Paris, D. C. (2015). Faculty leadership for integrative liberal learning. Washington, DC: Association of
American Colleges and Universities.
Froese-Germain, B., & McGahey, B. (2012). Optimizing conditions of teacher’s professional practice to support students with
special educational needs. Ottawa, ON, Canada: Canadian Teacher’s Federation. Retrieved from http://les.eric.ed.gov/
fulltext/ED532565.pdf
Gaston, P. L. (2015). General education transformed: How we can, why we must. Washington, DC: Association of American
Colleges and Universities.
Gay, G. (2011). Culturally responsive teaching: eory, research, and practice (2nd ed.). New York, NY: Teacher College
Press.
Goldrick-Rab, S., & Cook, M. A. E. (2011). College students in changing contexts. In P. G. Altbach, P. J. Gumport, &
R. O. Berdahl, (Eds) American higher education in the twenty-rst century: Social, political, and economic challenges (3rd
Ed.) pp. 254-278. Baltimore, MD: e John Hopkins University Press.
Gosselin, J., & Gagné, A. (2014). Dierentiated evaluation: An inclusive evaluation strategy aimed at promoting student
engagement and student learning in undergraduate classrooms. Toronto, Canada: Higher Education Quality Council of
Ontario.
Grove, J. (2016, September, 29). Mature students ‘do better with non-written assessment’. Times Higher Education.
Retrieved from https://www.timeshighereducation.com/news/mature-students-do-better-non-written-assessment.
Headden, S., & McKay, S. (2015, July). Motivation matters: How new research can help teachers boost student engagement.
Stanford, CA: Carnegie Foundation for the Advancement of Teaching.
Huber, L. P. (2010). Using Latina/o critical race theory (LatCrit) and racist nativism to explore intersectionality in the
educational experiences of undocumented Chicana college students. Educational Foundations, 24, 77-96.
Hurtado, S., & Halualani, R. (2014). Diversity assessment, accountability, and action: Going beyond the numbers.
Diversity & Democracy, 17(4), 8-11.
Hutchings, P. (2016). Aligning educational outcomes and practices. Urbana, IL: University of Illinois and Indiana University,
National Institute for Learning Outcomes Assessment (NILOA).
Jobs for the Future. (2016, February). Expanding competency-based education for all learners. Washington, DC: Author.
Jones, S. (2015). e game changers: Strategies to boost college completion and close attainment gaps. Change: e
Magazine of Higher Learning, 47(2), 24-29.
Jones, S. R., & McEwen, M. K. (2000). A conceptual model of multiple dimensions of identity. Journal of College Student
Development, 41, 405-414.
Juvova, A., Chudy, S., Neumeister, P., Plischke, J., & Kvinotova, J. (2015). Reection of constructivist theories in current
educational practice. Universal Journal of Educational Research, 3(5), 345-349.
Karp, M. M., Hughes, K. L., & O’Gara, L. (2010). An exploration of Tinto’s integration framework for community
college students. Journal of College Student Retention: Research, eory & Practice, 12(1), 69-86.
Kezar, A. (2011). Recognizing and serving low-income students in higher education: An examination of institutional policies,
practices, and culture. New York, NY: Routledge.
Khalifa, M. A., Gooden, M. A., & Davis, J. E. (2016). Culturally responsive school leadership: A synthesis of the literature.
Review of Educational Research, 86(4), 1272-1311.
Kim, E., & Diaz, J. (2013). Undocumented students and higher education. ASHE Higher Education Report, 38(6), 77-90.
Kuh, G. D., Ikenberry, S. O., Jankowski, N. A., Cain, T. R., Ewell, P. T., Hutchings, P., & Kinzie, J. (2015). Using evidence
of student learning to improve higher education. San Francisco, CA: Jossey-Bass.
Kuh, G. D., Jankowski, N., Ikenberry, S. O., & Kinzie, J. (2014). Knowing what students know and can do: e current
state of student learning outcomes assessment in US colleges and universities. Urbana, IL: University of Illinois and Indiana
University, National Institute for Learning Outcomes Assessment (NILOA).
Ladson-Billings, G. (1995a). Toward a theory of culturally relevant pedagogy. American Educational Research Journal,
32(3), 465-491.
Ladson-Billings, G. (1995b). But that’s just good teaching! e case for culturally relevant pedagogy. eory Into Practice,
34(3), 159-165.
National Institute for Learning Outcomes Assessment | 19
Land, S. M., Hannan, M. J., & Oliver, K. (2012). Student-centered learning environments: Foundations, assumptions and
design. In D. Jonassen, & S. Land (Eds.). eoretical foundations of learning environments (2nd ed.) pp. 3-26. New York,
NY: Routledge.
Lang, A. (1997). inking rich as well as simple: Boesch’s cultural psychology in semiotic perspective. Culture & Psychology,
3, 383–394.
Lara, R., & Wood, J. L. (2015). Impact of non-cognitive behaviors on student engagement for Latino male students attending
a designated two-year Hispanic-serving institution. In J. P. Mendez, F. A. Bonner II, J. Mendez-Negrete, & R. T. Palmer
(Eds) Hispanic-serving institutions in American higher education: eir origin, and present and future challenges. Sterling, VA:
Stylus Publishing, LLC.
Li, X., & Carroll, C. D. (2007). Characteristics of minority-serving institutions and minority undergraduates enrolled in these
institutions. Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department
of Education.
Lumina Foundation (2014). e Degree Qualications Prole. Indianapolis, IN: Author.
Macqueen, E. (2012). Studying the experiences of non-traditional students in teacher education. Sydney, Australia: Joint AARE
APERA International Conference. Retrieved from http://les.eric.ed.gov/fulltext/ED544492.pdf.
Mallory, A. (2009). Access and equity for all students: Meeting the needs of LGBT students. California Postsecondary Education
Commission. Retrieved from http://les.eric.ed.gov/fulltext/ED507508.pdf.
McKenzie, K. B., & Scheurich, J. J. (2004). Equity traps: A useful construct for preparing principals to lead schools that are
successful with racially diverse students. Educational Administration Quarterly, 40, 601-632.
Meents-DeCaigny, E., & Sanders, L. (2015). Framing research and assessment of co-curricular learning. New Directions for
Institutional Research, 164, 15-25.
Methvin, P., & Markham, P. N. (2015). Turning the page: Addressing the challenge of remediation. Change: e Magazine
of Higher Learning, 47(4), 50-56.
Miller, R., Bradbury, J., & Pedley, J. (1998). Academic performance of rst and second language students: Disadvantages and
underpreparedness. South African Journal of Science, 95, 30-34.
Montenegro, E., & Jankowski, N. A. (2015). Focused on what matters: Assessment of student learning outcomes at minority-
serving institutions. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes
Assessment (NILOA).
National Institute for Learning Outcomes Assessment. (2016, May). Higher education quality: Why documenting learning
matters. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment.
Newman, L. E., Carpenter, S., Grawe, N., & Jaret-McKinstry, S. (2014). Creating a culture conducive to integrative learning.
Peer Review, 16(4), 14-15.
Nunley, C., Bers, T, & Manning, T. (2011, July). Learning outcomes assessment in community colleges. Urbana, IL: University
of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).
Pascarella, E. T., & Terenzini, P. T. (2005). How college aects students. San Francisco, CA: Jossey-Bass.
Perez, W. (2010). Higher education access for undocumented students: Recommendations for counseling professionals.
Journal of College Admission, 206, 32-35.
Person, A. E., Goble, L., & Bruch, J. (2014, April). Developing competency-based program models in three community colleges.
Oakland, CA: Mathematica Policy Research.
Qualls, A. L. (1998). Culturally responsive assessment: Development strategies and validity issues. e Journal of Negro
Education, 67(3), 296-301.
Rankin, S., Weber, G., Blumenfeld, W., & Frazer, S. (2010). 2010 state of higher education for lesbian, gay, bisexual, and
transgender people. Charlotte, NC; Campus Pride.
Santiago, D. A., Galdeano, E. C., & Taylor, M. (2015). e condition of Latinos in education: 2015 factbook. Washington,
DC; Excelencia in Education.
Schuh, J. H. (2009). Assessment methods for student aairs. San Francisco, CA: Jossey-Bass.
National Institute for Learning Outcomes Assessment | 20
Schuh, J. H., Jones, S. R., Harper, S. R., & Associates. (2011). Student services: A handbook for the profession (5th Ed). San
Francisco, CA: Jossey-Bass.
Sedlacek, W. E. (1994). Issues in advancing diversity through assessment. Journal of Counseling & Development, 72, 549-553.
Shapiro, D., Dundar, A., Chen, J., Ziskin, M., Park, E., Torres, V., & Chiang, Y. (2012). Completing college: A national view
of student attainment rates. Herndon, VA: National Student Clearinghouse Research Center.
Singer-Freeman, K. & Bastone, L. (2016, July). Pedagogical choices make large classes feel small. (Occasional Paper No. 27).
Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).
Singleton, G. E. (2012). More courageous conversations about race. ousand Oaks, CA: Corwin.
Slee, J. (2010). A systemic approach to culturally responsive assessment practices and evaluation. Higher Education Quarterly,
64(3), 246-260.
Solorzano, D. G., & Yosso, T. J. (2002). Critical race methodology: Counter-storytelling as an analytical framework for
education research. Qualitative Inquiry, 8(1), 23-44.
Sousanis, N. (2015). Unattening. Cambridge, MA: Harvard University Press.
Spinelli, C. G. (2008). Addressing the issue of cultural and linguistic diversity and assessment: Informal evaluation measures
for English language learners. Reading & Writing Quarterly, 24(1), 101-118.
Strange, C. C., & Banning, J. H. (2015). Designing for learning: Creating campus environments for student success (2nd Ed.).
San Francisco, CA: Jossey-Bass.
Sullivan, A. L. (2010). Preventing disproportionality: A framework for culturally responsive assessment. Communique, 39(3),
24-26.
Suskie, L. (2004). Assessing student learning: A common sense guide. Boston, MA: Anker Publishing Company.
Tobolowski, B. F., & Cox, B. E. (2012). Rationalizing neglect: An institutional response to transfer students. Journal of
Higher Education, 83(3), 389-410.
Toomey Zimmerman, H., & Bell, P. (2012). Everyday expertise: Learning within and across formal and informal settings.
In D, Jonassen & S. Land (Eds.). eoretical foundations of learning environments (2nd ed.) pp. 224-241. New York, NY:
Routledge.
von Glasersfeld, E. (2005). Instroduction: Aspects of constructivism. In C. T. Fosnot (Ed.), Constructivism: eory, Perspectives,
and practice (2nd ed.), pp. 3-7. New York, NY: Teachers College Press.
Winkelmes, M., Bernacki, M., Butler, J., Zochowski, M., Golanics, J., & Harriss Weavil, K. (2016). A teaching intervention
that increases underserved college students’ success. Peer Review, 18(1-2), 31-36.
National Institute for Learning Outcomes Assessment | 21
NILOA National Advisory Panel
James Anderson
Interim Dean
Edward William and Jane Marr Gutsgell
Professor
University of Illinois Urbana-Champaign
Wallace Boston
CEO
American Public University System
Molly Corbett Broad
President
American Council on Education
Judith Eaton
President
Council for Higher Education Accreditation
Richard Ekman
President
Council of Independent Colleges
Keston Fulcher
Director of the Center for Assessment and
Research Studies
Associate Professor, Graduate Psychology
James Madison University
Paul Gaston, III
Trustees Professor
Kent State University
Mildred Garcia
President
California State University,
Fullerton
Susan Johnston
Executive Vice President
Association of Governing Boards
Norman Jones
Professor
Utah State University
Peggy Maki
Higher Education Consultant
George Mehay
Vice President for
Academic Leadership and Change
American Association of State Colleges and
Universities
Lynn Pasquerella
President
Association of American Colleges &
Universities
George Pernsteiner
President
State Higher Education Executive Ocers
Association
Mary Ellen Petrisko
President
WASC Senior College and University
Commission
Kent Phillippe
Associate Vice President, Research and
Student Success
American Association of Community Colleges
Robert Sheets
Research Professor
George Washington Institute of Public Policy
Ralph Wol
Founder and President
e Quality ASsurance Commons for Higher
and Postsecondary Education
Ex-Ocio Members
Peter Ewell
President Emeritus
National Center for Higher Education
Management Systems
Stanley Ikenberry
President Emeritus and Regent Professor
University of Illinois
Natasha Jankowski
Director, NILOA
Research Assistant Professor
George Kuh
Founding Director, National Institute for
Learning Outcomes Assessment
Adjunct Research Professor, University of
Illinois Urbana-Champaign
Chancellor’s Professor of Higher Education
Emeritus, Indiana University
Paul Lingenfelter
President Emeritus
State Higher Education Executive Ocers
NILOA Mission
NILOA’s primary objective is to
discover and disseminate ways that
academic programs and institutions
can productively use assessment data
internally to inform and strengthen
undergraduate education, and exter-
nally to communicate with policy
makers, families and other stake-
holders.
NILOA Occasional Paper Series
NILOA Occasional Papers
are commissioned to examine
contemporary issues that will inform
the academic community of the
current state-of-the art of assessing
learning outcomes in American higher
education. e authors are asked to
write for a general audience in order
to provide comprehensive, accurate
information about how institutions and
other organizations can become more
procient at assessing and reporting
student learning outcomes for the
purposes of improving student learning
and responsibly fullling expectations
for transparency and accountability
to policy makers and other external
audiences.
Comments and questions about this
paper should be sent to:
National Institute for Learning Outcomes Assessment | 22
About NILOA
• e National Institute for Learning Outcomes Assessment (NILOA) was estab-
lished in December 2008.
• NILOA is co-located at the University of Illinois and Indiana
University.
• e NILOA website contains free assessment resources and can be found at http://
www.learningoutcomesassessment.org/.
• e NILOA research team has scanned institutional websites, surveyed chief
academic ocers, and commissioned a series of occasional papers.
• NILOA’s Founding Director, George Kuh, founded the National Survey for
Student Engagement (NSSE).
• e other co-principal investigator for NILOA, Stanley Ikenberry, was president
of the University of Illinois from 1979 to 1995 and of the American Council of
Education from 1996 to 2001.
NILOA Sta
NATIONAL INSTITUTE FOR LEARNING OUTCOMES ASSESSMENT
Stanley Ikenberry, Co-Principal Investigator
George Kuh, Founding Director, Senior Scholar, and Co-Principal Investigator
Natasha Jankowski, Director
Gianina Baker, Assistant Director
Filip Przybysz, Communications Coordinator
Katie Schultz, Project Manager
Peter Ewell, Senior Scholar
Pat Hutchings, Senior Scholar
Jillian Kinzie, Senior Scholar
Paul Lingenfelter, Senior Scholar
David Marshall, Senior Scholar
Erick Montenegro, Research Analyst
Verna F. Orr, Research Analyst
Anthony B. Sullers, Jr., Research Analyst
Emily Teitelbaum, Research Analyst
Terry Vaughan III, Research Analyst
Karie Brown-Tess, Research Analyst
NILOA Sponsors
Lumina Foundation for Education
University of Illinois, College of Education
Produced by Creative Services | Public Aairs at the University of Illinois for NILOA. 10.032
National Institute for Learning Outcomes Assessment | 23
National Institute for Learning Outcomes Assessment
For more information, please contact:
National Institute for Learning Outcomes Assessment (NILOA)
University of Illinois at Urbana-Champaign
360 Education Building
Champaign, IL 61820
learningoutcomesassessment.org
Phone: 217.244.2155
knowledge accountability connection self-reflection educate action understand communicate listen learn access quality innovation success ingenuity
intellect curiosity challenge create achievement connection self-reflection educate action understand communicate listen learn access quality innovation
success ingenuity intellect curiosity challenge knowledge accountability connection understand communicate listen learn access quality innovation success
ingenuity self-reflection educate action understand intellect knowledge accountability connection self-reflection educate action understand communicate
curiosity challenge create achievement connection self-reflection curiosity challenge create achievement connection self-reflection knowledge accountability
connection self-reflection educate action understand communicate listen learn access quality innovation success ingenuity intellect curiosity challenge educate
innovation success ingenuity intellect curiosity challenge create achievement knowledge accountability connection self-reflection educate action understand
communicate curiosity challenge create achievement connection self-reflection understand communicate listen learn access quality action educate action
understand communicate listen learn action understand communicate listen learn access quality innovation success ingenuity intellect curiosity challenge
knowledge accountability connection access quality self-reflection curiosity challenge create achievement learn access quality innovation success ingenuity
self-reflection educate action understand intellect knowledge accountability connection self-reflection educate action understand knowledge accountability
connection self-reflection educate action understand communicate listen learn access quality innovation success ingenuity intellect curiosity challenge
connection knowledge accountability connection self-reflection educate action understand communicate listen learn access quality innovation success
ingenuity challenge create achievement connection self-reflection educate action understand connection self-reflection understand communicate listen learn
access quality action create achievement connection self-reflection educate action understand communicate listen learn access quality innovation success
educate action communicate listen learn access quality action educate action understand communicate educate innovation success self-reflection knowledge
accountability communicate listen learn achievement connection self-reflection educate action understand communicate listen learn access quality innovation
success ingenuity intellect access quality innovation success self-reflection curiosity challenge create achievement connection self-reflection understand
educate action understand communicate listen learn action understand communicate listen learn access quality innovation success ingenuity curiosity
challenge create achievement connection self-reflection understand communicate listen learn access quality action create achievement connection self-
reflection educate action understand communicate listen learn access quality innovation success educate action communicate listen learn access quality
action educate action understand create achievement connection self-reflection understand communicate listen learn access quality action create achievement
connection self-reflection educate action understand communicate listen communicate educate innovation success self-reflection knowledge accountability
connection self-reflection educate action understand communicate listen learn access quality innovation ingenuity intellect connection self-reflection
understand communicate listen learn access quality action create achievement connection self-reflection educate action understand communicate listen learn
