12 Steps to Making OER Count Using The Boyer Model and the DOERS3 Matrix

Kelly Arispe

Case study writer: Dr. Kelly Arispe, Associate Professor of Spanish

Institution: Boise State University, a doctoral university classified by high research activity and community engagement

Type of intervention: This case study will provide insight into using the OER Contribution Matrix with the Boyer’s Model of Scholarship for evaluation as well as highlight an aspect of OER ancillary materials that are even less understood and recognized by institutions.


Boise State University (BSU) is a metropolitan research university in the rural state of Idaho. I am an associate professor of Spanish and an applied linguist and teach undergraduate courses in Spanish and world language pedagogy and direct an online graduate certificate. In addition, I maintain an active research and service load commensurate with a doctoral university.

Currently, the criteria for promotion and tenure (P&T) are the traditional tripartite categories of teaching, service, and scholarly creative or research activities. University policy recognizes Boyer’s Model of Scholarship (BMS) according to (1) discovering knowledge, (2) integrating knowledge, (3) applying knowledge, and (4) scholarship of teaching, although the BMS is restricted for P&T application to research only. The university provides a web page with descriptions and examples for how faculty can communicate their work according to BMS;[1] however, open educational resource (OER) is not listed explicitly, and no guidelines or rubrics exist to support committees evaluating the merits of OER activities.

In 2019, BSU received state funding and granted 30 OER projects across campus to “encourage experimentation with OER…and grow our academic community’s awareness of how OER can complement or replace expensive teaching and learning resources.” I am the codirector of the Pathways Project (PP), which is a repository of more than 900 ancillary materials for world language teachers and was a recipient of this funding. Currently, I am participating in a university OER development series program where I am creating a Spanish conversational OER textbook to offer affordable access and enhance interaction and diversity in learning, which are critical to the Spanish program learning outcomes. As such, I am working with university leaders to create guidelines to evaluate OER activities for equitable integration in the P&T process. However, this case study is focused on my PP OER activities only.


The Institutional Context and Challenge

I primarily characterize my intellectual impact through high-engaged scholarship and community engagement that is fundamentally rooted in OER and open educational practices (OEP) with K–16 language teachers. Community engagement is a Carnegie distinction at my institution as well as one of the key pillars for the university’s strategic planning. I am applying for promotion next academic year and am working with university leaders to create rubrics for P&T committees that currently have no metric to evaluate OER activities included in the faculty portfolio. Furthermore, faculty have no guidelines for how to communicate evidence for OER-BMS alignment in their narrative, which is critical in helping make this work visible, equitable, and therefore, sustainable.

This case study highlights the process for creating both a guideline and rubric for OER activities according to the BMS specific to the Scholarship of Application, a category where I have demonstrated leadership and scholarly distinction. It is unique because it presents a case for OER research through the creation of non-textbook-related OER materials. In doing so, I propose a process and provide examples of how OER activities of all types can be aligned to standards of research evidence and evaluated equitably according to clear criteria.

High-Engaged Scholarship through Community Engagement with OER

PP is a repository of over 900 high-quality, standards-aligned OER classroom activities that support and engage the K–16 world language teacher community in OEP and on a global scale. This repository was built because of a needs-based assessment I conducted with the K–12 local teacher community in 2016 and 2017. The PP was founded in 2018 and has now grown to a community of more than 350 faculty, staff, and students who participate in OEP (to differing degrees) to impact the teaching and learning of 10 world languages across three levels of proficiency.

In addition to internal funding awarded in 2019, the PP received state funding in 2020 to pilot a case study in OEP to train teachers to retain, reuse, and revise PP materials. This served as proof of concept within a longitudinal design-based research framework. At the end of 2021, I was awarded a $100,000 National Endowment for the Humanities (NEH) Level II Digital Humanities Advancement Grant (DHAG) to evaluate teacher practices in the five Rs of OEP to retain, reuse, revise, remix, and redistribute PP materials. With my codirector, we led a cohort of 6 K–12 language teachers primarily in urban districts in the fall of 2022 and a spring cohort of 10 language teachers primarily in rural districts across the state of Idaho. In addition, I integrated the PP into two undergraduate and two graduate courses focused on world language theory, methodology, literacy, and assessment. In these courses, preservice and master’s students also engage in OEP and showcase the evidence of their work in their final professional portfolio. Furthermore, they are encouraged to use the PP activities in their field experiences in K–12 schools and, in so doing, are OER ambassadors by helping onboard their mentor teachers who benefit from the PP materials and learn how to locate and implement them for future use.

Finally, I am frequently invited to deliver local, regional, and international workshops and presentations to train in-service K–12 teachers on retaining, reusing, and revising PP materials for their learning contexts. In the last three years, I have delivered 36 workshops/presentations nationally and internationally. In addition to this work, I have coauthored peer-reviewed and praxis-oriented papers in open journals and coauthored a piece in a highly visible educational blog read by the teacher-educator community. Furthermore, the NEH award will contribute to several community-informed products, including a guide on how to lead community-engaged OER projects to impact teaching and learning at the K–12 levels.

In the following section, I explain the process I use for (1) communicating evidence to justify my OER as scholarship and (2) evaluating OER using a rubric aligned to the Scholarship of Application, which is a domain of the BMS. I contextualize the degree of success in the rubric according to high-engaged scholarship informed by Franz (2011).

Making the Work Visible and Measurable

One of the compelling reasons I joined my institution is for what is popularly known as “blue turf thinking”—a brand platform and a vision statement wrapped in one that was coined thanks to the unique color of the football field. “Blue turf thinking” is a way to say the university is committed to innovation and thinking outside the box. One of the challenges with innovative work is to ensure that policies and structural mechanisms within the university can support the rate of change that comes with innovation. This is one of the issues that confront BSU as it relates to the sustainability and quality of OER being produced. Faculty who engage in OER scholarship find it intellectually rewarding, and they know OER research benefits learning above and beyond affordability, which is the single greatest driver for institutional OER initiatives. And while there are many innovative leaders at BSU driving important philosophical and ideological conversations related to P&T and the BMS, the reality is that there must be a pragmatic approach to supporting faculty now. The litmus test for success in the short-term, and the approach I have taken in this process, is to consider what a non-OER faculty member would need to (1) understand the work according to the university standards of scholarship and (2) rate the work using a scale that aligns with the university strategic plan for community engagement and map to the Scholarship of Application according to the BMS. The rationale for using the BMS is that it is more holistic and inclusive of the integrated nature of my work in OER. Currently, there are scarce resources that include other types of OER activities like the PP, and this case study contributes both a process and evaluative material that can make OER community-engaged scholarship visible and measurable.

Step 1: Use the DOERS3 Matrix to Communicate Your Work

The DOERS3 matrix is helpful in identifying how one’s work aligns with teaching and service, and it helps faculty consider how to collect evidence for the adoption, adaptation, improved learning, and community categories in the matrix. In this way, the matrix can be an excellent reflection tool for faculty early in the P&T process; faculty who want to talk about their work with their chair and/or P&T committees for annual or periodic review can use the matrix to guide the discussion. For example, I requested letters of support from administrators and teachers in the community who directly benefited from PP workshops and presentations. The letters address the impacts she has made as an OER leader in the K–12 community in the state.

Step 2: Clearly Identify Evidence according to Standards of Scholarship

One way the DOERS3 matrix can become more inclusive of all OER-related projects is to think through evidence pieces related to tracking impact through usage. For example, faculty can document how their OER materials are being used by others in the larger, external community. Working with website tracking and data analytics, I have recorded the number of times a PP activity was revised or remixed as well as the number of times the project has been accessed and where (by city, state, region, etc.). These metrics are helpful in characterizing the adoption and, when possible, adaptation of OER materials for committee members, and they can help faculty chart these impacts over time.

The DOERS3 matrix may not be as helpful when it comes to the create category and specifically with presenting a case for OER as research. Indeed, this is often the pressure point for faculty doing high-engaged OER scholarship. At best, the matrix makes clear that creating OER can be a form of scholarship. However, in my context, there are no university guidelines that explain how creating OER counts as research, nor are there rubrics that support a clear and equitable evaluation process. This is particularly problematic for faculty who will be evaluated by peers (departmental and/or college committees) who have a general or simplistic understanding of what OER are. Often, non-OER creators/engagers tend to limit OER activity creation to teaching and service categories only.

At my institution, there are clear standards of scholarship that, although not explicitly worded within P&T policy, are strongly encouraged to be applied in the BMS evaluation process. As such, I use these standard criteria from our university website to communicate the process and/or document evidence as a rationale for their merit as scholarship. In the following table, the column on the left contains the eight standards of scholarship according to the BMS, and the column on the right is where I include examples or detailed evidence related to the creation of OER.

Table 1: A tool to map faculty-created OER to standards of research.
BMS standards criteria PP OER examples and evidence
Clear goals Teachers and students participate in the Pathways Project and come from different fields of study and schools across the globe to create open, digital activities that support the teaching and learning of world languages and promote intercultural competence.
Adequate preparation 1. Needs assessment to identify and prioritize biggest challenges and obstacles to aligning and delivering best practices in world language teaching across K–16 levels

2. Create a template for activities that integrate world language national standards, student-friendly (can-do) learning outcomes, teacher-friendly instructions, all material links with editable versions, open licenses, clear descriptions, and tags to improve online searchability

3. Create professional development materials (slide decks, on-demand video presentations, and online workshops) to train the OER PP team (undergraduate students, faculty, staff, and in-service teachers)

Appropriate methods Design-based research (DBR) methodology has informed exploration, pilot, and enactment cycles that allow the researchers, teachers, and design team to make incremental changes through multiple iterations that improve both the OER materials produced as well as the data collection and analysis.
Significant results Program evaluation and mixed-methods research data collection upholds Institutional Review Board requirements, testing validity, and reliability of data analysis. Usage data/analytics demonstrate that the goal of the project is being met.
Effective presentation Multiple ways of accessing PP materials: website, OER Commons, Pressbooks, Scholarworks.

All publications and presentations are open access and are shared via the website and Scholarworks.

Reflective critique The DBR is a methodology that integrates reflective critique in its design. Multiple iterations propel improvements/adjustments that are responsive to constructive critique and user needs.
Public dissemination Multiple publications and presentations nationally and internationally disseminate PP activities, professional development products, as well as research findings.
Peer review Publications and presentations are peer-reviewed.

PP activities are retained, reused, revised, remixed, and redistributed by a global teacher education community.

One of the potential challenges in advocating for OER as scholarship is with regards to peer review. In meeting with administrators and faculty leaders across a vast spectrum of units at my university, I encountered a great range of perspectives regarding the merits of peer review in the first place and not just specific to OER. For example, one leader argued that community-engaged scholarship shouldn’t be limited to traditional notions of “peer review” but that high-impact community-engaged scholarship must reflect the rigor of standards specific to and in concert with the community. Thus, this criterion (peer review) should, at the very least, be inclusive of a broader definition.

Precisely because many OER materials are self-published independent of editorial contracts, the onus can frequently fall on the faculty member to request or seek out peer review. Fortunately, my institution has recently catalyzed an OER development series with five pathways where faculty can participate in OER and receive support (mentoring and a stipend). One of these (pathways) provides a process for faculty to receive peer review for an authored textbook. At any rate, it is important to recognize that most faculty are bound to evidence of peer review, and each faculty member must determine early on how they will account for it in their narrative.

Step 3: Remix a Rubric If No Rubric Exists

One of the most challenging aspects of working with the BMS framework is anticipating how one will be evaluated in the absence of clear criteria or rubrics. Integrated research (referring to the BMS domains) has a crossover with traditional categories that have been entrenched, for better or for worse, in the university ecosystem and within the academic psyche. For this reason, it is imperative to delineate the degrees to which OER meet or do not meet scholarship standards. I identify my PP OER research as Scholarship of Application within the BMS domain and refer to my community engagement as high-engaged scholarship (Franz, 2011).

In my quest to adopt a rubric that can help me self-evaluate my progress, I draw from the University of South Florida’s community-engaged tool kit (Jordan, 2007) and adapt it to OER community engagement according to the university scale that is already used to evaluate faculty workload. I plan to include this rubric alongside the tool (Table 2) that maps the PP to standards of research.

OER Community Engagement Rubric

Each category can be assessed as

  • Does not meet expectations
  • Meets expectations (scholarship[2])
  • Exceeds expectations (high-engaged scholarship[3])
Table 2: OER Community Engagement Rubric
Evidence of High-Engaged Scholarship Description Assessment
Clear academic and change goals
  • Communicates the purpose of the OER and its need for the public good
  • Defines goals and objectives that are achievable
  • Identifies questions of inquiry in the discipline and in the community
  • Articulates research objectives
  • Articulates goals for teaching and learning
Adequate preparation in content area and grounding in the community
  • Invests time and effort in community partnerships for peer review, feedback, and OER-engaged pedagogy (OEP)
  • Facilitates training and professional development in OER-related competencies (e.g., the five Rs of OEP)
  • Conducts action or design-based research to evaluate OER extensibility to community teaching and learning contexts
  • Demonstrates OER leadership as an advocate in the discipline and OEP cycles
Appropriate methods, rigor, and community engagement
  • Enhances OER material by incorporating updated and real-world information from community members critical to student learning of course material
  • Deepens and contextualizes the learning experience in OEP/OER by involving community experts in design and implementation
  • Leverages grants for OER projects or an OER research project because of community involvement
  • Refines a research question or confirms its validity through cogeneration with a community partner
  • Involves the community to improve OER design, recruitment of other community members in OEP, instrument collection for OER-related research, or presenting and/or publishing because of OER project(s)
  • Disseminates the findings more broadly through partnerships with community organizations looking to adopt, adapt, or create OER in the discipline
  • Improves ethical credibility of OER for teaching and learning by directly addressing specific issues/concerns with the community
  • Encourages the extensibility of other OER for communities beyond the university
Significant results: Impact on the field and the community
  • The community contributes to as well as benefits from the OER research and/or OER materials
  • The OER makes progress toward diversity and inclusion for learning
  • Directly contributes to learning materials that did not previously exists
  • Secures increased funding to continue, expand, or replicate the initial OER project
  • Secures increased funding for community partners
  • Increases capacity of individuals in the community to advocate for OER and to use Creative Commons and/or fair usage licensing for their ongoing work
  • Enhances opportunities for students and staff to assume positions of leadership with the community
  • Opens up additional areas for further exploration and collaboration of OEP
  • Disseminates geographically limited work with clear discussion as to its generalizability to other populations or as a model that can be further investigated in other settings
Effective presentation/ dissemination to academic and community audiences
  • Publishes research results or teaching innovations in peer-reviewed journals, practitioner journals, professional blogs, or newspapers read by community members
  • Disseminates information through other media used by community members
  • Utilizes video, remote, or online programs to reach community members
  • Produces guidelines or policy documents directed toward administrators, content experts, and teachers
  • Presents at community events
  • Coauthors any of the above with community partners
Reflective critique: Lessons learned to improve the scholarship and community engagement
  • Conducts debriefing sessions, focus groups, and/or surveys with community members
  • Seeks evaluation from community members
  • Changes OER design or OEP engagement model based on feedback and lessons learned
  • Engages in personal reflection concerning, for example, issues of accessibility and diversity and inclusion
Leadership and personal contribution
  • Receives invitations to present to professional meetings, national or international conferences
  • Receives invitations to present to community audiences or facilitate professional development
  • Receives invitations to testify before legislative bodies regarding statewide OER adoption
  • Receives invitations to appear in the media
  • Receives invitations to serve on advisory or policymaking committees at national, regional, state, and/or community levels
  • Receives invitations to serve on editorial boards for OER initiatives
  • Receives awards or letters of appreciation from community-based organizations for contributions to teaching and learning
  • Mentors students, junior faculty, staff, and community partners
  • Is asked to be a peer reviewer of a colleague’s OER materials
  • Is asked to collaborate on OER university policy and initiatives
Consistently ethical behavior: Socially responsible conduct of research and teaching
  • Cultivates the conduct of “good science” using sound research techniques and appropriate engaged pedagogies that result in meaningful and beneficial contributions to the community
  • Follows the human subject review process and IRB standards for research
  • Engages communities in a respectful manner
  • Recognizes and values community systems and incorporates them into the research process and OER materials as appropriate
  • Appropriately acknowledges community partners in all dissemination of research and teaching materials whether in writing or presenting



Whereas OER are not listed as examples of scholarship across any categories (BMS or traditional) within university policy or guidelines, I have met with university leaders to advocate for its inclusion. Nonetheless, I was successful in discussing the merits of OER as an example of research according to the BMS at the departmental level, and as such, in 2018, OER was included as an example of a research product that would count toward P&T. Unfortunately, college and university policies have yet to include OER explicitly; although there has been no pushback from leadership when this has been discussed in conversation. There are ideological and pragmatic conversations related to OER initiatives and P&T alignment with the BMS happening across units, such as research and economic development, diversity and inclusion, the library, and eCampus. All parties are supportive of explicitly including OER activities as examples within the BMS while also acknowledging there is currently no way to evaluate them. In the meantime, I am contributing this case study as part of my OER advocacy to make this important work visible and sustainable so that faculty at my university are incentivized to stay committed to OER activities of all types. The following recommendations summarize ways faculty who create OER and are active in OER community engagement can communicate their work as scholarship and can adopt or adapt evaluative metrics for self-evaluation and, in the best-case scenario, for P&T policy or guideline adoption in the future.


  • Map your OER activities to the DOERS3 matrix early in the P&T process to identify how your work is characterized and take action steps to collect evidence where it is needed.
  • Determine if your OER engagement aligns to standards of research at your university. If so, utilize a table like Table 2 to communicate and justify how. Engage in conversations with P&T committee members early in the process and use this table to guide the discussion.
  • Request rubrics from key leaders across units, and if your institution does not provide one for OER, consider adopting or adapting one to put forward in your portfolio.
  • OER have great potential for community impact. To the extent possible, identify action steps to target high-engaged scholarship with a community that can partner and collaborate in your OER activities and research.


  • Franz, N. (2011). Tips for constructing a promotion and tenure dossier that documents engaged scholarship endeavors. Journal of Higher Education Outreach and Engagement, 15(3), 15–29.

  • Jordan, C. (2007). “Community-Engaged Scholarship Toolkit.” University of South Florida. https://www.usf.edu/engagement/faculty/community-engaged-scholarship-toolkit.aspx. Adapted with permission from Community-Engaged Scholarship Review, Promotion & Tenure Package, Peer Review Workgroup, Community-Engaged Scholarship for Health Collaborative, Community-Campus Partnerships for Health.

  1. The Boyer Model of Scholarship at Boise State University
  2. Scholarship is defined by Franz (2011) as “original intellectual work communicated and validated by peers.”
  3. High-engaged scholarship is defined by Franz (2011) as “engagement with communities that integrate scholarship in the process.”


Icon for the Creative Commons Attribution 4.0 International License

Valuing OER in the Tenure, Promotion, and Reappointment Process Copyright © 2024 by Kelly Arispe is licensed under a Creative Commons Attribution 4.0 International License, except where otherwise noted.

Share This Book