Background

Boise State University (BSU) is a metropolitan research university in the rural state of Idaho. I am an associate professor of Spanish and an applied linguist and teach undergraduate courses in Spanish and world language pedagogy and direct an online graduate certificate. In addition, I maintain an active research and service load commensurate with a doctoral university.

Currently, the criteria for promotion and tenure (P&T) are the traditional tripartite categories of teaching, service, and scholarly creative or research activities. University policy recognizes Boyer’s Model of Scholarship (BMS) according to (1) discovering knowledge, (2) integrating knowledge, (3) applying knowledge, and (4) scholarship of teaching, although the BMS is restricted for P&T application to research only. The university provides a web page with descriptions and examples for how faculty can communicate their work according to BMS;^[1] however, open educational resource (OER) is not listed explicitly, and no guidelines or rubrics exist to support committees evaluating the merits of OER activities.

In 2019, BSU received state funding and granted 30 OER projects across campus to “encourage experimentation with OER…and grow our academic community’s awareness of how OER can complement or replace expensive teaching and learning resources.” I am the codirector of the Pathways Project (PP), which is a repository of more than 900 ancillary materials for world language teachers and was a recipient of this funding. Currently, I am participating in a university OER development series program where I am creating a Spanish conversational OER textbook to offer affordable access and enhance interaction and diversity in learning, which are critical to the Spanish program learning outcomes. As such, I am working with university leaders to create guidelines to evaluate OER activities for equitable integration in the P&T process. However, this case study is focused on my PP OER activities only.

Approach

The Institutional Context and Challenge

I primarily characterize my intellectual impact through high-engaged scholarship and community engagement that is fundamentally rooted in OER and open educational practices (OEP) with K–16 language teachers. Community engagement is a Carnegie distinction at my institution as well as one of the key pillars for the university’s strategic planning. I am applying for promotion next academic year and am working with university leaders to create rubrics for P&T committees that currently have no metric to evaluate OER activities included in the faculty portfolio. Furthermore, faculty have no guidelines for how to communicate evidence for OER-BMS alignment in their narrative, which is critical in helping make this work visible, equitable, and therefore, sustainable.

This case study highlights the process for creating both a guideline and rubric for OER activities according to the BMS specific to the Scholarship of Application, a category where I have demonstrated leadership and scholarly distinction. It is unique because it presents a case for OER research through the creation of non-textbook-related OER materials. In doing so, I propose a process and provide examples of how OER activities of all types can be aligned to standards of research evidence and evaluated equitably according to clear criteria.

High-Engaged Scholarship through Community Engagement with OER

PP is a repository of over 900 high-quality, standards-aligned OER classroom activities that support and engage the K–16 world language teacher community in OEP and on a global scale. This repository was built because of a needs-based assessment I conducted with the K–12 local teacher community in 2016 and 2017. The PP was founded in 2018 and has now grown to a community of more than 350 faculty, staff, and students who participate in OEP (to differing degrees) to impact the teaching and learning of 10 world languages across three levels of proficiency.

In addition to internal funding awarded in 2019, the PP received state funding in 2020 to pilot a case study in OEP to train teachers to retain, reuse, and revise PP materials. This served as proof of concept within a longitudinal design-based research framework. At the end of 2021, I was awarded a $100,000 National Endowment for the Humanities (NEH) Level II Digital Humanities Advancement Grant (DHAG) to evaluate teacher practices in the five Rs of OEP to retain, reuse, revise, remix, and redistribute PP materials. With my codirector, we led a cohort of 6 K–12 language teachers primarily in urban districts in the fall of 2022 and a spring cohort of 10 language teachers primarily in rural districts across the state of Idaho. In addition, I integrated the PP into two undergraduate and two graduate courses focused on world language theory, methodology, literacy, and assessment. In these courses, preservice and master’s students also engage in OEP and showcase the evidence of their work in their final professional portfolio. Furthermore, they are encouraged to use the PP activities in their field experiences in K–12 schools and, in so doing, are OER ambassadors by helping onboard their mentor teachers who benefit from the PP materials and learn how to locate and implement them for future use.

Finally, I am frequently invited to deliver local, regional, and international workshops and presentations to train in-service K–12 teachers on retaining, reusing, and revising PP materials for their learning contexts. In the last three years, I have delivered 36 workshops/presentations nationally and internationally. In addition to this work, I have coauthored peer-reviewed and praxis-oriented papers in open journals and coauthored a piece in a highly visible educational blog read by the teacher-educator community. Furthermore, the NEH award will contribute to several community-informed products, including a guide on how to lead community-engaged OER projects to impact teaching and learning at the K–12 levels.

In the following section, I explain the process I use for (1) communicating evidence to justify my OER as scholarship and (2) evaluating OER using a rubric aligned to the Scholarship of Application, which is a domain of the BMS. I contextualize the degree of success in the rubric according to high-engaged scholarship informed by Franz (2011).

Making the Work Visible and Measurable

One of the compelling reasons I joined my institution is for what is popularly known as “blue turf thinking”—a brand platform and a vision statement wrapped in one that was coined thanks to the unique color of the football field. “Blue turf thinking” is a way to say the university is committed to innovation and thinking outside the box. One of the challenges with innovative work is to ensure that policies and structural mechanisms within the university can support the rate of change that comes with innovation. This is one of the issues that confront BSU as it relates to the sustainability and quality of OER being produced. Faculty who engage in OER scholarship find it intellectually rewarding, and they know OER research benefits learning above and beyond affordability, which is the single greatest driver for institutional OER initiatives. And while there are many innovative leaders at BSU driving important philosophical and ideological conversations related to P&T and the BMS, the reality is that there must be a pragmatic approach to supporting faculty now. The litmus test for success in the short-term, and the approach I have taken in this process, is to consider what a non-OER faculty member would need to (1) understand the work according to the university standards of scholarship and (2) rate the work using a scale that aligns with the university strategic plan for community engagement and map to the Scholarship of Application according to the BMS. The rationale for using the BMS is that it is more holistic and inclusive of the integrated nature of my work in OER. Currently, there are scarce resources that include other types of OER activities like the PP, and this case study contributes both a process and evaluative material that can make OER community-engaged scholarship visible and measurable.

Step 1: Use the DOERS3 Matrix to Communicate Your Work

The DOERS3 matrix is helpful in identifying how one’s work aligns with teaching and service, and it helps faculty consider how to collect evidence for the adoption, adaptation, improved learning, and community categories in the matrix. In this way, the matrix can be an excellent reflection tool for faculty early in the P&T process; faculty who want to talk about their work with their chair and/or P&T committees for annual or periodic review can use the matrix to guide the discussion. For example, I requested letters of support from administrators and teachers in the community who directly benefited from PP workshops and presentations. The letters address the impacts she has made as an OER leader in the K–12 community in the state.

Step 2: Clearly Identify Evidence according to Standards of Scholarship

One way the DOERS3 matrix can become more inclusive of all OER-related projects is to think through evidence pieces related to tracking impact through usage. For example, faculty can document how their OER materials are being used by others in the larger, external community. Working with website tracking and data analytics, I have recorded the number of times a PP activity was revised or remixed as well as the number of times the project has been accessed and where (by city, state, region, etc.). These metrics are helpful in characterizing the adoption and, when possible, adaptation of OER materials for committee members, and they can help faculty chart these impacts over time.

The DOERS3 matrix may not be as helpful when it comes to the create category and specifically with presenting a case for OER as research. Indeed, this is often the pressure point for faculty doing high-engaged OER scholarship. At best, the matrix makes clear that creating OER can be a form of scholarship. However, in my context, there are no university guidelines that explain how creating OER counts as research, nor are there rubrics that support a clear and equitable evaluation process. This is particularly problematic for faculty who will be evaluated by peers (departmental and/or college committees) who have a general or simplistic understanding of what OER are. Often, non-OER creators/engagers tend to limit OER activity creation to teaching and service categories only.

At my institution, there are clear standards of scholarship that, although not explicitly worded within P&T policy, are strongly encouraged to be applied in the BMS evaluation process. As such, I use these standard criteria from our university website to communicate the process and/or document evidence as a rationale for their merit as scholarship. In the following table, the column on the left contains the eight standards of scholarship according to the BMS, and the column on the right is where I include examples or detailed evidence related to the creation of OER.

One of the potential challenges in advocating for OER as scholarship is with regards to peer review. In meeting with administrators and faculty leaders across a vast spectrum of units at my university, I encountered a great range of perspectives regarding the merits of peer review in the first place and not just specific to OER. For example, one leader argued that community-engaged scholarship shouldn’t be limited to traditional notions of “peer review” but that high-impact community-engaged scholarship must reflect the rigor of standards specific to and in concert with the community. Thus, this criterion (peer review) should, at the very least, be inclusive of a broader definition.

Precisely because many OER materials are self-published independent of editorial contracts, the onus can frequently fall on the faculty member to request or seek out peer review. Fortunately, my institution has recently catalyzed an OER development series with five pathways where faculty can participate in OER and receive support (mentoring and a stipend). One of these (pathways) provides a process for faculty to receive peer review for an authored textbook. At any rate, it is important to recognize that most faculty are bound to evidence of peer review, and each faculty member must determine early on how they will account for it in their narrative.

Step 3: Remix a Rubric If No Rubric Exists

One of the most challenging aspects of working with the BMS framework is anticipating how one will be evaluated in the absence of clear criteria or rubrics. Integrated research (referring to the BMS domains) has a crossover with traditional categories that have been entrenched, for better or for worse, in the university ecosystem and within the academic psyche. For this reason, it is imperative to delineate the degrees to which OER meet or do not meet scholarship standards. I identify my PP OER research as Scholarship of Application within the BMS domain and refer to my community engagement as high-engaged scholarship (Franz, 2011).

In my quest to adopt a rubric that can help me self-evaluate my progress, I draw from the University of South Florida’s community-engaged tool kit (Jordan, 2007) and adapt it to OER community engagement according to the university scale that is already used to evaluate faculty workload. I plan to include this rubric alongside the tool (Table 2) that maps the PP to standards of research.

Results

Whereas OER are not listed as examples of scholarship across any categories (BMS or traditional) within university policy or guidelines, I have met with university leaders to advocate for its inclusion. Nonetheless, I was successful in discussing the merits of OER as an example of research according to the BMS at the departmental level, and as such, in 2018, OER was included as an example of a research product that would count toward P&T. Unfortunately, college and university policies have yet to include OER explicitly; although there has been no pushback from leadership when this has been discussed in conversation. There are ideological and pragmatic conversations related to OER initiatives and P&T alignment with the BMS happening across units, such as research and economic development, diversity and inclusion, the library, and eCampus. All parties are supportive of explicitly including OER activities as examples within the BMS while also acknowledging there is currently no way to evaluate them. In the meantime, I am contributing this case study as part of my OER advocacy to make this important work visible and sustainable so that faculty at my university are incentivized to stay committed to OER activities of all types. The following recommendations summarize ways faculty who create OER and are active in OER community engagement can communicate their work as scholarship and can adopt or adapt evaluative metrics for self-evaluation and, in the best-case scenario, for P&T policy or guideline adoption in the future.

Recommendations

• Map your OER activities to the DOERS3 matrix early in the P&T process to identify how your work is characterized and take action steps to collect evidence where it is needed.
• Determine if your OER engagement aligns to standards of research at your university. If so, utilize a table like Table 2 to communicate and justify how. Engage in conversations with P&T committee members early in the process and use this table to guide the discussion.
• Request rubrics from key leaders across units, and if your institution does not provide one for OER, consider adopting or adapting one to put forward in your portfolio.
• OER have great potential for community impact. To the extent possible, identify action steps to target high-engaged scholarship with a community that can partner and collaborate in your OER activities and research.

References

• Franz, N. (2011). Tips for constructing a promotion and tenure dossier that documents engaged scholarship endeavors. Journal of Higher Education Outreach and Engagement, 15(3), 15–29.

• Jordan, C. (2007). “Community-Engaged Scholarship Toolkit.” University of South Florida. https://www.usf.edu/engagement/faculty/community-engaged-scholarship-toolkit.aspx. Adapted with permission from Community-Engaged Scholarship Review, Promotion & Tenure Package, Peer Review Workgroup, Community-Engaged Scholarship for Health Collaborative, Community-Campus Partnerships for Health.