Evaluating the Mentoring Process and Experience
7.1 Mentor Evaluation of the Research Process and Experience
Evaluation by the mentor can be done at different levels. On the one hand, the mentor can design assessments that will allow mentees to demonstrate their achievement of learning outcomes specifically related to content knowledge of the research project. On the other hand, the mentor should evaluate the mentee’s performance and achievement of research attitudes, including but not limited to thinking critically, taking initiative, working independently, and accepting feedback.
The first level of evaluation can be attained by common assessment tasks such as writing essays, submitting written reports, developing a project, preparing an oral or poster presentation, writing a research journal, or preparing a portfolio. The assignment that is selected should align with the research project’s learning outcomes. It is also important that mentor and mentee discuss the assignment, that the mentee receives proper instructions, and that feedback is given during and after the preparation of the assignment. Providing the mentee with examples of similar assignments that highlight the expected quality of work will be beneficial.
The mentee must receive constructive criticism of his/her performance during the research experience. The mentor should be aware that comments and suggestions are usually highly regarded by the mentee so the mentor’s feedback must be honest and accurate. Evaluation on this second level requires the mentor to carefully reflect on mentee’s attitude gains. This evaluation can be done with the help of rubrics or evaluation forms, samples of which are presented in the tables below.
Unquestionably, both evaluation levels are closely interrelated; it is unlikely that a mentee will achieve content knowledge learning outcomes without applying the proper research attitudes.
Student-Centered Evaluation Form of Research Experience and Student Performance
Table 4: Student-Centered Evaluation Form of Research Experience
| CIRCLE ONE: 1=strongly agree; 5=strongly disagree | ||||||
| 1. Student was adequately prepared academically | 1 | 2 | 3 | 4 | 5 | n/a |
| 2. Student was engaged in the research project and took initiative | 1 | 2 | 3 | 4 | 5 | n/a |
| 3. Student worked cooperatively with other research assistants | 1 | 2 | 3 | 4 | 5 | n/a |
| 4. Student accepted feedback constructively | 1 | 2 | 3 | 4 | 5 | n/a |
| 5. Student contributed effort to establish rapport with me | 1 | 2 | 3 | 4 | 5 | n/a |
| 6. Student seemed comfortable working on my research project | 1 | 2 | 3 | 4 | 5 | n/a |
| 7. Student showed interest in graduate study and research | 1 | 2 | 3 | 4 | 5 | n/a |
| 8. Student’s interest in graduate study and research increased as a result of experience | 1 | 2 | 3 | 4 | 5 | n/a |
| 9. I would like to stay in touch with this student | 1 | 2 | 3 | 4 | 5 | n/a |
| 10. My experience with this program was positive | 1 | 2 | 3 | 4 | 5 | n/a |
| 11. I would be willing to mentor a student next year | 1 | 2 | 3 | 4 | 5 | n/a |
| Other comments:
|
||||||
Table 5: Student’s Performance Evaluation
| Short description of research performed by student:
|
||||||
| CIRCLE ONE: 1=strongly agree; 5=strongly disagree | ||||||
| Knowledge Application | ||||||
| Student applied knowledge to solve problems | 1 | 2 | 3 | 4 | 5 | n/a |
| Student searched independently | 1 | 2 | 3 | 4 | 5 | n/a |
| Problem Solving Skills | ||||||
| Student identified a real world problem as a member of a certain class of problems | 1 | 2 | 3 | 4 | 5 | n/a |
| Student was able to see underlying connections between concepts from different subject areas | 1 | 2 | 3 | 4 | 5 | n/a |
| Lab Skills | ||||||
| Student properly used laboratory equipment | 1 | 2 | 3 | 4 | 5 | n/a |
| Student followed laboratory safety procedures | 1 | 2 | 3 | 4 | 5 | n/a |
| Student designed and conducted tests | 1 | 2 | 3 | 4 | 5 | n/a |
| Student analyzed results testing | 1 | 2 | 3 | 4 | 5 | n/a |
| Student kept a lab record and gathered data | 1 | 2 | 3 | 4 | 5 | n/a |
| Teamwork Skills | ||||||
| Student was able to give and receive constructive criticism | 1 | 2 | 3 | 4 | 5 | n/a |
| Student took charge of and completed assigned tasks | 1 | 2 | 3 | 4 | 5 | n/a |
| Communication Skills | ||||||
| Student presented effectively | 1 | 2 | 3 | 4 | 5 | n/a |
| Student was able to write effectively | 1 | 2 | 3 | 4 | 5 | n/a |
| Ethics | ||||||
| Student was able to understand professional and ethical responsibilities | 1 | 2 | 3 | 4 | 5 | n/a |
| Subject Knowledge | ||||||
| Student gained knowledge of current issues in discipline | 1 | 2 | 3 | 4 | 5 | n/a |
| Comment on student’s strengths and weaknesses:
|
||||||
| Other comments:
|
||||||
Table 6: Research Attitudes Evaluation Rubric
|
||||
| THE PROCESS | ||||
| Beginning | Competent | Excellent | Evidence | |
| Thesis | Develops a relevant thesis for the course and assignment | Develops a manageable scope and focus; poses an interesting question or problem | Modifies thesis to incorporate initial findings and surprising insights | Paper/Project Bibliography |
| Search Tools | Uses basic tools such as the library catalog, search engines, and full text databases | Uses more tools, such as disciplinary databases, electronic journals, reference indexes or bibliographies | Uses complex tools, archival finding aids, and specialized databases | Paper/Project Bibliography |
| Search Techniques | Uses keyword searching | Modifies searches iteratively; identifies new keywords including synonyms, related terms, variant spellings; uses subject headings; follows footnotes and references | Modifies searches iteratively; uses search limits; identifies key authors; follows footnotes and references iteratively; uses cited reference searching | Paper/Project Bibliography |
| Library Collections | Uses school’s collections superficially | Digs deeper into school’s collections and services | Takes advantage of school’s collections and services to their fullest | Paper/Project Bibliography |
| THE SOURCES | ||||
| Beginning | Competent | Excellent | Evidence | |
|
Types and Formats
|
Uses basic sources: books, websites | Adds other source types used in the discipline | Seeks out a variety of source types used in the discipline |
Paper/Project
Bibliography |
| Possible source types include books, articles, conferences, government documents, dissertations, archives, manuscripts, technical reports, working papers, statistics, data sets, and audiovisual materials | ||||
| Depth and Breadth | Finds obvious references from familiar sources | Finds references from a variety of sources and disciplines | Finds references from multiple perspectives, pursuing comprehensive coverage | Paper/Project Bibliography |
|
Evaluation and Selection
|
Assesses the relevance of sources, which generally apply to the topic | Evaluates the quality of sources, which generally support the thesis | Carefully evaluates the quality of sources, which strongly support the thesis |
Paper/Project
Bibliography |
| Evaluation of the sources would take into consideration the authority, accuracy, currency, coverage, and potential bias in the sources as well as the overall appropriateness and relevance to thesis | ||||
| Citation Style | Cites all materials, but not in a standard or consistent way | Cites or credits all materials in a consistent way, for the most part | Cites or credits all materials correctly in a standard format, including annotations or notes as appropriate | Paper/Project Bibliography |
| THE PROJECT | ||||
| Beginning | Competent | Excellent | Evidence | |
| Synthesis | Connects several ideas from a few sources to the thesis | Draws on multiple ideas from several sources to form conclusions | Synthesizes ideas from many sources to reach original conclusions or novel insights | Paper/Project Bibliography |
| Originality | Interesting topic but not very original | A highly imaginative topic or approach | A new twist on previous research; an original contribution to the field | Paper/Project Bibliography |
| THE LEARNING | ||||
| Beginning | Competent | Excellent | Evidence | |
| Understanding | Develops a basic understanding of research | Develops a deeper understanding | Develops a thorough understanding that seems likely to persist | Paper/Project Bibliography |
| Initiative | Closely guided by faculty and/or support staff at all stages of research | Fairly independent throughout, though sought advice when necessary | Highly independent throughout, though sought advice when necessary | Paper/Project Bibliography |
| THE LAB WORK (WHEN APPLICABLE) | ||||
| Beginning | Competent | Excellent | ||
| Participation | Did the lab but did not appear very interested. Sometimes provides useful ideas when participating in the group and in classroom discussion. A satisfactory group member who does what is required. | Used time effectively. Stayed focused on the experiment most of the time. Usually provides useful ideas when participating in the group and in classroom discussion. A strong group member who puts in the effort! | Used time well in lab and focused attention on the experiment. Routinely provides useful ideas when participating in the group and in classroom discussion. A definite leader who contributes a lot of effort. |
|
| Problem Solving | Does not suggest or refine solutions, but is willing to try out solutions suggested by others. | Refines solutions suggested by others. | Actively looks for and suggests solutions to problems. | |
| Safety Procedures and Directions | Lab is carried out with some attention to relevant safety procedures and directions. | Lab is generally carried out with attention to relevant safety procedures and directions. | Lab is carried out with full attention to relevant safety procedures and directions. | |
| Working Area and Equipment | Had to be reminded to clean up area and equipment and/or to return materials. |
Good job on cleaning up working area and equipment. Returned all materials appropriately. |
Outstanding job cleaning up working area, and equipment. Returned all materials appropriately and responsibly. |
|
The previous tables provide evaluative tools for mentees’ learning outcomes. However, if a mentor chooses to devise his or her own evaluation rubric, the following table provides guidance on how to do so.
Table 7: Stages in the Process of Developing an Evaluation Rubric
| Stages | Comment |
| Decide the Dimensions of Performance or essential elements that must be evident in high quality work (these will become your rows). | Rule of thumb: If a student can score highly on all dimensions but not score well overall, you have the wrong dimensions. Revise. Discuss with colleagues. |
| Decide the levels of achievement—number and type (these will become your columns). | Ways of describing various levels of mastery include: – advanced, proficient, functional, developing; – sophisticated, competent, partly competent, not yet competent; – exemplary, proficient, marginal, unacceptable; – others that you choose (between 3 and 5 levels). |
| For each Dimension of Performance, distinguish between acceptable and unacceptable (failing) performance. | Write the criteria for acceptable performance clearly and unambiguously. |
| For each Dimension of Performance, write clear performance descriptors (criteria) at each achievement level. | Try to determine qualitative and quantitative differences that characterize work or performance at the different levels. |
| Include (if possible) the consequences of performing at each level. | For example, the standard of the work would (or would not) be accepted by the profession or a business (as in a charter) or a professional journal (as in publication guidelines), etc. |
| Add the rating scheme you will use and apply any weighting. | Consider: – awarding grades (analytical scheme) or not (holistic scheme). There are arguments for each of these approaches (see below). It’s useful to discuss this with your teaching team to ensure a consistent approach. – including weighting criteria if required. – whether grades should be awarded for work below the minimum standard. – the criteria for ‘failure’. |
| Evaluate and revise accordingly. | Few rubrics will be constructed perfectly the first time. They are developmental tools and need to be critically evaluated after use. |
Mentoring is a dynamic process. Therefore, if the mentor detects weaknesses in the mentee’s performance or attitudes during the research experience, the mentor should address those issues immediately and not at the end of the project when the evaluation is written. By including periodic assessment strategies in the research plan prior to progress meetings, the mentee is given the opportunity to address problems and realign the research project. This type of formative assessment allows for course correction while the summative assessment serves as the overall evaluation of the project.
