Evaluate-Compete: A Method for Assessing Student Outcomes in MATE’s ROV Competitions

Abstract:

MATE (Marine Advanced Technology Education) is an NSF-supported National Center that focuses on ocean themes to prepare students for technology-based careers. MATE organizes and manages regional and international underwater robotics competitions for middle school, high school, and college student teams. Working with faculty advisors, teams design, construct, deploy, and operate remotely operated vehicles (ROVs) in situations that require them to overcome a variety of obstacles. The ability to compete successfully requires students to understand and employ in a practical setting a wide variety of knowledge and technical skills that are critically important in the workplace. With funding from the NSF ATE program, MATE is adapting EvaluateUR, a method used to evaluate undergraduate research, for its use in evaluating the impact of its ROV competitions. The new method is called Evaluate-Compete. In common with EvaluateUR, Evaluate-Compete relies on an assessment with about a dozen major outcome categories each defined by several components. The components are scored using a 5-point rubric that reflects the frequency of the particular component. Students and mentors independently complete the assessments several times during the experience and afterwards meet to discuss their reasons for assigning a score. The conversations are an essential feature of the method and are intended to complement and extend the mentoring process, while at the same time helping students to further develop their metacognitive abilities. Evaluate-Compete’s design expands upon the feedback students generally receive through their participation in ROV competitions by making explicit the technical knowledge and range of skills they should be acquiring. This presentation provides background information on EvaluateUR, including its success in demonstrating gains in student learning, and describes how it lays the foundation for Evaluate-Compete. It also includes information about the MATE ROV Competition and the specific technical and workplace skills that it imparts to its student participants.