Lesson 2: Scoring Engineering Notebooks

Now that you know what teams submit, this lesson covers how you evaluate it. The Engineering Notebook Rubric gives you a set of descriptions to match against what is on the page, which is what keeps scoring consistent from one judge to the next and from the first notebook of the day to the last.

Download a copy to follow along throughout this lesson: Engineering Notebook Rubric – Letter Size or A4 Size.

How the Rubric is Organized

The rubric has five sections and sixteen criteria. Each criterion is one row, and each row is scored from 1 (Beginning) to 4 (Exemplary).

The three middle sections follow the Engineering Design Process used throughout the VEX Robotics curriculum. Together with Reflection and Iteration, they account for 48 of the 64 possible points. Organization matters, but most of what you score is the team's iterative work through the design process.

You score every row. A submitted notebook earns at least 1 on each criterion, so totals run from 16 to 64.

What the Rubric Is Looking For

You are reading for evidence that the team went through the Engineering Design Process: it defined a problem, tried things, learned from what happened, and improved. The Engineering Design Process is not a specific, sequential set of steps. The process can appear messy to those unfamiliar. Teams move freely between and among phases throughout the season. 

You are looking for the process itself, not for the words Define, Develop Solutions, and Optimize to appear on the page. A team that never labels a section "Optimize" may still document optimization thoroughly. The sixteen criteria in the rubric break the Engineering Design Process into pieces so that two judges reading the same notebook should arrive at similar scores.

Scoring a Criterion

Each row of the rubric gives you four descriptions. Your task is to decide which one matches what the notebook shows. Here is one row in full, from the Engineering Design Process – Develop Solutions section.

Criterion: Test procedures are defined

Reading across the row, the difference between levels is how much of the test the notebook actually captured.

• At Beginning, you can tell the team tried the robot, but there is no test to speak of.
  • Ex. “Ran the intake, seemed better.”
• At Developing, the team says it tested. What it measured, and how, is missing.
  • Ex. “Tested the intake today.”
• At Proficient, you know what was measured and how. 
  • Ex. Timed how long the intake took to pick up ten blocks."
• At Exemplary, you also know the conditions, closely enough that you could run the test yourself. 
  • Ex. “Timed the intake picking up ten blocks from the same starting position, three trials each, at full battery.” 

Some criteria ask about the whole notebook. Most rows are answered by finding the right entry. A few (whether the record spans the season, whether reflection appears throughout, whether iteration shows up in sequence) are answered by the shape of the notebook as a whole. For those, you are looking at whether the pattern holds across the season rather than at any single entry.

A few habits make this evaluation process go smoothly:

• Start with the Season Summary. Teams who submit a Season Summary are highlighting the areas of their notebook that they want you to pay special attention to. You do not need to start every notebook evaluation at page 1 or read every page of a notebook. The Season Summary guides you to what pages you should look at.
• Match the description, not your impression. When you are torn between two levels, reread both descriptions and pick the one the notebook actually meets. You can also ask the Judge Advisor for guidance or advice as needed.
• Find evidence to score and move to the next. If you find clear evidence to meet Exemplary for that criterion, you do not need to keep reading for that piece of information. You have a limited amount of time and are not required to read each notebook from beginning to end.
• Score the row you are on. Testing shows up in more than one criterion. Judge each row against its own description rather than carrying a general sense of the notebook across all sixteen.
• Score the record, not the robot. A team with a simple robot can document its process exceptionally well, and a team with an impresive robot can leave that process undocumented.
• Hold one standard all day. The first notebook you read and the last should be scored the same way. Your Judge Advisor will set a time limit that applies to every notebook.
• Use the Credit Summary when you reach the criterion on crediting outside sources. It tells you what to expect to find, which is faster than searching for credits yourself.

Remember that another judge is also reading each notebook, so you are not the only reader forming a view. If your scores differ on a criterion by more than one level, that is worth a conversation rather than an average. Your Judge Advisor can discuss how this happens.

Reading Across Different Teams

Notebooks arrive from teams that look nothing alike, even at the same event. A first-year team with two students and a long-term program with eight team members who have been participating since elementary school are both documenting the same design process at different levels of sophistication.

The rubric asks whether the process is documented, not how advanced the engineering is. That keeps scoring fair across very different teams, as long as you keep a few things separate from the score.

• Volume is not evidence. A hundred pages of meeting notes without decisions or testing, scores lower than twenty pages that capture both. A two-person team's notebook may be shorter, and shorter is not necessarily fewer points.
• Vocabulary is not understanding. A younger student describing why a wheel kept slipping is documenting the same reasoning as an older student using technical terms.
• Polish is not process. A formatted binder and a pencil sketch can each earn Exemplary on the same row.

What you are asking each time is the same question: does this notebook show the team's thinking?

From Scores to Awards

Scores stay within the judging team. Teams do not receive them, and they are not shared with coaches, Event Partners, or other volunteers. The score gives the judging team a way to rank notebooks so that a conversation can start. Final decisions come from discussion rather than from totals alone. Two notebooks a single point apart are not meaningfully different. The Judge Advisor will help guide judges through that deliberation process after teams have been interviewed.