This module leverages peer feedback/evaluations in many ways. In particular, we do several rounds of peer evaluations using TEAMMATES.
Admin Tools → TEAMMATES
Tool: TEAMMATES (for Peer Evaluations)
We use the TEAMMATES online peer evaluation system. TEAMMATES is a project run by NUS SoC students and used by over 0.5 million users from over 1000 universities.
Preparation: When the first feedback session is open on TEAMMATES, you will receive an eamil from TEAMMATES. There is nothing for you to do until then.
When you do receive that email, it will contain a unique link that you can use to access TEAMMATES without logging in first. Logging in to TEAMMATES using a Google account is optional (but doing so will allow you to see all your TEAMMATES sessions in one page).
Submitting peer evaluations is compulsory. If you routinely miss submitting peer evaluations, you can lose participation marks.
Session: Midterm Peer Evaluation
- Held about two weeks into the tP coding phase
Important questions included in the evaluation:
Some of these questions (e.g., contribution to DG) are omitted from the midterm peer evaluation but are in the final peer evaluation (they are given here for your reference)
Uses the Equal Share +/- N% scale for the answer
Uses the Equal Share +/- N% scale for the answer
Uses the Equal Share +/- N% scale for the answer
- Evaluated based on the following criteria, on a scale
Poor/Below Average/Average/Good/Excellent:
Peer Evaluation Criteria: Professional Conduct
- Professional Communication :
- Communicates sufficiently and professionally. e.g. Does not use offensive language or excessive slang in project communications.
- Responds to communication from team members in a timely manner (e.g. within 24 hours).
- Punctuality: Does not cause others to waste time or slow down project progress by frequent tardiness.
- Dependability: Promises what can be done, and delivers what was promised.
- Effort: Puts in sufficient effort to, and tries their best to keep up with the module/project pace. Seeks help from others when necessary.
- Quality: Does not deliver work products that seem to be below the student's competence level i.e. tries their best to make the work product as high quality as possible within her competency level.
- Meticulousness:
- Rarely overlooks submission requirements.
- Rarely misses compulsory module activities such as pre-module survey.
- Teamwork: How willing are you to act as part of a team, contribute to team-level tasks, adhere to team decisions, etc. Honors all collectively agreed-upon commitments e.g., weekly project meetings.
- Considered only for bonus marks, A+ grades, and tutor recruitment
- Evaluated based on the following criteria, on a scale
Poor/Below Average/Average/Good/Excellent:
Peer Evaluation Criteria: Competency
- Technical Competency: Able to gain competency in all the required tools and techniques.
- Mentoring skills: Helps others when possible. Able to mentor others well.
- Communication skills: Able to communicate (written and spoken) well. Takes initiative in discussions.
Session: Final Peer Evaluation
- Held at the end of the tP.
- This peer evaluation is compulsory. Not only it will count for weekly participation, those who don't submit will not get a chance to rebut peer evaluations received.
- This session includes all questions from the Midterm Peer Evaluation:
Admin Peer Evaluations → Session: Midterm Peer Evaluation Questions
Uses the Equal Share +/- N% scale for the answer
Uses the Equal Share +/- N% scale for the answer
Uses the Equal Share +/- N% scale for the answer
- Evaluated based on the following criteria, on a scale
Poor/Below Average/Average/Good/Excellent:
Peer Evaluation Criteria: Professional Conduct
- Professional Communication :
- Communicates sufficiently and professionally. e.g. Does not use offensive language or excessive slang in project communications.
- Responds to communication from team members in a timely manner (e.g. within 24 hours).
- Punctuality: Does not cause others to waste time or slow down project progress by frequent tardiness.
- Dependability: Promises what can be done, and delivers what was promised.
- Effort: Puts in sufficient effort to, and tries their best to keep up with the module/project pace. Seeks help from others when necessary.
- Quality: Does not deliver work products that seem to be below the student's competence level i.e. tries their best to make the work product as high quality as possible within her competency level.
- Meticulousness:
- Rarely overlooks submission requirements.
- Rarely misses compulsory module activities such as pre-module survey.
- Teamwork: How willing are you to act as part of a team, contribute to team-level tasks, adhere to team decisions, etc. Honors all collectively agreed-upon commitments e.g., weekly project meetings.
- Considered only for bonus marks, A+ grades, and tutor recruitment
- Evaluated based on the following criteria, on a scale
Poor/Below Average/Average/Good/Excellent:
Peer Evaluation Criteria: Competency
- Technical Competency: Able to gain competency in all the required tools and techniques.
- Mentoring skills: Helps others when possible. Able to mentor others well.
- Communication skills: Able to communicate (written and spoken) well. Takes initiative in discussions.
- In addition, it contains these additional questions:
Session: Responses to Peer Evaluations
- This is a chance for you to submit your objections to the ratings you received in the Final Peer Evaluation.
How peer evaluations are used
- Peer evaluations are rarely used directly to calculate marks. They are fare mostly used to flag cases that need further investigation.
- Before using peer evaluations in grading, we consider factors such as the following:
- Is there a consensus among team members? We do not want an extreme input from one team member to unduly affect the outcome. In many cases, we discard the highest and lowest ratings received before calculating the average.
- Do the other data points (e.g., LoC written, tutor feedback, commit history) corroborates the peer evaluations?
Guidelines for giving peer feedback
Giving constructive feedback to others is a valuable skill for software engineers. It is also an intended learning outcome of this module. Half-hearted/trivial feedback will not earn participation marks.
Here are some things to keep in mind:
- Assume you are giving feedback to a colleague, not a friend. Keep the tone of your feedback reasonably professional. Do not use offensive language or slang.
- The feedback should be honest and consistent. Giving positive qualitative feedback (e.g.
Thanks for all the hard work!and negative ratings (e.g.Equal share - 40%) to the same team member is not being honest. - State your expectations early. All too often students give positive/neutral feedback early (hoping that the team member will improve later) and trash the team member in the final evaluation (because the he/she did not improve as expected). However, this could be confusing to the recipient. It is better to give negative feedback early so that the team member gets a clear signal that he/she needs to improve.
Guidelines for interpreting contribution ratings
When you receive results of a peer evaluation question about contribution, use it mainly to compare the team view to your own view.
- Example 1:
Your view (of your own contribution) : E+10% i.e., 10% more than an equal share
Team view (of your own contribution): E+8%
Conclusion: The team's view is quite similar to yours. - Example 2:
Your view (of your own contribution) : E+15% i.e., 10% more than an equal share
Team view (of your own contribution): E+3%
Conclusion: The team's thinks you did significantly less than you claimed you did.