CPSC 508 Operations Background Schedule Reading Assignment Project Honesty
Home Operations Background Schedule Reading Assignment Project Honesty

Paper Reading Roles

This page contains information about the three roles a student will assume when reading the papers. For each day of class, each student will be a heavy reviewer for one paper and a light reviewer for the other. Among the heavy reviewers we select the discussion lead who helps to get the in-class discussion starting.

Every student will read every paper and is expected to actively participate in the discussions, both in-class and online. Be specific!

We list the roles in the assignment description in Canvas.

Light Reviewer

The task of the light reviewer is to read the paper and answer the following questions.

  • What kind of paper is this? Try to classify the paper. Is it a big vision paper? Is it a description of a system? Is it a survey? Something else?
  • Paper Summary (i.e., the fairytale) Provide a brief summary of the paper. The purpose here is to demonstrate that you've read (and understood) the paper. See paper summary of the heavy reviewer below for more details.
  • What did you learn? After having read the paper, reflect and think about what was the most surprising or interesting thing you've learned from the paper?
    Be specific. "I learned about system X" is too superficial. Rather say "I learned how system X uses concept Y to do Z."
  • What did you like / dislike the most? After having read the paper. There might have been things that you thought, "wow that was super cool" or "oh that was super disappointing". What was the single thing that you liked or disliked the most, and why?
    Be specific. "I didn't like the figures" doesn't say much. Rather say "I didn't like the overview figure, because it didn't help me understand the architecture described in the paper."
  • Questions for the Authors? Each paper usually has a conference presentation where the audience can ask questions about the paper to the presenter. After having read the paper, formulate at least two questions you would have liked to ask the authors. You should be specific to the paper.
  • What didn't you understand? Is there something technical that you didn't quite understand that may be worthwhile discussing in class?

Heavy Reviewer

The task of the heavy reviewer is to read the paper and provide a more detailed write up about the paper. It should cover the following points:

  • What kind of paper is this? Try to classify the paper. Is it a big vision paper? Is it a description of a system? Is it a survey? Something else?
  • Paper Summary (The Fairytale) Provide a brief summary of the paper. The purpose here is to demonstrate that you've read (and understood) the paper. Try to paraphrase and extract the essentials. Be specific!
    Write it as a fairy tale: "Once upon a time, ... . ... . ... . ... , and they lived happily ever after."
    Focus on the four parts (no more than 1-2 sentences for each part):
    1. Once upon a time... (Set the context of the paper)
    2. The Villain: What is the problem the paper tries to solve
    3. The Hero: How does the paper solve the problem?
    4. The happily ever after: What can we do now, how are the results improved, ...?

  • The Problem (Villain) Expand upon the summary and objectively elaborate on the problem the paper tries to solve. What is the problem and why is it important?

    Why has the problem not been solved before? The fact that the problem exists is an indication that previous work was insufficient in solving it, or it can be that the authors have identified a new problem. If the paper doesn't seem to address a particular problem, what is the primary motivation of the paper?

    This is a bit of a more detailed take on point two (the villain) of the fairy tale. Write about 4-8 sentences addressing the questions above (you can use bullet points with sentences). Try to paraphrase and extract the essential details from the paper.

  • The Solution (Hero) Expand upon the summary and objectively elaborate on the approach the authors took to provide a solution to the problem. How does the solution solve the problem? Is it unique or innovative? Is there an insight that has lead to the proposed solution?

    Again, this is a bit of a more detailed take on point three (the hero) of the fairy tale. Write about 5-10 sentences addressing the questions above (you can use bullet points with sentences). Try to paraphrase and extract the essential details from the paper.

  • Evaluation (Happily ever after) Expand upon the summary and objectively elaborate on the evaluation of the solution presented in the paper. What are the questions the authors wanted to answer and how did they go about evaluating their solution -- does it actually answer the questions? Extract the strengths and weaknesses of the solution. Are there even strengths and weaknesses in the evaluation itself -- is something missing?

    Again, this is a more detailed take on point four (the happily ever after) of the fairy tale. Write about 3-5 sentences focusing on the highlights of the evaluation (you can use bullet points with sentences).

  • What do you think? So what's your verdict about the paper? You can subjectively make a judgment!

    Did you like the paper, or did you dislike it? What precisely did you like or dislike and why? What would you have done differently?

    What do you think about the idea proposed in the paper?

    Try to construct an argument here that should convince others of your opinion. Write at least three sentences (you may write more if you like, and you may use bullet points.)

  • What didn't you understand? Is there something technical that you didn't quite understand that may be worthwhile discussing in class?

Discussion Lead

One of the heavy reviewers of a paper will become discussion lead. The discussion lead should prepare one slide that summarizes the main points of the paper to start the discussion in the class. Note, this is not a presentation! It's meant to be a quick refresher about the paper to get the discussion started.

Why are we doing this?

One of my colleagues that served with em on a program committee, when they were discussion lead for a paper usually prepared a single slide summary to get the paper discussion started. The slide contained information what the paper is about, the strengths and weaknesses the reviewers have identified and the scores that were assigned. This gave the rest of the PC a good overview of the paper.

What to include

Prepare to present the paper summary (i.e., its fairy tale -- no slide for this). Start with "Once upon a time, ..."

Read through what your peers wrote and combine this with your notes! Tell us what your colleagues and you think the kind of the paper is. Then, have one slide that has the following information:

  • Summarize three things liked about the paper
  • Summarize three things disliked about the paper
  • Summarize three interesting points made by other reviewers
  • Two interesting questions for the authors
  • Technical Questions: collect things that weren't understood

This shouldn't take too long.