Paper Reviews
Written paper reviews are a way to ensure you do the readings, and serve as the starting seed for class discussion. Anything you write in your review is fair game for me to bring up in class, so write as though all of your fellow students can see your reviews.

Reviews are due by 10 PM the night before lecture. This is to allow me time to digest what you all have written and adjust the next day’s discussion accordingly. You are welcome to meet in small groups to discuss papers, but each student must submit his or her own review.

Some lectures have more than one reading assignment. If you have to choose one among those,
please read and submit reviews for papers with ® mark, as we will discuss them in the class.

There is no late policy for reading assignments
. We will grade the best 20 reviews out of 26+ reading assignments. (I am changing this to 5 reviews in Fall 2009 to reduce the workload; however, I will expect higher-quality reviews that demonstrate a deep understanding of the material.)  

Review Guidelines
Bill Griswold, a first-rate software engineering researcher, has some excellent advice on how to read an engineering research paper. Keep his eight questions in mind and actively try to answer them as you read. If you cannot answer those questions by the time you are through, you have not truly read the paper. Try again or use the class mailing list to ask your classmates for help or clarification.

You can follow this format if you like. You will write four short paragraphs addressing the following points. Long reviews are not necessarily good reviews. Please limit your review to one page at most.

    1.    Stated goals and solution. What problem are the authors trying to solve? What are the bounds on this problem, i.e., what are they not trying to solve? What techniques or tools do the authors offer to solve the problem at hand? How do the authors know they have solved the problem? Do the authors test or validate their approach experimentally? Does the solution meet the stated goals, or does it fall short in some way? Avoid simply quoting the authors’ own abstract. Restating in your own words demonstrates your understanding.
    2.    Cool or significant ideas. What is new here? What are the main contributions of the paper? What did you find most interesting? Is this whole paper just a one-off clever trick or are there fundamental ideas here which could be reused in other contexts?
    3.    Fallacies and blind spots. Did the authors make any assumptions or disregard any issues that make their approach less appealing? Are there any theoretical problems, practical difficulties, implementation complexities, overlooked influences of evolving technology, and so on? Do you expect the technique to be more or less useful in the future? What kind of code or situation would defeat this approach, and are those programs or scenarios important in practice?
    4.    Note: we are not interested in flaws in presentation, such as trivial examples, confusing notation, or spelling errors. However, if you have a great idea on how some concept could be presented or formalized better, mention it.
    5.    New ideas and connections to other work. How could the paper be extended? How could some of the flaws of the paper be corrected or avoided? Also, how does this paper relate to others we have read, or even any other research you are familiar with? Are there similarities between this approach and other work, or differences that highlight important facets of both?
Please take the time to edit your reviews. Unclear or unnecessarily long prose will be graded accordingly. I credit Ben Liblit at the University of Wisconsin-Madison for a concise instruction on how to write good reviews in a graduate seminar course. 

Review Formatting
Please compose your review as plain ASCII text: no Microsoft Word, no PDF, just plain text. No attachment please.
Review Submission
Submit reviews using the UT blackboard system. Reviews are due by 10 PM the night before lecture.

Review Grading
3 points:
Demonstrates exceptional insight about paper or related work. Noteworthy effort above and beyond just reading the assigned paper. Few if any 3-point grades will be awarded for each paper.
2 points:
Clear and concise, demonstrating understanding of the key concepts of the paper. Ideas presented in your own words. Some evidence that the paper has been considered in the context of larger issues and themes of the course.
1 point:
Shallow, minimally-sufficient, or needlessly wordy. Key concepts misunderstood or missing. Author’s words echoed back to me with little effort to reinterpret or paraphrase.
0 point:
Late, incomplete, or never submitted at all.
There is no late policy. For final grade, the best 20 reviews out of 26+ reading assignments will be selected.