Option
(B): Literature Survey and
Tool Evaluation
The
following
is a more detailed description of the project timeline
and requirements:
Now:
Topic Selection
Start
thinking about what kinds of projects interest you and with whom
you might like to collaborate. I suggest taking a look at the course
schedule and talking with other students about mutual areas of
interest. You might also check out the list of suggested
literature survey topics on
the blackboard system. If you would
like me to help
you find a survey topic of your interests, please let me know.
1/27
Tuesday: Option and Group Selection (9:00 PM)
Please
send the course staffs an email regarding
which option you will
choose. If you choose to work in teams, please include the names and
email addresses of all the people
in your group, as well as one or two sentences describing possible
topics that you might be interested in pursuing. Please remember
that a literature survey cannot be done in team.
2/3
Tuesday: Literature Survey Proposal & Tool Evaluation Proposal
(9:00 PM)
*Literature Survey*: A
student must submit a two page proposal describing which topic you
would like to survey about. I expect to see sections on:
- Problem definition and
motivation. What are the goals of your literature survey
and why are these goals important?
- An initial discussion of
related work and a list of key papers:
select five papers
that
contain related ideas but are distinct from each other. The proposal
should provide reasons why you selected each paper.
- A list of milestones and
dates.
The
student should select a number of papers that contain ideas that are
distinct from each other, but are related in some way. The papers
should be research papers and not survey papers. Uplaod the proposal to
the blackboard system.
*Tool Evaluation*: Each
group should submit a short proposal about which software engineering
tool you would like to evaluate. A proposal must include a
sketch of your evaluation approach.
3/13
Friday: Tool Evaluation Report Due (9:00 PM)
Each group should submit a
tool evaluation report (max 10 pages) to the blackboard
system. Everyone should also submit a short summary of their
contributions to
the tool evaluation project (not required for groups of size 1). This
should be at most
a page long and can reference the final report.
- Background:
- What is the goal of the tool?
- Who can use the tool for doing what, and when and how?
- Describe a scenario of how the tool can be used in pratice
- Evaluation plan
- the objectives of evaluation
- a set of experienments or studies you designed to evaluate the
tool and why
- a set of subject programs (data sets) you used and why
- Evaluation results
- comparison between what you expected the tool to do and what
the tool actually did
- discuss strengths and provide rationales about why the
tool demonstrated strengths on certain cases (try to connect your arguments to the
algorithm described in the tool's white papers / research papers)
- discuss weaknesses and provide rationales about why the tool
did not work well on certain cases (suggest
your ideas on how to improve and augment existing algorithm)
- Future research directions
4/21
Tuesday: Literature Survey Draft Due (9:00 PM)
Each
student should submit a draft of their written literature survey. The
formatting of the draft report should match that of a final report.
(See below for what a final report should look like.) The draft should
be uploaded to the blackboard system.
It's
OK if you haven't completed your research by now. See the next
bullet for why you're turning in a draft two weeks before the final
report is due. Your draft should clearly specify what you plan to do
over the next two weeks.
4/28
Tuesday: Peer Reviews Due (9:00 PM)
We
will distribute your literature survey draft to other students in the
class, with the
goal of simulating a mini program committee review process. This means
you will also get comments from your peers. Consequently, you will all
have the opportunity to read and review drafts from other groups. This
process can be very educational unto itself, much akin to a miniature
"program committee meeting."
You
will be expected to read the draft reports that I give you (at most
three) and write detailed reviews of those papers. You are to upload
those reviews to the blackboard system by the above-specified deadline.
The reviews you write should be anonymous (i.e., not include your name
or other identifying information).
I
will then collect those reviews and send them to the authors
of the
relevant reports. There are several reasons we're doing this, but the
main goals are to (1) help you (as reviewers) gain more experience in
evaluating in-progress (as opposed to completed) research and (2) help
everyone improve the quality of their final written report.
5/3
Sunday: Final Report and Electronic Presentation Due (9 PM)
Each student will submit a written literature survey
(max 10 pages), as
well as a slide deck, to the blackboard system. Please submit the
report
and the slide deck as separate PDF files. You may include an
appendix beyond 10 pages, but your paper
must be intelligible without it. Submissions not in the ACM
format will
not be reviewed (this is to model program committees for conferences
and workshops, which have the option to automatically reject papers if
they do not comply with the submission guidelines).
Your report should be structured like a conference paper,
meaning that
your report should contain an abstract, a well-motivated introduction,
a discussion of related work (with citations), evaluation of surveyed
research, a discussion, open problems and so on.
The written report should reflect the
critical reading of
the papers. It will
present not only the material extracted from the papers but also your assessment
of it. An ideal report would include, in roughly equal measure:
- Summaries of the selected
papers and other background papers within a general framework.
- Identification of unifying
framework, evaluation or comparison criteria.
- A critical examination of
the ideas, methods, and significance of the research described in the
papers addressing such questions as:
- What is the impact of the
research?
- How widely applicable are
the ideas and methods of the papers?
- Why were particular
techniques used? particular research choices made?
- Are there different
contexts/ideas that would cause a change in those choices?
- Discussion of open problems
and future research directions within the area.
5/4-5/6:
Presentations (3:30 PM)
Each student will give a
short
presentation of their literature survey work. I anticipate
that each presentation will
be 10 minutes.