Twitter is a microblogging service that lets users send and receive short messages, called tweets. It is written in a combination of Ruby on Rails and Scala, but interfaces to it are available from many other languages.
Your software development team is developing several Twitter-based applications, and you've been asked to help build a test environment for Twitter apps. For both competitive and technical reasons you don't want to test directly with Twitter's own web site; instead, you want to build a substitute web site that you can test against. You also want the ability to build a Twitter proxy; that is, a web site that can pretend to be Twitter, and which is a proxy for the true Twitter service, but which logs all messages going to and fro, in an intelligent way.
You've been delegated to look into the Twisted event-driven networking framework as a candidate for building your Twitter test framework. Your boss thinks that this might be a good match for the problem, since Twisted's event-driven nature should allow for convenient integration of the various pieces in the Python environment. However, he doesn't know how well Twisted will really work in practice. In particular, he wants to know how easy is it to write applications using Twisted, and how maintainable and reliable those applications will be. He wants you to dig beyond the hype and really understand the pros and cons of using Twisted.
You dig around the net and find Twitty Twister, which is a Twisted-based library that lets you access the Twitter API from Twisted applications.
Figure out how to run Twitty Twister from the Unix shell. Run its "search" example, search for the string "Hello world", and record its results.
Modify the search example so that instead of merely reporting each matching tweet, it also reports all other information that is easily available about the tweet, such as its time stamp.
The current snapshot doesn't support the API methods trends, trends/current, trends/daily, and trends/weekly. Add support for these, and supply an example (just as there is already support and example for search).
Explain how your example works. Go through the overall flow of execution. Describe not only the code that you wrote, but also the code in Twitty Twister that gets exercised, and the code in Twisted that gets exercised.
Do some more research on Twisted as a potential framework for this kind of application. Your research should include an examination of the Twisted source code and documentation, and a small prototype or example code of your own that demonstrates whether Twisted would be an effective way to implement a test framework. Please base your research on Twisted 8.2.0 (dated 2008-12-28), even if a newer version comes out before the due date; that way we'll all be on the same page. (We suggest using Python 2.6.2, as Twisted 8.2.0 is designed for Python 2.6.x and we haven't tested it with Python 3 yet.)
Write a prototype Twitter proxy server in the Twisted framework, using pure Python. Implement at least the search and trends (including trends/current, etc.) APIs described above; if you have some problems implementing the rest of the API, describe them in your report. Your server should log as necessary into a file, using a format of your design. You can use the logs' data in your reports.
Test your prototype by running the "search" and "trend" client examples against it. If you need to modify the search client, explain why the modifications are needed.
If you can think of some similar project you'd like to do, that resembles the current project but is cooler, ask the T.A. for permission to do the similar project, and then go for it.
Write a report that summarizes your research, recommends whether Twisted is a suitable framework for this kind of application, and justifies your recommendation. Describe any problems you ran into.
Prepare and give a brief oral presentation of your research, and present it in discussion section. Please make sure your talk fits in the time allotted, as we don't want to cut you off abruptly. Focus on the parts you did differently from everybody else; you need not, for example, introduce Twitter or Twisted in your talk, or talk about the standard examples.
Your research, report, and presentation should focus on language-related issues. For example, how easy is it to write in Python Twisted-based programs that run as Twitter proxies? What are the performance implications of using Twisted? Don't worry about nontechnical issues like licensing, or about management issues like software support and retraining programmers.
Your report and talk are an important part of this assignment: they are not mere afterthoughts. Please see Resources for oral presentations and written reports for advice on generating high-quality reports and presentations.
Your report should use standard technical academic style, and should have a title, abstract, introduction, body, recommendations/conclusions, references, and any other sections necessary. Limit your report to at most five pages. Use the USENIX '09 style, which uses a two-column format with 10-point font for most of the text, on an 8½"×11" page; an example of the output format and an example student paper are available.
Your paper is not expected to be just like that example student paper! That was written by a graduate student, she was writing a conference paper describing months of full-time research, and the paper is too long for us. It's merely an example of technical style and layout.
If you need to run servers on SEASnet to do your research, please let the TA know how many TCP ports you need, and we will allocate them for you. Please do not use TCP ports at random, as that might collide with other students' uses.
You can grab a copy of the Twisted source code from the Twisted web site. You can also find a copy of the Twisted source code in the directory ~eggert/src/Twisted-8.2.0 on SEASnet; a distribution compressed tarball is in the file /u/cs/fac/eggert/src/tarpit/Twisted-8.2.0.tar.bz2. A copy of Twitty Twister as of 2009-05-19 is in the directory ~eggert/src/twitty-twister/.
You can run Twisted on SEASnet by prepending /usr/local/cs/bin to your PATH in the usual way.
Submit a file named report.pdf containing a copy of your paper in PDF form.
Submit a file named talk.pdf containing a copy of your presentation's visual aids in PDF form. Also submit a file containing the original format of the aids (talk.odp, talk.ppt, etc.), so that we can have them ready to go for your talk all using the same laptop, and not have to waste time connecting your laptop to the projector. Please consult with the TA well before the talk if your visual aids require special software (e.g., if you have a prototype server that you wish to demonstrate).
Submit any other supporting work (e.g., your source code) in a gzipped tar file named project.tgz.
The following references are all Safari online books, so their contents should be viewable by anybody on the UCLA campus.
Abe Fettig, Twisted network programming essentials, O'Reilly (October 2005), ISBN 0-596-10032-9.
Kevin Makice, Twitter API: Up and Running: Learn How to Build Applications with the Twitter API, O'Reilly (March 2009), ISBN 978-0-596-15461-5.
Tim O'Reilly & Sarah Milstein, The Twitter Book, O'Reilly (May 2009), ISBN 978-0-596-80281-3.