ND Home | COE Home | Site Map

Course Project 1 Suggestions: MEASUREMENT

Fundamentally, Operating Systems manage tradeoffs. These tradeoffs are driven by observed user behavior. For example, if all the processes were observed to be short, then the OS schedulers are optimized for short tasks. Note the chicken and egg problem here - the first studies don't have enough users and hence initial policies are built based on unrealistic models. Once you optimize the OS for small tasks, future application writers change their program that could have benefited from large tasks to instead use small tasks so as to achieve good performance. Thus, the initial workload defines future systems (perhaps unintentionally).

Regardless, there had been some classical measurement papers which have heavily influenced systems development. For example, the file system measurement study (scheduled for 11/4/08) have had enormous impact on file system study. An excellent presentation on how to measure Computer Systems was presented by Prof. Seltzer @ Harvard (PDF). A word of caution, as half of a course project, I am not expecting you to measure something to the same scope as what Prof. Seltzer mentions. Regardless, a simple measurement project that is scientifically valid is preferable to a large measurement project where we have no idea on what is being measured.

Measuring systems tend to be the first step before proposing solutions to solve some problem. On the other hand, it pays to be open to go where the data takes you. The key to a successful measurement project is: a) have a clear idea on what you want to measure. Try to understand what you are measuring, what the expected behavior will look like and how you might recognize such a behavior. Try to think of what might be surprising and what might be expected. b) Device a data collection methodology which will capture your system. This involves researching some exiting tools to measure the system. More importantly, you need to clearly understand the limitations of what you are collecting. You ability to convince others that you had measured something interesting depends on your ability to convince others that your collection methodlogy was sound. For example, suppose you want to measure file system write calls. Suppose that you observe that a particular system was creating a million writes a second. You need to clearly understand (and be able to explain) how your measurement system did not perturb the system and that the million writes are not really caused by you (say by your data collection tool logging write updates to the same file system). c) Clearly describe what you learnt from the collected data. Elementary statistics suggests that you need to repeat experiments in order to produce statistically valid results). Such an understanding will make the solution to this problem fairly obvious.

  • Compare the performance of two different OS: Many students have opinions about the various operating systems (Windows, Linux, FreeBSD, Mac OSX etc.) One project is to put their prejudices to the test. Choose comparable hardware (or the same hardware), perform the same operation under both the operating system and try to understand the system behavior.
  • Measure the Idleness of machines: Modern machines are likely to be continously online. They also run daemons, virus scanners and other stuff. Measure what really goes on when a system is idle. This might help you decide on how to optimize machines to run longer while "idle"
  • Measure file system activity in your favorite scenario: There hasn't been much research on the file system access of modern devices. An interesting project would be to measure file system activity in Servers/desktops/laptops/iPhones/PDAs etc. You need to find the right tool and the right demographics (e.g., file system activity by CS grads, CS ugrads, CS secretaries etc.). You can measure the typical behavior, idle behavior, behavior under heavy load etc.
  • Measure availability characteristics of your favorite device: (FARSITE from Microsoft research reported a long term study of the behavior of corporate desktops). You can measure the availability behavior of certain class of machines (e.g., lectern PCs)
  • You get the idea, measure energy consumption, user availability, you name it - you can do it - just drop me a note and discuss before you go into full swing.

Note that, once you started working on project 1 and find that you can extend it to perform a more thorough measurement that might be useful/publishable on its own right, talk to me.

Surendar Chandra
Last modified: 09/23/2008 9:14