Projects #

Students may choose either the Research Project track or the Mini-Projects track to earn points for the “Project(s)” portion of the course grade.

Final project track decisions must be made by Fri, Sep 6 - 11:59pm (deadline of the first research milestone).

Mini Projects #

While all students are encouraged to choose the Research Projects track, some students may prefer a more directed, hands on set of assignments. The Mini-Projects track provides a series of smaller projects that relate more directly to the course material. There is one project for each major topic focus of the course. The projects require a range of programming as well as open-ended investigation.

Mini Projects Grading #

The Mini-Projects track is out of 400 points distributed as follows:

Below are the assigned projects as well as a example selection of possible upcoming projects in this course:

Project 1: Cryptography #

Due: Fri, Sep 13 - 11:59pm MP1

The goal of this assignment is to provide an introduction to using cryptographic APIs. Specifically, you will need to specify a secure mode of operation (we are using GCM), correctly using initialization vectors, and ensuring both message integrity and confidentiality. You will also be getting first-hand experience in how Diffie-Hellman works, and its susceptibility to middle-person attacks.

See the assignment description for more details.

Project 2: Network Security #

Due: Fri, Oct 4 - 11:59pm MP2

The goal of this project is to learn how to perform an audit on a network with the intention to discover interesting characteristics and phenomena. The project is mostly open-ended and students will be required to be creative and learn new tools.

See the assignment description for more details.

Project 3: System Security #

Due: Fri, Nov 8 - 11:59pm MP3

The goal of this project is to gain practical experience with least-privilege access control policy. Specifically, you will be exploring the sandbox permissions provided by Flatpak, an emerging package distribution format for Linux desktop applications. The project emphasizes security and usability trade-offs, as well as the importance of policy specification when creating a secure system.

See the assignment description for more details.

Project 4: Privacy #

Due: Tue, Dec 3 - 11:59pm MP4

The goal of this project is to gain a deeper understanding into mobile application privacy and to develop skills for empirical investigation. Specifically, students will use the Amandroid static program analysis tool to identify privacy leaks in Android applications. Amandroid will be applied to a small corpus of applications, and a report will classify and describe the findings.

Research Project #

The research project requires that students execute research in {systems or network} {security or privacy}. By completing the research project, students will learn to think critically about security problems and solutions. All solutions have limitations, and understanding the ramification of these limitations is critical to understanding the security and privacy of an environment.

Research Project Overview #

The research project track milestones mimic the steps required to create a conference-quality paper submission. Be realistic about what can be accomplished in a single semester. However, the work should reflect real thought and effort — projects executed in the closing days of the semester are unlikely to be well received. The grade will be based on the following factors: novelty, depth, correctness, clarity of presentation, and effort. There is no specific rubric, because some factors (e.g., novelty) can out weigh other factors (e.g., effort). The final written project will be assessed similar to how papers are reviewed for conferences.

Project teams may include groups of up to two students; groups of two will be expected to make greater progress. I will advise each team/individual independently as needed. The project grade will be a combination of grades received for a number of milestone artifacts, a presentation, and a final written paper.

Research Project Grading #

The research project track is out of 400 points distributed as follows, see also the class schedule for milestone due dates.

Milestone 1: Project Proposal (25 points) #

• Due: Fri, Sep 6 - 11:59pm RM1
• Submission: Submit your PDF via Gradescope (supports group submissions).

Milestone 1 is possibly the hardest milestone of the research project, so do not be discouraged. The purpose of this milestone is to settle on 1) a project idea/area, and 2) a project team. While the specific project may change slightly during the course of the semester in response to the related work survey and implementation/experiment findings, it is important to have a strong direction. Projects can be in any area of systems security, but must be approved by the instructor.

Example project areas include:

• cloud computing
• hardware improvements for security
• operating system security enhancements
• program security mechanisms
• web browser security
• smartphone security
• network security
• privacy

For ideas, students are encouraged to browse the last several years proceedings of the 4 top-tier security conferences:

• USENIX Security
• ACM CCS
• IEEE Security and Privacy
• NDSS

Note: You might also find this blog post by Zhiyun Qian very helpful.

The qualities of a good research project, at a minimum, include the following (this is my own subjective view):

• It provides new insight
• It is important, or at least matters to someone
• The findings are non-obvious or provide practical significance

In addition to the above qualities, you should consider the following questions when choosing a research topic, at least for this course:

• What data will you need?
• If you plan to collect this data, what access will you need, and whose permission will be required?
• What resources (computers, storage, etc.) will you need?
• What skills / depth of knowledge will you need?
• Do you need to involve users in experiments / data collection?

What to turn in: For the first milestone, each student should turn in a PDF document that includes at least 3-5 unique project ideas (not slight variations).

• If an idea is to conduct an empirical study, the project idea should include:
  • title
  • research question(s)
  • proposed methodology
  • expected findings
• If the idea is more along the lines of building a solution for a problem, the project idea should include:
  • title
  • problem
  • solution idea
  • solution approach
  • expected findings

After the project proposal is submitted, you must meet with the instructor to determine the final project. You are also encouraged to meet also with the instructor before turning in the project proposal to discuss project ideas. Schedule a meeting during office hours.

Examples: Prof. Enck provided example project proposals (PDF). You can download the .tex template for the examples here.

Milestone 2: Related Work (50 points) #

Due: Fri, Sep 27 - 11:59pm RM2

One of the most critical and often overlooked portions of a research project is a sufficient investigation of related work. For this milestone, you will write a related work section. A good related work section is not simply a laundry list of papers and corresponding summaries. Rather, a good related work tells a story of how technology and research has advanced to address problems and topics related to that considered by the paper. It is also critical to contrast your paper with the prior work, not just state what the prior work does. While your research is not yet complete, you should have enough of a grasp on the idea to contrast it with the prior work. You may also revise your related work for the final written paper. Finally, during your related work investigation, take note of how the related work section of those papers is written. Many papers have poorly written related work sections. Identify what makes a good and bad related work section.

Template: When formatting your related work, use the provided template. Include your title and an abstract on the first page. Do not change the font size, margins, or any other formatting.

To receive 40 of the 50 points, the related work must be at least (but not significantly more than) two full columns of text (using the provided template) and contain at least 30 citations. Websites (i.e., not academic work) count as one-half a citation. The remaining 10 points will be based on the quality of the document, including the writing, quality of citations, number of missing well known citations, etc. Going well beyond the minimum 30 citations will help achieve the full 50 points for this milestone.

Note on Length: The related work section in a security paper is typically between 0.5 and 0.75 page. For this milestone, your target is 1 page. Writing a concise related work section takes practice, and the related work section that most students write on their first attempt is more verbose than it needs to be. At the same time, your related work section for this milestone should not be more than 1.5 pages. If it is, you are writing too much, or you have chosen too broad of a scope for your paper. After receiving feedback on your related work section for this milestone, you are encouraged to write a more concise 0.5 to 0.75 page related work section for the final project.

What to turn in: PDF of the related work (lastname-relwork.pdf) using the provided template.

Note about files: Only .pdf files generated from the provided template using LaTeX may be submitted. Failure to comply with any of these file format and naming requirements will result in an automatic ten point deduction from the milestone grade.

Milestone 3: Research Plan (75 points) #

Due: Fri, Oct 18 - 11:59pm RM3

At this point, you have identified a problem and have at least a vague idea of your solution or approach to answer an empirical research question. An idea is of little value if it is not evaluated. For this milestone, you will report on how you plan to evaluate your idea.

You must describe the following:

• Problem Statement (10 points): A short description (one paragraph or less) of the problem you trying to solve. Note that the problem may have been refined from previous milestones. If there are significant changes, please discuss with the instructor.

• Solution Idea / Study Goal (10 points): If your project is to propose a solution to a problem, name this section “Solution Idea” and provide a short description (one or two paragraphs) of how you propose to solve the problem. If your project is an empirical evaluation of some sort, name this section “Study Goal” and describe the significance of it. Note that the idea may have been refined from the previous milestones. If there are significant changes, please discuss with the instructor.

• Threat Model (10 points): A description (at least several paragraphs) describing the security assumptions for your solution idea. A good threat model should describe:

  1. Who is the adversary
  2. What are the goals of the adversary
  3. What are the capabilities of the adversary
  4. What is the trusted computing base (TCB)

  Note, when describing the adversary capabilities, if is often useful to describe assumptions of what the adversary cannot do (e.g., does not have physical access to a device). Even if you are performing an empirical study, you should strive to describe a threat model under which the study is valid. If you have questions, please see the instructor.

• Research Questions (15 points): A list of at least three (more desired) research questions that inquire about the problem and/or solution idea. Research questions should be specific, concrete, and unambiguous questions. For example, research questions may inquire about protection against specific threats, performance overhead, scalability, and usability.

• Methodology (10 points): A high level description of how you plan to answer the research questions (at least a paragraph for each). For example, a project might design and implement a protection and then empirically evaluate the protection in some way.

• Evaluation Plan (20 points): A description of how you plan to answer the research questions. The evaluation plan may mirror the research questions, or multiple research questions may be answered by a single part of the evaluation. The proposed evaluation may be split into both the design and a more formal evaluation section. If graphs or tables are appropriate, provide mock versions that you expect to have in the final paper. For each experiment, you should describe:

  1. Experimental setup (e.g., hardware, software, and datasets used)
  2. Specific measurements and metrics you plan to use
  3. What constitutes success

Note: In system security research papers, the design section often provides a form of evaluation by describing how the solution defends against potential attacks. As such, think of your design section as if it provides an argumentative evaluation. For example, the sub-sections of your design are the specific security properties that your design needs to achieve. Each sub-section then makes makes an argument (though the design description, referencing the threat model where appropriate) for the specific security property. The evaluation section can then argue why the composition of the individual security properties is secure. Your evaluation plan should make an initial attempt at this high level structure so that you may receive feedback before the final paper. Finally, systems security papers also have more explicit evaluation sections that consist of several experiments (e.g., performance, compatibility, usability).

What to turn in: PDF of the research plan (lastname-plan.pdf) using the provided template.

Note about files: Only .pdf files generated from the provided template using LaTeX may be submitted. Failure to comply with any of these file format and naming requirements will result in an automatic ten point deduction from the milestone grade.

Milestone 4: Abstract/Intro (25 points) #

Due: Fri, Nov 8 - 11:59pm RM4

The abstract and introduction are crucial to a paper. This is where you motivate and pitch your idea and present the high-level results. If you fail to get the reviewer’s interest in the abstract and introduction, it is hard to recover. Furthermore, the abstract and introduction are the most-read parts of published paper.

Abstracts should be relatively short: one paragraph, around 200 words. A wandering and off topic abstract can confuse the reader. Consider the following template for writing an abstract (approximately one sentence per point):

• Area
• Problem
• Solution Idea
• Methodology
• Results
• Take-away

An introduction is a longer version of this same outline, often dedicating about a paragraph per point. A good outline for your introduction follows (approximately one paragraph per point):

• General technology area and high level problem (sometimes two paragraphs)
• Most important related work and why it fails to fully address the problem
• High level goal or vision for the paper (sometimes broader than the specific provided solution)
• “In this paper, …” (specifics for this paper)
• Major results
• “This paper makes the following contributions:” and three bullet points
• “The remainder of this paper proceeds as follows. Section 2 … Section 3 …”
• The abstract/intro document should be between 1 and 1.5 pages using this format, including the title and author block.

What to turn in: PDF of the abstract and intro (lastname-intro.pdf). Use the final paper template, commenting out the \input commands for the other sections in the skeleton file.

Note about files: Only .pdf files generated from the provided template using LaTeX may be submitted. Failure to comply with any of these file format and naming requirements will result in an automatic five point deduction from the milestone grade.

Milestone 5: Presentation (25 points) #

Due: Mon, Dec 2 - 11:45am RM5

Towards the end of the semester, students electing the research track will present their work to the class. The duration of the presentation will depend on the total number of projects.

• The presentation of your research project should be around 5 minutes
• The presentation should use slides (but you can use a tool of your choice, e.g., PowerPoint, Keynote, Beamer, reveal.js, …)
• Please submit a PDF of your slides to Gradescope, so that I can assign points for the presentation via that submission (also a good idea to submit your slides before your presentation as backup).
• You are free to structure the presentation to your liking. In general, I would recommend following the research paper structure, i.e.:
  1. short intro + background for your research area
  2. description of your methodology
  3. presentation of results, some discussion
  4. outlook / this research going forward based on your findings so far.
• The presentation may also include a demo (confirm with the instructor). While live demos are nice, recorded demos are more reliable.

As an alternative to the in-class presentation in case you are: following along online, can’t make it to class, sudden illness, etc. You can either:

• Submit a recording of your presentation to me via email (preferably before the end of the in-class presentation day). Be aware of email size limits, a URL to a recording on Google Drive or similar is probably the best way.
• Book an office hour to present live via Google Meet or Zoom (preferably on or before the in-class presentation day).

What to turn in: PDF of the presentation slides (lastname-presentation.pdf) on Gradescope.

Milestone 6: Final Written Paper (200 points) #

Due: Tue, Dec 3 - 11:59pm RM6

The written version of the final project is a conference-quality report, consisting of 8–10 pages (not including references), 1-inch margins, two column, 10-pt font. Use the provided LaTeX template. Suggested outline:

• Abstract (around 200 words)
• Introduction (includes references to highly-relevant related work, i.e., state of the art for the problem you are trying to solve)
• Overview of Approach (a nice and accessible “English” description of your approach, often including a “fluffy diagram” to orient the reader)
• Protocol/Architecture/Design/…
• Evaluation (don’t forget to interpret your data)
• Discussion (discuss some of the important simplifying assumptions, and suggest possibilities for future work)
• Related Work (“somewhat related” work goes here; directly related work goes into the Introduction)
• Conclusions (don’t summarize your work here. That’s what the abstract was for. Instead provide some philosophical ruminations of your work and future possibilities, i.e., conclusions that you have arrived at as a result of your work.)
• References

What to turn in: PDF of the final paper (lastname-paper.pdf) using the provided final paper template.

Note about files: Only .pdf files generated from the provided template using LaTeX may be submitted. Failure to comply with any of these file format and naming requirements will result in an automatic 20 point deduction from the milestone grade.