Write, Read, or Fix? Exploring Alternative Methods for Secure Development Studies

Document Type

Conference Proceeding

Publication Date

1-1-2024

Abstract

When studying how software developers perform security tasks, researchers often ask participants to write code. These studies can be challenging because programming can be time-consuming and frustrating. This paper explores whether alternatives to code-writing can yield scientifically valid results while reducing participant stress. We conducted a remote study in which Python programmers completed two encryption tasks using an assigned library by either writing code from scratch, reading existing code and identifying issues, or fixing issues in existing code. We found that the read and fix conditions were less effective than the write condition in revealing security problems with APIs and their documentation, but still provided useful insights. Meanwhile, the read and especially fix conditions generally resulted in more positive participant experiences. Based on these findings, we make preliminary recommendations for how and when researchers might best use all three study design methods; we also recommend future work to further explore the uses and trade-offs of these approaches.

Identifier

85204873400 (Scopus)

ISBN

[9781939133427]

Publication Title

Proceedings of the 20th Symposium on Usable Privacy and Security, SOUPS 2024

First Page

81

Last Page

100

Grant

CNS-1801545

Fund Ref

National Science Foundation

This document is currently not available here.

Share

COinS