Caitlin Sadowski
Dr. Caitlin Sadowski is an engineering manager at Google in Mountain View, California, where she leads the Chrome Data team: a cross-functional team focused on tools, infrastructure, and insights related to measuring Chrome browser usage. In the past, she made static analysis useful at Google by creating the Tricorder program analysis platform and co-founded a team focused on understanding developer productivity. She has a PhD from the University of California at Santa Cruz where she worked on a variety of research topics related to programming languages, software engineering, and human computer interaction. She enjoys baking with her two young children.
Authored Publications
Google Publications
Other Publications
Sort By
A World Wide View of Browsing the World Wide Web
Aurore Fass
Emma Thomas
Jon Azose
Kimberly Ruth
Mark Pearson
Zakir Durumeric
Internet Measurement Conference (IMC), ACM (2022) (to appear)
Preview abstract
In this paper, we perform the first large-scale study of how people use and spend time on the web. Our study is based on anonymous, aggregated telemetry data from a major web browser with several hundred million users globally. Our study considers only users who chose to explicitly enable sharing URLs with the browser vendor and have usage statistic reporting enabled. We analyze the distribution of web traffic, the types of popular sites that people visit and spend the most time on, differences between desktop and mobile browsing behavior, geographical differences in web usage, and the websites popular in regions worldwide. Our study sheds light on both online user behavior and how the measurement community can better analyze the web in future research studies.
View details
Enabling the Study of Software Development Behavior with Cross-Tool Logs
Ben Holtz
Edward K. Smith
Andrea Marie Knight Dolan
Elizabeth Kammer
Jillian Dicker
Lan Cheng
IEEE Software, vol. Special Issue on Behavioral Science of Software Engineering (2020)
Preview abstract
Understanding developers’ day-to-day behavior can help answer important research questions, but capturing that behavior at scale can be challenging, particularly when developers use many tools in concert to accomplish their tasks. In this paper, we describe our experience creating a system that integrates log data from dozens of development tools at Google, including tools that developers use to email, schedule meetings, ask and answer technical questions, find code, build and test, and review code. The contribution of this article is a technical description of the system, a validation of it, and a demonstration of its usefulness.
View details
Web Feature Deprecation: A Case Study for Chrome
Ariana Mirian
Geoffrey M. Voelker
Nik Bhagat
Stefan Savage
International Conference on Software Engineering (ICSE) SEIP track (2019) (to appear)
Preview abstract
Deprecation is a necessary function for the health and innovation of the web ecosystem. However, web feature deprecation is an understudied topic. While Chrome has a protocol for web feature deprecation, much of this process is based on a mix of few metrics and intuition. In this paper, we analyze web feature deprecations, in an attempt to improve this process. First, we produce a taxonomy of reasons why developers want to deprecate web features. We then provide a set of guidelines for deciding when it is safe to deprecate a web feature and a methodology for approaching the question of whether to deprecate a web feature. Finally, we provide a tool that helps determine whether a web feature meets these guidelines for deprecation. We also discuss the challenges faced during this process.
View details
What Predicts Software Developers’ Productivity?
David C. Shepherd
Michael Phillips
Andrea Knight Dolan
Edward K. Smith
Transactions on Software Engineering (2019)
Preview abstract
Organizations have a variety of options to help their software developers become their most productive selves, from modifying office layouts, to investing in better tools, to cleaning up the source code. But which options will have the biggest impact? Drawing from the literature in software engineering and industrial/organizational psychology to identify factors that correlate with productivity, we designed a survey that asked 622 developers across 3 companies about these productivity factors and about self-rated productivity. Our results suggest that the factors that most strongly correlate with self-rated productivity were non-technical factors, such as job enthusiasm, peer support for new ideas, and receiving useful feedback about job performance. Compared to other knowledge workers, our results also suggest that software developers’ self-rated productivity is more strongly related to task variety and ability to work remotely.
View details
Do Developers Learn New Tools On The Toilet?
Edward K. Smith
Andrea Knight Dolan
Andrew Trenk
Steve Gross
Proceedings of the 2019 International Conference on Software Engineering
Preview abstract
Maintaining awareness of useful tools is a substantial challenge for developers. Physical newsletters are a simple technique to inform developers about tools. In this paper, we evaluate such a technique, called Testing on the Toilet, by performing a mixed-methods case study. We first quantitatively evaluate how effective this technique is by applying statistical causal inference over six years of data about tools used by thousands of developers. We then qualitatively contextualize these results by interviewing and surveying 382 developers, from authors to editors to readers. We found that the technique was generally effective at increasing software development tool use, although the increase varied depending on factors such as the breadth of applicability of the tool, the extent to which the tool has reached saturation, and the memorability of the tool name.
View details
Modern Code Review: A Case Study at Google
Emma Söderberg
Luke Church
Michal Sipko
Alberto Bacchelli
International Conference on Software Engineering, Software Engineering in Practice track (ICSE SEIP) (2018) (to appear)
Preview abstract
Employing lightweight, tool-based code review of code changes
(aka modern code review) has become the norm for a wide
variety of open-source and industrial systems. In this paper,
we make an exploratory investigation of modern code
review at Google. Google introduced code review early on
and evolved it over the years; our study sheds light on why
Google introduced this practice and analyzes its current
status, after the process has been refined through decades of
code changes and millions of code reviews. By means of 12
interviews, a survey with 44 respondents, and the analysis
of review logs for 9 million reviewed changes, we investigate
motivations behind code review at Google, current practices,
and developers’ satisfaction and challenges.
View details
Discovering API Usability Problems at Scale
Andrew Head
International Workshop on API Usage and Evolution (WAPI) (2018)
Preview abstract
Software developers’ productivity can be negatively impacted by using APIs incorrectly. In this paper, we describe an analysis technique we designed to find API usability problems by comparing successive file-level changes made by individual software developers. We applied our tool, StopMotion, to the file histories of real developers doing real tasks at Google. The results reveal several API usability challenges including simple typos, conceptual API misalignments, and conflation of similar APIs.
View details
When Not to Comment: Questions and Tradeoffs with API Documentation for C++ Projects
Andrew Head
Emerson Murphy-Hill
Andrea Knight
International Conference on Software Engineering (ICSE) (2018) (to appear)
Preview abstract
Without usable and accurate documentation of how to use an API,
programmers can find themselves deterred from reusing relevant
code. In C++, one place developers can find documentation is in
a header file, but when information is missing, they may look at
the corresponding implementation (the “.cc” file). To understand
what’s missing from C++ API documentation and whether it should
be fixed, we conducted a mixed-methods study. This involved three
experience sampling studies with hundreds of developers at the
moment they visited implementation code, interviews with 18 of
those developers, and interviews with 8 API maintainers. We found
that in many cases, updating documentation may provide only
limited value for developers, while requiring effort maintainers
don’t want to invest. This helps frame future tools and processes
designed to fill in missing low-level API documentation.
View details
Lessons from Building Static Analysis Tools at Google
Edward Aftandilian
Alex Eagle
Liam Miller-Cushon
Communications of the ACM (CACM), vol. 61 Issue 4 (2018), pp. 58-66
Preview abstract
In this article, we describe how we have applied the lessons
from Google’s previous experience with FindBugs Java analysis,
as well as lessons from the academic literature, to build
a successful static analysis infrastructure that is used daily
by the majority of engineers at Google. Our tooling detects
thousands of issues per day that are fixed by engineers, by
their own choice, before the problematic code is checked into
the codebase.
View details
Advantages and Disadvantages of a Monolithic Codebase
Andrea Knight
Edward K. Smith
International Conference on Software Engineering, Software Engineering in Practice track (ICSE SEIP) (2018)
Preview abstract
Monolithic source code repositories (repos) are used by several large tech companies, but little is known about their advantages or disadvantages compared to multiple per-project repos. This paper investigates the relative tradeoffs by utilizing a mixed-methods approach. Our primary contribution is a survey of engineers who have experience with both monolithic repos and multiple, per-project repos. This paper also backs up the claims made by these engineers with a large-scale analysis of developer tool logs. Our study finds that the visibility of the codebase is a significant advantage of a monolithic repo: it enables engineers to discover APIs to reuse, find examples for using an API, and automatically have dependent code updated as an API migrates to a new version. Engineers also appreciate the centralization of dependency management in the repo. In contrast, multiple-repository (multi-repo) systems afford engineers more flexibility to select their own toolchains and provide significant access control and stability benefits. In both cases, the related tooling is also a significant factor; engineers favor particular tools and are drawn to repo management systems that support their desired toolchain.
View details
An Empirical Study of Practitioners’ Perspectives on Green Software Engineering
Irene Manotas
Christian Bird
Rui Zhang
David Shepherd
Lori Pollock
James Clause
International Conference on Software Engineering (ICSE), 1600 Amphitheatre Parkway (2016) (to appear)
Preview abstract
The energy consumption of software is an increasing concern as the use of mobile applications, embedded systems, and data center-based services expands. While research in green software engineering is correspondingly increasing, little is known about the current practices and perspectives of software engineers in the field. This paper describes the first empirical study of how practitioners think about energy when they write requirements, design, construct, test, and maintain their software. We report findings from a quantitative, targeted survey of 464 practitioners from ABB, Google, IBM, and Microsoft, which was motivated by and supported with qualitative data from 18 in-depth interviews with Microsoft employees. The major findings and implications from the collected data contextualize existing green software engineering research and suggest directions for researchers aiming to develop strategies and tools to help practitioners improve the energy usage of their applications.
View details
A Cross-Tool Study on Program Analysis Tool Notification Communication
Brittany Johnson
Rahul Pandita
Justin Smith
Denae Ford
Sarah Elder
Emerson Murphy-Hill
Sarah Heckman
Foundations of Software Engineering (FSE) (2016)
Preview abstract
Program analysis tools use notifications to communicate
with developers, but previous research suggests that developers
encounter challenges that impede this communication.
This paper describes a qualitative study that identifies 12
kinds of challenges that cause notifications to miscommunicate
with developers. Our resulting theory reveals that
many challenges span multiple tools and multiple levels of
developer experience. Our results suggest that, for example,
future tools that model developer experience could improve
communication and help developers build more accurate
mental models.
View details
How Developers Search for Code: A Case Study
Kathryn T. Stolee
Sebastian Elbaum
Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering (ESEC/FSE ), 1600 Amphitheatre Parkway (2015)
Preview abstract
With the advent of large code repositories and sophisticated search capabilities, code search is increasingly becoming a key software development activity. In this work we shed some light into how developers search for code through a case study performed at Google, using a combination of survey and log-analysis methodologies. Our study provides insights into what developers are doing and trying to learn when performing a search, search scope, query properties, and what a search session under different contexts usually entails. Our results indicate that programmers search for code very frequently, conducting an average of five search sessions with 12 total queries each workday. The search queries are often targeted at a particular code location and programmers are typically looking for code with which they are somewhat familiar. Further, programmers are generally seeking answers to questions about how to use an API, what code does, why something is failing, or where code is located.
View details
Tricorder: Building a Program Analysis Ecosystem
Emma Soederberg
International Conference on Software Engineering (ICSE) (2015)
Preview abstract
Static analysis tools help developers find bugs, improve code readability, and ensure consistent style across a project. However, these tools can be difficult to smoothly integrate with each other and into the developer workflow, particularly when scaling to large codebases. We present Tricorder, a program analysis platform aimed at building a data-driven ecosystem around program analysis. We present a set of guiding principles for our program analysis tools and a scalable architecture for an analysis platform implementing these principles. We include an empirical, in-situ evaluation of the tool as it is used by developers across Google that shows the usefulness and impact of the platform.
View details
RLint: Reformatting R Code to Follow the Google Style Guide
Alex Blocker
Andy Chen
Andy Chu
Tim Hesterberg
Tom Zhang
R User Conference (2014)
Preview abstract
RLint (https://code.google.com/p/google-rlint/) both checks and reformats R code to the Google R Style Guide. It warns of violations and optionally produces compliant code. It considers proper spacing, line alignment inside brackets, and other style violations, but like all lint programs does not try to handle all syntax issues.
Code that follows a uniform style eases maintenance, modification, and ensuring correctness, especially when multiple programmers are involved. Thus, RLint is automatically used within Google as part of the peer review process for R code. We encourage CRAN package authors and other R programmers to use this tool. A user can run the open-source Python-based program in a Linux, Unix, Mac or Windows machine via a command line.
View details
Programmers’ Build Errors: A Case Study (at Google)
Hyunmin Seo
Sebastian Elbaum
Edward Aftandilian
Robert Bowdidge
International Conference on Software Engineering (ICSE) (2014)
Preview abstract
Building is an integral part of the software development process. However, little is known about the errors occurring in this process. In this paper, we present an empirical study of 26.6 million builds produced during a period of nine months by thousands of developers. We describe the workflow through which those builds are generated, and we analyze failure frequency, error types, and resolution efforts to fix those errors. The results provide insights on how a large organization build process works, and pinpoints errors for which further developer support would be most effective.
View details
How Developers Use Data Race Detection Tools
Evaluation and Usability of Programming Languages and Tools (PLATEAU), ACM (2014)
Preview abstract
Developers need help with multithreaded programming. We
investigate how two program analysis tools are used by developers
at Google: ThreadSafety, an annotation-based
static data race analysis, and TSan, a dynamic data race detector.
The data was collected by interviewing seven veteran
industry developers at Google, and provides unique insight
into how four different teams use tooling in different ways to
help with multithreaded programming. The result is a collection
of perceived pros and cons of using ThreadSafety
and TSan, as well as general issues with multithreading.
View details
Does Bug Prediction Support Human Developers? Findings from a Google Case Study
Chris Lewis
Zhongpeng Lin
Xiaoyan Zhu
Rong Ou
E. James Whitehead Jr.
International Conference on Software Engineering (ICSE) (2013)
Preview abstract
While many bug prediction algorithms have been developed by academia, they're often only tested and verified in the lab using automated means. We do not have a strong idea about whether such algorithms are useful to guide human developers. We deployed a bug prediction algorithm across Google, and found no identifiable change in developer behavior. Using our experience, we provide several characteristics that bug prediction algorithms need to meet in order to be accepted by human developers and truly change how developers evaluate their code.
View details
Preview abstract
We have met many software engineering researchers who would like to evaluate a tool or system they developed with real users, but do not know how to begin. In this second iteration of the USER workshop, attendees will collaboratively design, develop, and pilot plans for conducting user evaluations of their own tools and/or software engineering research projects. Attendees will gain practical experience with various user evaluation methods through scaffolded group exercises, panel discussions, and mentoring by a panel of user-focused software engineering researchers. Together, we will establish a community of likeminded researchers and developers to help one another improve our research and practice through user evaluation.
View details
Proceedings of the ACM 4th annual workshop on Evaluation and usability of programming languages and tools (PLATEAU)
Preview
Emerson Murphy-Hill
Shane Markstrum
Conference on Systems, Programming, and Applications: Software for Humanity (SPLASH) (2012)
No Results Found
