Early work in Data Science for/in/about SE

Neil Ernst

2026-05-04

Early Days

“Computers”

Still from the excellent movie ‘Hidden Figures’

“Computers” → women doing slide rule calculations as shown in Hidden Figures.

Note that as this job became more and more lucrative, it stopped being seen as “women’s work” and women were driven out of the field

Era of Hubris: mainframes and the software to run them

Proprietary data from big projects: the A7 software project, IBM’s OS/360 project. An era of positivism and Taylorism: if only we can measure what people are doing, we can manage it like we would building pyramids.

The Mythical Man-Month and Peopleware

one does not meme

• “adding more people to a late project makes it later” (Brooks)
• “the logical structure of a system reflects the organization that built it” (Conway’s Law)

This was ultimately shown to be false in two seminal analyses: Brooks’s mythical man-month project and Peopleware. It paralleled similar revolutions in sociology, epistemology, critical theory that pointed out the many flaws in a strictly empirical approach to culture, science, literature, the “counter culture” if you will.

The Role of Data

• cannot manage what you cannot measure?
• Goodhart’s law: “When a measure becomes a target, it ceases to be a good measure”.
  • lines of code == productivity
• But surely something is worth measuring?
  • “Who writes code faster” (10x)
  • “How long is this compile cycle”
  • “What file is worth fixing?”

People like Brooks are hardly hippies; what he saw was - I think - deeply distressing for him, someone raised on math and engineering.

The reality was that scientific management - the current trend, the idea that humans are these cogs that can be precisely measured and moved into the correct place to maximize efficiency - failed utterly in the software process (it failed everywhere else too, but that’s another story[^Of course nowadays companies are arguably reinventing Taylorism 2.0, with extremely precise monitoring of workers, how long they spend in the bathroom, how often they are on the phone, forcing people to download company spy software onto personal phones, etc.] ).

Data has therefore been part of software development and software management since the beginning, as part of this cultural legacy of measure what you manage: how long does it take to compile, how many lines of code are people writing, etc. The 10X study is a good example.

The Open Source Era

Data Everywhere

• plenty of data to was available for studies like the 10x study,
• challenge for researchers was (and is) getting access to data that is cheap to acquire (i.e. licensing) and still representative of the problems in practice.
• Paper by Tu and Godfrey (readings) represents one of these early approaches.

The rise of open source software like Linux was therefore very important in developing data science approaches, and more broadly, the field of empirical software engineering. They look at how Linux has changed over time, using a specific theory (Lehman’s laws) as guide. They then also report on what parts of Linux change most (the drivers).

Show the paper and its graphs.

Empirical Software Engineering

Novel in the 90s to even be able to get source code!

Triggered a focus on Empirical SE, like Kitchenham outlines.

Now there was enough data to be able to systematize what we were studying and how we were studying it.

One issue that remained, of course, was to be able to move between industry relevance and academic tractability.

Bringing Insights into Practice: The Sapienz Example

From the SCAM Facebook paper by O’Hearn and Harman:

1. Irrelevant: ‘Academics need to make their research more industrially-relevant’,
2. Unconvincing: ‘Scientific work should be evaluated on large-scale real-world problems’,
3. Misdirected: ‘Researchers need to spend more time understanding the problems faced by the industry’,
4. Unsupportive: ‘Industry needs to provide more funding for research’,
5. Closed System: ‘Industrialists need to make engineering production code and software engineering artifacts available for academic study’, and
6. Closed Mind: ‘Industrialists are unwilling to adopt promising research’.

Insights into the human aspect

Developers are, in practice, not short of lists of fault and failure reports from which they might choose to act

Thus the tool - the data science output—must ensure it produces work that has:

• Relevance: the developer to whom the bug report is sent is one of the set of suitable people to fix the bug;
• Context: the bug can be understood efficiently;
• Timeliness: the information arrives in time to allow an effective bug fix.
• Debug payload: the information provided by the tool makes the fix process efficient;

Summary

• people, and managers, have a need to understand what is actually happening
• software metrics are one (limited) tool for this
• there is more than just metrics - need to consider the wider perspectives
• enormous amounts of data (of varying quality) available