Hello! And Welcome back to The HPS Podcast where we talk all things History, Philosophy and Social Studies of Science.
I am Samara Greenwood and today it is my pleasure to have one of my all-time favourite philosophers of physics on the podcast, Dr Sophie Ritson.
Sophie focuses on the way contemporary physicists – of both the experimental and theoretical kind – work together to not only develop robust and reliable knowledge, but also to find creative ways to further expand our fundamental understanding of the universe.
Sophie is unafraid to jump right into the deep end on these questions. She began her career examining the string theory controversy. While string theory is widely regarded by its practitioners as the only viable option for extending the standard model of physics, since the 1980s critics have continually questioned the scientific status of string theory and its likelihood of success. This dispute has often been called ‘The String Theory Wars’
More recently Sophie has studied first-hand the high stakes work on experimental particle physics happening at the Large Hadron Collider at CERN in Switzerland.
In High-energy physics there is widespread agreement that the standard model is both successful and yet incomplete. In the absence of new results, experimental physicists are actively hunting for new directions, searching for 'unknown unknowns’, often using creative data analysis techniques to do.
Sophie’s approach to topics such as these is to engage as much as possible with practicing scientists to gain more direct insight into the intellectual, social, and even emotional dynamics at play in these large-scale, complex and evolving situations.
In this week’s episode, Sophie discusses her fascinating research on these topics, as well as on the larger question of ‘how do large scale research collaboration’s generate knowledge?’
Samara Greenwood: Welcome to the podcast, Sophie.
Sophie Ritson: Thanks very much.
Samara Greenwood: Starting as we always do, we'd love to hear a little bit about your backstory. How did you make your way to history and philosophy of science?
Sophie Ritson: Well, it actually started right here at the University of Melbourne. I was an undergraduate doing a science degree with a major in physics, which I was very much enjoying and then I took a semester off to travel. When I came back, I discovered one of the key subjects that I had to do was not in the right sequence, and I had to fill a hole. So, I literally, in the office, looked down at a list of subjects, and I saw a subject that was called ‘From Plato to Einstein’. I was like, wow, that sounds like fun, why not do something different? Then I just fell in love.
It was a great subject; it was a great introduction to HPS. So many things to get your teeth into, so many things that were just so interesting, so many things I wanted to know more about. After that I added a second major to my undergraduate degree and then went on and did honours at the University of Melbourne and a PhD and haven't looked back since.
Samara Greenwood: What did you do your honours on? Was it the same as what you did your PhD on?
Sophie Ritson: Yes, they were closely linked because I discovered, as many people do in their honours year, there was no way I could fit in everything I wanted to do.
So, in my honours year, I wrote about the string theory controversy. The first level of this controversy was considerations about whether string theory should be considered scientific or not. These are classic questions for a historian and philosopher of science. You begin studying the demarcation problem, can we come up with some necessary and sufficient conditions to say this is science and this thing over there is not science. Here was this incredibly expert community with researchers at the most prestigious institutions in the world, at Harvard, at Oxford, at Cambridge, and people were saying that this activity that they were doing shouldn't be considered science. They were grappling with these considerations. So that's where I started in my honours thesis.
Samara Greenwood: I know you went on to do that in your PhD as well. Was there anything controversial in what you came up with?
Sophie Ritson: In general, the string theory controversy could be very controversial. The debates were very heated at times. There were occasions where physicists made death threats against other physicists. So, this is the level of emotion that was part of this controversy.
At the very end of my PhD I actually got blogged about. My anonymity was taken away, and suddenly all these high energy physicists were writing about me on the internet, which was slightly concerning but at least it happened at the end of my PhD. I can see why people like to have their historical actors having already passed away. Having them call you up and have them tell you what they think about things is a little bit more confronting.
Samara Greenwood: I bet that is true. And so, what are you currently working on?
Sophie Ritson: So, there was a really interesting episode that happened in two of the experiments at the Large Hadron Collider.
At the end of the year, they do this thing called the CERN Jamboree where all results are announced from the whole year. There were two really big experiments at CERN who both independently of each other announced some evidence for a previously undetected experimental result, essentially announced evidence of an undetected particle. But not conclusive evidence. Not such that they would say we have discovered something. But they had both independently of each other found some evidence and this set the high energy physics community into an uproar.
It was incredibly exciting. Hundreds and hundreds of papers were published about this potential result. But the timing of it also coincided with a shutdown period for the experiments. So you've got these announcements and then you've got no more data coming in, and no more data coming in for months.
A few months later, both experiments again presented to the experimental and theoretical physics communities, and they had managed to slightly increase the signal. So, perhaps there's a bit more evidence to suggest that there is something really exciting going on here.
In the meantime, the rest of the world caught up. It was in The New York Times, it was published absolutely everywhere because this was potentially an extraordinarily ground breaking discovery. Because, whilst the discovery of the Higgs boson was incredibly important and novel and Nobel Prize winning and interesting, it was to an extent expected. It came from predictions from the 1970s. What these two experiments had found was evidence of something completely unexpected that nobody saw coming and had the possibility to be incredibly transformative.
However, what ultimately ended up happening was, once the two experiments started collecting data again, the signal completely disappeared. It was just a statistical fluctuation. There were no mistakes in interpretation, this is just what happens when you deal with statistical phenomena. And it disappeared.
Then the two communities were left grappling with, what do you do now? If something disappears that you never had in the first place, has anything changed? Has the content of your knowledge changed? There are a few different things we can think about because we tend to think about the impact of something rather than the impact of nothing. So, I've been investigating this case study.
Samara Greenwood: Mm, and it is my understanding that you've interviewed people related to this. How has that gone?
Sophie Ritson: Yes, so I spent quite a bit of time at CERN and I interviewed the teams that were involved in both of the experiments. What was, for them, one of the most stressful periods of their professional lives as everyone was incredibly excited and really wanted to know what was going on.
They helped me piece together the picture of what was going on inside the collaborations. How they get to a publication, which can take three to five years, is an extraordinarily extended, convoluted process. You'll have 3,000 authors being listed on one of these publications of which a small proportion will actually have contributed to the publication. So, figuring out who actually is involved and is responsible is a very difficult to do.
Samara Greenwood: That leads us well into the next question which is, could you introduce us to the topic which we'll be discussing today?
Sophie Ritson: Yes, so, funnily enough, the topic is research collaboration.
So, I have looked at some really big research collaborations. The experiments at CERN at the Large Hadron Collider, the two experiments I just mentioned, they have over 3, 000 collaboration members. But in general, modern science is overwhelmingly collaborative these days. It's sort of rare to find individuals, not all disciplines, but rare to find individuals working in isolation now.
Samara Greenwood: So, what are some central themes in research on scientific collaboration?
Sophie Ritson: Well, there are lots, as you can imagine. Collaboration is a very broad umbrella, and it will be understood differently in different disciplines and then again differently in interdisciplinary approaches. But I think there's a few things we can think about and broadly it all stems from thinking about, how is it that groups as opposed to individuals generate knowledge? And, how things might change if we're thinking about things as a group instead of an individual? How we might think of, what are the conditions for knowledge formation if it is something that you can't do alone, if it is something that you have to do as a group? We can think about who is responsible? So, if there's some kind of knowledge claim made, and it was made by 20 people, who do we hold responsible for that knowledge claim?
There's been some recent discussions around this in terms of ChatGPT actually. So, when we think about authors of a paper, we tend to think of the person as literally sitting down and clapping on the keys. But that's not really what authorship means in the context of research. It means that they've contributed to a paper, even though they might not have actually written down a single word. Ultimately, we tend to hold authors of papers responsible, which is why ChatGPT can't be an author. Even if they have contributed to the text, ChatGPT is not responsible for the content. So, when you sign on to be an author of a paper, you are in part signing on to be responsible for what's in there.
But then if we think about this in the context of the experiments that I look at, there are 3,000 people on those papers. So, are we really going to hold 3, 000 people responsible for the content? We can also think about how peer review could work in this situation. So, if you're asking one person to review the work of hundreds, are they able to engage? Are they able to assess? Is that even possible? Does peer review really make sense here? We can also think about the role of identity when we think about collaborations. Identity seems to be simpler when we talk about individuals, it never is, but it seems simpler. So, what is the identity of a research collaboration?
You have almost got the modern-day version of Theseus's ship. So, say you've got a collaboration of a hundred people, and then over the period of five years, everyone who was part of that collaboration has left and been replaced by different members. Is that still the same research collaboration? I think we're inclined to say yes, not just because of the name, but because of the conventions and the rules and the structures.
What this gives us a bit of an insight into, is that collaboration is far more than just the individuals added up. It's more than the sum of its parts. Now we can think about the role of governance structures, membership, other collaboration structures, lots and lots of different things.
Samara Greenwood: With your own research, are there any of those themes that you really delve into? And what's your own position on those?
Sophie Ritson: One thing I find really interesting when we're thinking about the structures of collaboration is, what is the ideal set up that will promote the possibility for novelty whilst also protecting the credibility of the collaboration? There is this inherent worry that if you've got a really big collaboration and you have to get to some form of consensus do you strip away all the things that are interesting or potentially novel or controversial or disruptive by requiring so many people to be involved in some kind of consensus formation. Does this mean that you're inherently limiting boundary-pushing science, novelty-producing science? At the same time, these processes are absolutely crucial in order to say that you have robust knowledge, in order to have a credible claim being made.
So, there's lots of different ways in which you can structure or organise or make up a collaboration and then balance that tension between the novelty and the robustness of what is ultimately produced.
Samara Greenwood: Oh, that's really interesting. When you think more generally about innovation, you often think of small, agile teams or individuals as being the ones that can be creative because there is less structure, right? So, I can see how that can be really difficult when you also have to have very large teams. Are there strategies that these large teams use to keep that innovative thinking?
Sophie Ritson: Yes and no. The focus is often on making sure the results are robust and credible, which is absolutely understandable. For example, in the big experiments at CERN, the internal review processes for papers are that in principle, anyone can contribute to the critical review of a paper, and at different points in time. So, often when these papers go out to external review, there's nothing to be done, because they've been through this extraordinarily robust and rigorous process.
In terms of ensuring the potential for novelty, it's really difficult because if you sit down and think, ‘how would I design even just a simple experiment to find evidence for something I've never thought of before?’ how would you even begin that task? How do you do it when you don't know what it is you're looking for? Donald Rumsfeld's 'Unknown Unknowns'. It's very difficult. But there are some very clever ideas that they've got.
Samara Greenwood: What about from outside the experiment? So, I know theorists are happier, perhaps, to speculate about things. Are there ideas coming from that area that may eventually come to be experimentally tested?
Sophie Ritson: There are certainly lots and lots of ideas from theorists, so there have been decades of research on - it's called "beyond standard model physics". Their price of speculation is much lower as you kind of alluded to. It's not to say it's unlimited. It's not to say that they could postulate just anything. But they don't have to design a multi-billion-dollar experiment in order to just think of something.
But none of these approaches have come to fruition, so far. There have been thousands and thousands of beyond standard model physics papers, all suggesting different approaches. One of the very popular ones is supersymmetry, which takes on various forms. But, as of yet, no experiment has found any evidence for any of these approaches. And so, there is a little bit of a shift of maybe this progress might come from experiment rather than theory.
Samara Greenwood: What first got you interested in studying collaboration?
Sophie Ritson: I first got interested in studying collaboration because I was interested in the Large Hadron Collider and the experiments there, which are, as I mentioned, incredibly collaborative. But, more broadly, I was interested in problems that you can't do alone, that require multiple epistemic agents in order to come together, each knowing and understanding and producing different things and then what that means for the generation of knowledge.
When you look at modern research, it is so rare to find someone advancing something alone. This so-called Nobel Prize model of research, where you have individual geniuses making extraordinary contributions, just really didn't match up with research in practice. So that's what got me interested.
Samara Greenwood: Definitely, is there anything about scientific collaboration that you've then found surprising? Or you think others might find surprising?
Sophie Ritson: One thing in particular that I was really surprised by was when I was spending time at CERN, there was so much discussion of creativity. And in my interviews, I had never asked a single question about creativity. This came completely unprompted from the researchers I was talking to who were really trying to impress upon me the importance of creativity. Particularly in more challenging experimental circumstances. They would note that if we're going to be able to do the experiments we want to do, we're really constrained by the experimental conditions. So we're going to have to think of ways of being able to do more with the data that we've got.
That was the way in which they were, surprisingly consistently, referring to creativity. This is really different from how we might ordinarily think of creativity, which is really linked to sort of artistic expression in lots of other disciplines. But here they were saying, what's absolutely needed is you've got a hundred people working on something in particular, and if you can transform your understanding of the uncertainty by 0. 5 percent and then everyone across that group is able to transform how you understand this experiment and your understanding of uncertainty and all these other things, you can transform a result from ‘evidence for’, ‘suggestions of’ to a discovery claim, without actually changing the input at all. It's really just how you understand it. You do that collaboratively as a team with these iterative, tiny little improvements and you can have an extraordinary transformation.
Samara Greenwood: That is fascinating. A very directed form of creativity, I guess.
Sophie Ritson: Directed in the sense of there's an understanding of what the problem is. That you need to be able to do more with your data without increasing the amount of data. But the actual transformation is very much undirected. ‘I have to think of ways of doing things that haven't been thought of before’ or slight iterations that are transformations.
Samara Greenwood: One thing that surprised me, when I've come to see your presentations on some of these topics, is the emotion involved. We often classically think of science as relatively unemotional and perhaps physicists as being the epitome of the classic view of the scientist. And yet there is so much emotion. What do you make of that?
Sophie Ritson: Yes, certainly these topics do evoke a lot of emotion in the high energy physics community, and it does seem to be against stereotype.
It's hard to wrap your head around other people's emotions and I'm certainly no psychologist, but one of the things that I do see is how much this is tied up with individuals’ identity. So, I can sort of understand how emotions would be involved there.
String theorists were being told ‘that thing that they are’, a string theorist, was not a scientific thing, they're not scientists, as well as saying string theory is not science. There's so much credibility and prestige and all these things linked up with this term science. So, to have that taken away from them or potentially threatened, I can see how that would invoke a very emotional response, because it was essentially saying that these claims that they were making about the world didn't have credibility,
Samara Greenwood: Is there anything in the collaboration research that you think practicing scientists might find of particular interest?
Sophie Ritson: Certainly the normative aspects of it. People who model research collaborations say there are better and worse ways of doing this. [For example] kind of a distribution between like radicals and people who support the process. So that you've got people that are suggesting change and prompting things as well as people that are making sure that things are consistent and robust.
One of the difficulties in this research is that there's no one size fits all approach to collaboration and what really determines a successful collaboration is the local conditions. There are broad lessons that can be drawn as long as you always think about the local conditions. What are your research goals? Who are you collaborating with? Who are your partners that are also really important? In general, when we look at research collaborations in terms of what might make a particular collaboration successful, it really is the local conditions that are extraordinarily important.
Samara Greenwood: And what about the broader public? What might be of interest and value to everyone?
Sophie Ritson: I think one of the really important lessons that you can take from the research on collaboration is that diversity is an epistemic good. There are so many social and ethical reasons why we would say diversity is really good, but it's not only that. It's an epistemic good. It results in more rigorous, more robust, more impactful knowledge. That's a really important finding, I think, because sometimes it's not drawn as attention to as much.
Samara Greenwood: Within the diversity and collaboration research, is there a sense of different kinds of diversity? Are there particular kinds of diversity that are more beneficial or ones that we should particularly be focusing on?
Sophie Ritson: Tough to say. It brings me back to thinking about the local conditions when you're trying to think about something. So, if for example you're an Australian researcher and you're working towards some research that might be impactful in the public health space, there's a few different ways in which you could think about diversity there.
Public health is an extraordinarily varied area of research. One of the first things you might think about is, ‘Actually, who is the relevant expert in this area?’ Because It's not always obvious. These things look obvious once the research is done and you look backwards and you say, ‘Oh, expert A was the right expert in this situation’. But when you're confronted with something that you don't understand yet, it might be that you need to bring in experts from four different fields. And they're all going to say, ‘Oh, I've got the perfect research methodology for you. This is how you can know more about X.’ But that actually isn't clear. When we have problems, the problem doesn't tell us what methodology we should use to try and understand it.
That's the diversity of the experts involved in that particular example. But then, this might be an issue that faces a particular group, more than it faces other groups. So, you can think about, in your design of your approach, how you want that group to be involved in the design. Because you could set up a project where you make sure that you invite a whole diverse group of experts, and then you get to the end and you say, ‘okay, this is our approach’ and then the group that it's relevant for says, ‘Oh, that doesn't work for us. Why didn't you just talk to us?’ So, you can think about how they could be far more involved in the research design, in the process, because they have extraordinary insights into this that the researchers might not have.
The word co-production gets thrown around a lot. Not in the old STS kind of sociology of science, co-production of knowledge sense, but in terms of when we collaborate, we should co-produce our knowledge with the people for whom it's relevant. Thinking about how you can have genuine co-production of knowledge also gets thrown around all the time. I'm not just co-producing, I'm genuinely co-producing. What that really means is, particular to your epistemic goals, you really need to think about all the different people that could be contributing to this thing, because these are problems that we can't do alone.
Samara Greenwood: So, my final question, are there facets of scientific collaboration that you believe are under researched at the moment? In other words, what would you like to see more emphasis on in the future?
Sophie Ritson: One thing I'm finding at the moment particularly interesting is thinking about trust in research collaborations and representations of uncertainty.
Naomi Oreskes and others for some time have drawn attention to how trust can be undermined in research, or doubt can be created, to use her terminology, and have argued that what's really important is that scientists understand the limitations of their own research and they need to think about how they communicate uncertainty.
I think that's really interesting to think about in the context of collaborations, because broadly modern science is for the most part collaborative. So, If we shift questions from ‘do I trust this individual researcher?’ Person A, Person B, to questions of ‘how do I decide if I trust this research collaboration?’ Which five years ago might have had a 50 percent different population than it currently does now, how is it that you evaluate that? How can you think about trusting a collaboration? These are also questions for people who might not be actively engaged in research, but also in terms of policy, in terms of researchers who just need to look to other areas of research that they don't know very well. How do you decide whether to trust a collaboration or not?
Also, in the context that collaborations can be opaque to the external world, all the machinations and the decision-making processes. If one individual can't comprehend the whole picture, if it requires multiples, then how does another individual from outside the collaboration say, ‘oh, I trust that or I don't trust that’.
Samara Greenwood: That would be great to know more about. Are you planning on researching in that area in the future?
Sophie Ritson: I am. Yes.
Samara Greenwood: Excellent. Thank you so much for joining us on the podcast, Sophie, and it has been wonderful to talk to you.
Sophie Ritson: Thanks so much. It's been so much fun.