Skip to main content
SearchLoginLogin or Signup

Shifting Scientific Behaviors

🎧 Leveraging bureaucracy, pre-registered reports, and shame to change scientific publishing practices (56 min with transcript).
Published onDec 01, 2022
Shifting Scientific Behaviors
·

Welcome to Convos on the Common, a Commonplace podcast. In this episode, we chat with Bart Penders, an Associate Professor at Maastricht University in the Netherlands about his recent publication: “Process and Bureaucracy: Scientific Reform as Civilization.” Our conversation focuses on different reform movements, pre-registered reports, how shame motivates publishing behaviors, and the plurality found at every level of bureaucracy. 

You can also find this podcast on Anchor!


Sarah Kearns 00:27

So maybe you could briefly introduce yourself and then give us a brief summary of this paper that we're talking about.

Bort Penders 00:35

My name is Bart Penders. I'm a sociologist of science. I was at some point in the past, also a molecular biologist, and retrained into sociology of science, which I currently research and teach at Maastricht University in the Netherlands. I'm mainly interested in the credibility of science, how scientists achieve it for themselves for their claims, their scientific claims, and how they maintain it once they actually have achieved it. And in the process of studying that I tend to go abroad in the sense of asking, Which alliances do scientists engage in which collaborations do they pursue to help them build credibility for themselves and their claims? But also, which behaviors do they promote, or at least see that they promote? And the rhetoric that goes along with that includes, for instance, stressing something like research integrity. If you're slightly pessimistic about it, you can use it incrementally to build credibility for yourself. You could also, of course, argue that there are principled reasons to maintain research integrity. At least it helps in in establishing yourself as a legitimate voice worth listening to, and builds sort of a trustworthiness surrounding the identity of the scientists, the team, that they're a part of, perhaps even the institute that they're part of, and the scientific utterances that they actually do.

The paper that we're talking about is in line with that. It's called “Process and bureaucracy: scientific reform as civilization.” And that title contains quite a lot of key words that ultimately the paper ties together. What I'm trying to do here is look at something that we call the scientific reform movement, we'll get into that at some point, I assume, in this conversation. I'm trying to understand, not necessarily whether they're right or wrong, but how they seek to expand their view on science as it should take place, and slowly but steadily, influence their surroundings, in a sense that they must, or it becomes almost unavoidable, to follow the example that they set.

Elements of this changed reformed science are heavily bureaucratic, so they exist out of bits of paper (not actually paper, it's all digital and practice) and forms that need to be filled out that account to a very high degree of detail and a lot of standardization to fulfill this articulated idea of good science. That's where the bureaucracy comes from, and the use of that bureaucracy is to conform to these types of processes and are actively performed in a way as the the right thing to do. [These bureaucratic processes are] something that is, modern, new, civilized, and as such, in contrast to the things that are old fashioned and uncivilized, or at least not as civilized. The general idea of the this civilization process is that it is continuous, it never ends. It sort of points in a certain direction that you can identify or see. But it's nevertheless still difficult to predict. Usually research on civilization processes is historical research. There is no prediction required. But at this particular case, it's an ongoing process, which we can look back at in order to understand.

Because [the process of defining “civilized science”] is ongoing, it would be great if we could also sort of slightly anticipate within certain degrees of probability, what route it might take. So that's how it's all essentially tied together. There are also alternative trajectories. And even when you pursue certain values, you can still discuss whether or not those are the ones that need to be prioritized, or even the right ones to pursue in the context of improving sites.

“Messy” Reforms and Paradigms

Sarah Kearns 06:40

Yeah, there's a lot there. It almost sounds like there's like a this like meta scientific reformation here, like beyond the particular like paradigms of a scientific field, but more like a paradigm shift of how scientific values and behaviors are performed. Is that kind of what you mean,

Bort Penders 07:01

Up to a certain point, yes, but that makes it slightly sound slightly uniform. And that's one of the things that I really want to stress is that this this reform movement in itself is not uniform. So they struggle with different concerns that they seek to alleviate or, or solve problems that they seek to solve. Their concerns are different, plural, also the proposed solutions, and proposed prioritized values that guide those solutions are different. You can still call all of that reform, because it is it is about changing these particular practices and these institutes, but it's not particularly uniform.

It's not one paradigm shift, because that's what what Thomas Kuhn came up with paradigm shift. What he means is indeed this dominant theoretical corpus in a field that is at some point replaced by an alternative theoretical corpus that guides our understanding of the world. And there it is very much uniform, and here it is not. So, we are talking about shifts, and they are paradigmatic in the sense that they are the change how we see essentially everything in the conduct of of science, but it is not as neatly, actually, most of the realities isn't actually conformed to Thomas Kuhn said anyway, but in practice, this is not as neatly organized as one paradigm shift.

It is very much meta, yes. So there is an entire this entire field of meta science that, depending on how you understand, it is either part of this reform movement or fuels it or informs it at least provides evidence. What is reformed? Is science itself and not what science studies. But yeah, not all of it is matter. Of course, a lot of it is very practical. All of it is about day to day, things that you do pragmatic, how does my day look like as a scientist? If you are fully committed to this reform process, you'll your day will look differently, because you're, it's expected of you to do slightly different things.

Sarah Kearns 09:38

Yeah, that makes sense. Like there's particular instances of scientific behavior.

Concerns of Reformers

Maybe stepping back a little bit, what concerns are these different performers trying to address and how and does this messy reform movement converge towards changing what like Science capital S is? Or sort of like, what were the sort of the motivations of performers here,

Bort Penders 10:09

There are multiple ones. And I, we can't go through all of them sort of as as part of this conversation. But I could highlight like two.

One of the concerns is essentially scientists are insufficiently rational and that they are, in some form or another, biased whether it's confirmation bias or something else. There's actually a really nice paper by Ivan Flis, a Croatian sociologists of science, who said that it's not so weird that it started in psychology, because this is up to a certain point it’s a psychological dualization of psychology in the sense that all of the things that we've learned about how the human mind works — when you apply it to scientists doing science — you lose trust in what they're actually doing, because we cannot be as rational as scientist as we are expected to be based on sort of traditional ideas of what science is this hyper rational knowledge production process in which all external influences are kept at bay, at all cost. Only that way, can we in some way or another produce anything of value. Otherwise, it simply lacks value. That's one concern. And it's very concrete, of course. You can imagine all sorts of things that you can do in response.

Another concern, and that's something that you see in the UNESCO recommendations for open science and open science is also very much part of this reform, corner of science. There, it's not the bias of, of individual scientists. But essentially the concern that science itself is not actually offering enough to certain parts of the world to certain parts of the globe's population, that different voices are not represented, different communities are not represented. As a result, what is produced in science lacks value, because it is not something that we could all use, and that represents all of us. It’s sort of about diversity and inclusivity, very much on the open science agenda, but completely different from the motivation of not sufficient rationalism.

Both of these ideas, both of these concerns (you could call them even sort of a moral panic of a very specific group) they do not result in the same proposals for changes for science. One calls to streamline the process to get all the weird things out. And the other means you need to put more in more voices, more communities more, essentially, it needs to be more social. And the former one, at least needs to be a lot less social. That's, of course, difficult to combine, and also difficult to achieve any sort of convergence on because they seem a polar opposite, they seem to point in different directions. One is very much driven by sort of an isolated idea of science and scientists. The other one is driven by a perception of science as part of culture writ large. It really depends on what you think sciences where its boundaries are, where they should be, and and how you navigate that boundary space.

The proposed solutions that are connected to the first moral panic (the not sufficiently rational, scientific process) those are the ones I've discussed in this paper. If you step back from the scientific community and look at the both of them, then they're far more balanced, but from the inside of science it really seems like first one is is absolutely dominant and the other one is largely absent. But that's sort of a very specific niche that you're then in and look at the world through. So those are key differences that we cannot forget. And there are, of course, multiple concerns far beyond those two.

Sarah Kearns 15:30

Yeah, like you said, there, there's almost like this plurality of not just the problems, but the solutions to and that they might not necessarily be in conflict with each other. But definitely could be, like you pointed out that the issue like the problem of lacking objectivity means scaling down, and then addressing the issue of diversity means scaling up. So and those are probably multi-dimensional, as well.

Facets of Objectivity

Bort Penders 16:04

Well, you're not connecting the word objectivity to one of the two, what actually is connected to both, but it means different things in both settings. So objectivity and the first is essentially the liberating of the scientific process from all the influences, sort of a mechanical form of objectivity. If you let a computer do it, or a machine, that is sort of what it would do. The other one, the other sort of inclusivity based idea of objectivity, essentially states that you can't get rid of bias anyway, it's inevitable. But you can balance biases, to ensure that not one type of bias becomes dominant, and therefore unduly influences the entire process. If you have sufficient balancing or countering forms of perspective, essentially, you wouldn't call it bias in that system but you could call it perspective. And if you have sufficient perspectives, then it becomes more objective. Because they bit by bit constantly each other out. It's also a great position, defended usually by Naomi Oreskes, who you probably all know because she writes beautiful books. And her last book, Why trust science? really sort of builds on this idea of why science is trustworthy, because it, you can't perfect the scientist. You can protect the institution by allowing all those other things to become part of it, to make it better. That's a completely different idea, but it's still objectivity. The word means something different in both of these settings. So we tend to only use it in the first meeting, and rarely use it and the second one, and I would argue that we should use it more in that second meeting.

Sarah Kearns 18:11

That's a good point. I appreciate the call out.

Paper Pushing

Coming back to the paper a little bit, what bureaucratic processes are, what infrastructures that exist that bolster that trust in science? And what are maybe some of the critiques that the reform movements are, are pushing forward to address the weakness use the two examples that you've brought up?

Bort Penders 18:50

So there actually have been that type of institutions for a very long time — things like institutional review boards, or medical ethical committees —they're all part of the infrastructure and they will help scientists taking care of specific values that they might not prioritize themselves, but essentially, they're forced to prioritize them because they have to go through these hoops of work with the systems. That can be annoying. And that can be bureaucratic. All scientists have ever sort of asked for ethical permission to do experiments, know what bureaucracy means in science. But it is about protecting participants and ensuring that you don't perform experiments that you really shouldn't and that sort of thing.

Especially when it comes to sort of the promotion of rationality in science in the process different types of procedures have been built because there weren't that many of them. The whole thing is a trust based system. It was and it actually still is. But now, distrust have been built into the system: you have to prove that you're doing the right thing at a certain points in time.

Pre-registration & Registered Reports

I used two examples in in the paper. One is preregistration of a study that you might want to do. And the other one is a registered report, which is sort of the 2.0 version of the pre-registration. Both are essentially digital forms that you fill up in which you make certain commitments and promises about what you are planning to do. Then, because you've put them on paper and made them traceable, people can compare what you've promised and what you've actually done. If there's discrepancy, that might be perfectly okay, but at least you have to explain it. That means rationalizing of the process and incentives or awards for cheating are removed. And then cheating. Sounds like a really big thing in science, but it could also be really small. It's not about usually at least, fraud or fabrication of whatever. What about tweaking something a little. And, and not being transparent about how you did that, because some tweaks are fine. But not being transparent about how you did the board, why you did them, that's less fine, at least in this, this idea of how to do this.

That concern [of the lack of transparency and honesty with regard to publishing] is not insanely new. It's not like like five years ago, scientists came up with the fact that this is a problem. In 1830 or so Charles Babbage wrote, “Reflections on the Decline of Science in England and Some of its Causes.” Nobody titles a book like that anymore, but it's quite obvious what is in there based on that title. It's essentially the same diagnosis, as is repeated now, in the context of this of this reform movement, and in that sense it's historically continuous. It's a struggle that's been part of science for a long time.

Additional Labor (and Fear) Required

Due to the possibility of now, building digital infrastructures, something that was far more difficult a long time ago. A registered report in a non digital age, that would slow down science to the point of nothing. But now it can be done because it's digital. That registered report is different from pre-registration, in the sense that people still pre-registered their study. But before they do anything, they share part of the article or paper that they seek to publish, an introduction, literature review, methodology, some commitments with respect to statistics, and sort of the things that you want to do without actually doing them. You submit that to a journal, you have to pick a very specific journal to do that, because not all journals actually work that way. But there is quite a list that facilitates this process.1

Once submitted [the journal editorial board] will review or ask someone to review the promises, essentially, that you've made. And you might get feedback just like it would always in peer review, and requests to make changes or adaptations or updates. Once it's accepted, you go and actually do the experimental work, list all the results that you came up with and then finished the paper, essentially built a second version of the paper, and then resubmitted to the journal. The journal usually commits itself to publication of that second version by accepting the first version.

It's not about the results, it is about the methods and how you do the science. And that help hopes to make it easier to publish negative or inconclusive results. And well, every scientist knows a lot of the things that you do — and in some fields most of the things that you do — come out negative or inconclusive. That's not without value. It's actually quite valuable, but it's difficult to sell. And this particular system, you immunize the infrastructure to the outcome.

But that requires extra work, reviewing something twice, having editor's who know what to do with all of that, you need to have a differentiated publication, a forum where you can differentiate between the two types of studies. And you need to be able to host all of those documents, essentially forever, for people to continue to be able to check up on whether version one and two are actually the same and whether the pre-registration and the study actually done corresponds with one another.

In a negative sense, you're building a panopticon, the possibility of absolute surveillance. And even if nobody actually does look your way, if your paper is never checked, the simple simply the possibility that it can be checked scares people into complying. From the perspective of maximizing rationality, that's a good thing. The more scared they are, the better it will work. That's not the vocabulary that, that most of the people in the reform movement would use. But that is a way to interpret that effect.

[This process] is not about inclusion, it’s a very different social process that then produces compliance here. But it does require a lot of infrastructure that did not exist before. And that's only imaginable now in our digital age, with most of the servers, well placed, essentially, where people have room and money. It's also not free, digital infrastructure costs a lot of money. And some of that has to be paid by scientists, but a lot of it is currently paid for by a number of quite wealthy foundations. Whether they can continue to do that indefinitely is of course a question, and what agenda is behind it is also a question, probably quite a legitimate concern for the quality of science.

In addition, the medical ethical committee in the Institutional Review Board is also going nowhere. It's still there. And you still have to go through those hoops as well. So there is added bureaucracy, to science. I think I explicitly also wrote it in the paper, that's not always a bad thing. Bureaucracies are great ways to scale up things, to wield power in an abstract sense over vast areas. Well, if it wasn't Napoleon, then it's just a big empire. It is science. It's a large virtual sphere in which you sort of wield that power, and you need it if you want to go big. You can't do it without bureaucracy. So it's not always a negative thing. But it can also become a negative thing, because it really can slow things down, and also standardize things to a point that thinking differently is no longer facilitated.

Sarah Kearns 29:20

Yeah, that makes sense. I feel like you you mentioning that. You know, even though things are more digital now that these things still cost a lot of money. And that and when you were talking about the preregistered report, I feel like that what a preregister report is reminds me a lot of writing a grant. I come I also come from like a biological science background. And I feel like to really be able to get money to do any of the experiments that you want to do. You have to like, write a proposal of the things that you'd like to do in your forthcoming research. So I guess I'm wondering, a little bit how, how is the period distillation different than then writing a grant and maybe diving into a little bit more of the particulars about like, what kind of culture and technological infrastructure sort of facilitates that type of like change towards making that process easier?

Bort Penders 30:17

Yeah. We all write research grants, we hardly get any of them funded, of course. But I think that's also a big difference in the sense that a research grant is also a promise, you promise to go and do something, I need to sell that promise by, first of all explaining how good it is, but also how relevant it is. And thirdly, how likely you are to successfully completed and the but it's usually not very detailed in the sense of exactly which measurements you're going to do to what, under what conditions or why? Because there's simply no room. And in that sense, a pre registration, or in combination with a registered report is far more concrete. So it details specific studies, individual experiments, sometimes even with exactly which controls are used, and why as much detail as possible, about what's going to happen to the point that somebody could repeat it. No grant proposal contains information that even comes close to reaching that sort of objective. That's why probably, if you write a grant, and you actually get it awarded, then you'd hire ref to possibly connected if you're committed to this way of working to like attend pre registrations in order to encompass the entire thing that you're trying to achieve, all of which depend on one another. So you could never file them simultaneously, you'd have to work through them, one by one, or perhaps into parallel routes.

Impacts on Peer Review

Sarah Kearns 32:09

Yeah, that makes sense. I guess you kind of mentioned this about having like these types of like pre registrations and register reports having these new types of editorial boards. I guess it kind of makes me wonder how that would change peer review. I think people now realize that peer review is requires a lot of effort and time, and they want recognition for for that work. So I guess I'm kind of wondering, maybe a little bit more culturally, how this would impact? How review would work and how publishing would work?

Bort Penders 32:52

Yeah, review is actually a great example of a system in which the second moral program that I talked about, that's more about inclusivity. Is is gaining ground. Because one of the things that we see in this hyper rationality program is that essentially, reviewers are also not trusted anymore, because they're just as bias. And so whatever they find about or assess about your paper is probably biased. So why would you trust it? So peer review is broken from that type of of risk perspective. But one of the ways we can fix it is by Well, changing what peer review is, for instance, by doing open review, meaning that you publish the reviews alongside the paper, and through that facilitate surveillance of the quality of reviews, you could also not review at all in the sense that you publish the first version of a paper, whether that's on a preprint server or on a specific journal platform that simply does things that way, and then allow anyone to comment on it, all voices are included, or could at least potentially be included, in order to improve that analysis. And then a revision of the paper would take a majority of them, never everything, but a majority of them into account. That also would be visible, it wouldn't be public. And participation is facilitated there. And in that sense, that will be more inclusive than open a review, which is still about a select few voices being in control, even though you could see what happens and then you could start complaining or whatever, but that's a lot more difficult than if you would open it up. So we see all those changes happening right now. And What if you're invited to be a reviewer on a paper nowadays? You never really know what's going to happen? Because there are so many different strands of possibilities that open up. So is it closed? Is it open? Is it blinded? Is it not? Is it participatory? Is it not? You have to check for some journals, you know, for others you don't. So there's little surprise elements to it, which is not necessarily a bad thing. It also exposes all of us to different ways of working. And you can learn from all of them, and invest those lessons again, at some point. Yeah, but peer review is a great example of a culture that is changing as a result of these moral pressures on sites, trying to respond or responding in multiple plural ways at the same time. And I hope that no one is going to win, that the plurality continues that we have a sustained plural identity there. Because some types of work simply fit types of review better. And if you find the right combination, then you stick with it. But it doesn't mean that everybody else has to do it that way, all the time. So it's a great example of something that's also quite visible to most people in science, because the Reform Movement is something I still have never heard about it. But changes to peer review. everybody notices.

Sarah Kearns 36:35

Yeah, I feel like what you what you said earlier about the panopticon, and having that like fear of being being watched. I've read accounts of open peer reviewers feeling that they're like, "Oh, I can't be as harsh as I would like to be because like my name is going to be published alongside this this review." So I don't know if that ties into what you're saying.

Bort Penders 37:00

When you think about it more than two minutes, then you notice that you can be very critical and still polite. Yeah. And if that is your way, then that's fine. And it's also because it's physical. As long as you stay polite, and all your arguments make sense, then it's going to be very difficult for people to disagree with your diagnosis. Or they might disagree on details. But it's not then that they're that they'll be accusing you of something doing something terrible. Yeah. Or is important and style. Well, in the context of this paper, civilization.

Sarah Kearns 37:47

Yeah, for sure. And at the end of the day, like any any critique is, ideally it's towards bettering science and bettering the method or whatever it is they're critiquing.

Flexible Systems

But kind of what you said about you don't want there to be a [single] solution. I feel like having that type of plurality makes it very challenging to have effective bureaucracy or infrastructure that sort of accommodates for, for that type of flexibility.

Bort Penders 38:24

Well, most of the infrastructures that we have are more sort of omnipresent, they don't apply to everyone at all times. Something as simple as a preprint server to Biomedicine scholars. It's a it's a standard thing that they've been using now for a couple of years, maybe the better part of a decade, so everybody's gotten used to it. In the humanities, people have no clue what it is, and probably will not use it because it's so new If you're a physicist, then preprints are just the way you've always worked for decades and decades. So that infrastructure already existed in physics for such a long time, without necessarily influencing how all the other disciplines were working. Some people didn't even know. That plurality has always been there, and sustaining it or updating it by making it slightly more plural, and making the boundaries slightly more visible. It's not going to make anything impossible.

Just pointing out that [pre-prints] have value is something that's relatively new, but it's not going to make it more difficult to do our research. It just means that some people commit to one model of credibility building or trustworthiness building and others seek out different tools to do the same thing and then they use a slightly different infrastructure or build their own if they want to innovate. But there are plenty of solutions to do that. And it's not at all sort of revolutionary, to be plural.

Sarah Kearns 40:18

At the beginning, we talked about how there seems like to be this mistrust in the system at large. Does that amount of plurality across the different fields make more people receptive to that type of plurality?

Bort Penders 40:39

So when I said, concerns about scientists at large, indeed, there is this plurality of the type of concern, but also how visible or how present the discussion about those concerns is in different fields. So some don't really talk about it very much — chemists, physicists — they're not too concerned and feel that the way they work is quite efficient. Then there are fields that are heavily engaged — psychology, life sciences, for the most part — who recognize various degrees of concerns, internalize them, make a big show out of trying to solve it all. Digital humanities, by the way, also quite extensively engages with with this matter. But the non digital humanities largely ignore the debates. If they engage with the debate, it's because they find the debates interesting, but not necessarily that they feel that it applies to them.

When it comes to publishing infrastructures, there always was this plurality, some publishing books, others in journals, and others, again, in conference proceedings, so very much different. It's not something to be afraid of. It's just something to embrace up to a certain point at least.

Motivated by Shame

Sarah Kearns 42:32

We talked a little bit about how, you know, fear motivates people. How does shame, influence people's behavior and scientists behavior?

Bort Penders 42:43

Yeah, so if you do not conform to a very specific norm that prescribes how you should behave, then that can make you feel weird. That's not something that's specific to science that is about daily life, but it also applies to science. I always give a few examples to students, for instance, and I asked them what our norms are guiding our behaviors right now in a lecture hall, or in a laboratory or whatever, and people come up with various obvious things you sort of exhibited respect to one another, and if you don't do that you're called out for it and then you'll probably be ashamed of your own behavior, or you're ashamed of somebody else's behavior.

It's not so different in in science.If a new norm for what is considered to be the right thing to do in a given situation, is developed, whether that's a pre registration or not, or something else, it could also be something really tiny, not living up to it in a context where most do or at least where you feel you should, even if others don't, that can induce feelings of shame, shame about you feeling that you are not doing enough. It's very much an internalized thing.

One of the beautiful examples that Elias, one of the authors that I rely on, who came up with this social explanation drives civilization. Already in the 1930s. He uses an example of a spoon and a pot of soup. The way we've used the spoon in a pot of soup has changed. So, a long time ago, we all use the same spoon to eat out of the same pot of soup. When you update that practice to give everybody their own spoon but still share the same pot of soup, if you don't forget, and use somebody else's spoon, then you'll notice (and somebody else might need not even notice) but you’ll notice yourself and you feel shame because you're not living up to this [new] norm, this ideal of civilization. And because table manners advance, certain point, everybody gets their own pot, and you use your own individual spoon to eat from your own individual pot that allows further different degrees of civilization.

In this case, table manners, which is quite banal up to a certain point, illustrates how the process works and how shame is a driver and constant reminder of conformity. You can imagine this process going forward with different further specializations of further civilization in how you do that, how your table manners develop, where you put the pot of soup, where you place the spoon. Do you need a napkin? How do you fold it? Where do you place the spoon on the napkin? You can develop in formal rules, informal manners continuously. But every time you break one, or you don't live up to when further increase in ambition, with respect to these developments, shame confronts you, when you're confronted with shame, and that drives you. The other idea is disgust. If you see somebody else, put the put the individual spoon in the shared pot of soup, then you're disgusted by it. Because it does not live up to how you should behave in that situation. So that works both ways.

[Shame and disgust are] a strong driver of changing scientific processes of complying to the use of a pre-registration of a registered report of living up to the bureaucratic requirements associated with science. And it sounds boring. But it's the boring things that make systems tick.

Culture Shifts

Sarah Kearns 47:54

Yeah, that makes sense. I love that soup analogy. But that I feel like, up until recently, open science felt like that uncivilized behavior, but it feels like more and more — and maybe that's just because I'm in I work for an open publishing group — that more open science behaviors, like pre registered reports, feels like it's becoming more of the normal. So I guess I'm wondering how is that sort of shame or disgust flipped?

Bort Penders 48:26

Yeah, so first of all, again, there is plurality. These forces are not equally powerful in every place. Some feel that pressure very strongly to the point that any tiny deviation might be punished immediately. Psychology and cognitive sciences, they are the furthest in that type of development, with life sciences, for instance, racing behind by quite a bit. The “more advanced,” that's also one of one of the terms used by some of the theorists that that I quote, and the more advanced your process is, the more disgust or shame and the more intense you will react to somebody else's process.

If you're trailing behind, then you don't necessarily feel shame or disgust on behalf of those in front of you, but because they might represent an elite that you seek to copy and copying has costs additional labor participation embryo accuracy. But it also has rewards in the sense that you might become part of that elite. As you try, the elite moves ahead as well. So it's a race that you're never going to win. It does get you rewards, it does get you something, but the cost can be high in terms of labor, not in terms of money.

Sarah Kearns 50:24

Yeah, I'd like to think, though, that the value like scientific values are changing in that, maybe it's a little bit easier to reward these different types of behaviors.

Bort Penders 50:40

The digital infrastructures that we now have, they make things visible that before were not so much. So some of the invisible labor behind writing papers, coordinating teams, designing research animal, also just executing that research, a lot of that invisible labor, labor can be made more visible through those infrastructures. It's not perfect and absolute in any way. But at least it can facilitate that.

One of the great things, of course, is that that publishing too much has become something that some actors are disgusted by, because it represents conformity, certain norm that is now frowned upon. And if you don't reward people in extremists, at least for publication, then you can also start to reward them for something else. And that could be participation in this invisible labor. And up to a certain point that's happening, but not as much as could happen.

Sarah Kearns 51:49

Yeah, that makes sense. You saying that there’s a push back to publishing too much, reminded me of the question I had about the registered reports. You said that these processes inevitably take time and effort. With the pandemic, still in my mind and trying to publish things quickly, is there a limit to that type of bureaucracy, in cases like that [where it’s critical to publish a lot quickly]?

Bort Penders 52:35

The general idea is still the pursuit of efficiency, at least especially in this this hyper rationalist model. The efficiency is achieved by publishing more and ensuring that what is published actually has the value that they seek. In that sense, publishing less can still be associated with more efficiency, and you just sort of cut off all the bits that you don't need anyway and ensure that the parts that you have are trustworthy and have high quality.

Enhancing publishing, and then this other moral program — which is far more about diversity, respecting diversity, and inclusivity — also means that you don't publish more, but that in the publications that you do achieve, you honor, that diversity of perspectives. That's also work. That doesn't happen by itself, it's actually quite a bit of work to try and achieve that. It also requires bureaucracy to organize. It's a completely different bureaucracy, but it's still a bureaucracy. So in that sense, again, there, it's not contrary to another, but it does. It really asked of us that we imagine this scientific process to look different and and targets different goals.

Sarah Kearns 54:21

Yeah, that makes sense. But I guess I wonder who's deciding what these goals are? Is it a community organically oriented thing? Bureaucracy to me implies sort of like a top down sort of structure, but I don't know if that necessarily is what you mean here.

Bort Penders 54:39

It's not extremely top down because some of these bureaucracies are built bottom up by this one journal trying out a new idea for publishing and then it catches on because it aligns with many of the values that people in the community hold. Deciding the goals is still something that's done by the community, and also means that those who are not in the forum might disagree, either a little bit or violently. And we see communities of pre-registration skeptics, we see discussion about the value of all those bureaucracies, we see competition between the pursuit of efficiency and distributing through preprint servers things as quickly as possible. Calls for slow science suggests at least, almost go in the other direction. That is not necessarily in agreement with each other all the time. So there are different articulations of those what the goals are. And that brings us back to the plurality that I've been talking about all this time.

Sarah Kearns 55:51

Yeah, I think that's always going to be the the answer. It's a multitude, a multitude of things. I know we could talk about about all these things forever, but I don't want to take up too much of your of your evening.

Bort Penders 56:06

Thank you for having me in this podcast. And I look forward to hearing what people think. And everybody is, of course, invited but not required to read the paper that that we talked about. And yeah, thanks for the interesting discussion. And see you soon. Yeah, thank you.

Comments
0
comment
No comments here
Why not start the discussion?