Balancing Trailblazing and Tradition

In this podcast interview, we learn from Lennart Nacke, a professor of Human-Computer Interaction in Games at the University of Waterloo. Being so interdisciplinary, he has an interesting perspective on what kind of work is recognized within a department, which we explore in depth here, in addition to considering the impact technology has on disciplines and the importance of balancing “trailblazing ideas” with the core tradition.

SUMMARY KEYWORDS: evaluate, interdisciplinary, citations, tenure, understand, university, papers, guidelines, teaching, games.

Sarah Gulliford 00:30

Thank you for joining me. I guess to start off with this podcast, could you briefly introduce yourself?

Lennart Nacke 00:42

Yeah. Hi, Sarah. Thanks so much for inviting me to the podcast. My name is Lennart Nacke, I'm a professor and research director of the HCI Games Group at the University of Waterloo in Canada. I study what makes games engaging, and how we can use them to improve products, systems, and services, or our own health through Exergames, for example. And, I also advise organizations on effective gamification strategies and user experience strategies. Those are my fields.

Sarah Gulliford 01:12

I first came to hear about you and your work through a series of tweets that you've had about your strategy for developing, citeable papers.

Given your background in game theory — and maybe that gamification a little bit makes sense there —

where does that experience about publishing come from? What has been your motivation to share those tips? What's the community response sort of been with that kind of public engagement?

Lennart Nacke 01:44

Yeah, so it, of course, comes from experience, I think I've been in the professor game now for more than 15 years. I've published more than 200 academic articles. I've edited textbooks, I've been an Associate Editor for several journals. I've actually run and organized several conferences in my field. In HCI, we publish a lot in conferences. Doing a conference is a big deal and have done like papers, chairing, and things like that as well. That's kind of where a lot of it comes from, just from working as a professor and also like being on tenure committees and things like that. Just understanding what people are doing, what people are looking for.

And of course, from my own experience, having sort of navigated between the disciplines, I'm a very interdisciplinary scholar, like even my Bachelor’s degree was in Computational Visualistics, it wasn't even in proper computer science. It was an interdisciplinary degree that had lots of psychology and humanities angles in it, actually medicine as well. Yeah, very interesting, radiology, you know like image processing for radiology.

I ran into this issue of a lot of students asking me, “How can I write better papers, how can I be more effective as an academic?” So I started mentoring my own students. And then at some point, at some of these conferences, I was sharing people were asking me, “Hey, can you give a workshop about that, I think it’s very useful.” And so I started giving these workshops. At some point, I was just looking at the numbers of people in these workshops, and looking at the money that conferences are making off of me running these workshops, I'm like, I should probably do this as my own thing. So I started my own little business, giving some of these things out and like, recording little courses and things like that.

If you start your own business, then you got to be on Twitter, you got to be on LinkedIn, you got to be talking about that stuff to get attention. That's a double-edged sword. Academic Twitter is not the nicest of places. A lot of the times I actually know what I’m doing, [but] I never actually expect how controversial [what I say is] going to be until it until it hits. But at the end of the day, all I want to do is I want to share expertise, and help people write better papers, be more strategic about the way that they approach their careers.

Sarah Gulliford 04:48

Yeah, I mean, I'll have to confess that was probably one of those negative reactors at first. Like my assumption of, oh, you're a game theorists so you must be gamifying publishing, which means playing the citation game, which is its own, has its own history of issues, but we'll definitely get into that later. But I think [something you said is] interesting: you said that you created your own business out of this type of work. It's kind of funny.

Even when you're helping academics write papers for the academic system that you're still kind of being exploited in a way for the conferences.

Lennart Nacke 05:35

I think nobody knows exploitation as well as academics, right? Like we're being exploited 24/7 in our jobs. I think I've done pretty well for myself being a full professor having tenure, it's pretty hard to achieve, but even then you feel like you're giving away a lot of your time for free. And there's just like expectations to do a lot of stuff for free and just for the goodness of your heart. I've done it for 15 years, and a lot of it, I still enjoy it. But [when you have a] beer with your friends and industry and see what's happening there, they’re like, “how are you not charging for this?” And I'm like, “I don't know, I never thought about it.” And that's when you kind of start thinking, okay, you know, maybe, maybe it makes sense to be a little bit more business-savvy when it comes to some of these things. Like you can help people, but you can also get paid for it.

Sarah Gulliford 06:39

You mentioned that a lot of your work is interdisciplinary. Could you go into a little bit more about that?

Lennart Nacke 06:45

Yeah, for sure. So I think at the heart of it is, of course, my original degree. And then when I wanted to go into games, this kind of happened after my Master’s, it was a degree where you're looking at creating computer graphics, where a lot of math is kind of like a math-heavy, computer science idea. But I kind of started this degree because I wanted to work at Pixar someday because they make such cool movies, but also do something with computers since I thought that’s where the future is at. And then, at some point in between, I got captivated by games, and thought games [are] really cool way to do this as an interactive style. And then, as I studied games, I was really looking at the impact that it has on humans, and then [I] kind of started this love affair with psychology. And then at some point in between there figured out there's an entire field that does this, which is user experience and human-computer interaction, where we’re looking at computers, but also how can we make computers better so that they actually work for humans? When I realized this, I kind of switched mid-career, which wasn't necessarily the most strategic move. If you switch before tenure, you're giving up a lot. I gave up a sabbatical pretty much, because I did that.

I ended up at the University of Waterloo, and I ended up at what is a very interdisciplinary department, the Stratford School of Interaction Design and Business. As you can see in the name, it already has interaction design, and it has business: two separate fields, right. There's also a lot of people very interested in UX, which has both design and user experience. So as part of that, I've kind of worked with different requirements that we have when we are scouting for new faculty. So when we're looking for new faculty, it's not like a field like computer science, where you look at somebody with a math background. You can come to us and you can try to understand interaction and user experience from a lot of different angles, that can be health background, or working with textiles, or working in privacy, like one of my colleagues is working in privacy and security, but she’s very interested in the User Experience aspects of that. And then we have the Fine Arts. Our current director has a background in Fine Arts. So trying to make a lot of these backgrounds work together has, of course, its own sets of challenges for how do you evaluate scholarly productivity? Again, coming back to that h-index. Depending on your field, the h-index might just not be a thing. Or, in the humanities, that just doesn’t matter; it’s just a thoroughly argued book that has a lot of substance and quality that can be a career-defining move, and it’s not as important to publish in X number of journals.

Sarah Gulliford 10:38

Right. I mean, something that you had mentioned in a brief introductory conversation that we had before this is that, you said that you're aware that citations aren't the only or even best metric to evaluate work.

How do you maybe maintain balance of paper citations with that perspective? How are you creating these like more holistic systems around evaluating work within your department?

Lennart Nacke 11:30

Yeah, I think for me, citations and things like that have been a powerful driver of my career. But that's really because nobody taught me otherwise. My supervisor taught me that, at the end of the day, it does matter that you have good output in that regard. So, for me, that was the starting point where I came from. So a lot of the advice is coming from understanding that, for a lot of the hard science fields, that’s how it works.

For a department hire, though, I think that's also where you really have to understand the person that you're working with [and the systems that are valuable to their background]. The trick is to look at the scholar specifically. It was a while ago that we were tasked with a ‘redoing’ of the annual performance and evaluation guidelines. And I think that's when a lot of the stuff happens that we're looking at, in an interdisciplinary department we're looking at.

In Canada anyways, it's always evaluated according to three fields: teaching, research, [and] service. Depending on your hiring contract, it might be more emphasis on teaching or more emphasis on research. Very rarely [do you spend] more on service unless you take [on] a dean or associate director role or something like that. I'm currently [the] Associate Director, so I have a higher weighting on service. And so essentially, then you're looking at the outcomes. How can we find evidence that the teaching is good? How can we find evidence that the research is good? And how can we find evidence that the service is good? And really, teaching and research are kind of the two big things that people [are] looking at because once you're doing service, it's internally in the university; it's easier to evaluate that because you're working a lot with colleagues.

Let's talk about teaching for a bit because there's a lot of talk about, on the academic end, how do we properly evaluate teaching. In the past, it was a lot with student evaluations and like, just the numbers that the students give you. And everyone knows if you've been in the game for long enough, you can pretty much bribe students with high grades, it's actually quite horrible. If they fill it out in person, they're usually better than if they fill it out online. White men are privileged over women and BIPOC people as well. So this is just inherent bias. So you're already getting worse evaluations just by fitting into any of these categories. So that's not good. That's not a great tool for evaluating teaching, but it's still, to a degree, it's being used. But it's getting better, at least at our university, it's getting better in terms of like having a peer actually come into your lecture, looking at the style that you teach, and evaluating that.

For scholarship, obviously, you can't just write a paper with somebody to see if they're doing good scholarship. So it's a little bit trickier. I had to do a lot of argumentation in my department, the beginning because we are part of the Faculty of Arts, so they're actually more used to the humanities-style publishing model. And I came in there [and] said, “Well, I only do conference publications, I don't even do journals that much.” So, it was a lot of educating that this is the highest impact in our field. There's a lot of citations coming from this. And of course, when you have that argument, and you're going back into that old thinking: “oh, yeah, I understand citations. I understand that metric.” For me, that still publishes a lot and gets cited a lot, that argumentation works well. But what about somebody that doesn't do that? And so how do we evaluate somebody that does exhibitions, screenings, festivals, curatorial practices, performance art? And then, we're pretty much summarizing that studio-specific research. I mean, even [on] social media where you’re educating the public, that also should count as scholarly work if it's done to a degree of thoroughness. It's up to the evaluator, but the pointers are there to say that these are the types of things that you should be looking at.

Sarah Gulliford 18:10

That sounds like a lot of the conversations that we've been having with people in the digital humanities, or art space where each thing you make is a distinct entity. So like, how are you measuring that new digital platform? Or that exhibit that happened to be during COVID?

How are you evaluating an experimental piece that doesn't have an evaluative metric, yet?

Lennart Nacke 18:48

It's tricky, right? Because at the end of the day, you're looking at impact, right? Like, does the work have impact? And how do you evaluate impact? It's simple: It's really eyeballs on something. If your work doesn't produce a response, in some way, then it's hard to evaluate, it's hard to see the value. You can ask is the media talking about it? Is there a public response? At the end of the day, when you go up for tenure, every one of these processes is all about proof of impact or discourse. So even [if] you publish the book, maybe it's not getting a lot of citations, but it really stimulated an interesting discourse. And you have the evidence that shows that discourse with the public or maybe it formed a policy or even have political implications.

Sarah Gulliford 20:26

Right. So I guess you mentioned that

Your university or your department put together promotion guidelines recently. Who was involved in that?

Lennart Nacke 21:08

Yeah, I think it's, a lot of our stuff is committee-based, right? Like you form a committee, the committee works on creating these things and gets information, of course, from the entire department. In our case, I think it's nice because I've actually been in two different departments where this kind of interdisciplinary work has happened. It really is a collaborative process. I also think Canada's a little bit nicer than United States where you find that universities hire people without [the] intention of giving them tenure, right? Like they hire people on tenure track, and then it gets really competitive. In Canada, that is not really the case: when we hire somebody on the tenure track, we fully expect them to get tenure.

It's young departments, specifically in the school that I'm in now, I'm the only full professor. So it's a very young faculty, [with a] lot of hires in the recent years. And so we want to make sure that everyone gets evaluated properly, and everyone has a clear career path forward. In the first couple of years, we’ve had trailblazers advocating for the impact in their field. And then you take that feedback, and you use it to inform these new guidelines. And so, you know, okay, somebody has this background coming from this department. The interesting thing for our school at the beginning, everyone was cross-appointed, because we weren't actually our own entity yet. We became our own school/department just a couple of years ago which had a lot of benefits, like now we have our own structures and processes in place. But it also meant need[ing] to have these proper guidelines. Being in that position, we put something in place that is really comforting and worked out for everyone that is already part of this. It's a lot of negotiation and information gathering with other departments in terms of [learning how each of them put their tenure and promotion together and using bits of each].

Sarah Gulliford 24:11

Yeah, that sounds like the newness of the department helped keep that energy or sparked that change.

How do you think that more established organizations or universities or departments could perform that kind of same re-evaluation process?

Lennart Nacke 24:39

Yeah, I think it's super tricky. If you're an established department, let's take like our English department, or like some departments that have been around for a long, long time. The department has been around, the more established profs you have, the more established process you have, the more resilience to change you have in that department. You'd need somebody in that department that has the accolades, to be trusted by the more senior staff and folks, but also has the willingness to change something for the newness of things. So hopefully, you know, you'd find a person that fulfills all of these criteria, but it is very difficult. Depending on the age of the department, even if people can see how it's sensible, [it still requires a lot of change]. But if it's like very clear how your discipline does things, then then why change it, right?

This opens up a whole different discussion, should we even have disciplines in the university of the future. To me, the really good research really happens at the fringes where you’re infusing your search for something from another discipline. If you're bringing a method or an approach from another discipline into your own discipline, you usually have a hit paper on your hands, and you're bringing something that's very likely going to be very popular, because it's new. Maybe I'm speaking from a very privileged position, I just want to acknowledge that, you know, that might not be as easy as I think it might be.

Sarah Gulliford 27:07

Yeah, no, that's totally fair. I agree that, if you're borrowing a concept from a different field, it's like innovative to the field that you're bringing it to, because it's like this new way of looking at something.

It almost sounds like because like you're so interdisciplinary, that’s helped you be successful in the traditional sense but also continue to do that trailblazing.

Lennart Nacke 27:41

It's hit or miss, right? If you look at Academia, there is a big traditionalist wall that you feel from all sides.

I think people, for example, people that are currently experimenting with AI. This is a very difficult topic for a lot of academics to wrap their head around, because it will change - it already changes our students do their work. And it will change how we do our own work in the future. And I think that is very scary for a lot of people that a lot of the established processes are. There's a lot of questions about data security and privacy that people that are trailblazing are not thinking through. Just because it's company-driven right now, it also means it's not research-driven, so there's a lot of pushback from traditional researchers. I'm always amazed by it, I use a lot of these tools in my day-to-day and I'm always amazed when I see a new update or something come out. I'm scared now and also excited.

Scholarly disciplines are changing, technology is changing, and our whole environment that we do research is changing, and I think it's happening more rapidly than we're willing to acknowledge. There's still traditional holding on to traditional values, but I think it’s changing the same way with we've seen through the LGBTQ++ community, rethinking of gender roles and how we perceive gender and sexuality in the last couple of years. In the public eye, I think in a similar way. These trailblazers for disciplines will change how we're perceiving disciplines because the traditionalist views might not be right. The young scholars are ready for a change and ready for mixing things up so, at some point, it's going to change the way that things are set up. I think in the end we’ll all be better for it, specifically in research.

Sarah Gulliford 31:37

Any paradigm shift is always going to be, I think, uncomfortable for anybody, probably even the trailblazer type people like. Even though they have an aim, there's no ground yet to know where you're walking or so. I personally struggle with this. Our organization is always trying to anticipate or push the boundaries of publishing, but it's always hard because even though we know where we'd like to go, there's nothing there yet.

How do we create something that is stable and secure enough for people to trust such that it’s rooted in tradition?

So you can be like, we understand what you need and what you'd like. But also a thing over here could be much better.” So it's like trying to create that invisible path that doesn't yet exist, to get people to change to something that also may or may not work, right?

Lennart Nacke 32:43

If you ever read Karl Popper's works on science, right? That's what science is all about. It's all about doing things that can be wrong. And it's like, sometimes we forget that because we're so focused on the significant result, but it’s okay to have an idea and test it and that just doesn't go anywhere. But, between all the hassle for tenure and promotion, and doing things that are meaningful, it creates a whole different discussion. I think that doing work that's meaningful can also mean finding nothing and reporting properly.

Sarah Gulliford 33:30

I had a friend in grad school, who told me is this: if you're not failing at least 50% of the time then you're not, like, learning anything.

Lennart Nacke 33:37

I think that's how we get a thick skin, too, as researchers, right. In some regards, we're losing some of that. I feel like a new generation that's coming in — this might be through COVID — I think there's a lot initial fear of this rejection, or not doing it right the first time, or not being able to look it up on YouTube. You would hope there would be more of a willingness to make the mistakes and own the mistakes as well. But I guess, let’s bring it back to social media. This is a generation that grows up with social media, and it's so easy and so quick to get bullied on social media for something that I think people are just, like, more afraid to make mistakes in the public eye.

Sarah Gulliford 34:51

Yeah, coupled with it's so easy to share anything.

Lennart Nacke 35:05

Everyone's doing it. And it's not always going to be perfect. And that's okay. I think school is doing the opposite, because they want everything we do to keep perfection. I'd love schools to be more focused on just doing something for the sake of creation and not so much for everything needs to be perfect, but you're learning something as you're creating something and you will learn from it not being good. There's a way to give feedback and be courteous about it. But let's give feedback so that we can improve, and let's be open to receiving feedback.

Sarah Gulliford 36:02

Right, that could come back into the tenure promotion guideline update.

Is that shift in educational structure, like, built into the teaching component?

Lennart Nacke 36:22

I think it's really important for us to build programs and, and even build class structures that are very open. And specifically, in my department, we've kind of moved away from the midterm structure. I haven't had the final exam, in like 10 years of teaching: I've done project-based teaching. So you give people a problem, you help them critically think through that problem and build a solution often together with their peers in the project.

I don't see math ever going to a project-based work. It's very clear how you do something and it's very clear the steps that you need to go through to get something done. And I think that's still important knowledge, right? But I think there's also a need to understand that as tools for technology. Technology permeates every one of the other fields. If you were an English major, you should be playing around with ChatGPT, and you should see how that can infuse your poetry or what it can do with your writing. Because if you don't, like, there's a lot of that is going to be happening, whether we want it or not. And the tools are really becoming quite powerful. It's really important for people to have access to that technology and have access to that understanding. But with that, of course, also comes the quest for validity and teaching people that mindset of being able to evaluate what is a fact and what's not a fact, because, obviously, that's getting much harder, specifically in the US. This ability to distinguish bogus from fact, I think, it's going to be one of the key skills of anyone with a higher education going forward. It's going to be so important.

Sarah Gulliford 39:58

Yeah, that's such a tough thing to learn. I remember back in computer class in middle school, it was the thing that we talked the most about. And we're still talking about, like, decades later. It's hard, and technology's always going to be make it harder and different. So it's always something that we need to keep learning.

How were the guidelines taken by other departments within your university? Or how has your university supported your efforts?

Lennart Nacke 40:41

Yeah, I think the wonderful thing is that, within the Faculty of Arts, we have a lot of different departments here at the university. And so there is talk between departments as other departments are looking at overhauling their guidelines and considering hiring more interdisciplinary scholars. I do think, within the faculty level, that's still pretty relevant. There's a lot of cross-pollination happening, I would say. I think it's a little bit different, more difficult once you hit the university level. So one faculty might not be talking to another faculty the same way as within the faculty, the different departments talk to one another. So then it can become much more difficult to do that cross-pollination, and they might also not have the same interest, right? Like, for example, we have a very successful engineering department, and they might be very proud of what they've set up [so change is less likely there]. I can say that there was a lot of interest in health, for example. And so some of the engineering departments really infused a lot of their stuff with health, because we don't actually have a medical faculty here at the University of Waterloo. So there was more interest in so how can we bring technology to health. And you know, I think that's super, super cool. Super interesting to see that. Even in things that you might think are more traditional, there is cross-pollination happening, just because people are interested in it.

Sarah Gulliford 43:08

It sounds like what you were talking about with English. You can try and preserve the core of what it is, but kind of no matter what it's gonna change.

Lennart Nacke 43:19

I can’t speak for that department, but I think what I'm seeing is a lot more interest in digital media right there. In the Games Institute here, where I work, there's a lot of English scholars, they're interested in interactive fiction, and interactive storytelling, right? Like, why not play around with these things?

Sarah Gulliford 43:52

Right! That's all really exciting.

Is there anything else that you want to mention before we wrap up this podcast?

Lennart Nacke 44:01

I think one of the things that I would like to give future scholars on their way as they're trying to go into these fields is to be able to work interdisciplinarily, you're also - you need to be aware of your discipline first. So don't just dive right into interdisciplinarity, as much as this might sound exciting. For this to work well for you, I think you need to first work on an understanding of this discipline that you're interested in, whether it is you know, English, medicine, biology, or engineering. So try to research that a bit to try to understand it a little bit. So that you know, what currently works in that discipline. And then from that knowledge, build your “trailblazer idea” so it’s infused with something new. And don't be afraid of that! Because again, like we said, it's going to happen either way. So be open about it. Look at all the opportunities for learning. Look at all the evolvement of tools that's happening out there and see how you can bring that to your discipline, because one way or the other, it will come into your discipline. So understand first, and then innovate.

Sarah Gulliford 45:17

Yeah, I think that's very good advice. Thank you for meeting with me. This is a really great conversation.

Lennart Nacke 45:29

Yeah, thank you so much for inviting me. And for anyone that wants to know more. You can follow me on Twitter and I'm actually @acagamic on most social platforms on TikTok and YouTube and what have you. Also, if you want you can find me with first name, last name on LinkedIn, and on my website.