Last month I had the joy of attending the Australasian Research Management Society (ARMS) 2024 Conference in Darwin, Australia and presenting on my work in peer-reviewed podcasting.
Putting the talk together was a trip. I must have rewritten it at least six times and thought often of the “success iceberg” – the drastic difference between the final result and the amount of work put in. It was a process of refinement. Many equally good talks, interesting insights, and enlightening tidbits were abandoned along the way. I’ll aim to take you down some of these rabbit holes in coming posts but for this one I thought it easiest to just share the talk I gave, albeit abridged.
This is what I ended up presenting on the day and I hope it’s of worth to you:
“I’ll start by acknowledging the traditional owners of the land on which we meet and pay my respects to their elders. It is a deep respect I hold because our First Nations people know the value of oral knowledge systems. They have passed their knowledge down through oral transfer from generation to generation for over 60,000 years. It’s a real testament to the power and stickiness of verbal communication.
Now, we’ve heard a lot about inclusivity today, and I’m asking, ‘What are we doing practically to change the way we communicate so we are not excluding so we’re making knowledge more accessible to students and non academics, and even to academics of different disciplines?’. Scholarly audio, is part of the answer.
I want you to imagine a world where academic ideas aren’t confined to text based journals behind paywalls. A world where complex theories are as accessible as your favourite playlist. That’s the reality I’m working to create.
My name is Kate Murray. I work for University of Melbourne where I’ve been making research-based audio outputs since 2014. So 10 years this year, that kind of crept up on me. These are some of the podcasts I produced where I’ve done multiple episodes for each and down the side of this slide, some of the affiliations I’ve worked with. Urban Studies is the newbie up there because this year I produced a very special podcast series in collaboration with them. It’s a research podcast, formerly peer reviewed through a journal. It’s the first peer reviewed podcast in Australia, definitely the first in the discipline of Urban Studies, and I do believe it’s a world first as well.
In making this I’m challenging the norms of traditional and non traditional outputs and working towards a dominant design for peer-reviewed podcasting that can be scaled up and adopted by other journals and organisations.
I’ll take you through our process, but first I’m going to put some definitions in place, because we’ve heard quite a few podcasts mentioned today. What I’m talking about are podcasts made for research purposes by academics or research institutions. Not made by journalists, or enthusiasts, and not about Higher Education industry.
So I’m not talking about Hidden Brain, which is a fabulous psychology podcast made by Shankur Vendantem, who is a journalist interviewing researchers. Amazing, but we’re not talking about that. Not talking about enthusiasts like Lindsay Green, who loves escalators, and decided to make a podcast all about escalators called People Movers which was very successful and wonderfully niche. And we’re not talking about industry podcasts like On The Reg by Inger Mewburn and Jason Downes which is a wealth of knowledge for academics but I’m not talking about that today.
I’m talking about scholarly audio and that is audio content capable of being peer reviewed. It doesn’t have to be peer reviewed, but it’s capable of being because it’s part of the research process, and creates new knowledge. Scholarly audio is permeating the research process and can now exist across all stages.
So you might think field recordings, oral histories, interviews with subjects, archival audio – all of these are capable of being peer-reviewed as part of the research process. These are all in the data collection stage of the research process.
In the analysis stage are a growing number of audio databases, some open access and online, others exclusively for research purposes such as The Big Australian Speech Corpus documenting variations and evolution of Australian English or the Australian Acoustics Observatory in Queensland which captures nature soundscapes for animal conservation.
Research outputs are also scholarly audio but the important definition here is they are recognised outputs and not a promotional tool for pre-existing text-based outputs. So scholarly audio is across the research process, it’s becoming part of the methodology and I even know of one academic striving to complete an entire research project without any text involved across the process. I’m keeping an eye out to see how it goes.
Let’s look at some examples. Dallas Rogers is an amazing academic at the University of Sydney, who makes City Road. This podcast is a great example of multidisciplinary practice, because he gets academics from all different disciplines to talk to him about issues regarding his research and interests in urban studies. And through that process, new ideas are generated, collaborations are formed, and it all folds back into his research work. Not only his work, but his interviewees and listeners too. It really is a flourishing network of idea generation.
Another example, this from Connected Cities podcast which I produced. Here we have Jodie McVernon in public health and Roger Keil in urban studies. Now, Roger was an academic visiting from Canada, we held a small graduate research get together, but we also recorded it and then edited it and turned it into a podcast. This is a good example of how you can fold podcasting into the activities that are already happening at a university. It doesn’t have to be in the studio. It doesn’t have to be difficult. It doesn’t have to be a new thing you are doing.
The other cool aspect of this event – it was called Disease and the City, we held it late 2019 and I put it in the vault for a bit, thinking I can publish this whenever. Then 2020 happened and suddenly it was very timely content. Of course we hadn’t talked about COVID in this recording so I contacted Roger (Jodie was now very busy) sent him the edited episode and asked “can you just talk into your phone and send me a voice memo of what are your thoughts right now, given all we discussed and now pandemic and lockdowns?” He sent it through, recorded from his home as everything was shutting down, and we put it on the end of the episode. It was that simple. It wasn’t high quality audio but it was raw and relevant. I really loved that.
One more example I’ll give quickly is Climate Talks by Jackie Peel in Law at University of Melbourne and Cathy Oke who runs Melbourne Centre for Cities – again a cross-disciplinary collaboration. This podcast is based around the Conferences of the Parties (COPs) held annually to address the United Nations Framework Convention on Climate Change and is specifically designed for building partnerships and collaborations and for climate change activism. It’s won multiple awards and is a shining example of the impact scholarly podcasting can have on partnerships and diplomacy.
Now, accessibility is a big part of why I’m passionate about audio. It’s essential for inclusive research and teaching to have multiple modes of communication. Podcasts are more than just an audio file attached to an RSS feed. They are images, they are chapter breakdowns, timestamps, transcripts, websites so they are actually various types of communication packaged together. Even real-world events can be part of podcasting which builds community too.
Other important benefits real quick, I will highlight a few on these slides. One of the things I love about podcasting is you’re welcome to geek out. You can go down rabbit holes off rabbit holes off rabbit holes, and your audience will come with you. So it’s about quality, not quantity, with the audience. I’ve heard in a few presentations today questions around listener numbers and audience stats but to me, I don’t care so much about how many people are listening, as long as it’s the right people, as long as we are getting collaborations and generating new ideas. That is the return on investment I’m wanting to see.
Okay, so we know scholarly podcasts are already out there in a variety of ways – why peer review a podcast?
I don’t know how many of you were in Steve’s talk earlier today, but he was talking about how he didn’t get permission to make his amazing podcasts and videos for Monash University, he just made them. Actually, every single one of the examples I’ve shown so far, not one of them has ‘podcast’ in their job title. Not one of them has FTE committed to podcasting. I don’t even think any of them gets funding for their podcasting. So this is really unrecognised work being done by our academics. Yet very impactful.
Next, how are we ensuring academic rigor? Because there are a lot of podcasts out there. Ultimately, peer reviewed podcasting seeks to revolutionise academic communication, making research more accessible while maintaining scholarly standards. It is part of the digital scholarship movement, and part of legitimising academic work.
Peer reviewed podcasts have existed for a few years, and they’re in many different forms, and they’re very experimental. One example is the Open Peer Review Podcast made by Lori Beckstead in Canada. In this podcast, the author of a journal or a book chapter or whatever, sits down in the studio with their reviewers and gets a live peer review that’s recorded. That is a nightmare scenario for a lot of academics, but the results are actually friendly and fascinating. It’s a really radical idea challenging what is peer review, its structure and purpose.
Many peer-reviewed podcasts are very innovative like this and I love them but I didn’t want to do that. I wanted to make something that was more familiar, something akin to what we already understand as a traditional academic output, to bridge the gap between experimental and traditional.
And I cannot talk peer-review podcasting without mentioning the book Podcast or Perish which delivers a great analysis of the current state of peer-review publishing and how podcasting can act as a disruptor to better the industry. Lori Beckstead is also a co-author on this book as are two other impressive scholars in this area: Hannah McGregor and Ian M Cook. All worthy of some online following if you want to learn more about scholarly podcasting and see some peer-reviewed podcasts in practice.
As I said, I wanted to create something different and the path ahead was not easy, nor was it clearly defined. It took me about a year of just talking about scholarly podcasting to everybody I could, finding my idea, honing it and trying to find how I could make my peer-reviewed podcast and who would join me.
I joined forces with Urban Studies and the wonderful Michele Acuto and we started working our way through a series of tough questions together: How to implement feedback? How much soundscaping is allowed? How to ensure diversity of voices? Who qualifies as interviewee? Will it be open access or paywall? What type of peer-review? How do we split up sections? How do we reference other works? How long should an episode be? How do we apply identifiers and what information gets published? We even unpacked what is a podcast.
It was a true collaboration. I didn’t always like the answers but compromises were made on all sides to get this prototype out. The result was “Walking the city with…” a six episode series of peer-reviewed podcasts.
We saw the series akin to an issue of a journal, each episode like an article therein. The podcast series is not aimed at the public, so we’re not doing explainers of high level theories of urban planning. The target audience are Urban Studies academics, the same who would read the journal.
And here is the slide probably everyone wants to see, this was our peer-review process in a very simplified manner.
Now that first green block is the biggest because it encapsulates most of the making of the podcast. Interviewing, editing, scripting, studio recordings, editing again, music choices, transcribing and more. The first submission was a series of audio files, transcripts, and images sent to the Editor. In this first round they reviewed all episodes as a set and if acceptable in this initial form each episode was sent off to peer-reviewers identified by the journal.
I then received feedback from reviewers and went to re-edit each episode accordingly. However, not all the feedback could be implemented immediately due to the nature of audio and our choices to interview academics in the field. Recording in place made for wonderful engaging listening with layered meanings, but it constrained us when it came to editing. So some feedback we could implement in this series and some we had to take as iterative.
An example of iterative feedback was that all the interviews were conducted by the male co-host (Michele Acuto) and they wanted the female co-host (Caitlin Morrissey) to conduct more. We couldn’t go back and re-conduct interviews with seven academics from around the world on location and we couldn’t conduct a series of new interviews to create gender balance within this series. Instead we listed this as iterative feedback and in the second series, which is already in-train, Caitlin is conducting the majority of interviews.
We haven’t quite figured it all out, so it’s still an emerging process, there’s still questions to be answered, and there is a second season that’s going to come and we’re going to pull that iterative feedback in, we need to make decisions about how we’re going to scale that up. So what does this look like when people are submitting podcasts to the journal? How do we streamline that process as well, we are grappling with those key questions.
My ultimate goal here is working towards a standard of practice which could be adopted by anyone and scaled up.
Along the way I want to keep exploring those big questions and possibilities around peer-review podcasting and what it can do for research communication and for academic work practices.
Thank you.”