Journal Selection: Looking Beyond Traditional Metrics

Is there a correlation between a higher journal impact factor and increased reach and engagement? The answer has always been, "Of course!" But now with companies driving their own engagement, and alternatives to the journals becoming more mainstream and accepted everyday, the answer is more nuanced. Join our expert guests from Digital Science to discuss the NEW metrics of journal publication.

Transcript

Garth Sundem  00:01

Welcome to this episode of the Medical Affairs Professional Society podcast series, Elevate. I'm your host, Garth Sundem, communications director at maps. And today we're talking about the new face of journal metrics with Carlos Areia, Senior Data Scientist at Digital Science. And Julia Mutygullina, Product Solutions Manager at Digital Science. So, Carlos, Julia, we have spoken before. And there's a question that I've always wanted to ask the both of you. And that is, should we still be relying on impact factor with journal? So? Is there still a correlation between a higher journal impact factor and increased reach and engagement? What do you think?

Julia Mutygullina  00:50

I can start on that. And obviously, I will rely heavily on other feedback that I keep hearing from our clients. The answer is not really, it is tricky. I think that and the latest, actually the latest roundtable that we had, at maps in Madrid recently with a lot of our clients and maps, community members, it sort of evolved this question, right. So we're in the current age of commute, digital communication, and really increasing impact of other types of communication than just publishing in journal. It poses this question in front of actually in front of Pharma. Is it enough? And the answer is lately that it's moving to be not enough to just to look at the journal impact factor in their strategic planning in their decision making and further in communicating. So that's, that's why we think that it is really important to touch on that.

Garth Sundem  02:01

Are you saying not enough? So does that imply that impact factor is still a factor in the decision making for journal placement, but not the factor? Not the only factor?

Julia Mutygullina  02:13

Obviously, I don't think and again, this is my opinion, but I think that a lot of a lot of our clients and pharma will echo that, that impact factor, it is still something that is very important that stays on the scene, and there is no probably forecast when it's going to be completely replaced, maybe never. But what we're hearing is that even some bigger companies, they are already not relying only on this metric KPI and smaller companies. What we know is that they in some cases cannot rely on it at all, because of the nature of their activities, the size, and other specifics. So I think that that's probably the best answer at the moment. It is still important, but we need to look into other metrics and other factors

Garth Sundem  03:10

too. Okay. And especially, I guess, reach and engagement. So the I guess the question was, is Impact Factor correlated with reach and engagement? If it's not necessarily Carlos, what, what is correlated with reach and engagement?

Carlos Areia  03:26

Well, thank you again. And Hi, Julia. Thank you, again, for inviting me for for this podcast. So yeah, I think that there is obviously some a bit of correlation, in the long term, do have the dissemination plans to have like engagement and public reach? And that will even eventually probably translate into into the Impact Factor? A couple years, well, in the next year, a couple of years after, but like Julia was saying, yes, there is like this move now with, with all the digital tools and all the data that's available to move a bit from or complement the impact factor with other metrics that are more dynamic, like real time public reach and public engagement or, or doesn't need to be necessarily real time but like not waiting until the next impact factor to see how are we changed. So I think increasingly, our customers are looking to get like some more real time insights. And their use cases can be quite specific. And I think one of the limits of the impact factor is it's basically a number and there are some information that you can't Well extrapolate just from a number. And so I guess when we develop some of our tools, we took that into consideration and will help help our clients defined other ways of impact that some of them can still be a bit static plus some of them are more dynamic and change through time.

Garth Sundem  04:57

That's interesting. You know, I think for medical for As publications teams, it always has been, you know, you place a study at a journal with a high impact factor. And the high higher impact factor is your score. It's like a hockey game or a baseball game or a basketball game, or like Euro 24, which I'm watching right now, you get a higher score, you get a higher impact factor, and that shows that you've won. But now, we're looking at different kinds of metrics to show how well we do. And it's almost on the team itself to show this region to show this engagement. And it's not just saying, Hey, I got an empty JM. It's saying, Here's how many people we drove to engage with this content? Is it now on Medical Affairs teams to be driving the engagement? And not just dropping a study in a journal with high impact factor? Do we have to be driving engagement?

Carlos Areia  05:58

Yeah, I think it's a whole team effort. And I love that analogy of a game score. But what I'm asking is the Impact Factor doesn't quite tell what game is it playing is baseball is it probably is a football. And I think that's where other types of metrics come in. So for example, a journal with high impact factor does not necessarily have the highest attention in news or in a particular stakeholder group inside x. So for example, if the game that you want to play is to reach a particular audience, a particular stakeholder group in social media, your metric might not be reflected in the impact factor. And then you need like another set of metrics that are perhaps more dynamic, and can give you a bit more information to basically give you the insights into play the game that you want to play in place in play that baseball game instead of playing football, like the rest of the the other teams.

Garth Sundem  06:56

That's interesting. So maybe our strategy or our game depends on our purpose. You know, if we want to, you know, we talked about reaching, we talked about resonance, right. And I imagine that your NEJM study still has significant resonance in certain communities, even if the reach and engagement is smaller. However, I can imagine trying to change clinical practice or public opinion, you know, it's a different game, it's a different purpose. And in that case, you really would be looking at reach and engagement. So does the Julian, do you think that a publication teams choice of of journal depends on the kind of study or the type of impact that they're trying to make? You know, I'm wondering if reaching scientists is different than reaching the public? And if our purpose drives our journal choice, what do you think?

Julia Mutygullina  07:56

Well, absolutely, this is very fair question. And this is the question that we try to answer every day, actually. And it's not only the type of the article, the type of the publication, obviously, some articles are really right to publish in this top tier journals with the highest impact factor, and they will be perfectly fit into the audience that these journals reach. However, there are many publications, and this is actually I was reading recently, this very active discussion online that, okay, you publish this one great article, and then you need to publish, but immediately some follow ups. And you're looking into publishing this follow up somewhere. So in this case, the strategy to try to publish the following up research in the same journal, it may may delay Actually, your sort of response collection of the response because well, obviously, the time to publish in is longer in a talk to your journals, and then maybe these types of the follow ups, they don't necessarily match with the audience of the journal. Right? So again, depending on what, what information you publish, right, will it be research clinical trials, some follow up on that, and also, it heavily like falls into which therapeutic area you're probably operating in, because again, for some therapeutic areas, these top tier journals, they won't always be the best choice. In some cases, you would need to look into something that is more specialized and reaches the audience that is like maybe 10 people, but they are really interested in what you're talking about.

Garth Sundem  09:41

You know, I know that researchers or teams kind of create relationships with journals, or they'll publish the continuing installments of their research in a single journal. And it's almost an interesting way for people to be able to follow along with the developments of that of that research, you know, follow the thread through. But I hear you saying that maybe if you have one High Impact Factor journal publication, then teams could be looking to more specialized publications for those follow ups. Is that is that is that a new strategy? Or, you know, I mean, I guess it would be if people are just trying to get high impact factor. And now we're talking about going specialized. But is that something you would recommend to people is trying to have one central study and then a bunch of satellite studies in different journals?

Julia Mutygullina  10:36

I think that this is the tendency that yeah, the industry is looking into now, again, I wouldn't be in a probably in a position to recommend anything, but what we observe in our data is that sometimes not necessarily like the cannons in one journal, but actually timeliness of providing information that can make a difference. Because what we see in like, in our data that we have, and it is sometimes interesting to look at, right, when you have this one big article that shoots and then the follow up that is published, very, very late, or they don't necessarily attract the same attention versus the follow ups that are published almost immediately, like faster after the after the core publication. So again, I think that this is still a good area for research. And we keep constantly looking at it. Carlos is one of the people who keeps investigating the data and gives us these insights. So I think we are still also curious to what is the right answer to the question, right? Is this? What is the right strategy?

Garth Sundem  11:38

Okay, I keep getting sucked into publication strategy. And I apologize for that. I want to get back to metrics to measure the impact of our publications. So Carlos, you talked about sort of real time publication metrics. But what I'm wondering is if that data is also coming from different sources now, you know, what does reach and engagement mean, in today's landscape? Is it? I don't know. Is it still? What how many? I don't know, comments you get on various forums? Or? Or where are we going to look for reach and engagement now?

Carlos Areia  12:15

Well, it depends we're looking at so I'm thinking from, like journal selection perspective. And for example, using Julie's example, on these, like satellite studies, if you want to publish elsewhere, and outside of the court journal, and the more specialized, like, for example, public engagement and reach would be like, not only how many, not only the hard numbers, but like, what type of people are reading it, and what type of authors are publishing on it. So for example, you can create trends and specialize the journalist. So for example, even inside very particular narrow therapeutic areas, or a particular brand or disease, or even a product, we can calculate, like the proportion or the number of papers that every journal is published on a particular product or very, very rare disease, a very narrow field, if you want to see that, if you want to call it that. And then for example, we can see like, like the trend of the types of people that publish it, like, are they mostly affiliated with universities, or hospitals or pharma, and now is that is developed, developed over time, for example, there are some journals that are more resistant to publishing with corporate, that then if you go like, through on the trend through the years, they have like, kinda shifted, and then are collaborating more and more, with companies as well, and are publishing more companies study than they were, like, 10 years ago. And then you can look, for example, at the readership as well. For example, are the people that are sharing data, the clinicians? Are they are the researchers? Are they both? So it's a bit of a mix between quantitative and qualitative metrics, because for example, particular narrative, like for example, in rare diseases journal, if it's probably better, if like, even if it's a small group of people, but there are clinicians in that field, are aware of it and are sharing it and they're talking about it then, like a big journal, where you have like 10 times of people of people sharing and talking about the journal, but none of them are in the field. So this again, this is like the kind of flexibility that we work with our clients to help them define what's important to them and in in the case of journal selection, what kind of metrics and what if called impact are they aiming for?

Garth Sundem  14:51

Okay, that's interesting couple of things. I always I must be outdated and thinking of journal metrics as sort of One study thing, how did our study do? Oh, there's one publication in a journal, and it did this well, and you give it a score or whatever. But you're talking about sort of measuring, or using metrics to describe the impact across a product, you know, or or across a company's activities, where it's not just a one off thing that you're measuring. But you're talking about measuring the impact of almost like an entire publication strategy. Is that is that the case? Yeah,

Carlos Areia  15:32

so in this case, I think, I think our previous poll as a few months ago was about publication impact, or read wealth, liking these types of metrics, like, how did your publication reach certain audiences and, and at the end, I did engage with the public and he was saying, etc. Now here, I'm talking through a journal lenses. So, for example, to be able to make an informed decision in terms of selecting a journal, besides traditional metrics that we saw here, like the Impact Factor, I'm now trying to branch out and think about, what's the readership? What's the authorship? How many, for example, inside of a particular product, or many publications that they share? If they published within that product line, or competitors, or etc, within for the therapeutic area, inside particular journals. And this can be really powerful insights in terms of like how to select the best venue for your next publication.

Garth Sundem  16:34

Okay, my next uninformed question, series about informed questions. So does your eventual journal selection influence, your author choice, you're talking a lot about, you know, impact metrics based on authorship. And I'm wondering how early in the process, your eventual journal choice needs to influence the development of a publication? You know, if you're looking at journal choice, when when do you look at journal choice? It's not only when you get your study results out, right? Is it even at the point of authorship, or when you start thinking about journal choice? I'm

Carlos Areia  17:14

a bit biased on that. Because before joining digital science I do while I'm still I'm a bit of a researcher, myself, and usually on the publication, I start, during the study, basically, like gaming, where do we want to what do we want to publish? And so yeah, because for example, I come from academia, academic background, and there were some times that we were pushed to be able to publish in, in the journal that we wanted to publish to involve, like, patients from the beginning of the study. And I think, well, increasingly, this is going to happen. Well, more and more as time goes by. But But yeah, no, I think I think yes, I think that should be, in my personal opinion, fairly early in the in the process. And I like to like to explore, like, what journals are out there to understand their metrics to understand Understand their reach, and, and create like a pool of, of journals that will, will reach help reach the objective overall objective, so to know, to know, if they're going to reach the overall objective? Well, I think people will need to know what the objective is beforehand.

Garth Sundem  18:27

Okay, and this kind of goes back to Julia thinking about one High Impact Factor study, and then a bunch of satellites studies or whatever you want to call it, I can imagine that publications team, sitting down with a clean dev team and thinking, okay, these are all of our studies that are going to come out, here's where we would like to place them for different purposes, and different kinds of impact with different kinds of audiences, and maybe even different kinds of authors who would be involved in each of them. Is that how it works now? Or Or, again, kind of the Carla's point, are people being involved early in the strategic planning for where a hypothetical study might be pleased? Or are we just getting into the series of clean Deaf Studies and then seeing where we can place them?

Julia Mutygullina  19:20

I think that while answering that question, and I wanted to add something on top of what Carlos said, that's, again, what's interesting to learn from the field, answering your question, Garth. I think that that's where the field is now move into, or at least that's that's what I see based again, on the insights that are shared with us and the questions that we are asked. I don't think that the industry is there already yet. And I think there is a very good combination of just old fashioned approach which works because it is still something that is very solid But especially probably, again, let's talk about let's not talk a lot about pandemics, but we cannot not mention it right. I think especially after COVID, there was a significant shift in this in this area, right. And the industry started looking very attentively into change in their approach to try to make it more successful because we keep hearing it again and again, that their approach that they used before everything going digital is not working now, for them, even like even personal meetings are not working as well as before. So again, there is definitely the recognition that the shift needs to happen, but it is still in process. And I think that again, just to add on, continue answering and add on what Carlos has mentioned. And he already mentioned about the flexibility of additional approaches except from Impact Factor. Right, right part part of using this, I think that this is the main area of investigation where the opportunities lay right, again, if you want to do with a prospecting and planning Impact Factor is a little bit static to change your strategy on the goal, let's say halfway through, through through through the year. And sometimes this is exactly what is necessary. If the planning should be done a little bit more a child, I would say, in this case, you definitely need to look at something else than just impact factor or classic approaches that you have been following for years.

Garth Sundem  21:36

Right and back to the Carlos is an analogy of what game are you playing? You know, what is your purpose? What are you trying to do with this study? It's not just general impact, it might be a more specific purpose or or people that you're trying to impact or, you know, some sort of change in the in the clinical landscape. All right, let's let's leave it there for today. Thank you, Julia and Carlos for joining us. To learn more about how your organization can partner with Digital Science Visit Digital-Science.com MAPS members don't forget to subscribe, and we hope you enjoyed this episode of the Medical Affairs Professional Society podcast series, Elevate.