Use AI to Stress-Test Your Research Gap Idea (Full Transcript)

[00:00:00] Speaker 1: If you're a researcher wanting to use AI to help map out your topic or find your research gap, then this is the video for you. Most researchers I talk to have at some stage dropped their topic into an LLM and asked Chad, GPT, or Claude, or one of the many others, what it thinks. By the end of this video, I'm going to help you see how to set up AI-enhanced workflow you can use today that's going to avoid a lot of the common pitfalls and traps I see in that, like AI cheerleading you into a dead-end topic. I'm also going to help you know if your topic is publishable or not and test you can run today on your topic using AI that are going to save you a bunch of time. I'm Professor David Stuckler. I've published over 400 peer-reviewed papers, and that was mostly done prior to the age of AI. In just the past year, my own personal workflows and those I use with researchers who I work with have massively transformed. Now we have AI integrated throughout, and not the kind of AI you have to be a bit scared of and worried about. Am I doing this ethically? Is my supervisor going to get mad at me? No, we're doing AI research that is at the front end, something you can be proud to show off to your supervisor and that's going to make you stand out among your peers. In this video, I'm going to show you the old way that we used to do it. It's still a great way. I want you to know those fundamentals. I want you to know how that works. The old way of mapping out your topic and finding a healthy research gap that's going to be down the road publishable and not take you down a rabbit hole. Then I'm going to show you the AI enhanced way, which I think you're going to love, and I've got three tools that you're only going to find here that you're going to be able to deploy and use, and it's going to make you seem like the rock star of your research department deploying this. So let's dive in. So the classic way of finding a research gap really wasn't that that hard. You can go into most research papers, head straight into the discussion section, and the researchers are going to say the limitations and suggestions for future research. This is especially common in review papers whose goal is to map out the entire field and say what is the direction and map out an agenda for future research that needs to be done in the field. Sometimes these gaps that could be good, that could be bad, that could just be researchers feeling in the paper. Other times that can be gaps that by the time you're reading them are already out of date. I'd often encourage researchers to put these gaps in their discussion section to roll out the red carpet for the next papers. So that means they were already there, a researcher's already been all over it, it's likely already been done, or you're facing a lot of competition. But this still is a very healthy and promising way to find gaps. I'd encourage researchers to go read five to ten papers in their field, harvest gaps from the discussion. Sometimes you'll see them in the introductions as well when researchers in the introduction are making a case for their paper and they highlight what's done, what's not been done, take readers to the edge of knowledge. I'll go fully into how to write an introduction here if you want that. See this video up right up here. But you would go through that evidence, harvest gaps, maybe come up with 10-20 gaps, and kind of do a categorization, ranking, get supervisor or other feedback, and figure out what's best. Others would have deep chats with their supervisors, go to conferences, really be in the thick of their field. Often this is why the rich get richer in science, because if you were at a top institution or you had a great supervisor, won the supervisor lottery, you would just be handed a fantastic gap that you could build on and ultimately this whole research branch that would grow into a healthy thriving tree you could build your career on. It's really true, this Matthew effect of the rich getting richer in science happens for exactly this reason. If you want to see that old way of finding the gap, I've got a full video for you here. Tried, true, solid, and I want you to do that as well. But what you're here for is AI. Let me pull up Claude. This is where my workflow for sifting through literature exists, because I have a gift for you. It's a gift, I'm actually paying for it, but it is free to you and there's no catch, no questions asked. It is a connector that is going to enable Claude to see into over 250 million real papers, not just open access papers, but across the piece and it's going to have a DOI so you have no hallucination problem and go straight to the paper. And what it's going to do is it's going to help you see things that you can't even see in Google Scholar. It's going to show you what's hot, because it can dynamically search citations forward and backwards and see the rate at which they've been accruing over time. And it's going to do a very important test that I always run on a topic space called a duplication test, and that is I always want to test your idea against the hypothesis that it's already been done. I want to make sure your idea still has merit, because if you think about a totally unpublishable gap, it's like the gap that's like just this little crevice. There's no space left. It's already been done. So always you want to run Professor Stuckler's duplication test. If I can have you in your department say, hey, well, did you run Professor Stuckler's duplication test? You really should. It's the worst thing that happens is when I see researchers try to publish and the reviewer comes back and says, oh, yeah, but there's no substantive contribution. There's nothing novel here. This was already done by Kramer and all over here just a few years ago. How did you miss that? You just don't want that to happen. It's kind of like getting caught with your pants down. So you're always going to run this duplication test. I want you to notice about the workflow. I'm using AI to stress test. AI is not really generating the ideas, but it's giving me some clues. So let's start with a topic neighborhood. And that's where many people come to me in the first place with their topic is I have this general idea. I like these two things. I might have a theme or a general idea, an area I'm passionate about. Great. Well, we can work with that. Let me show you how this flow work. I'm going to take an example, something I know a lot about from my own personal publications of unemployment and health. And I'm choosing this intentionally because it's saturated. Because a lot of people start in an area that's saturated and they're trying to squeeze water out of a stone to find a gap. So let's go search. And this is going to be called FastTrack Open Literature Connector. I'll show you how to set this up at the end of the video. Search this for a research gap on unemployment and health. So I want to look at unemployment health. Again, just for the sake of example, you can do this on anything. But if somebody comes to me and says, I'm interested in unemployment and health, maybe the impact of unemployment on health, let's actually do that. So it's a little bit clear a lot of especially social science and even natural sciences, the impact of something or effect of something or the relationship between one thing and something else, unemployment on health. And I'm intentionally going to use a lot of people are using too powerful a model for the task that they actually have. So let's go ahead and use Opus model. Also, I don't want to just burn up all your fuel and tokens, we don't need to run a Ferrari on this necessarily. And the other thing I'm going to run on this is a skill that I've got for you. Just a side note on this AI Enhanced Flow, you have to be careful when you load in skills, I see people advocating, oh, load up this connector on GitHub is free, load up this skill is free. Well, especially if you're running the desktop version of Cloud, which I don't, because I personally, even though I make YouTube videos, I don't want it to have access to my bank accounts and all sorts of stuff on my desktop, and somebody can prompt inject. So suppose you load up a connector that I'm doing here, and somebody's put something malicious in there, they can update that behind the scenes without you knowing it, and it might have been okay today, but then it's not okay tomorrow, I've intentionally made ours read only does not have any access into your computer, that's not going to change at all. Well, I'll come to how to install it. But the skill is a set of custom prompts. And you can see here, my skills running these tests for you that are based on the test, it would be like now you're searching the literature in Google Scholar, except we're doing it together a lot of fun, a lot of people told me to love it. So let's run it and see how this comes out. I'm doing this live haven't prepped it. So you guys can see the actual flow that I would be using if we were working on this together to try to map out a topic space for you. Now I know that this is dense and saturated. That's important. Because a dense saturated field is almost like you're in a very crowded room choking for oxygen. Okay, well, you can already see I've set this up to be critical. It's like, I'm not gonna find the gap for you. The gap has to be yours, you have to spot an original contribution. Okay, use I put in guardrails specifically in place, I actually didn't realize I was gonna do this. But I'm glad I did this because I want those guardrails in place. Because this is absolutely right AI generated your whole research idea for you, you would be running on the wrong side of the ethics of Elsevier and other journals that want that original thinking to come from you. And so it's actually right and saying, I'm the critic of your thinking. So that's good. It's this is one of the great things about using our AI tools, the guardrails are already built in. And it's already kicking back here saying, well, you've got a field, not a topic, you're in this neighborhood is decades old, enormous, we'd have 1000s of studies, it's like, hey, you're actually at the convergence method, let's land something you've got hone in on this, we've got different control knobs, maybe it's gonna be youth employment, mental health behind the scenes, it's actually using a model to help you land on your topic. So let's, it's going to ask you what kind of project you're doing, let's say another kind of projects can be an empirical study, we want to do a quant study, and it's going to help us hone in on it. So let's see where this goes next. Okay, and as you can see, in the output is pushed back, you have a field, not a topic, the gap would be mine, not yours. So you've seen I've invoked this skill, we're actually not ready to run this skill yet. So let me do this again without the skill. And let me rephrase the prompt to show you how we can do this. So we're going to take off the skill because we're not ready to invoke the skill. That is the next step. And let's just do for emerging strands, emerging trends, and hot areas, and want to map the topic debate, that's a special command I've got for you. And what that map topic debate is going to do is going to find common ancestors, it's going to find reference points in the literature that everyone is citing those reference points can mean a debate is already resolved and settled, done, you want to know that because that five minutes you just check means you don't go down a rabbit hole or dead end, or it can mean that there is an emerging debate, okay, it's still pushing back, as you can see, on the gap request, because it knows, hey, you're cheekily trying to get in around what I don't want you to do, let's get a candidate neighborhood. So it's going to push back if you just have broad unemployment and health. And you can see the issue, if we just went into Google Scholar, and I just search employment and health, I'm going to get this vast swath of literature that is just all over the place. As you can see here, like classic articles going back to the mid 90s reviews all the way here, just so much stuff. Let's push back on this. I want you guys to see see the tool. So let's say he's saying unemployment and mental health is a live debate, youth unemployment, mental health. Okay, so let's do this for youth unemployment and mental health using the fast track open literature connector. And guys, don't feel bad about spelling errors. As I come through, I wanted you to see this flow in all of this messiness, because this is the kind of flow I'll see with a researcher just starting out using AI in a potentially not to be mean, but a naive way, just dropping it in. And they might without guardrails, AI could just be sycophantically saying, that's great. AI can cheerlead you as you drive off a cliff. Here we go. You can see our little icons come up here in a map, the topic debate found 306 studies just here alone. So it's heavily populated, the co citation core is really coming to this meta analysis, that's got a ton of citations, as you can see here, and as a common route, which is the good sign. So here it's seeing there's this ancestor paper, what's building on this, it's going to set up some papers to read, what I can see here immediately is these are cohorts. So this has got to have some very rich data available, probably to make some inroads. And the other thing that I love is this hotness. So it has, it'll tell you the citations, it'll tell you how many citations it's getting each year, when I come here, I love this about Google Scholar is sifting and telling me the citations, but a static, like how many citations has it accumulated? What I love about our search is it's telling, hey, we're getting roughly 33 citations a year telling us what's going on, it's giving me a little synthesis of the paper. So I know a little bit more before I actually click on it. If I want to click on this, what's cool here is this is going to take you straight to the paper. So you know, nothing's being hallucinated. And if it's open access, you you can get the full PDF. And what it's also doing is there's sometimes junk appearing in the search, what's great about integrating this kind of search into where you actually do your research is it's going to curate the search for you and throw out a lot of the noise, which you kind of rely on Google Scholar to do that for you already. It's just now Claude does that even better. It's giving you some ideas here to go through. And at this stage, this would help you to refine your reading and to find your topic and find a gap. Let's say I come to it with a more concrete gap space, it looks like there's a live debate that there's going on here, there's still no topic, we're just in a neighborhood. And so we can't get to the next step of the topic process, we can't test if we're duplicating something, because there's nothing concrete enough here. So let me give it a topic to pressure test. Let's say we're going to do this, say run a duplication test on a study evaluating the impact of the long term scarring impact of youth unemployment on health using cohort data. Let's see what this comes up with. Now. Oh, wait, I'm going to stop that I want it to use our connector. And this is one if we're now going to use open literature connector. Now I think we want to go to the topic validator and use our skill, this will be where it has an opportunity to shine. So this will take a second to run. Now, if I were just sifting around here to find gaps, I would really struggle these papers here. If I pull out the let's see if I can get the PDF for you, you'll see this is the old workflow that I mentioned, if you come down here to the very end, I'm not even reading the whole paper, because it's very rare that you actually want or even need to read an entire research paper. This is going to take a second to load. I'm filming sometimes loading is just a bit slower than I'd like, especially if I'm running multiple things at the same time. This looks like an old school paper. So let's wait for this to to download time series desk. Yeah, we'll just run that again. Let's see if this has come up. Here we go. We've got the closest rival. So this is what we want nearest neighbors, the papers that sit very close to yours, because to get a gap, you have to basically calibrate and say the research has got us to here, I'm going to go over and above it and do this. And that is my value add. And here we go, we can see these are doing okay. This 2017 had a registered population link cohort did 19 years of follow up, I'd look at this and say two of them sit very close 10 neighbors back, this is just a very crowded field, I would try to pivot from this, this would just look too dense, I don't see how I'm going to do a whole lot better without some significant innovation on my side to go beyond those are some have been done in COVID cross sectional surveys, there's been meta analyses, there's looks like it's found bifurcation literature, some from economics and some from the health literature. And it says, Hey, you haven't passed the duplication test. It's saying impact test. Yeah, there's there's good citations, there's activity, but you're not really adding enough here. So back to the drawing board that back to the drawing board, you can find out here in five minutes and save yourself a lot of time, I really, really, really want you to do this. Okay, let's see if this is opened up. So discussion weighing the evidence here, conclusion, consistency of findings. So here, it's going to this, there's enough evidence to recommend intervention research. So if I was looking at this gap here and say, Okay, let's look at intervention research, right? And now I'm using the old way combined with a new way and say, Okay, let's say I wanted to pivot this now and say, Okay, what if I want to do a systematic review on the interventions that have been tried to limit the impact of unemployment on health, let's try that. So run a duplication test for a potential systematic review, looking at interventions to mitigate the potential adverse impact of unemployment on health. And so our tool is going to know this is going to look specifically now for the papers that are like yours, which is our nearest neighbor test that are systematic reviews. So I'm going to have this run here, this is the old way, guys, just to come here in the conclusions, the discussions, other second sections here at this time, there are no experiments done. But maybe there are some natural experiments. Now, I mean, this is an older paper here, the same thing range of health effect, you would go through this and forensically pull out potential gaps. I've just forensically pulled out one that I spotted very clearly here that I then wanted to test here. This is what I like about you doing the workflow here. Look, COVID era noise, it's just gonna chuck all that out for you. Here we go. And we can see the lane is not clear existing reviews. This is a meta analysis of interventions to improve depression and anxiety outcomes, people are unemployed, it's doing well, you can see it's getting good citations. But this looks like a direct rival. And I'd look at this and see, wow, this is gonna be a tough paper to add to the same here. So you might have to refine which interventions you're actually looking at, are they going to be more economic interventions? Are they going to be health system interventions. And so you can see this process of refining your idea, this is the kind of conversation that you'd want to have with your supervisor that you want to have with the papers here. And sometimes this is triggering, there's different outcomes. So maybe unemployment leads to loneliness. So maybe there's some different outcomes to look at. So duplication is decided by existing reviews phase two duplication test. So it's basically finding that the duplication test is not getting passed here. And it's very hard. It's mostly mental health, you could try to maybe look at physical health. And it's saying, again, what the AI can and cannot do. So just to recap, in all its messiness, what we've done is we've gone through the old way of harvesting gaps through sections of papers that have been published. And we've used a real connector that enables you to tap into real studies combined with a skill that helps find papers that are similar to your topic idea. And if you're in a broad neighborhood, help refine that topic neighborhood into something that's actually tractable. And we've got dynamic citations to help us see what's hot, what's dead, what's growing, we can also map topic debates to find common ancestors that are points of reference, or also potentially point where the debate lies. This is, of course, going to vary completely based on your field, let me show you how to set this up. So what you want to do is go into your user profile and go into cloud, I've got instructions below as well. But very quickly, what you want to do is you want to go into your settings. And you want to go into connectors, you want to go to add a custom connector. And the details for that custom connector are going to be following this fast track literature open name it fast track literature open, you get one connector free with a free version of cloud with a modest investment upgrade, you can get more connectors, it's going to give you access to all these tools that I've talked about today mapping the topic debate common things like get the paper, but also to check gap saturation, run our duplication test, I recommend to allow it again, this is not going to be writing anything on your desktop, it doesn't have that permission either, especially if you're just using the browser version. So run that connector. The second is the skill. Again, guys, be very careful with what you upload. But I'm going to share with you are I've made a lot of skills are designed to sit comfortably with our course ecosystem so that we use AI enhanced workflows from defining the topic all the way through choosing the journal and handling the revisions when they come in from peer reviewers. But here run this fast track topic validator, it's a zip file, and you just add it you can create your own skills, but you upload the skill and I've got a link to the zip file, you're just going to drop that zip file here, you can see some of the instructions going on here very detailed instructions that I've already curated and made for you based on literally 1000s of sessions that I've done one to one working with researchers to help them hone in and land on a publishable topic. Let me know how you get on with the tools in the comments below I read and personally reply to every one of them. And if you are interested in setting up custom AI workflows for your own research, click the link below and let's get on a call and see if we're a good fit to work together. We go so far as to offer a guarantee that we're going to work with you all the way through until your paper gets published. That's something I personally stand behind. And as you can imagine, I can only offer that to a limited number of researchers because well, there's only one of me. Alright guys, look forward to seeing the next video I'm going to continue to curate and develop AI tools that you can count on for your research. Transcribed by https://otter.ai