Deep Research AI and the Future of Literature Reviews (Full Transcript)

[00:00:00] Speaker 1: Are Traditional Literature Reviews Becoming Obsolete? I'll say that again. Are Traditional Literature Reviews Becoming Obsolete? That is the theme of this week's live workshop. I would have never thought this to be the case before, but I believe we might be seeing right before our eyes the end of Traditional Literature Reviews as we know it. We are going to test drive the new Chet GPT Deep Research Model and let me tell you, it is a game changer. So much so that I am about to launch an exciting experiment I can't wait to share with you and give you guys a sneak peek of what I'm about to do that is really going to shake things up. But before we get started, a big welcome, I know some of you are joining from YouTube, some of you are joining from our new Research Collective. I've got a QR code to check that out exclusively for researchers in developing countries where we are seeking to break down the barriers to access to elite mentorship and training. And what we're going to do today is I'm going to cover for you where literature reviews are, how they might fit into your life right now and some of the frustrations you might be feeling with them. Some of the problems with AI powered research as we know about it, some of the myths around them and introduce you to deep research and why I think it's so much of a game changer, what you need to know about it, how you can use it in the right way. And then we're going to explore a little bit how far can we really push these new systems. So any of you who have thought about using AI in your research, this is going to be a tremendous session for you today. And before we start, quick tip. If you are going to use AI, master the research fundamentals first. Using AI, and I see this with a lot of students, especially if they're at the very beginning of their PhD or they're in their masters, they're feeling lost, they're feeling frustrated, a little bit confused, and kind of the drug of choice that they turn to is AI. And it might give you a quick hit, it might feel like a really great, you might even get some positive feedback taking those steps, but creeping in the back of your mind might be this lurking sensation like, am I cheating? Am I a fraud? Am I going to get found out and maybe even kicked out of my program? Am I doing something ethical? And the problem is, if you use AI and you don't have good research fundamentals, well, bad research plus AI is just accelerated bad research. You've got to master the fundamentals and know not just how to use AI power and let AI run with things, that's tempting, but there's a quick magic bullet fix, which is going to leave you feeling like an imposter. Master research fundamentals, then add AI in the right way, and that's powerful. We're going to show you how. All right, let's dive in. So traditional literature reviews, and I know we've got some professors jumping in on this channel as well. I mean, comment below what your frustrations are about traditional literature reviews. Because they can feel very slow doing a literature review now, especially if it feels fast moving, and you get your literature review done by the time you go to publish it, it's already out of date. This definitely happened in the recent COVID pandemic, when the race was on, and there would be a review, and then another review, and then another review published. And it can take, you know, there can be tons of literature to wade through, so that even if you set up a scope of your review, you feel like you're doing something partial. And it can feel that traditional literature reviews are something now, and I can tell you for myself, as a professor, that sometimes I might hand off to a grad student or a PhD student, and even for them, it might be a little bit of a rite of passage to get some mastery of the field to go on to do bigger and greater things. But they still do play a valuable role for giving the current state of knowledge, for evidence debate, understanding gaps, and trying to understand where next. Now, the first generation of AI tools, really, frankly, and I'm going to show you a second in a direct comparison, they were exciting, because they were fresh and new. But the chinks in the armor, the gaps, the limitations were just all too obvious. They would hallucinate, for one. So when the literature reviews, if you ask Chad GPT, or now DeepSeek or others, they would just make up references. And they were also weak. They would give you kind of shortish writing that was all too obvious. Now, I can often sniff out the smell of a Chad GPT content or submission, or if a student's been working on something and sent it to me, I know instantly if Chad GPT has had its fingerprints on it. And it might have been descriptive, but it really lacked, often what was coming from Chad GPT, type LLM, large language processing models, was really kind of just a rehash of what was already there. So it wasn't really able to get to the higher orders of cognition of synthesis. And so that, at least, is potentially about to change. According to Chad GPT, they've now introduced this model of deep research that they believe can replace 10% of all human work, and they argue is at PhD level intelligence, but not just PhD level intelligence, but PhD in every subject level intelligence combined with this power to now search. So I want to share with you, I decided to test drive this, and I want to share with you the results of that. I'm going to share my screen here in a second. On an area, on a domain that I'm quite familiar with, so my original core background training is health economics and social epidemiology. So I looked at a topic that has a lot of theoretical brambles, has been well discussed over and over for decades, is interdisciplinary, cross-sex political science, sociology, epidemiology. So I picked a topic right at this interface on social capital and health. So I wanted to test drive the lit review potential. I'm going to share my screen here. I'm going to pull up Chad GPT so you can see what I've been looking at. Looking at his deep research feature. Okay. Now, what you're seeing now is not the deep research feature. This was the first literature review that I was talking about before of just using the GPT 4.0 model. And I'm not doing this in real time because especially when the U.S. comes online, things tend to slow down. But just to show you, if we have some time later, I'll try to get a deep research running in the background. So you can see it's thought process as it goes forward, which is pretty incredible. But here we go. Please conduct a literature review. I tried to specify a little bit of what to do in the prompt. Conduct a literature review on the health effects of social capital. Use academic references and draw on PubMed specifically. Identify potential gaps in literature for future research and current methodological weaknesses of existing studies. Often thinking about this as a conventional literature review as a funnel. By the end of your funnel, you go broad in your literature review and you hope to kind of get to the end to a specific research question, kind of rolling out the red carpet for what research you might want to go do next, mapping out a research agenda. So often for early stage researchers, they sometimes make the mistake of the start of their literature review is where they want to be with their empirical studies, quantitative or qualitative, whatever they want to do, when sometimes they need to go broader in their funnel and step back for a second. Okay. So I tried to give it a little bit of guidance where to go and you can see this wonderful literature review that we got here is four paragraphs long. Now some things have started to improve, but not perfectly. So we can see that it has, it's starting, because I said PubMed, it's starting to come up with some studies. And I'm going to pull this up and we'll see if this is actually giving us a real study. And the problem with this GPT 4.0 is it actually, that citation takes you to a study by DeRose and Varda. It is not correctly cited. It has a citation, but it hallucinated the citation again. This is the well-known problem of GPT. And you can see this is just a little bit thin. This is, you know, if somebody tried to submit this to me, this would be, even at an undergraduate level, kind of interesting, well and good, but a fail. Just wouldn't be getting anywhere. But again, not completely useless. No. It could give you some ideas. It might be a sounding board. Might be able to cross-check some things. Did you think about this in making up studies? Maybe there are some interesting points saying that the majority of studies have been conducted in the West, only few multi-level studies in non-Western settings. So definitely some points that are true, especially about methodologically, but really at a point in serious professional research, almost useless. All right. Let's now go to this. I did the exact same search here, but this time I did it with deep research. So to do that, you just activate deeper research button here, get detailed insights. And what it did surprised me. It said first, to conduct a comprehensive literature review, I need more info. And info it asked was the info that I would typically press researchers who are trying to get clarity on their topic to define. So we often use, to ensure that a model is on point, a PICO model, where you have to define your population, the intervention or exposure you're looking at, a comparison group if relevant, an outcome. These are the letters, the abbreviations of PICO. The outcomes you're looking at, the time dimension, the research designs, you need to think through these parameters so you construct a playing field and understand what's in and what's out of your review. Otherwise you can snowball and just drown yourself in masses and masses of studies. And that's just a recipe for frustration and also not seeing you for a very long time. You're going to lose your friends and not have a very nice life. Just be buried in the library, which I was a victim of myself when I was a graduate student. Anyway, I digress. That's another story altogether. But it asks me, do you have a specific time range for the studies? Are you interested in specific health outcomes? Do you want to just look at mental health? Do you want to look at cardiovascular health? General well-being? Should we focus on specific populations, youth, elderly? These are dialing in the PICO control knobs, so the population, I'm going to hone in here, I'm going to zoom out there. You want to include, important question about the inclusion criteria, do you want to include systematic reviews and meta-analysis or do you only want the original empirical studies? Do you want them to be structured in a specific way? So in my first iteration here, I said, yes, let's go ahead and include systematic reviews, but don't treat them as original unless they have a meta-analysis. And meta-analysis is systematic reviews are treating articles like data. And meta-analysis takes it a step further and kind of uses that data, makes a data set and starts analyzing the data like it was any other quantitative data set. And I said, I wanted to do this thematically, like a narrative thematic review, highlighting key mechanisms. And I wanted to leave it the space to find the optimal structure to highlight what it believes to be the most important evidence. And so it just clarified that I was going and here it went. You see the time it took, five minutes, one through 23 sources. All right. And here you can see we are dealing with something fundamentally different. I mean, let me just compare just this introduction to what we had before. All right. I mean, this is just, I hope you can see we are miles away. So already just the clarifying questions tell me I'm no longer speaking to some neophyte in the field who hasn't got a clue what's going on. I was just having a conversation with an experienced researcher who's probing me and forcing me to clarify, much like in a conversation one of my researchers or colleagues would be having with me, those kind of probing questions that I would do to clarify things. So here we go. Check this out. So introduction. And here we have citations. And these are real citations. These are real citations and they're not incorrect citations. I'm going to show you this in a second. Like this here, a double edged phenomenon. This is pointing to the dark side of social capital. This citation is correct. This is a subtlety in the literature saying, hey, social capital actually can have some, it's not just a universally good phenomenon, it can actually hurt things. This is an emerging strain in the literature and it picked up on it. It developed a conceptual framework to understand key distinctions going on about bridging bonding linking social capital and the multilevel analysis. And it kind of set up like why this review was needed. So that's pretty important. Structure. It broke this into social capital, mental health, organized the strands of literature here. Physical health and mortality. And I'm going to read some of this in a second just to show you. Variation by population and context. And negative health effects. Taking that dark side I was talking about. I want to go through this too because what's cool about this is when it quotes something, the way it's citing these references, when you click on this reference, it takes you to the place in the article that it says what it's saying. So it's not just citing the article. It is citing and taking you to the specific part in the article where that's being said. This is the opposite of hallucination. This is even more precise than our traditional referencing methods. I mean, that right there just kind of blew my mind because oftentimes researchers are looking to cite and they want to know you're saying the right thing. This is citation 2.0. Okay. And again, if you want, let me know in the chat. I'll go zoom in and I can show you the check I did on these citations. This is correct. And this is detailed and precise and accurate. Okay. Now, mechanisms, mechanisms. This is, this is great. This is brilliant. I mean, look, this is not just an A. This is a research review that I would expect. This would be hard for a first year PhD student to do. I mean, this is the quality of a review and synthesis that I would expect from an experienced researcher in the field who has maybe five years of experience in the field to be able to do this. It's well-written, it's clear, it's engaging. Psychosocial support and stress buffering, health information and norms, access to resources and services, collective efficacy, social concern, control, psychosocial factors. Look, you know why, it's so easy to critique the early stage literature review of GPT. This is more difficult to critique. There's still critiques and I'm going to talk about the weaknesses, but I just want to kind of share with you, and I hope you can immediately see, maybe you're having the same reaction as me. If not, let me know below what you think. But I was really quite floored by this. So this clearly will save a ton of time. But it is at a point where it does start to beg the question, are traditional literature reviews becoming obsolete? This can live update. You don't have to go through a lengthy, protracted process to get this out. You can update this almost in real time or, well, the five minutes that it took to generate something for a human to do may be a bit better, and I'll talk about where I think humans are still better in other ways, but the speed, the speed is tremendous. Five minutes to do something that will take an experienced researcher to do well would take about a week. That's an experienced researcher. So yeah, let me keep going here. Policy implications and gaps. Came up with gaps. Let's look at the gaps because this is something like, I like to see clarifying causality. Great. That's what I would see. Standardizing measurement. That's good. Mechanistic studies, life course, understudy populations, addressing the dark side, and digital age. So, I mean, these are actually really not bad. Really not bad as a synthesis goes through, like I said, methodological weaknesses. This is, I measure, this is about 10,000 words. This is, I mean, this is like a master's thesis. Publication bias, selective reporting. So I'm not going to go into the details here. I'm just going to turn to the chat to a second to see what you guys think about this, and I can see, great. I love this community. It's truly international. We've got people from Sydney, people from Jakarta, Malawi, Rwanda, India. I mean, it's just such an honor to work with you from such an international community. And some of you are asking, does it follow our peer writing system? I'm going to show you that yes, in many ways. By the way, if you are wanting to use AI, we do have a tool here, FastTrack Mentor. We have trained an AI in our systems. One of the weaknesses of AI is it doesn't work from a coherent system. It's just kind of optimizing for whatever you're sending it. So the feedback that you'll get from FastTrack Mentor is it's trained on all of our data. It's going to give you much more focused and precise feedback. If you're interested in checking that out, let me know. I mean, these are tools that we want to create to the public to help you. And what's great about this FastTrack Mentor is it turns the way it's supporting you into a learning opportunity by pointing you to which of our videos and trainings is going to actually help you go deeper to understanding that step so that you begin to master the research fundamentals before you just start hitting yourself with that drug, that quick hit, quick fix drug of AI. Now, all right. Some of the problems that we have here, and there are still problems. There are still problems. Where is this weak? So what's weak is the method. There's no replicability. There's no reproducibility to this method. Or is that the case? Can I fix that with a prompt? That's what I'm about to test and show you next. The other weakness, it relies on open access articles. So I said, go to PubMed and it's only getting the PubMed articles that are open access. I can see Louisa. Hey, Louisa. It's good to see you joining us from Malaysia. I'm blown away. Yeah. Stunning. It's a game changer. I think we're going to be at the end of literature reviews in five years. As we know it, with some exceptions. So originality. These are not original ideas. These are ideas that have been said in articles elsewhere. So are we at the truest highest order of cognition that a top tier researcher can do? Not yet. The originality is not there. The creativity is not there. The higher level cognition and the synthesis is not there. The clustering and the thematic analysis is at a high level. It's very good on structure and it's very good on being comprehensive. Like I said, it captured that emerging strand of the literature on the dark side of social capital. So it passed the test there. This would definitely, no doubt, get an A. And it doesn't have that smell of chat GPT. I'm not going to show you here, but I went to AI detectors. And the AI detectors haven't caught up yet. It's kind of like whack-a-mole, new ways of using AI and new ways for students to unethically cheat and then whack-a-mole and try to get it. The AI detectors catch up. This is just out. So they haven't caught up yet. This was detected as only 61% AI written or 61%. So that's really not, you know, I mean, before chat GPT, if I would have copied that other one in, it's 100%. 100%. Some of you are asking, hey, Prof, there's no deep research option. I'm glad you mentioned that. Yeah. It's because you've got to subscribe to Pro and it's 200 bucks a month, which is a hefty price tag. And honestly, for most research students, this is like taking the drug. If you took, I don't know, the analogy I can give is like, you might've smoked marijuana a little bit and you suddenly went to heroin. That's what like using chat GPT and going to deep research is. This is a very tempting, dangerous drug. And I'm going to show, I've got an experiment, so stay tuned and follow my channel. I'm going to experiment to see how far can we go? How far can we push the limits of this? And so I'm going to show you the next iteration. I'm going to take a couple of questions though. After the end of the literature review, what will the researcher do? Yeah. I'm glad you asked. So typically it's this funnel idea. At the end of the funnel, you want to use, for most researchers, you want to use a literature review to justify that there are gaps in your field, kind of you've summarized, you've taken stock. You said, here's what we know. Here's what we don't know. And here's what we need to know. So it's going to kind of flesh out, roll out the red carpet for your idea that you want to do next in your empirical studies. So yeah, gaming hub, this is what I'm saying. You don't, you don't have the deep research option because you've got to subscribe to chat GPT pro, which is about $200 a month. I don't know if they vary their pricing plans. I don't think, and I don't recommend investing into this. It's got a hefty price tag and it's like the heroin drug. I want you guys to feel confident in using AI going forward in the right ways. And also to get to originality and creative synthesis on your own, to get to those higher order cognitions. Because if you are doing a PhD or you're a researcher, you're writing grant proposals, you want to develop yourself into being the leading expert in the world. Now, that said, I think our fields are going to have to adapt to these tools and the kind of traditional literature review. If you're an experienced researcher and you want to get up to speed in the literature very quickly, this can save you a ton of time in knowing what to read and really getting key information fast. Okay, so big problems, the references, you still can't get the citations right. I really pushed chat GPT hard to get the references right. Let me show you what that looked like. So like, okay, please number the citations and create a reference section based on Vancouver style. It started, actually started to do a very nice job here, one, two, three. And so as this was going on, I'm like, this is great. I mean, honestly, who doesn't like doing references? Comment below what the most frustrating thing is for you in a literature review. I hate doing references, even with Zotero. If you're not using Zotero, you need to use Zotero. Even with Zotero or EndNote, saves a ton of time. It's just painfully boring. It's just like clicking around, feels like any administrative assistant can do it. I will say there is something satisfying when you insert a reference in the paper and you feel like you've armored up your piece, but it still is kind of tedious and clerical and not my idea of a good time. Okay, so it started here. It was great. But then, oh no, additional sections follow the same citation structure. So I mean, these are real references. These are real papers. This is pretty good, except it just decided it didn't want to play anymore. And it was like, I'm taking a nap. You figure it out. It's almost like Chad GPT was mocking me, saying, hey, you know, you're a professor. You can do this stuff. Don't make me do the work for you. All right. All right. Thanks. I need the full manuscript. It's like, okay, I'll update it for you when it's ready. Oh, no, I don't. I don't want to play anymore. And I kept going. And then it went through the deep research process again. The short story of this, I didn't get it to work. It wouldn't do the references. So that's okay. It still is a vast improvement. It can do it. It can. It just doesn't want to. So there might be a hidden prompt that I haven't cracked. It's still early days that will do this. But again, these are, these are real studies. These are real studies. These are real authors. And I later tested to say, hey, can you come up with recommendations for reviewers? And he came up with real authors and their email addresses and a justification for why I chose them. I mean, I hope you guys can see this is, this is, let me just go back for a second to, this is where we were a week ago. This was a week ago, right? The world has changed. And we need to change too. Um, experimenting coming up, what, what does this mean now? What does this mean for us? I hope you can see that this is profound. Chuck is saying it seems like pitfalls can easily be resolved with a human, human touches. Yes. And Chuck, you are anticipating where I'm going with my experiment coming up. So stay tuned. And do you reckon there will be an explosion in literature reviews across the board? Will people be submitting deep research generated work? And you know, it does make you wonder, you know, somebody in the background and we've seen how quickly DeepSeq could replicate Chet GPT. These new models, the plagiarism and the AI detectors can't pick up. It does make me wonder how much AI related material is already there in our journals. Now, Elsevier has a very good, I'm going to show you in a second here. Let me share my screen. I'm going to take us out of Chet GPT. I want to show you Elsevier's AI policy. And uh, I think that's going to help you guys to see what we're looking at and what you need to think about. So AI's policy here is, this is Elsevier. So Elsevier has basically said, no, Chet GPT AI is not an author. They're basically to the stance. Well, it's not human. Can't be an author. You can't have Chet GPT be an author. Fair enough. But it does say, right, when you use it, when you use AI, it should be to improve readability and language, not replace key authoring tasks such as scientific, pedagogic, drawing scientific conclusions, drawing clinical or medical recommendations. But that's vague. Can you do a, is it a scientific insight if you're just thematically organizing what's already been said? It's unclear. This does say, I think, err on the side of transparency and disclose what you have done. And they kind of affirm it, declaring the use of these technologies supports transparency, trust between authors, readers, reviewers, editors, so on and so forth. So it also goes on to say that this is new content creation only. And it should not be used on previously published material. That's just a little bit unclear what it's trying to say, because these are, some people say are big plagiarism machines, the LLM models, it's training data is previously published material. So that's just a little bit unclear. But lack of clarity aside, this is quite exciting and quite powerful. And it's just one of those moments of, is it AGI where the bots are going to come take us over and destroy the world? No. But is this leading to a change in what our value added is as researchers? Yes. I think about this as the moment, some of you, I like chess, all right, I'm pretty nerdy, no bones about it. I like chess. But I remember when Garry Kasparov would kind of, he was the leading chess player, he's still one of the world's leading chess players, and he was just destroying everybody. But the computers were catching up. And I remember the turning point, I believe it was in the 90s, when the computers beat the best human in the world. And the chess community responded in a way, not getting down, it's not like they stopped playing chess, they adapted. And now what routinely happens is chess grandmasters, they train and they learn with engines as a co-pilot. And they're stronger chess players than they would have ever been before, because they are accelerating their learning, they're accelerating their advancing through the use of computers. Is any human as good as a computer? No. Not even close. But they adapted, adapted to the game. So let me keep going, I'm going to go through some of your questions. And Daniel had an idea coming through. Actually, I can see a lot of questions. So guys, if I miss a question, do bear with me. And hey, Katija, awesome, I'm pleased to hear you're learning a lot with FastTrack. Yeah, I mean, we will show you the right way to use AI to leave you feeling confident and to do better research and save a ton of time and not run and feel like you're running into plagiarism or ethical brandles or doing something wrong that's gonna make you feel like a fraud. So yeah, definitely check out the collective. Is it following our peer system? Yeah, the writing, the academic writing is pretty good on this. You'll see our peer system is our writing system. Each paragraph makes one point. First sentence is a topic sentence, I won't go into that. You can see on my channel, some of our great trainings on academic writing, if you've never had any academic writing training, it's going to make a big difference for you. But yes, it's following a lot of that. Right now, it's not really able to do a method section that well. I'm going to continue the live demonstration and show you when I did ask it to try to piece together some methods, what it came up with. So I'll share that with you. And Efaz asked, which GPT should I use for literature review? Look, I haven't done the full comparisons with DeepSeek, which is apparently cheaper. The ChetGPT 4.0 is good, but again, I hope you're saying, I don't ever recommend having these tools just completely do your literature review for you. Maybe in certain circumstances, if you want to compare the literature review you did to what the system comes up with on its own as a sounding board, but very specific instances. So Daniel's saying, after the bibliography, prompt it to discuss according to key themes in the discussion. So I'm not sure I understand what you're saying. I think what you mean is you want it to... I'm not sure I follow you here, Daniel. I think what you're maybe saying is you're giving it some of the raw material already and saying, hey, turn this into key themes for my discussion. So I'd like you to do the originality of the analysis and come up with the themes yourself, and then ask ChetGPT, hey, are there any themes I've missed? Do you agree with these themes? Kind of using it as a sounding board and not for it to replace the originality and creativity, which is what really makes you stand out, which is what is uniquely human. If you're ceding that power to the machine, you are, in a way, doing yourself a disservice because you're not going to get to the higher order cognition, the thing that gets you from grading papers to get you to the very highest marks. The thing that's uniquely human. You've outsourced your best abilities. If you're doing something like that to save time, that's another conversation. I'm glad these are helpful insights. Yeah, Louise, the app is something we're still in the process of developing, but right now, the training data that's continuously improving that we've used is in the FastTrack Mentor ChetGPT version that's available, that's trained on our data, and we'll use all of our systems. So you're working from a coherent system. And I think Mary D asks a very important question. Is it minimizing the use of the human brain or we build up from AI? So I think like the chess players, the idea we have in research is to stand on the shoulders of giants. So if these are tools that make our life easier and better, we need to find important ways and the right ways to integrate them to stay ahead of the curve. It's kind of like when we used to have a slide rule to do math. Well, once calculators came out, no one used a slide rule. Beyond that, it wasn't necessary anymore. So that technology kind of faded away and we upgraded and incorporated the new technology and even forgot about the old one. And this is a direction that some of the things that we do are going to get replaced and outmoded by AI. And this is just a giant leap forward. And AI is on this exponential accelerating curve. So I mean, I can show you the difference of what a week, literally what a week has made in this release. And imagine where we're going to be in two, three years, five years, 10 years at this rate. Yeah. You're worried about your personal data if you use DeepSeek? Yeah. ChedGBT, I opt out. You can opt out so that what you put in is not used in their training data going forward. Daniel, I think you're saying that you want to compare and I agree with that idea of comparison. I think that's a really, really nice way to tap the power of the GPT here as a comparison, as a sounding board. One of the best ways that I recommend is to accelerate your learning. So I just had a postdoc who we wanted to use a synthetic control model and we had a coherent system. We had selected the tool, but he was a little bit uncertain on how synthetic control worked. And so he got in dialogues with ChedGBT that really helped him accelerate his learning, gave him instant feedback on what he didn't understand. He could tell it to, hey, explain this to me like I'm a first grader, like I'm 10. And instead of going to textbooks and hunting around for the material, he got, he was there instantly. I mean, that kind of encyclopedic knowledge, some of you will remember. There used to be encyclopedias, the Britannicas, some of you may remember in the US and people buy them. They look great on the mantle, on your wall, above your TV. And then Wikipedia came out and it was gone. Well, now with ChedGBT, even the idea of the encyclopedia is gone. And I think in a way you might be in a world where the textbook is gone in the future because this can update the textbook so fast. Again, there's more work to be done, given the right prompts, given the right training data. But this creates a potential for live reviews that continuously update. Imagine, you have a literature review on what's the state of the impact of climate change and the attribution studies that show what climate change has done as an impact. And every morning it updates. And you have a continuous living review. And this is the idea of some systematic reviews, continuous living reviews. Speaking of reviews, I want to go back and show you now the methods because there are different types of reviews and different purposes of them. So some reviews, kind of quick and dirty, you might be doing the review, just kind of support the research you already want to do to make sure you've cited key papers in introduction, justified your gap. Other reviews you might want to do in a completely replicable, reproducible way. And this is where ChedGBT is falling short, but it's made some improvements. So let me come back to the screen and see if I can get back to where we were. Here's where we were. And it's going to take a second to load up. And I'm going to end here by showing you a live demonstration of deep research. So guys, if any of you has a specific topic you'd like me to test drive here with you live, we can do that together. This is just going to take me a second to load. So here we go. There we go. All right. One second. All right. Let me pull this up. I'm going to show you where I had it do a method section, or tried to do a method section. Let me pull that up for you. Hang on a second. I had a little bit of a long conversation trying to get there, trying to pull out different things here. Here we go. Methods. So I tried to see if I can make it do methods. All right. So here we go. You can kind of see this. So it kind of started to say, but it lied. It said, we searched PubMed, Web of Science, and Scopus for articles published up to December 2023. I don't know why it chose that date. And it said it combined these search terms in the article titled Abstracts. It also said they did a snowball search. These are real articles, by the way, of key articles by these folks. And it set up inclusion, exclusion criteria, study selection, data synthesis. So why do I say this? So the highest form of systematic review, and what's sometimes considered the top of the evidence hierarchy, the top of the evidence pyramid, what informs medical guidelines for life and death decisions about how to treat patients, are systematic reviews. Because they review all the papers together, assess their quality, decide what the best evidence is, and make recommendations on those. And it follows a reproducible format so that if Daniel does something, that Louise can take Daniel's steps, reproduce them, and come up with the same conclusions. This is a little bit, I'm not actually sure. So I have a doubt now. I said to do PubMed. I don't know if it actually searched Web of Science and Scopus. I'm not sure if it's just making that up now or not. That's a real doubt. But this is not reproducible, is the problem. It's not like you can go follow and have the AI do the exact same thing. Can that be fixed in the future? Potentially. Potentially. But it's starting to describe methods following the method. I can't verify if it actually did this. So what someone would need to do is take this, and we'll probably do this in our FastTrack AI lab, is take this and test this and apply these criteria and see if we get the same answer and same result. If it is following a reproducible method, or is it just making that up? So systematic reviews follow this very rigorous process, which will detail which articles were in and out, why, trace the reasons and numbers, and often have a flowchart to map through that kind of recipe for baking a cake, in this case baking a literature review. So I don't know. We're going to have to do some additional testing here. But it's making up some numbers. So it says we had 3,000 citations approximately. We cut out these. We got out these for these reasons. We cut out these for these reasons. And yeah, I think it's a little scary, because there is a lot of potential for some unethically minded researchers to try to fool the system and get these through peer review. And this is much, much closer now to having a novice in the field might not be able to sniff out the weaknesses in these methods that a seasoned veteran can. So I think that's fascinating. Okay. Would anybody like us to do a live... Daniel has got a potential topic here. This is why I love having you in the community, Daniel. Daniel's like the dark side of AI for academic research. Try this topic. Yeah, that would be a very cool one to do. I want to do one that's maybe... Here, the impact of transport and energy consumption on environmental degradation. I like that. Sana, let's go with that. So I'm going to open up a new GPT. I'm going to share my screen and say, Sana, it's going to ask you some clarifying questions. So stick with me. Again, this is just for illustrative purposes. I don't recommend that you use this. So we'll just say, please conduct a... I don't think you need to say please, but anyway. Guys, be nice in general in life. People appreciate niceties. I don't know if AI appreciates niceties. That's a different sort of testing that we'll work on. There have been studies saying, and the problem is like, hey, Chet GPT, I'll pay you $100 if you do this. But it actually performs better. So somehow it's trained to respond to niceties like a human. So we're going to say please. So please, I'm going to share the screen again. Please conduct a literature review. So let's get your prompt up here on the impact of transport energy consumption. And I think it's going to make you clarify what transport energy consumption is on environmental degradation. I want to see exactly. So if we were having this conversation, I would probe you probably about the PICO and the topic and the boundaries you want to put on the review. I want to just leave this for deep research to engage the process so you can kind of naturally see what route it's going to take. So here we go. We're going to do this. Let's see. You can see I've clicked to activate deep research. Okay. Can you specify the scope of your route? So Sana, help us out with this. Sana, help us out. So it wants to know recent studies, geographical focus. Are you looking at air pollution, carbon emissions? It's clarifying the outcomes. Transport modes is covering road, air, rail, shipping. Methods. You want specific methods. You want empirical studies, theoretical models, policy analysis, a mix. So I know what I would say, but Sana, I hope you'll jump in and let us know what you'd like to see here. I don't know if there's a lag, but we might have lost Sana. So that's okay. So Sana, I'm going to fill this in for you and what I think is right. So let's just say broad historical review. Let's say global. I want it on transport. I'm a little bit less worried about rail. Let's stick with air and let's go with air pollution. Let's go all environmental impacts and let's do empirical studies. All right. I hope you guys can still see the screen. And this is going to go on in the background. Oh, okay. Sana, I just saw your comment here about last five years, carbon emissions. And Daniel, sorry guys, we can play with this a little bit later in some of our live workshops. Again, if you haven't checked it out, we have multiple workshops a week where we actually have the opportunity to work together on your research and tap the full range of our training systems and methods. That again is multiple times a week. Again, go check out the research collective if you haven't already, it is awesome. And I've published over 400 papers. I'm now in a phase of I want to empower the next generation and our FastTrack AI lab is testing out a lot of these tools to bring the best and the greatest to you as we go along. Now check this out. This is what I want to share with you. It shows you what it's doing. And it's thinking. Look at this. I'm digging into transport emissions data. And we can see what our little assistant is doing here as it's piecing together sources. I'm looking at how transport and energy use has grown since the 1950s, focusing on road air rail shipping and its impact on CO2. So this is going to take a little bit, guys. This will be at least, this is at least a five minute exercise. And I'll be able to share a link afterwards to you to where you can go into ChetGBT and look at the result of this. So I will paste a link to this on the video if it starts to run too long. So you'll be able to find this live on my channel. I'm exploring road transport, non-OECD and aviation data to understand their emissions. So in this case, we didn't specify which search engines to search. So this, we would have needed to say a little bit if you wanted to be more academic to search Google Scholar and give it that kind of prompt about where to search, which is why I wanted it to go to PubMed before. But you'll still see, this will piece together more of this gray literature search. And actually, that brings up a point. The gray literature search is something that's unstructured, kind of difficult. By gray literature, I mean literature that is not published in peer-reviewed journals. So it might be in World Bank reports and other documents. One application we're using of ChetGBT a lot at the moment is for policy surveillance. So I have a team of researchers, and we wanted to understand, we're testing out an idea about obfuscation. This is in the political science literature, but that politicians make policy changes, but they obfuscate them. They don't exactly make clear what changes are going to happen. This is especially the case with one of the biggest components of government spending for social support and welfare, which is on pensions. And so we're looking in Europe and getting all the pension policy changes data and all the pension discussion and manifestos to see how much was the public aware of what policy changes were being discussed on the table, and they were even voting for, to their pension system. And, you know, that's in different languages and takes some real deep digging in gray literature, and that's something great. We're developing some awesome policy surveillance with the aid of ChetGBT. Now again, we're keeping the original human component of our ideas and what we're bringing to the table, but we're having the machine do what the machine can do faster, what it can do better than us in ethical, appropriate ways. So that example, the example of my postdoc going in and learning about synthetic control models much more rapidly, having a customized textbook to his level of knowledge, these are the right ways, among the right ways that we use. We've also got a checklist, our FastTrack AI checklist, which is going to help you ensure if you're interested in getting a copy, comment checklist below, send me an email. You can reach me at davidstuckler at fasttrackgrad.com. And if you want to explore the possibilities for working together, just do get in touch. Always love to hear from you. I always love to try to connect you to the right resources that are just for you. We also have an open access Facebook group you can check out, although we're shifting our new content into our collective so that no researcher anywhere in the world has to feel isolated anymore, they can tap into the power of the community. I actually want to show you how that looks. Okay, this is just done. I'm going to show you what that collective looks like in a second. We've got lots of templates that you can use for writing your papers. Wow. And you can see, this is a massive undertaking of what we got here. Okay, I need to digest the scale. Hope you guys were watching along as I was speaking. Okay, here we go, a literature review. Let's check this out. Let's check this out. Now, for me, as somebody who knows nothing about this topic, a little bit of background, I mean, it's captured the stylized facts. Transport was responsible for a quarter of global CO2 emissions. I didn't know that. I thought the cows were doing a whole lot more by being flatulent, having methane emissions, but there we go. So this is really fascinating. And then the review has gone through a structure. Now, I can already see from this review, it's kind of got a similar flavor to that first literature review I did, suggested to do in style, and that's going to be something interesting to see in our future testing, how that comes out. Historical trends for rapid growth in energy is well-structured and clear. It's got some signposting here, which is nice, and you can see it's parallel, follows the parallelity saying here and there and there, where you're taking the reader, so really good practices. It's following our peer system where each paragraph is making one clear point. So here, for example, this is an example of peer. Many countries implemented these stricter policies. It's going to explain what they were and what impact it had. So it's making, it's very clear following peer. Yes, Anna, what do you think about this? This is very good. Is this, this is kind of a narrative review structure. It's not quite a systematic review. It doesn't have any method saying how it got this information. You might have to prompt it to ask it, but I wanted to show you how this definitely works. Okay, here the structure's going a little bit funky. Not sure I would go with this structure. I'm scanning through this very quickly, guys. I'm going to leave the link for you to check this out. And future implications, folks on technological transformation. For somebody who's an expert, I'd love to see what you think. Let me know in the chat below. Is this something that you could see yourself using? Is this something that would be helpful for you? How would you use it? For those of you who have subject knowledge, is this saying anything new? How would you grade this? I, like what I graded in my subject area of knowledge was good, but it was not outstanding. But a massive step forward. It didn't get to real originality or creativity. It was just kind of repeating some of the platitudes in the field, although it was very comprehensive in its scope and coverage. And it had good depth. It was not superficial like before. But yeah, this is quite long. I want to see what Sana says. Sana says, this is insane. I mean, I am... Hang on, let me get this up and see if I can pull this up. Sana says, this is insane. I am speechless. Yeah, and it says it will guide where to go. So a really good use here can help you if you don't know all the themes. If you're a novice in your literature, this can help point you to where to read. It can help you hone in and say, oh, you know what? I actually am interested in the technological transformation here, and then be able to dig deeper on that side and on that front. So I just want to share with you guys, just in closing, again, I'm plugging once again my very shameless self-advertising, The Collective. If any of you guys are in The Collective and you're loving it, let us know here in the comments. But yeah, check it out. We're running a challenge right now where people in the community pair up for peer feedback. And we have these constant challenges going on. We have some writing sprints. We have our, again, our step-by-step courses. It's a really powerful community. It's really special. We can be together in touch in the direct messages. I mean, you're just going to love it. And it's just, it makes research fun. And I think a lot of students who have, there's a lot of students, exactly why we created the Fast Track community is that a lot of students just haven't gotten the support that they needed to truly thrive. And our mission, I hope you'll join us in this global movement, is to end that. And my plea to you is to master the research fundamentals in the right way. I think we are rapidly approaching a day and age where traditional literature reviews as we know it are at their end. And like we used to say, I would not have survived without you and Courtney. And yeah, if you are struggling in silence and you are feeling alone, get in touch with me. You've got my email. You know where to find me. My email is available in the DMs and we are here to help. I'm going to drop links below here so you can check out how to find more resources and you can find our checklist and our best trainings on how to use AI in the right way. So you feel confident. You don't have anxiety. You don't feel like a fraud and you can really next level up your research. Thanks everybody for joining us today. Stay tuned for the experiment that I'm about to release. Chuck has anticipated where we are going to be going with that. We are shaking up the world of publishing and love to have you be a part. That's it for today. Have a great week everybody. See you at next week's workshop.