Prompt Engineering Tips for Better Research Outputs (Full Transcript)

[00:00:00] Speaker 1: Hello, and welcome to Conversations for Research Rockstars. If you're tuning in, you're probably a researcher and have, by now, likely had at least some experience using AI tools like ChatGPT, Gemini, Clod, or Copilot maybe, for brainstorming, drafting a questionnaire, or a discussion guide. Maybe for summarizing data, or even just as a general purpose editor. But as excited as many of us are about what AI can offer, let's be honest, haven't we all had experiences where we felt frustrated by what it actually delivered? Like you spent way more time correcting errors than you would have expected? Sound familiar? It happens to all of us. And that's why we have to talk about prompt engineering. First, let's start with a distinction that's not always clear, that is, what is the difference between prompting versus prompt engineering? Anytime you type or speak a request into an AI, you're prompting. That can be as simple as, write a 45-minute discussion guide about dental insurance perceptions. Very conversational, very little instruction. And that's what I would call winging it, just entering a brief description and hoping for the best. But here's the difference. Prompt engineering is more deliberate. It's about using known frameworks for designing, refining, and testing your instructions so the AI delivers targeted, high-quality output from round one. There are many well-documented approaches to prompt engineering. Here, I'll cover two commonly used ones, but do note that there are other options, such as RAG, zero-shot prompting, and more. Here is what is important. When we engage in prompt engineering, we're intentional. We lay out steps, add context, and guide the AI through iterations. We provide specific types of inputs using optimized input sequences. It's a garbage-in, garbage-out kind of thing. The better our input to the AI, the better the results. And certain prompt structures consistently perform better than others. The bottom line is this. In the context of qualitative and quantitative research tasks, basic prompting often leads to uneven or disappointing results. So let's look at two options, both known, structured, prompt engineering approaches, that can elevate the quality of what you get back from AI, and perhaps even save you time. First up, prompt chaining. Sticking with our dental insurance example, instead of asking for the whole guide in one go, with prompt chaining, we build it step by step. Step one, ask the AI. Design six logical sections for a 45-minute discussion guide about dental insurance awareness among employed adults without coverage, using section titles and time suggestions. The first section should be a warm-up section. Two, review the outline carefully. And this is important. Give it your feedback as precisely as possible. Maybe you want to shorten the duration of one section and add a projective exercise to another one. Or maybe you want to move section three to go after section five and add laddering instructions in section four. These are all nice and precise examples of feedback so that the AI can iterate exactly as you want. And even when feedback is more nuanced, the AI can still iterate effectively if we're specific. For example, imagine you gave the AI the feedback, quote, it feels like this version assumes a very high level of awareness about dental insurance. We will have participants at varying levels of awareness. The AI will apply that adjustment throughout the guide. And don't you just love how quickly it updates based on feedback? Step three, once the outline looks right content, timing, sequence, tone, you can tell the AI. Using this outline, write the full 45-minute discussion guide. Step four, review the returned draft and prepare to iterate again. Just like with the outline, we will give the AI precise feedback about the discussion guide. Even with optimized prompting, we'll likely go through a few iterations before the guide is almost client-ready. And this is important. Even with prompt chaining, the AI will never return a draft that is 100% exactly to your satisfaction. You, as the human researcher, will always make some edits before the guide is final. Prompt chaining is widely used across many applications. And for researchers, it applies nicely to designing instruments and summarizing data. For our second prompt engineering option, we have reflection. And that is reflection spelled with an X, R-E-F-L-E-X-I-O-N. In this approach, you ask the AI to critique and refine its own draft, hence the term reflection. For example, your input might be, review this discussion guide and revise any questions that are leading, confusing, or biased. Then list what you changed and why. This technique asks the AI for a second pass using your criteria. And when asked, it can find its own errors surprisingly well, saving us time. Now, I know what you're thinking. Why doesn't the AI just avoid these mistakes automatically? Wouldn't it know, for example, that leading questions are bad? Well, it has the information, but AI works off patterns, not judgment. So unless you explicitly instruct it to check for bias or confusion, you get what you get. With reflection, we can go through as many rounds as we like. First, we checked for bias and leading questions. Then upon review of the new iteration, we may ask for additional improvements. For example, we might notice that the guide seems a little boring. So we might tell the AI, review the guide and identify any parts that might feel boring or onerous to participants. Then you, as the researcher, can decide which identified parts should, in fact, be replaced. Of course, regardless of your prompt engineering approach, whether you use prompt chaining, reflection, or something else, we researchers have to be detail-oriented. AI-generated outputs can look polished at first glance, but when you really look closely, you will find mistakes. And we need to catch and correct those errors before clients or stakeholders do. As researchers, we own the final quality AI is a tool, not a substitute for our expertise. And we absolutely cannot risk assuming it is 100% correct. All of this brings us back to the main lesson. Winging it just prompting can be a quick start, but frustrating and ultimately take longer to get to a useful output. Prompt engineering delivers far better results for most research tasks than just winging it. Now let's close with three points to remember. 1. Try out known prompt engineering techniques like prompt chaining, reflection, and zero-shot. They can dramatically amp up your AI outputs. 2. Treat the AI as a co-creator. Run it through several rounds. Ask it questions, point out errors, give it your success criteria, ask it for variations. With AI, you can do multiple iterations in minutes, not days. 3. Review the AI's output closely, word for word. The human is always responsible for accuracy and quality. I hope this conversation was helpful, and if you want more practical examples and resources, check out researchrockstar.com and the other videos on our YouTube channel. And for current Research Rockstar students, don't miss the AI job aids in your training portal. Thanks to all of you Research Rockstars out there, and please share any questions or feedback in the comments.