What a PhD Thesis Critique Reveals About Rigor (Full Transcript)

[00:00:00] Speaker 1: And whatever it is you think you're good at in your life, I'd probably take about a year and become an authority in this field in which you think you're good. So whatever ideas you have about how I can improve, just f*** right off.

[00:00:08] Speaker 2: I'm tougher than you. I can drink closer to failure than you. I can do it for longer. I have more willpower than you. I'm funnier than you. I'm smarter than you.

[00:00:13] Speaker 3: Hey, Professor Stuckler here. Today, we're reacting to a video about the PhD of Dr. Mike. He's a famous YouTuber on fitness who describes himself as a preeminent expert, the smartest coach in all of the fitness industry.

[00:00:27] Speaker 1: I'm both on a raw IQ scale smarter than almost every coach, maybe every coach. And has an IQ over 160. And so my IQ is above 160. We don't know how high because they don't do standardized IQ tests above that.

[00:00:40] Speaker 3: We're going to compare that to a video which got a copy of his original PhD, went through it in detail, and tries to make the case that the whole PhD is bogus, and it wouldn't have passed muster at a respectable university. Listen, the reason I'm making this video is to help PhD students and researchers, many who follow my channel are at various stages of their research journey. And I want to help you safeguard yourself against potential critiques in the future and also better understand what the thresholds and milestones really are for a PhD. I don't want to delve into a critique about who's right, who's wrong, should this PhD have passed or failed. That's just not my purpose. So what we're going to go through in this video is first, I'm going to set some background about two axes of confidence and ability, one known as the Dunning-Kruger effect and another commonly referred to as imposter syndrome. Then I'm going to go through what are some common criteria that were in the video I'm reacting to for attaining the threshold of a PhD. And look through some of the critiques it made of Dr. Mike's thesis and what that means for you. And finally, I'm going to highlight how you can avoid some of the rookie mistakes that were spotted in this PhD through some simple steps that you can take today so that you're safeguarded now and well into the future. This is important because we are seeing these kinds of attacks on people's theses more and more as some seek to discredit those perhaps who are in positions of authority or power. The most extreme example of this recently perhaps has been the critique of Harvard's former president, I should say PhD, finding there were some misattributed citations and sources that ultimately got her accused of plagiarism and led to her resignation. So with that, let's dive straight in to our background about the Dunning-Kruger effect and imposter syndrome. Each axis I want to label ability. And on one, I want to put here perceived ability. It's kind of how good you think you are versus your actual ability, which is how good you actually are. Ideally, as your actual ability goes up here, you perceive yourself to have more ability. But there's a phenomenon sometimes known as the Dunning-Kruger effect, which really is a cognitive bias, which was used to describe the situation where some people, especially at low levels of actual ability, perceive themselves to be much more capable, more of experts than they indeed really are. And that's when people fall here where their perceptions outstrip their actual ability. On the flip side of that, there's a phenomenon known as the as imposter syndrome. This can be feeling like a fraud, feeling like you're not good enough, you don't belong, even that you're never going to make it. And it can happen when your perceived ability is really down in the gutter, that you feel like you're not good enough in line with your actual ability. OK, so with that backdrop, let's dive in and see what the whole thesis is all about.

[00:03:34] Speaker 4: Mike Israel's PhD thesis titled The Interrelationships of Fitness Characteristics in Division 1 Athletes investigates how various fitness traits, including strength, rate of force development, power, vertical jump height, short distance sprinting speed, and body composition, relate to one another in Division 1 collegiate athletes. The overarching aim is to explore the extent to which athletic performance can be understood as the summation of these interrelated physical attributes.

[00:04:03] Speaker 3: So on the surface, the research is plausible, looking at the intercorrelations of different athletic characteristics to see perhaps if there's some core set or constellation of these characteristics that then relate to overall academic performance in various domains. Efforts like this have been done for intelligence as well, where some theorists try to think of not as kind of book smarts, street smarts, all these different manifestations of intelligence as being different, but having some underlying g-force that relates to all of them.

[00:04:32] Speaker 4: Let's keep going. Before we critically examine Mike's thesis, we need objective criteria to assess it by. Although the core expectations and assessment criteria for a PhD thesis are remarkably consistent globally, my alma mater Melbourne University can serve as a guide. To be awarded a pass, a doctoral thesis must a. Demonstrate authority in the candidate's field, and show evidence of command of knowledge in relevant fields. So you have to inspire trust in the reader. The reader should get the impression that you're a reliable investigator.

[00:05:07] Speaker 3: Yeah, so this is true, definitely. One way to think of a PhD is like getting a driver's license, so that you're a card-carrying member of the field. And typically, think about what the flow of the PhD is. You do some original research, it gets sent out to independent examiners, who are supposed to be at arm's length and objective in our other card-carrying members in the field, who then evaluate it and decide if it is good enough or not to meet the bar, which I think this reaction video is going to cover next.

[00:05:37] Speaker 4: b. Demonstrate a thorough grasp of the methodological techniques appropriate to the research question, and an awareness of their limitations. So it's not enough just to choose a methodology. You have to persuade the reader that you chose it for a good reason. c. Make a contribution to knowledge that rests on originality of approach and slash or interpretation of the findings, and in some cases, the discovery of new facts.

[00:06:05] Speaker 3: This is really important, because there are different kinds of contributions. There can be a great contribution in just demonstrating something that we've already believed but never been able to show to be true. So that can on the surface look like, well, this is just common sense. Yes, but perhaps we haven't had the data to be able to show that yet, or we haven't had the methodological techniques or research design that powered such a study. And that's actually a very nice contribution to the field, and definitely nothing wrong. And I think that's why this says, in some cases, discovery of new facts, because that's not essentially necessary or even sufficient for a PhD.

[00:06:47] Speaker 4: So you have to offer a genuinely new contribution that meaningfully advances your field. Wait, wait, hang on.

[00:06:54] Speaker 3: That's not 100% true. In research and science generally, there's a notion of standing on the shoulders of giants. And that means we're building on what came before and often taking incremental steps forward. At the level of a PhD, that can indeed be reproducing existing work. It can be failing to verify existing studies, maybe using new data or new methods or a different design. But I often say, PhD is not the time to try to win a Nobel Prize with some vastly original contribution. It's the time to get points on the board, maybe a base hit or single, to use a baseball analogy. And often you'll see work that is in this terrain, which is low risk, low reward, just performing right at the forefront of the field so that card-carrying members say that, yes, this is actively contributing.

[00:07:46] Speaker 4: D. Demonstrate the candidate's ability to communicate research findings effectively.

[00:07:51] Speaker 3: Yes, 100%. It is important to be able to write clearly and present well. I do find this is a problem that beleaguers a lot of graduate students because they've never had any writing training. And scientific writing isn't like the writing that you might have done for your English class back in the day. I do recommend checking out our writing system, peer system. I often say, if you give me five minutes of your life, I'll completely transform your academic writing for forever. Check out the links below.

[00:08:19] Speaker 4: So you can't bury your findings in jargon or submit something that reads like an unedited draft. Your thesis has to be clearly written, professionally presented and free from basic mistakes, whether in language, formatting or data handling. E. Demonstrate an understanding of and commitment to research ethics and integrity. Yeah, 100%.

[00:08:41] Speaker 3: This is pretty standard stuff.

[00:08:43] Speaker 4: And F. Be a careful, rigorous and sustained piece of work, demonstrating that a research apprenticeship is complete and the holder may be admitted to the community of scholars in the discipline.

[00:08:54] Speaker 3: Yeah, this is exactly like that PhD as a driver's license analogy, that in a way you've ticked the box to where independent members of the research community who are card carrying members holding that license themselves, look at your work and say, yes, this is meeting the standards of rigor and scientific practice that we expect from another card carrying member of the discipline. This can happen in a few ways. So the most conventional way is through submitting your thesis to have independent review from examiners. There's also something called a PhD by publication, where you write together papers and submit them to independent journals who then send them for independent peer review. And those experts in the field decide whether they pass muster, whether they merit publication in those journals. What you can do is then stitch those together and submit them to be recognized that, yes, this does classify as a contribution to the field and thus merits a PhD. But the point is that experts in the field are judging your work and saying if it's good enough to be in and that good enough is not always perhaps as black or white as people would lead you to believe. There definitely can be variations there.

[00:10:06] Speaker 4: You can't just stitch together a few chapters and call it a thesis. The whole thing needs to be coherent, meticulously edited, internally consistent and attentive to detail, which includes, of course, how your data are presented, interpreted and integrated across chapters.

[00:10:26] Speaker 3: 100% PhD is a massive undertaking. It is not for the faint of heart. I don't recommend it at all. If you just think I don't know what to do with my life, maybe I should do a PhD. Plenty of studies now constantly demonstrated that your mental health will decline on average if you take on a PhD. So you really want to dive into this if you're ready and willing to work very hard for a prolonged period of time to develop deep expertise in a subject matter. So now let's turn to what the crux of the critique is. Now that you've all seen what the criteria for a PhD are, having that out of the way. And these are pretty standard criteria across the board.

[00:11:06] Speaker 4: In this chapter, I'll argue that according to standard assessment criteria, Mike's thesis would have failed or at the very least been sent back for major revisions at any high ranking university. Mike's thesis falls short of fulfilling one of the most fundamental requirements for doctoral research, a significant contribution of original knowledge. The stated goal of Mike's thesis was to explore interrelationships among fitness variables that he claimed were less clear in the existing literature.

[00:11:37] Speaker 3: Okay, so this is important. He's saying here interrelationships among them are not well established. There's a part of a thesis where you define your gap. And it's basically to say, look, the field has brought us to here, but what they haven't done is this. And that distance between what I'm doing compared to what's already been done, that is the gap and it directly links to your value add. You have to define what's the missingness of existing research. And this reads to me like a very clear gap statement that these interrelationships haven't been looked at.

[00:12:13] Speaker 4: However, the core findings that stronger athletes are more muscular and powerful, that more powerful athletes jump higher and sprint faster, and that leaner athletes perform better in these tasks are not only unsurprising, but also extensively documented in prior research.

[00:12:31] Speaker 3: So look, I mean, there are, you go through the library and look at theses. There are a lot that prove or try to reinforce the evidence on things that we believe to be true or have already been shown. Often they might be doing it with new data, taking a new spin on it. Again, it comes back to fulfilling that gap and making a clear articulation of the value add over and above existing research.

[00:12:59] Speaker 4: Even by 2013, you don't have to take it from me. Just listen to Mike himself, who openly concedes this throughout the document. For instance, in chapter three, Mike explicitly notes that the finding that stronger athletes produce higher rates of force development has been previously well supported in the literature. So this in and of itself is not a concession.

[00:13:22] Speaker 3: In fact, this is ideal. What you typically want to do in any research paper is say, hey, look, in my data, which no one's maybe looked at this idea before or haven't tested these relationships, is this borne out? So is this consistent? Does this cohere with existing scholarship? Yes, no, if no, where does it depart? And so commonly you will say in this, add some credibility to your data that, yes, I did corroborate what others have said and maybe say, no, I didn't find support for what somebody else had found. Listen, I mean, let me give you another example from the health research. There was research on tobacco and heart disease done in the U.S. and people in India said, well, that might not apply to India. So we need, we don't believe the U.S. data. We have to test it out here and guarantee you there's been a lot of theses that just replicated what was shown in the U.S. in a different context in India to show those same effects. And lo and behold, they found tobacco had the same effects on heart disease. I won't get into that, but I think this is a genuine misunderstanding of writing a section that is common in conclusions and interpretation of coherence with existing literature.

[00:14:26] Speaker 4: The finding of higher power generating abilities being associated with stronger athletes is well supported by previous data.

[00:14:34] Speaker 3: Again, same points as before. I'm going to go through this because from my point of view, I don't see this as necessarily a strong critique, but again, I haven't dived into exactly what the value add was in terms of meeting that gap. And I haven't scanned through the whole thesis to see what all the gaps were that the thesis promised to fulfill and how effectively it delivered on that. And I also don't have the in-depth knowledge of the specific literature to say, well, was this validation in this dataset a contribution? Because that validation in a novel dataset here being elite athletes, that could very well be a contribution to the field if it had been done in un-elite athletes, and now they've done it in elite athletes. That could make a lot of sense plausibly. Let me go on here to the second big critique about lack of independent thought.

[00:15:22] Speaker 4: Another way Mike's thesis could be said to lack originality is its dearth of independence in thought and approach. While it is expected that a PhD candidate will ground his or her work in existing literature, Mike's thesis rarely moves beyond mere reiteration. As I'll detail later, the literature review in Mike's thesis is uncritical.

[00:15:43] Speaker 3: So this critical point is typically you want to critique existing literature and kind of drive the argument to roll out the red carpet for your upcoming data analysis to say, well, the literature is great on this, but weak and hasn't done this. That leaves your gap and rolls out the red carpet for what you're going to do next in your thesis.

[00:16:02] Speaker 4: It's just an inventory of past findings rather than an evaluative synthesis. And that kind of passive engagement with sources continues throughout the thesis. Mike's research questions are framed in terms of small population specific gaps without articulating why these gaps matter or whether existing models should...

[00:16:22] Speaker 3: So this here is articulating a type of gap called a population gap. They haven't looked at this in large teams or teams of athletes. This fits in this category of that US-India tobacco heart disease example before. I suspect this thesis, as I kind of alluded to earlier, might have been trying to find kind of this underlying core of intercorrelations that could lead to some core G of elite athletic performance. I don't know if it actually went that far. If I were going to advise Dr. Mike, I think that would be a very cool way to approach this study, not being familiar with the academic literature in this area at all. But yeah, this is a very plausible set of gaps and a potential value add that on the surface of it, I haven't looked at the results yet. The thesis is likely able to deliver on. Let's go in here to the next critique.

[00:17:15] Speaker 4: A doctoral thesis isn't evaluated just on the originality or significance of its content, but also the clarity and accuracy with which that content is presented. Mike Israel's thesis fails comprehensively on this front. Across the thesis, Mike presents impossible and contradictory data.

[00:17:34] Speaker 3: Okay, so this we'd have to look at. It's kind of if you get in an age group in years of people that doesn't make sense or something. So if you saw an age in years, I'm looking at elite athletes and age was 80 years old or something, that could look funky, but let's see where this goes.

[00:17:51] Speaker 4: Inflates simple points with jargon and allows hundreds of basic grammatical errors.

[00:17:59] Speaker 3: Big rookie mistake. Look, I see this even at high levels. Guys, simple fix. Get Grammarly. It's free. It's easy to use. It will catch these howlers. And when you're doing a big project like a thesis, these kinds of things, composition and a misplaced dot, when you have multiple layers of editing upon editing and maybe, excuse me, track changes from your colleagues, this stuff can creep in. It happens to everybody. Get Grammarly and stamp it out. That's tip number one for today.

[00:18:23] Speaker 4: Hypergraphical and formatting errors to remain uncorrected. The following sections unpack these issues in detail. And by the end, you'll see that it's impossible to view Mike's work as anything but fundamentally compromised. Some of the data Mike presents in the extreme group comparison tables are statistically impossible. On page 77, Mike acknowledges the large standard deviations in tables 4.6 and 4.7. But how large are we talking? Well, let's examine. I'll put on screen tables 4.6 and 4.7, which are on page 71. They compare the top 20 and bottom 20 jumpers. Tables 4.6 and 4.7 report that for the 20 lowest performers, the mean body masses were 73.9 kilos and 75.3 kilos respectively.

[00:19:18] Speaker 3: So I think this is going down the path of trying to make a data critique. Look, these are small numbers. I won't go through a statistical example of how you can get very large standard deviations with small numbers. I mean, this suggests to me seeing these high standard deviations could be just trying to reconcile without actually having the data. In good transparency, it is good. I encourage all the researchers we work with to be at the avant-garde of transparency and publish their data sets with their work. And there's an old school way of wanting to keep your data close to your chest so that people won't run off with them. This suggests there could be some big outliers in the data set. So if I were guiding Dr. Mike here, I'd flag this as, hang on, wait a second, let's take a look. But again, it's not to undermine the scientific project and its credibility, but there are often even just clerical errors that slip into data analyses all the time. One thing I'm increasingly recommending the researchers I work with, as it comes to tip number two, to avoid potential rookie mistakes and errors, is to perform an independent AI peer review of your paper. And this has become increasingly important because reviewers are getting lazy and doing that themselves. So you want to know what the AI or the chat GPT is going to come up with. But it'll also catch howlers. And it'll catch howlers not just in the grammar, which grammarly will do, but it'll catch the equivalent of grammar errors in data entry in your data sets. And a thesis might have even hundreds. Mine had a ton of tables and figures. And look, this could be simple typing errors that can creep in that you might not spot. Your viewers might even not spot because they're human. Okay, let me move on to the next critique here. This one on copy pasting editorial negligence.

[00:20:50] Speaker 4: One of the sloppiest features of Mike's thesis is the rampant copy pasting of entire sections across chapters with identical errors left untouched. The methods section in chapters four and five are near verbatim clones of those in chapter three.

[00:21:07] Speaker 3: Yes. I mean, if you've just copied and pasted blanket chunks of text, that could be a problem and maybe an error. But there are two styles of PhDs. So one is kind of a traditional narrative book style that goes kind of introduction literature, literature review methods, and then empirical chapters with results. And another style that is that contained almost PhD by publication like style that contain paper, contain paper, contain paper, and each of those would have its own method section. I think this is of the latter type. And what I can see here is these are not verbatim. It says, if I look at the second column here, it looks at the relationship between vertical jumping and a variety of other fitness characteristics. In the third one, it says muscularity and a variety of other characteristics. It's not implausible that you would have very similar methods to describe the same subjects and data collection procedures that were common across the board.

[00:21:58] Speaker 4: Producing the exact same typos, the exact same formatting mistakes, and the exact same.

[00:22:04] Speaker 3: Now, take a note about these formatting mistakes again, because when those creep in, they undermine your credibility as an author and can lead people to falsely believe that you did a cursory job when you didn't. So really, again, this just reinforces to make sure you use Grammarly because this should never happen. Let's go on to the next critique here. Okay, this looks like on references.

[00:22:26] Speaker 4: Mike's thesis, written in 2013, was required to comply with the APA 6th edition, the mandated style guide of East Tennessee State University's sports science and coach education department. The thesis repeatedly fails to follow even the most basic rules of APA 6 citation and formatting. In-text citations are riddled with errors. Author groups are separated by commas instead. Okay, hang on.

[00:22:53] Speaker 3: I see this all the time, and I get why this happens. You have hundreds of references, and if you're trying to do this by hand manually, you are setting yourself up for failure. Simple fix. Get Zotero. It's free. This type of project has such a large undertaking that you will get lost with your references. Yet with Zotero, you're going to avoid misattributing sources because you have them right there in a reference manager to deploy. And with a click of a button, you can go from APA to Vancouver to Harvard. These are different citation styles that are commonly used, and avoid this kind of critique or problem to come out like a bogeyman and invite you into the future. Look, I see this even happen on people submitting papers to journals. So it's a very simple fix, and don't let that happen to you. So I'm going to go ahead and skip through to the end because this seems to repeat the same core critiques. Let me head out this way.

[00:23:43] Speaker 4: Last, this analysis belongs to a larger project of understanding Mike Israel's delusions of grandeur. Mike Israel speaks as if he's a towering intellect in the fitness space.

[00:23:55] Speaker 1: I'm both on a raw IQ scale smarter than almost every coach, maybe every coach. And I know more about physiology, body responses, etc. than they do. You've never met anyone with as much willpower as me. And whatever it is you think you're good at in your life, I'd probably take about a year and become an authority in this field in which you think you're good. And also in every category of human behavior, I'm tougher than you. I can train closer to failure than you. I can do it for longer. I have more willpower than you. I'm funnier than you. I'm smarter than you, and I'm better.

[00:24:27] Speaker 3: Okay, so in marketing, there's something called a puff when somebody says, we have the best fish and chips in the whole world. I don't want to dive into whether this is accurate or not, because a PhD can't tell you that. A PhD is weakly and imperfectly correlated with intelligence. I know some bright PhDs who can't find their way out of their own driveway. And I know some people who don't have PhDs who have made stunningly impressive innovations and contributions to our society, pushing the borders of even scientific understanding. So yes, I don't want to go fully into this. Let's just watch a little more.

[00:25:05] Speaker 1: So if you talk to me about anything with resistance training and hypertrophy, yes, I consider myself to be one of the world's permanent experts on all of that, especially in its application interaction with science.

[00:25:14] Speaker 3: So he says something here about where he locates his preeminence is the interaction of research with science. And I think one way to think about this is there may be doctors who are the best in the world who are surgeons who implement a procedure. They have a very steady hand, and they're great at following the guidelines and doing a very clean job. But that's different from the expertise that designs a surgical procedure, designs the tools that they use. So again, the standards for that academic contribution would be recognized through some objective indicators around citations, around awards, around different kinds of accolades. When it comes to fitness YouTuber, there isn't kind of the same standard for what that preeminence means. One thing that a PhD will do for you is it should cultivate a degree of healthy skepticism to claims, especially scientific claims. I use the Italian phrase, it's very common. I live a lot in Italy. Trust but verify. Trust but verify science. So if you're in any domain consuming scientific information on YouTube, be skeptical about it and try to validate it for yourself, ideally going back to the original scholarship.

[00:26:27] Speaker 1: I make no excuses for that. I will debate anyone in the world about that. That's easy. I haven't met a whole lot of people in the bodybuilding industry with whose raw intellect I was blown away. You are presuming you know more than me about the thing I am one of the top experts in the world in and also the thing I'm literally involved in myself. So whatever ideas you have about how I can improve, just right off because I've taken enough IQ tests and you can only write so many books and be on so many podcasts until people are like, you're pretty smart. I've taken the Raven's Advanced Progressive Matrices test and I pegged it scale high.

[00:26:59] Speaker 3: One of the biggest gains I made from doing a PhD and gaining expertise has been the confidence to say when I don't know something. That simple phrase, I don't know but I can find out for you. I think it's very powerful because it begins to acknowledge you know where the borders of your expertise are. Again, in the spirit of healthy skepticism, I'm always cautious when someone is going to lengths to prove to me or say to me how good they are at something. Just simple cliche that actions will speak louder than words and the top what I've seen of top researchers in the field is that they have nothing to prove. I've never heard a top researcher have to tell me how smart they are because their actions, their publication record, their profile speaks volumes. Fitness, YouTube, different kind of world. So different kind of norms, different kinds of standards. That would not be the way that researchers would engage and from what I'm hearing, again not going into it, I don't think Dr. Mike is intending to be a researcher when referred to as PhD. Just saying like others that he got a driver's license. Whether he uses that driver's license to drive the car is a different question. Stepping back for a second to wrap up guys, I hope this video has helped you to avoid common rookie mistakes that can come back to bite you. Get grammarly today if you don't have it already to avoid any of those typographical errors that will inevitably creep in along the way. Use AI in a smart way to provide an independent check on the robustness of your work and catch any howlers in the data that might have crept in and also get Zotero. It's lovely, it's free, it's going to save you a ton of time. And listen, as you progress and navigate on this journey from wherever you're at in your research career, it's tremendously rewarding, it's tremendously powerful and transformational. If you would like guidance along that journey to avoid the pitfalls of Dunning-Kruger or imposter syndrome, I'd encourage you to click the link below, speak to a member of our team and let's see if we're a good fit to work together and help you move faster in that next stage of your journey, whether that's to finishing graduate level research or to publishing and writing grants and beyond. I look forward to seeing you in the next video.