WRS: In my lectures and writing, I often tell health care and fitness industry practitioners to seek out the scientific source articles when it comes to the latest greatest medical breakthroughs trumpeted across news headlines. But even with the articles in hand, what do the numbers mean? For some, so-called “statistical significance” is a shining stamp of approval. But you absolutely, positively MUST put the findings into context and, even if the results are “significant,” what do they mean in real terms? For example, many years ago, I authored a claim by claim debunking of the dietary supplement pyruvate using the VERY same research articles that marketers were using to sell it! All I had to do was look at the experimental conditions, numbers of subjects, humans vs. animals, and the significant p values relative to the actual numbers. While they spouted off that the product could help you shed 48% more fat and 37% more weight, the ACTUAL difference between the supplement and placebo group were very small.
In this second installment from guest blogger Christian Thoma, MSc, PhD Candidate, he discusses the meaning of the word “significant” and what p-values really mean in practical terms. Over to you, mate!
Introduction: Is It Significant?
Everyone from scientists through to the interested public often misunderstands significance as it applies to research findings. Despite over eight years in research, I've only recently developed a true appreciation for the importance of good statistical methods. I can appreciate why it makes some people's eyes roll, so this post isn't about explaining p-values or confidence intervals. However, I do highly recommend that you read the Wikipedia entry for p-value - subheading: Frequent misunderstandings.
“Significant” in Practical vs. Scientific Terms
In common language, to say something is significant can mean it is important or substantial. In scientific writing, it usually relates to a statistical measure trying to determine if two sets of numbers - results of some study - are genuinely different or only look different because of normal variation independent of the study itself. For example, if we put 20 people on a strength training program and another 20 on cycle-based interval program, looked at their cholesterol after three months, and found that the average changes were different in each group, we would want to know if this difference was likely to be 'real' or just a chance finding (i.e. due to cholesterol varying day-to-day). Using this meaning of significant (i.e. statistically significant), a significant finding is one in which we have good confidence that an observed difference was real.
Consistency vs. Variation
At this point we need to look at the actual findings. In most cases not everyone in a group is going to experience the same change. Some may get a big drop, some a small drop, and some a small rise. If that's the case, the average change would need to be quite big for the statistical tests to show us that it's statistically significant. If on the other hand everyone in the group had a small drop, the average change could be quite small and we would still get a significant result. It's a case of consistency vs. variation.
Statistical Blender vs. Real World Meaning
Let's say we get a 'probably' real difference. We still need to ask - is it significant in the common tongue? Was it substantial and/or important? The answer depends on your perspective and on which of the two scenarios above apply. As a researcher you would be happy to have found something be it big or small. But let's say the second scenario applies where we are fairly confident that there was a drop, but the drop was consistently small. In such a case, you as a patient, clinician, or trainer might not care because there are other ways of getting a much bigger drop. How you thought about it would also depend on whether a small drop was important in your context. Had the study been about a supplement causing fat reduction, but only by a very small amount, you might think it useful for dropping the last 1-2% body-fat for a physique contest, but not for helping a client drop 10 kg (22 lbs).
Clinical vs. Public Health Significance
The above example is often called clinical significance, but it applies outside the clinic. If you work one-on-one or use research to inform your own lifestyle, you benefit the most by first confirming statistical significance and then applying your own criteria to clinical significance.
There is another kind I like to call public health significance. It's important because the rules are different when you apply a small change to a very large group. Let's say there was a cheap, safe, and easy way to lower people's blood pressure by 1-2 mmHg, a way that could be implemented so that a large portion of the population benefited. Chances are public health people and health economists would be pushing it. Why, given that no doctor treating individual patients would care about such a small change? Because by shifting the average of a large group a little, it will also shift a lot of people from a higher risk category to a lower one. Here it's the size of the group being affected and not the size of the individual effect that matters. This last example should help explain why some very modest findings, in the right context, stir up excitement.
Take Home Messages
If you're interested in the results of research, check for statistical significance, but then go further. Know that even a highly statistically significant result doesn't mean the result was substantial or important as this is context specific. Be clear on the relevant context for you, or your client/patient. Understand that in a different context, the findings may have a different importance or relevance.
WRS: The eloquently written guest blog you are about to read was authored by a friend and colleague Christian Thoma and tackles the popularly accepted notion that if it was good for our paleolithic ancestors, then it must be good for us now. Over the years, we have shared many colourful discussions in the areas of nutrition, exercise, health, and disease. One of the (many) things I admire about Christian is his critical thinking and analytical abilities. Enjoy and please feel free to post your comments at the end of the blog. Thanks!
About the author Christian Thoma is a former food regulatory scientist who has recently chosen to pursue his hobby of exercise science by undertaking a PhD on exercise and non-alcoholic fatty liver disease. He is an American College of Sports Medicine certified Clinical Exercise Specialist, holds a BSc in microbiology and immunology, and an MSc in human nutrition, both from the University of Otago, Dunedin, New Zealand. He presently resides in the UK, where he spends his spare time purchasing exercise equipment and any reading/viewing materials that use evidence-based reasoning to advocate a change in the status quo in nutrition, exercise, and how science is conducted and reported.
Cave Man is SO Back in Style!
It has become somewhat popular to explain things in both biological and psychological sciences in terms of their evolutionary basis. In the biological sciences, advocates of the paleolithic diet spring to mind. In psychology, the often used example is how chronic stress is the result of a fight or flight response suited to acute danger now being continuously activated by everything from time pressure to road rage. You'll no doubt have your own examples you've heard, read, or even thought up. I'll focus on the biology as I'm not a psychologist.
Cave Man Debate: Proceed With Caution
I'm convinced by the evidence that we did indeed evolve, and am not here to start a debate about that any more than I'm prepared to debate a man claiming to be Julius Caesar demanding the keys to Rome. Nor do I intend to argue that the various theories based on evolutionary concepts are wrong. I would, however, caution anyone from accepting a line of reasoning that starts with "we evolved to," purely because it sounds plausible/sensible. The reasons that I think you should be cautious are that: 1) it's really hard to know how people conducted their lives 20,000 years ago; 2) even if we did evolve in this or that way, it doesn't mean that is optimal - it just means it was good enough for our ancestors to survive long enough to reproduce; and 3) we haven't stopped evolving and we don't have a good handle on how quickly it happens.
One Size Does Not Fit All
Anthropologists do have their techniques to study past cultures and they do an amazing job given what little they often have to work with. It is, however, very difficult to verify the hypotheses they form. Even where these are robust (or we assume them to be), our ancestors didn't all live in the same places, in the same way, eating the same food. Therefore, to promote one lifestyle or diet purely on the basis that this is how our ancestors lived lacks broader perspective.
Because Cave Men Survived, Doesn't Make Their Lifestyles Optimal
Evolution is driven by reproduction. The strongest don't always survive, however the genes of those that reproduce certainly do. The easier it is to survive in a given environment, the greater the diversity of people who pass on their genes. Ecosystems appear to move towards a state of balance, or they collapse. These systems don't create the 'best' anything as that would create imbalance. Look at new species introduced into ecosystems where they have no predators and/or an abundant food supply - they thrive and take over. To suggest that we will thrive because we go back to what was adequate for our survival ignores this one obvious observation entirely.
As I said above, the evidence for processes of natural selection has certainly convinced me. But the details are an area of ongoing intense study. There is evidence that the nutritional state of the mother during pregnancy has effects on her grandchildren and perhaps beyond in ways that are not part of the classically understood models of the passing on of genes. We simply don't know how much and in what way we may have evolved in the last 10, 20, or 50 thousand years even though there is good reason to view these as short time frames in evolutionary terms.
Is the Total Equal to the Sum of the Parts?
Being able to devise a hypothesis based on evolutionary reason may well be a good justification for further study, but it shouldn't be regarded as sufficient evidence in itself. It may be that a diet based on what is believed about pre-agriculutral times is healthy, or that a lifestyle involving movement similar in quality and quantity of that performed by stone-aged tribes is healthy. It may also be that it wasn't so much the diet as the calorie cycling - from bountiful to limited - has value over the actual foods, their ratios or quantities. Being more active/less sedentary is good advice, but the movement quantities of our ancestors may lead to considerable wear and tear over the long life-spans many now lead. Lastly, the need for food probably drove activity, this is no longer the case. There is no guarantee that if one takes a component out of one system and slots it into another it will work the same way or even be appropriate at all.
"Natural" Does Not Always Mean Safe or Beneficial
This brings me to the closely related assumptions by many that 'natural' is good and 'unnatural' or 'man-made' is harmful. Presumably because we evolved for the former and not the latter. I used to work for a food regulatory agency. We proposed legislation and as such had to be very clear with our definitions as they needed to be enforceable in court. If you can't define what you do and what you don't mean without creating a very long list, forget it. Funnily enough, we never managed to define 'natural'. It means one thing to you, one thing to your friend, and something slightly different to your neighbour. Definitions aside, nature abounds with very toxic things from animal and insect venom, bacterial and fungal toxins, to volcanic ash and sulphuric acid lakes. Even some of the nastiest man-made stuff is just a concentrated version of nature's finest. In toxicology, our motto was the difference between a medicine and a poison is the dose. Do be careful about what you put in your body and advise clients as to what they put in theirs, but please don't assume that if it's natural in your definition it's also health promoting.
Take Home Message
My take home message is always look at the evidence and think about it critically. Don't fall prey to things that suit your biases. And be very wary of any reasoning based on a philosophy such as natural is good and man-made is bad or, if it worked for our ancestors, it will do us all good now. Take advantage of what science, viewed with a critical and inquisitive eye, has to offer. We don't yet have a thorough enough grasp of physiology to be able to consistently predict how eating or doing something will affect us, and reality doesn't always fit into nice philosophical frameworks.
Kelly Slater Wins His 10th Association of Surfing Professionals World Title
What a week for professional surfing. Kelly Slater nails down an historic 10th ASP world title, Stephanie Gilmore stitches up her 4th title, and Andy Irons unexpectedly passed away.
This blog was prompted by an article in The Australian, a few passages of which reminded me of my two personal encounters with the surfing alien commonly known as Kelly Slater.
The article goes: ON the Gold Coast in 1984, Australian surfing team coach Paul Neilsen got a call from Quiksilver founder Alan Green. It was after the Pacific Cup, an international teams event, and Green wanted to get the goss on some kid from Florida.
"What's this kid like?" Green asked.
"He's the best surfer I've ever seen," Neilsen replied.
------------------------------------------------------ First Encounter With the Prodigal Son WRS: Fast forward three years to 1987. Some friends of mine were down surfing in mainland Mexico that summer and happened to meet up with a group of Floridian surfers tearing it to bits. Then about a month later, we were hanging out on the beach in front of the lighthouse in Cape Hatteras, North Carolina during the East Coast Championships when all of a sudden these same guys turn up, only they had another guy in their company: a 15 year-old Kelly Slater.
15 year-old Jedi
I'd seen his name in the surf mags a couple of times and heard he was a good surfer, so while my friends were talking with his friends, he and I exchanged a few words. Nothing too deep, more just about the swell, wind directions, etc. But there was one thing that is forever burned into my mind that I thought made the encounter unforgettable. Though he was quite friendly and happy to talk, his eyes were intensely focused on watching the surf, keenly studying the waves at the contest site, mentally preparing for his upcoming heat. It was as if he was doing some sort of Jedi mind exercises. It was all so adult-like, yet here was a kid without even so much as a whisker on his chin. I knew the kid was different but, at that moment, I don't think anyone knew just how "different" he really was.
I particularly liked that he was somewhat quiet and soft-spoken, not about spouting off to the world how great he was. Seeing just how far he's come, his contribution to the sport of surfing, his (perhaps unwilling) position as a living cultural icon.....well, his humility only makes the story all the more poignant.
Surfing Time Warp
During that same trip, I crossed paths with him again while surfing Frisco Pier. I was paddling over the back of a lefthander just as he was taking off. I looked down just as he was driving hard off the bottom and, just as he passed out of my view, I looked back and saw him launch a backside aerial, land it, and then proceed to do another one, and after that he systematically dismantled the wave to oblivion. Whoa, real deal! I was sold. Now, 23 years later, I can honestly say, his surfing on that day was radical even by today's standards. He was unquestionably the best I'd ever seen. I mean, yeah, Tom Curren, Tom Carroll, and Martin Potter were my heroes back in those days, and they clearly ripped, but this kid was my age, my generation, blazing a trail for a new generation of surfers. He raised the bar on what was believed possible on a wave, and most impressively, he did so in a controlled and functional manner. He wasn't just launching big airs and then flailing to recover in the whitewash. No, he was landing everything with convincing authority.
The Dynasty Begins
Once he got a taste for Hawaii and nailed down his rep in some serious shit, I can't say I was surprised he bagged a world title on his rookie year on the pro tour! One title turned into two, turned into three, turned into four. Only Mark Richards had nailed down four world titles and here Slater matched him. Then four turned into five, turned into six titles. The more he kept winning, the more I realized we had an extraterrestrial living amongst us.
After that, he seemed to change gears and backed off a bit, with other guys (including Andy Irons) winning the championship over a seven year period. But I think what really amazed me was that with all the new generation of surfers on the WCT, he was still able to come back and win it four more times, making him the youngest and oldest world champion surfer. God only knows how long it's going to take for another surfer to come along and win 10 titles.
The 2004 Encounter: Surfing's First Rock Star
I next crossed paths with Kelly Slater 17 years later in 2004 at a pro contest at Maroubra, Sydney, Australia. Of course, by that time, he both literally and figuratively walked on water and couldn't go anywhere without a mob of rabid fans following his every move. Here's a Youtube video of that very contest. Note how he barely exited the water and the crowd just descended on him, with security having to escort him back to the contest scaffolding. I was with some friends that afternoon at a local restaurant and was a bit distracted by a large group of people along the main drag. Then I noticed Kelly Slater buried in the middle of this moving scrum of humanity. Wow, things had certainly changed in his life since 1987.
Taking Down the Tiger
The article in the Australian states, "While Slater takes time to think about whether he wants to aim for an 11th surfing world title, the three-handicapper revealed he wouldn't mind giving professional golf a shot.
"I've been thinking about that, I'd love to win a pro golf tournament one day," Slater said.
After seeing the look in his eyes that September day on the beach in Cape Hatteras, something tells me he just might give Tiger Woods a run for his money!!
Unless you've been living under a rock the past 24 hours, there's almost no way you haven't heard the news of Andy Irons' unexpected passing. As of this writing, it is not known for certain how he died, but speculation seems to be settling in on two possibilities: 1) dengue fever; and 2) medications found on the bedside table. In a few of the news items I've read, he had been in recovery for drug addiction, so no telling what's going on just yet until the report is released. In any event, it is not my intent to slur the name of the 2002 - 2004 ASP world surfing champion. What he was able to accomplish on a wave was mind-boggling to say the least and, by many accounts, he was the only real competitive threat to nine- (almost 10) time world champion Kelly Slater. I'm sure I'm not the only one scratching my head and asking why why why.....
Behind the legend, Andy Irons was only human
The reason for this blog was prompted by a short video by Billabong which highlights Andy Irons and contains some interesting, candid admissions by the man behind the public image. The bits that caught my attention start at the 50 second time stamp. You can view it here:
I often marvel at the minds behind individuals of immense talent. I find it both inspirational and, at times, admittedly a bit disturbing. The inspirational part comes from hearing their stories and how they rose through the ranks to reach the pinnacle of their career. I love to hear their methods and techniques they've used for keeping mentally anchored, focused, grounded in order to weather the storms of life on their meteoric rise to the top.
On the other hand, I find it disturbing because I realize that, when the lights die out and the crowds go home, it leaves something to be desired, a persistent longing. You're a hero for the moment in the eyes of adoring fans, but then the next day, average Joe is like, "ok, that was fun. Right, now I need to go to my J.O.B. to pay the bills." How do you process and reconcile all this? It's a common story really. How many famous musicians, actors, sports stars have you heard of that have had complete meltdowns, later admitting that the pressure and expectations from the public, team owners, sponsors etc just got to be too much? You get high on the high, but then come to expect it. When it's not there, it can be downright depressing.
Never get drunk on your success
One of the things I've realized in life is when great things happen, you NEVER get drunk on your success. Never start beating your chest and blowing your own horn. Appreciate what's happening in the moment, be grateful for it, live it, feel it, I mean truly experience it in that time space, but never let that fleeting moment rule your mind, as it only sets you up for a massive let down. Great, so you won the Super Bowl. Awesome! On Monday morning, everyone is back to their own lives and nobody really cares anymore (or not as much)!
Lowrider?
On the other hand, never ride the lows too low. I've coined the phrase "Mood Proofing" and tend to use it quite regularly as it allows me to handle the ups and downs in my own mind in such a way that it keeps me mentally and emotionally stable, balanced somewhere in the middle. Sure, it's easy to get sucked into the void and throw a pity party for yourself when you're having a rough patch, perhaps letting it infect your thinking while it drags you away from potentially positive growth experiences. As a keen observer of life, I've come to realize that no matter who you are, no matter how high you've climbed in life (or are climbing), everyone is subject to tough times and inner mental turmoil. I am convinced everyone has their inner demons. That Achilles heel, mental Kryptonite that can either make or break you. God knows I certainly have my own demons I wrestle with, but it's how you process and manage those demons that can send you down a path of success or destruction.
Surfing as medicine
Andy Irons: "I have a lot of inner demons. If I didn't have surfing to get those out of my system I would self destruct. Surfing's the only reason that keeps me going in the normal state. It keeps my life on an even keel. without it, it would tip into oblivion."
WRS: Here's a three-time world surfing champion putting himself in a very vulnerable position, cathartically confessing his inner demons to the world. I can SOOO relate to this comment. I swear there are days where it all gets to be too much and the minute I get to the beach and smell that salt air, before I even hit the water's edge, I just feel like I'm back to being human again. It reminds me of an old advertisement in the surfing mags back in the 80s, maybe a Billabong ad, that said, "sometimes life sucks....then we go surfing!" I realize not all readers of this blog are surfers, but whatever your therapeutic version of surfing is, you absolutely unequivocally MUST embrace it. If you're into other sports, music, art, dance, whatever, I cannot overemphasize the need for your own form of mental therapy!
Overcoming mental fatigue Andy Irons: "I swear, I'll be honest, I wasn't having fun. I was going out putting a jersey on, doing what I was supposed to do and it was just repetition. I really lost my will to compete. I wouldn't be doing it right now if I didn't have the will. I mean it's not as strong as it used to be but I know it's growing....."
WRS: I think this was filmed a few years after Andy had won his three world titles and had been having a rough patch, working on battling his way back to the top of the crop a la Occy in 1999. These comments in particular really resonated with me because the very thing that he loved had lost its luster to some extent. Maybe it was the ultimate let down going from the very top of the EXTREMELY competitive surfing world to losing the alpha male position in the hierarchy. I loved that he was getting back to his roots and embracing the very thing that had propelled him to surf stardom. I also saw that he was awaiting the birth of his first child which, no doubt, would have been an important catalyst to keep him on track. But alas we'll never know.
On Resilience Andy Irons: "....I lost so many heats to get to where I was. I lost to a girl in my first contest ever. And I quit for a year and never touched a jersey. And I had to pick up the jersey after that and get back in there. And that's the only reason I got to where I'm at, or I was at, five years ago when I won those titles. And, you know, I haven't won the title in a long time, but you know I'm getting back in the jersey and I'm not winning heats yet, but I gotta start somewhere."
WRS: Again, another testament to the age old value of resilience. The people that win in life are not always the smartest, biggest, strongest, or whatever, but are actually those who can commit errors, learn from their mistakes, harness the wisdom, and keep getting up after each beating to eventually rise to the top (if you're not making errors, then you're probably not pushing your boundaries). And they unquestionably appreciate their spoils when they do arrive! It's not the actual winning but the winning when viewed through the lens of just how much hard work it took to accomplish that win. Anything worthwhile in life takes hard work.
Myth of the Leprechaun
When I look back over my own life and reflect upon my successes along the way, I also pay a sincere and heart felt homage to the failures, difficulties, sacrifices etc that it took to accomplish each of them. In very few cases did so-called "luck" have anything to do with it. I love the expression, "I'm a big fan of luck....the harder I work, the luckier I get."
I can recall a personal story from 1996 when I was living in San Diego. At the time I was working on my masters degree at San Diego State University, living in a nice house on ocean front in Mission Beach, surfing nearly every day, had control over my work schedule, great network of friends, a wonderful social life, I mean just the quintessential perfect Southern California lifestyle in the eyes of all outside observers. One day a friend visiting from the east coast came over to catch up, reminisce about old times, have a surf out in front of the house. During our conversation, she paused, looked around and, upon reflecting on where I was in life at that time, said to me, "Wow, what a life....you really got lucky!" I was shocked, I mean thoroughly offended. Luck? LUCK? What did luck have to do with anything in my life? They didn't hand me a winning lottery ticket when I crossed the California state line. I went through a tremendous amount of personal hardship by leaving my former life on the east coast, showing up with minimal funds, no job, no social network, no guarantee of a successful academic transfer to SDSU. And to push through those storms of life by taking it a day at a time, all the way through to the point of finishing my PhD, well, deeming what I'd worked very hard for as "luck" was a dismissive slap in the face and only served to slight the blood, sweat, and tears associated with coming to California (and later New Zealand) and "making it."
Getting back on track, it is this spirit of well-guided RESILIENCE that must be harnessed and properly employed in each of our lives. Though he passed on much too soon, I am convinced that had he lived, Andy Irons would have come throttling back to a top spot in the ASP pecking order. But if anything "good" (given the unfortunate circumstances) can come from this tragedy, I am hopeful that his story, the good, the bad, and the ugly, will leave a legacy which will help educate and inspire generations to come.
William Sukala is a Clinical Exercise Physiologist with two decades of experience in both clinical cardiac rehabilitation and preventive health care settings. He holds a master's degree in Exercise Physiology, a bachelor's degree in Nutrition, and is currently completing his PhD with a research focus on type 2 diabetes, obesity, and associated metabolic syndrome risk factors. William is a popular international presenter on medical exercise topics and has authored articles in major publications both domestically and internationally. He is frequently cited as an expert in his field by magazines, newspapers, and television news media.
Posts
