The Quantified Self (www.quantifiedself.com) Curated by Curated by Wired Wired cofounding editor Kevin Kelly and Gary Wolf, a managing editor of cofounding editor Kevin Kelly and Gary Wolf, a managing editor of Wired Wired, this is the perfect home for all self-experimenters. The resources section alone is worth a trip to this site, which provides the most comprehensive list of data-tracking tools and services on the web (www.fourhourbody.com/quantified).
Alexandra Carmichael, "How to Run a Successful Self-Experiment" (www.fourhourbody.com/self-experiment) Most people have never systematically done a self-experiment. And yet, it"s one of the easiest methods for discovering what variables are affecting your well-being. This article shows you the five principles that will help you get started in running successful self-experiments. Bonus: an 11-minute video from Seth Roberts, discussing experiment design. Most people have never systematically done a self-experiment. And yet, it"s one of the easiest methods for discovering what variables are affecting your well-being. This article shows you the five principles that will help you get started in running successful self-experiments. Bonus: an 11-minute video from Seth Roberts, discussing experiment design.
CureTogether (www.curetogether.com) CureTogether, which won the Mayo Clinic iSpot Compet.i.tion for Ideas That Will Transform Healthcare (2009), helps people anonymously track and compare health data to better understand their bodies and make more informed treatment decisions. Think you"re alone with a condition? Chances are you"ll find dozens of others with the same problem on CureTogether. CureTogether, which won the Mayo Clinic iSpot Compet.i.tion for Ideas That Will Transform Healthcare (2009), helps people anonymously track and compare health data to better understand their bodies and make more informed treatment decisions. Think you"re alone with a condition? Chances are you"ll find dozens of others with the same problem on CureTogether.
Daytum (www.daytum.com) Conceived by Ryan Case and Nicholas Felton, Daytum is an elegant and intuitive service for examining and visualizing your everyday habits and routines. Conceived by Ryan Case and Nicholas Felton, Daytum is an elegant and intuitive service for examining and visualizing your everyday habits and routines.
Data Logger ( Data Logger for iPhone enables you to store and graph any data of your choosing along with a time-stamp and location. It can be used for anything, whether food-related, animal sightings, or temperature sensor readings around your neighborhood. If you can think of it, it can be recorded and tracked. Data Logger for iPhone enables you to store and graph any data of your choosing along with a time-stamp and location. It can be used for anything, whether food-related, animal sightings, or temperature sensor readings around your neighborhood. If you can think of it, it can be recorded and tracked.
SPOTTING BAD SCIENCE 101.
How Not to Trick Yourself Nothing is more irredeemably irrelevant than bad science.-John Polanyi, n.o.bel Prize winner in chemistry [image]
JIM BORGMAN Cincinnati Enquirer Cincinnati Enquirer. Reprinted with permission of UNIVERSAL UCLICK. All rights reserved.
"PANEL U URGES H HOUR OF E EXERCISE A D DAY"
-New York Times, SEPTEMBER 2002 2002 "WHY E EXERCISE W WON"T M MAKE Y YOU T THIN"
-Time, AUGUST 2009 2009 "LOW-CARB F FAD F FADES, AND A ATKINS I IS B BIG L LOSER"
-Washington Post, SEPTEMBER 2005 2005 "LOW-CARB D DIETS C COMBAT M METABOLIC S SYNDROME"
-Washington Post, July 2007 It gets tiring, doesn"t it? Scientists seem to change their minds every six months. Eggs and b.u.t.ter will kill you, so you turn to margarine and turkey bacon. Now margarine will kill you and one egg a day is okay?! It almost seems better to opt out completely and live in ignorance.
Fortunately, science isn"t arbitrary. In fact, you just need to learn a few simple concepts to separate the truth (or probable truth) from complete fiction.
Most research is presented to the public through media or propagandists with agendas. Since diet is most often hijacked for selling newspapers and ideologies, we"ll use almost almost-believable diet nonsense to develop our BS meter. To create the most perfect you, you need to know which science to follow and which "science" to ignore.
After reading the next eight pages, you will know more about research studies than the average MD.
The Big Five The Big Five are important to understand, as they are the tools most often used to exaggerate and brainwash.
They are also critical to grasp so you don"t trick yourself or waste time with false leads as a self-experimenter. I"ve phrased each concept as a question you should ask yourself when looking at diet advice or the "latest research."
1. IS A RELATIVE CHANGE (LIKE PERCENTAGES) BEING USED TO CONVINCE?.
This concept is best ill.u.s.trated with two potential news headlines.
"STUDIES S SHOW P PEOPLE W WHO A AVOID S SATURATED F FAT L LIVE L LONGER"
Should you start avoiding saturated fat?
First, find out exactly what "longer" means. Based on available data, it turns out that reducing your saturated fat intake to 10% of daily calories for your entire adult life would add only 330 days to your lifespan. Considering this, is the trouble worth it if a rib-eye is one of your pleasures in life? Probably not.
"PEOPLE W WHO D DRINK C COFFEE L LOSE 20% M 20% MORE F FAT T THAN T THOSE W WHO D DON"T"
Should you start drinking coffee?
Leaving aside the question of whether or not this is an observational study (discussed next), it"s worth looking at that mighty impressive 20%.
Relative increases or decreases, most often expressed as percentages, can be misleading. increases or decreases, most often expressed as percentages, can be misleading.
Relative isn"t enough. It"s critical to ask what the isn"t enough. It"s critical to ask what the absolute absolute increase or decrease was-in this case, how many pounds of fat did both groups actually lose, and over what period of time? In most cases, percentages are used in media and sales brochures to mask the fact that changes were minuscule. increase or decrease was-in this case, how many pounds of fat did both groups actually lose, and over what period of time? In most cases, percentages are used in media and sales brochures to mask the fact that changes were minuscule.
If it were 0.25 pounds lost for the control group and 0.30 pounds (20% more) for the coffee group over eight weeks at three cups per day, is picking up the coffee habit worth the side effects of high-dose caffeine? Nope.
Distrust percentages in isolation.
2. IS THIS AN OBSERVATIONAL STUDY CLAIMING TO SHOW CAUSE AND EFFECT?.
This is the mother lode. If you learn just one concept in this chapter, learn this one. It"s the cardinal sin.
Observational studies,3 also referred to as also referred to as uncontrolled uncontrolled experiments, look at different groups or populations outside the lab and compare the occurrence of specific phenomena, usually diseases. One example is the often misinterpreted "China study." experiments, look at different groups or populations outside the lab and compare the occurrence of specific phenomena, usually diseases. One example is the often misinterpreted "China study."
Here is the most important paragraph in this chapter: Observational studies cannot control or even doc.u.ment all of the variables involved. Observational studies can only show correlation: A and B both exist at the same time in one group. They cannot show cause and effect.4 In contrast, randomized and controlled experiments control variables and can therefore show cause and effect (causation): A causes B to happen.
The satirical religion Pastafarianism purposely confuses correlation and causation:
With a decrease in the number of pirates, there has been an increase in global warming over the same period.Therefore, global warming is caused by a lack of pirates.
Even more compelling:
Somalia has the highest number of Pirates AND the lowest Carbon emissions of any country. Coincidence?
Drawing unwarranted cause-and-effect conclusions from observational studies is the bread-and-b.u.t.ter of media and cause- or financially-driven scientists blind to their own lack of ethics.
Don"t fall for Pastafarianism in science.
It is critical not not to take advice based purely on observational studies. In 2004, a commentary published in the to take advice based purely on observational studies. In 2004, a commentary published in the International Journal of Epidemiology International Journal of Epidemiology t.i.tled "The hormone replacementcoronary heart disease conundrum: is this the death of observational epidemiology?" highlighted the dangers of doing so. Observation of one group of women using hormone replacement therapy (HRT) showed lower heart disease, and media and HRT proponents were fast to promote this sound-bite conclusion: t.i.tled "The hormone replacementcoronary heart disease conundrum: is this the death of observational epidemiology?" highlighted the dangers of doing so. Observation of one group of women using hormone replacement therapy (HRT) showed lower heart disease, and media and HRT proponents were fast to promote this sound-bite conclusion: HRT reduces heart disease! HRT reduces heart disease! Sadly, randomized and controlled trials (RCT) later showed no protective effects, and even a slight increase of risk, for heart disease among those using HRT. Sadly, randomized and controlled trials (RCT) later showed no protective effects, and even a slight increase of risk, for heart disease among those using HRT.
How was this possible?
It turns out that the observational studies didn"t sufficiently account for different socioeconomic statuses between groups, or for the influence of doctors selecting women for HRT who were less predisposed to heart disease to begin with. The latter is an example of how failing to randomly a.s.sign subjects to groups (randomization) leaves observational studies open to bias from experimenters.
The 2004 hindsight commentary stated, rightly:
The differing results between observational studies and RCT [randomized and controlled trials] in the a.s.sociation between HRT and CHD throw this idea [that well-conducted observational studies can produce similar estimates of treatment effects as RCT] into question and may signify the death of observational epidemiology.
Observational studies are valuable for developing hypotheses (educated guesses that can then be tested in controlled settings) but they cannot and should not be used to show cause and effect. To do so is both irresponsible and potentially dangerous.
3. DOES THIS STUDY DEPEND ON SELF-REPORTING OR SURVEYS?.
In 1980, scientists at an isolated research station in Antarctica had test subjects weigh and record all of the food they consumed. Once a week, the subjects were asked to recall what they had eaten the day before (food, keep in mind, they had weighed and recorded in their notebooks). Despite all the attention to logging meals, the men still underestimated their intake by 2030%.
Granted, these might have been blizzard-blind men. Hardly normal circ.u.mstances. Let"s look at how the real professionals do it.
The Women"s Health Initiative (WHI) was a ma.s.sive $415-million project conducted under the auspices of the National Inst.i.tutes of Health (NIH) and involving nearly 49,000 women. It was designed to investigate a number of health issues, including the effect of low-fat diets on cancer, over an eight-year period. It was a large-scale intervention study, often viewed by media as the gold standard in nutrition research. Dr. Michael Thun of the American Cancer a.s.sociation went so far as to call the WHI "the Rolls-Royce of studies."
The New York Times New York Times announced the results in 2006 with a headline that was crystal clear: announced the results in 2006 with a headline that was crystal clear: "LOW-FAT D DIET D DOES N NOT C CUT H HEALTH R RISKS, STUDY F FINDS"
Though I agree with the conclusion based on other data, the WHI study couldn"t possibly conclude this. Let"s look to Michael Pollan for a peek under the hood at one of the most common weaknesses of nutritional studies: self- reporting.
To try to fill out the food-frequency questionnaire used by the Women"s Health Initiative, as I recently did, is to realize just how shaky the data on which such trials rely really are.... It asked me to think back over the past three months to recall whether when I ate okra, squash or yams, were they fried, and if so, were they fried in stick margarine, tub margarine, b.u.t.ter, "shortening" (in which category they inexplicably lump together hydrogenated vegetable oil and lard), olive or canola oil or nonstick spray? I honestly didn"t remember, and in the case of any okra eaten in a restaurant, even a hypnotist could not get out of me what sort of fat it was fried in....This is the sort of data on which the largest questions of diet and health are being decided in America today.
Other WHI questions include:
When you ate chicken or turkey, how often did you eat the skin?Did you usually choose light meat, dark meat, or both?In the last three months, how many times have you eaten a half-cup serving of broccoli?5 How would you do? Think you can pull off recollection within 20% of reality?
Let"s test the last 24 hours.
Do this: estimate the number of calories you ate and drank yesterday, without using any references, as well as the fat calories you consumed. Then eat the same meals and snacks, but weigh it all in grams on a portable kitchen scale. Measure drinks using a measuring cup. Use www.nutritiondata.com to determine the 100-gram caloric value of each food and do the math. to determine the 100-gram caloric value of each food and do the math.6 Unless you are a professional athlete and can distinguish between a 150-gram serving of steak and a 200-gram serving, the difference between your self-reporting and your weighing a.n.a.lysis will shock you. Combine that misreporting across 49,000 people and you can imagine the rather Pica.s.so-like picture it creates.
Granted, self-reported data can be useful, especially when recording in real time (i.e., recording something as it happens). This has become easier with technology that takes memory out of the equation, such as DailyBurn"s FoodScanner iPhone application and the Withings wifi scale.7 To the greatest extent possible, avoid studies that depend on after-the-fact after-the-fact self-reporting. Trust your own data. Just record it when things happen. self-reporting. Trust your own data. Just record it when things happen.
4. IS THIS DIET STUDY CLAIMING A CONTROL GROUP?.
Desirable as it may be, it is almost impossible to change just one macronutrient variable (protein, carbohydrate, fat) in a diet study. It is therefore almost impossible to create a control group.
If a researcher makes such a claim and vilifies a single macronutrient, your skeptical spider sense should tingle.
Let"s suppose that someone claims that a study on the effects of a low-fat diet proves it"s healthier than a higher-fat diet. There is a control group.
First things first: if you decrease fat or cholesterol in a whole-food diet, you"ll be removing fat and and protein, which means you must add carbohydrate to make the diets equal in calories. If you don"t, you have unequal calories as another variable. If you make this caloric correction by adding carbohydrate elsewhere, now you face a conundrum: not only are you comparing a higher-fat and a lower-fat diet, but you"re simultaneously comparing a higher-protein and a lower-protein diet, as well as a higher-carb and a lower-carb diet. protein, which means you must add carbohydrate to make the diets equal in calories. If you don"t, you have unequal calories as another variable. If you make this caloric correction by adding carbohydrate elsewhere, now you face a conundrum: not only are you comparing a higher-fat and a lower-fat diet, but you"re simultaneously comparing a higher-protein and a lower-protein diet, as well as a higher-carb and a lower-carb diet.
How can we know which is responsible for what?
We can"t.
Self-experimentation actually offers an unexpected advantage here.
The "control" is everything you"ve tried up to a certain point that hasn"t produced a desired effect. Isolating one variable is often less important than the sum impact of a group of changes. In other words, has your your bodyfat percentage gone up or down in the last two weeks of replacing diet A with diet B? If you weren"t losing fat on A and now you are, A was your control. bodyfat percentage gone up or down in the last two weeks of replacing diet A with diet B? If you weren"t losing fat on A and now you are, A was your control.
In an ideal (but unattractive) test, you would go back to A and see if bodyfat then moves in the other direction. Then repeat the switch again. This would minimize the possibility that the first change in bodyfat just happened to coincide with the change in diet to B.
Alas, this switching would also maximize your likelihood of going insane. If something seems to be working, just stick with it.
5. DO THE FUNDERS OF THE STUDY HAVE A VESTED INTEREST IN A CERTAIN OUTCOME?.
Beware of unholy unions between scientists and funding sources.
Fred Stare, founder and chair of the Nutrition Department at Harvard University, obtained a $1,026,000 grant from General Foods in 1960. These manufacturers of the sugar-rich Post cereals, Kool-Aid, and the Tang breakfast drink would be subsidizing the "expansion of the School"s Nutrition Research Laboratories." In the decade that followed, Stare became the most public and reputable defender of sugar and modern food additives, all the while receiving funding from Coca-Cola and the National Soft Drinks a.s.sociation, among others. Does this prove malfeasance? No. Does it show that funding, support that can be discontinued, often comes from those most likely to benefit? Yes. People respond to incentives.
Simple due diligence: Peruse the required "conflicts of interest" section in studies or reviews related to diet and look for "consulting fees." If McCorporation or the XYZ Lobby decides it"s too obvious to fund studies directly, hiring researchers as consultants can be an indirect means to the same end. James Hill of the University of Colorado, as one example, is well known for attempting to discredit the relationship between obesity and higher insulin levels caused by sugar consumption. In his stated conflicts of interest, you will see that he has received consulting fees from Coca-Cola, Kraft Foods, and Mars Corporation (makers of Snickers, M&Ms, and Mars Bars). Peruse the required "conflicts of interest" section in studies or reviews related to diet and look for "consulting fees." If McCorporation or the XYZ Lobby decides it"s too obvious to fund studies directly, hiring researchers as consultants can be an indirect means to the same end. James Hill of the University of Colorado, as one example, is well known for attempting to discredit the relationship between obesity and higher insulin levels caused by sugar consumption. In his stated conflicts of interest, you will see that he has received consulting fees from Coca-Cola, Kraft Foods, and Mars Corporation (makers of Snickers, M&Ms, and Mars Bars).
Does this make him guilty of skewing data to serve corporate interests? No. But it should make you look closely at the studies themselves before accepting the headlines and making behavioral changes in your life.
The Goal of this Chapter vs. the Goal of this Book Understanding how to act under conditions of incomplete information is the highest and most urgent human pursuit.-Na.s.sim Taleb, The Black Swan The Black Swan Are the experiments in this book bulletproof? Far from it. All studies are flawed in some respect, often for legitimate cost or ethical considerations.
I use myself as a single subject, and (with a few exceptions) I neither randomize nor create a control. Some scientists will, no doubt, have a field day picking these self-experiments apart.
This doesn"t bother me, and it shouldn"t bother you.
The goal of this chapter is simple: since we often use published research as a starting point for self-experimentation, we want to ensure that we don"t take false leads from tricksters or misinformed journalists with good intentions. Understanding the Big Five questions and the hallmarks of sensationalism puts you in a rare group: those who can depend on themselves, not the media, for nutritional guidance.
This opens doors that we can then crowbar and leverage for incredible effect.
My goal for the book is not, first and foremost, to identify the single variables that produce target changes. That is often the goal of clinical research for publication, but experimentation for self-improvement is a different beast.
It may be that alpha-lipoic acid does nothing in a given fat-loss c.o.c.ktail; perhaps it"s the angle of an exercise and not the load that triggers muscular gain; or it could be that spinach has none of the effects I predict, but other foods in the prescribed meal do. The exact mechanism doesn"t much matter if we get the effects we want...without side effects.
Dr. Martin Luther King Jr. famously wrote that "justice too long delayed is justice denied." In the world of self-experimentation, where the outcomes are of personal importance, results too long delayed are results denied. This doesn"t mean being haphazard. It"s more than possible to tinker without hurting yourself. It means, however, that waiting for perfect conditions often means waiting forever.
In the world I live in, people want to lose fat or improve s.e.xual performance now, not in five or ten years.
Let the journals catch up later-you don"t have to wait.
[image]
[image]P-Value: One Number to Understand