A Genetic Testing Dupe?

The government says I am being misled by useless information about my genes. I disagree.

"Misleading and of little or no practical use to consumers" is the way that the Government Accountability Office (GAO) described the results of direct-to-consumer genetic screening tests in a report unveiled last week. To reach this damning conclusion, the GAO sent in genetic samples from five people for testing by four leading direct-to-consumer testing companies. For each donor the GAO sent two DNA samples, one sample using the person’s actual profile and one using a fictitious profile. Although the testing companies were not identified in the report, it’s pretty clear that they are 23andMe, deCodeMe, Navigenics, and Pathway Genomics.

However, being a customer of two of the four companies, I was puzzled by the GAO’s claim that I had been duped. I found the information obtained from my two genetic profiles neither misleading nor useless. (There is one criticism from the GAO that is right on, however. Most genetic studies so far have been done on ethnically European populations, and the testing companies have been remiss in failing to warn customers who are not ethnically European that many results may not apply to them.)

The GAO reached its conclusions, which were presented at a Capitol Hill hearing before the House of Representatives Subcommittee on Oversight and Investigations, by comparing the test results from the four companies for 15 common diseases and conditions: Alzheimer’s disease, atrial fibrillation (a type of irregular heart beat), breast cancer, celiac disease (a chronic digestive problem caused by an inability to process gluten), colon cancer, heart attack, hypertension, leukemia, multiple sclerosis, obesity, prostate cancer, restless leg syndrome, rheumatoid arthritis, type 1 diabetes, and type 2 diabetes. Not all four companies tested for all 15 disease risks and conditions. The GAO found that “each donor’s factual profile received disease risk predictions that varied across all four companies, indicating that identical DNA can yield contradictory results depending solely on the company it was sent to for analysis.” In addition, the risk predictions “often conflicted with the donors’ factual illnesses and family medical histories.”

To illustrate the “contradictory” risk predictions it found, the GAO reports that one 48-year-old male donor received conflicting information “that he was at below-average, average, and above average risk for prostate cancer and hypertension.” Another donor, a 61-year-old male who had had a pacemaker installed 13 years ago, was told that he was at below-average risk for an irregular heartbeat by two companies and at average risk by two others. Different results from the same DNA and results that diverge from the donors’ actual medical conditions—must be pure snake oil, right? Well, maybe not.

The GAO acknowledged that the “contradictory” results it uncovered “can be attributed in part to the fact that the companies analyzed different genetic ‘markers’ in assessing the donors’ risk for disease.” Generally, these “markers” consist of slight differences in genes called single nucleotide polymorphisms (SNPs). SNPs are relatively common genetic variations in which a single DNA base pair differs among individuals. Some SNPs affect the susceptibility of people to certain diseases or influence their response to certain drugs. That’s what the companies are testing for.

Different companies select the markers they think most relevant and the studies they believe are most accurate. For example, 23andMe offers customers results based on “established research” which is defined as being based on studies with a minimum sample size of 750 cases with appropriate controls and which has been independently replicated by at least one other study. The company also offers results based on less stringent “preliminary research” which is generally based on fewer cases and has not been replicated independently. Pathway Genomics reports results based on studies using 1,000 cases and 1,000 controls.

Many of the "contradictory" findings are actually artifacts of the agency's ranking of results as "below-average, average, or above-average." Comparing my 23andMe and Pathway results, one finds a continuum of information. Part of the "differential" in reported results (as opposed to "contradictions") occurs because the two companies test and report different alleles associated with each condition. In general, Pathway reports on more markers than 23andMe does.

That being said, scoring my personal results using the GAO's rather crude tripartite system I find that the two companies agree on 12 out of 20 disease risks they both report. Confining the results to just the ones compared by the GAO, the agreement comes to 8 out of 11 disease risks they both test. To get a sense of why results differ let’s compare the risk predictions for my chances of colorectal cancer, melanoma, and heart attack.

With regard to colorectal cancer, both 23andMe and Pathway Genomics find that my risk is about average. 23andMe tests for three different variants and calculates that 6.2 out of 100 men of European ethnicity who share my genome will get colorectal cancer between the ages of 15 and 79. The average risk for this population is 6.1 out of 100. Pathway Genomics screens for the same three gene variants plus seven others and calculates that my genetic profile is “typical” of the general population for colorectal cancer risk. However, I had a small polyp excised a few years back suggesting that I might be at a slightly higher risk for colorectal cancer. I plan to keep those colonoscopies coming.

The companies disagree about my risk for melanoma. Pathway Genomics reports that my genetic profile for the disease risk is typical, whereas 23andMe reports that my genetic data suggests that my risk is slightly below average at 2.2 out 100 versus 2.9 out of 100. Considering that at age 10 during my first visit ever to an ocean beach, I suffered a second degree sunburn over my shoulders and upper back, I’m not going to be too sanguine about my melanoma risk

Finally, the companies also disagree about my risk for a heart attack. 23andMe tests for only one variant which indicates that my risk of heart attack is slightly below average at 20.9 out 100 Caucasian males, whereas the average is 21.2 out 100. The GAO report worried that “a consumer with a strong family history may be falsely reassured by below-average risk predictions related to heart attacks and consequently make poor health choices.” Does the GAO really think that a difference of three-tenths of a percent in risk will encourage me switch to a diet of Double Bypass Burgers? Besides, Pathway Genomics tested for 11 variants associated with heart attacks and reports that I am “somewhat more susceptible” than the average to a heart attack. Given my family history—my father died of a heart attack at age 70 and my mother died of stroke brought on by atrial fibrillation at the same age—I’ll go with Pathway Genomics’ analysis. By the way, both 23andMe and Pathway find that I am at greater risk than average for experiencing atrial fibrillation. Note to self: Eat more green leafy vegetables and get to the gym!

Given personal and family history of polyps, severe sunburn, and heart disease, do the genetic tests add anything? Considering that I am likely to suffer heart disease, it’s a good thing to know that people with my “genetic markers receive significantly greater benefit from intensive statin therapy (such as atorvastatin—Lipitor) than people who do not have these markers,” according to Pathway Genomics. Pathway also reports that I am unlikely to have a bad reaction to taking statins. (I’ll be talking with my doctor about this soon.) And if I do survive a heart attack, I will definitely mention to my cardiologist that if she’s considering treatment with warfarin, my genetic tests suggest that I should start out with a lower dose of that blood thinner.

The differential tests results do not bother me, and I would be surprised if many gene testing pioneer customers find the information they receive all that confusing. The results are probabilistic calculations based on a selection of low risk susceptibility alleles. The right way to think about the current direct-to-consumer genotype screening tests is that they are a preliminary technology. They offer supplementary, not dispositive information about various health risks. The tests are not perfect, but they are the beginning of the process through which consumers, physicians, and purveyors will learn how to better interpret and use genetic information over time.

We are in the Apple II era of genetic testing. It would have been silly to ban the Apple II just because it was not as easy to use or immediately comprehensible as the MacBook Air. Standardization of test results will come as more information about the interaction between genetic variants and environmental influences accumulates. The current tests function as training wheels for curious consumers who will be using the whole genome and epigenetic screening tests that will be widely and cheaply available by the end of this decade. As one of those curious consumers, I don't want or need federal regulators to protect me from my own test results.

Ronald Bailey is Reason's science correspondent. His book Liberation Biology: The Scientific and Moral Case for the Biotech Revolution is now available from Prometheus Books.

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  • Fist of Etiquette||

    Without this testing, how will I know if I have the caveat emptor gene?

  • SLO||

    Thanks for a writeup on this. I saw the GAO report and hoped you would put it in perspective.

  • Subsidize Me!||

    After you become a customer, do these companies continue to monitor your major health issues and/or cause of death? Or are they reliant solely on studies (ahem... government-funded studies) in order to accumulate data? I'm wondering if these companies aren't paying off the right people.

    I plan to keep those colonoscopies coming.

    They plan to keep you... oh, nevermind.

  • ||

    SM! 23andMe periodically updates information based on new studies. In addition, customers like me (with informed consent, of course) answer various surveys which the company plans to use tease out other gene/phenotype/environment correlations. 23andMe just published its first study in PLOS Genetics based on this model of research.

  • Subsidize Me!||

    Wow. It looks like over the years they'll get a ton of data to mine if their customers are willing to continue participation... and the positive externality is that the general public benefits if the company publishes their studies - I'm not surprised the government is looking to curtail this.

  • ||

    Yes, 23andMe uses your health and trait information to build their data base and keeps members informed of new genetic information. It is true that this science is in it's infancy, but that does not make it useless.

  • Binky||

    "Misleading and of little or no practical use to consumers"

    Unlike "Amount/Serving" nutrition info, "Unemployment" figures, ....

  • DDavis||

    How low can we go for health freedom?

    License doctors, so that we can't be treated by who we want. Restrict access to medical equipment to licensed doctors. Restrict access to medical procedures and medications without permission of government deputized doctors. Restrict free speech rights of medical companies. Distort the market, diverting our health dollars into insurance instead of care, then let the government regulated insurance lackeys restrict what care we can have.

    Soon, we won't be able to spit in a tube and have a company tell us what our dna is. What next, make bathroom scales illegal?

    I'm just happy I've at least got some of my dna tested and my raw data downloaded through 23andme before the FDA decides that it's illegal for me to know true facts about myself.

    I'd recommend everyone do the same. Get your raw snp data before the govt makes it illegal to do so. Yes, the health predictions aren't currently so accurate, but the snp data is, and as long as you have it, you'll be able to take advantage of it as the health predictions get more accurate over time.

  • Chad||

    No, you aren't being misled, Ron. But you have to remember that you are a 99th percenter. Most people aren't like you. They are, in fact, pretty much mathematically inept.

    These types of figures ARE misleading to the vast majority of people, just like credit card contracts and mortgages. And of course, the libertarian "solution" to this is to let the strong prey on the weak.

  • ||

    Chad: And the authoritarian "solution" is to treat everybody as though they are "weak" and control everything for "their own good."

  • Chad||

    Likely not. But there is a vast middle ground between doing absolutely nothing and banning something.

  • Contrarian P||

    With that logic, I should never tell my patients that they are at risk for anything until after they've had it. All these companies are doing is giving you a risk profile. Even if the odds are against you getting a disease genetically, you can still get it. Though the odds are against people winning money in Vegas, they still do. Should we ban gambling in Vegas to protect people who can't do the math that slot machines will take their money? How exactly is a company screening your DNA for relative risk, possibly encouraging you to make healthier choices or get those screening tests everyone seems to be talking about a bad idea? How is that the strong preying on the weak? Seems to me that you're placing yourself in the Godlike position of deciding that the rest of us are too dumb to understand something. Why not wait for a case where there has actually been harm caused by these tests?

  • Chad||

    How exactly is a company screening your DNA for relative risk, possibly encouraging you to make healthier choices or get those screening tests everyone seems to be talking about a bad idea

    It depends on your ratio of false positives to real ones. If there are thirty people sent into a tizzy and through a gauntlet of invasive tests fruitlessly chasing a false positive for each person who actually finds something useful, you could easily wind up in a net negative situation.

    And to pretend the companies offering these tests aren't finding ways to sell you things based on the results, and exploit the false positives....well, you may as well believe in tooth fairies.

  • Subsidize Me!||

    It depends on your ratio of false positives to real ones. If there are thirty people sent into a tizzy and through a gauntlet of invasive tests fruitlessly chasing a false positive for each person who actually finds something useful, you could easily wind up in a net negative situation.

    So the one guy who gets a colonoscopy earlier than usual, finds cancer, and starts treatment early based on a genetic test that he voluntarily paid for somehow evokes the ire of the utilitarian health police (and do-gooders like you) who are upset that another 30 people got negative test results after voluntarily purchasing a genetic screening: this is definitely a negative outcome that must be corrected through oversight and regulation.

    Is this how nationalized healthcare will work? Will the government save money by only testing people that have a near-certainty of sickness?

  • Contrarian P||

    I don't think there's been any kind of longitudinal study done on the results of the genetic testing at issue here to generate a false positive to false negative ratio. There really is no research as this is a brand new field. What you're suggesting would require decades to measure and frankly if a consenting adult wishes to get a genetic test, who are you to say he or she can't because you believe it will generate too many false positives? There are definite genetic markers for certain conditions (breast cancer is a particular case) making it much more likely that you will develop that disease and there will undoubtedly be more such indicators found in the future. As a physician, I do not have to order a test that my patient demands, but if they are willing to pay for it out of their own pockets and understand the potential risks involved, plus have a report which raises a reasonable question that a test might answer, I believe they should have the ability to choose for themselves.

    I think your main problem with this is twofold. One, under the health care system you'd apparently like, the rest of us would get stuck with the tab for elective testing, which I agree is a bad plan, but would not be much of an issue if patients could just pay for the testing they desired. The second is a bit more insidious...your belief that between my patient and me, we can't determine by research and evaluation what tests are worthwhile to order, your belief that only some enlightened D.C. politico can make that determination. I just find that attitude repugnant, especially given the wonderful things that Washington has forced on the medical profession over the years that do nothing for patients other than drive up the cost of their bill.

  • DDavis||

    ### All these companies are doing is giving you a risk profile.

    They are giving you your actual SNP data as well, and that is really where they are helpful.

  • Subsidize Me!||

    Nothing like a little ego massage...

    No, you aren't being misled, Ron. But you have to remember that you are a 99th percenter. Most people aren't like you. They are, in fact, pretty much mathematically inept.

    ...to make the belligerent stupidity sound less objectionable.

    These types of figures ARE misleading to the vast majority of people, just like credit card contracts and mortgages. And of course, the libertarian "solution" to this is to let the strong prey on the weak.

    I hope someday that your political base realizes the contempt you pseudo-smart elitists hold for them. Doesn't the bell toll for you too, Chad?

  • ||

    To people incapable or unwilling to handle probabilities*, announcing these figures are 'misleading' is misleading, too.

    * not constituting a 'vast' majority anywhere, and probably not constituting any majority in any developed country.

    People are actually pretty smart - I've been able to explain, for example, the concept of NPV to a drunk hobo. :) (no, seriously, happened once.)

    I'm inclined to think that by saying financial contracts are complex and people are 'mathematically inept', you're simply giving people an excuse.

  • ||

    Yes, that has always been my policy....eat the weak!

  • ||

    Yes, that has always been my policy....eat the weak!

  • ||

    So you think the customer base of these companies is likely to be people who are too stupid to understand the results and can't afford the tests?

    And to combat this 'risk' you surely want to regulate these industries.

    Stop your damned statist impulse to control other peoples' access to information.

    Let people buy reports they want to buy. People buy and follow zodiac reports which are completely worthless and misleading. People buy and read the NY Times which is completely worthless and misleading. Want to ban them too?

  • ||

    The GAO is correct. Most of these genetic profile tests are worse than worthless. At present, the only reliable genetic tests are those where one or a few genes are responsible for a specific disease or condition. Conditions such as atherosclerosis, heart arrhythmias, diabetes mellitus, most cancers, Alzheimer's disease, etc. have both genetic and environmental risk factors, and the genetic risk factors involve many genes. Assessing risks for those conditions based only upon a few gene assays is essentially impossible with the knowledge we have today. Sorry, Mr. Bailey, but you wasted your money.

    -- A clinical pathologist

  • ||

    Dr. T: Just curious - have you had one of these genotype screening tests?

  • ||

    And we've learned about all there is to know about genetic risk factors, so people who have their profile on file won't ever be able to get better analysis than we have right . . . this. . . minute.

  • ||

    Dr. T: Just one additional comment -- I rarely find expenditures made on learning new things a waste of my money.

  • Tony||

    Learning that a snake-oil salesman thinks his snake oil is great isn't worth that much. I agree with your sentiment, but I worry about an immediate connection between profit and genetic testing, especially in a for-profit healthcare paradigm.

  • ||

    Tony,
    You should worry about being a fucking idiot....but it won't change the fact!

  • Jason||

    but I worry about an immediate connection between profit and genetic testing, especially in a for-profit healthcare paradigm.

    What, you want rents instead?

  • Neu Mejican||

    The information from these test can be BOTH "misleading and of little or no practical use to consumers" and interesting enough that Ron didn't waste his money on them.

    I must admit to still being confused why Ron feels being able to pay money for this interesting information about his own genes is a good thing, but GAO report information (also allowing people to learn new things) is a bad thing. Ron can learn about his genes and learn that most of the information these companies provide has no practical medical value. It's win win.

  • DDavis||

    ## Sorry, Mr. Bailey, but you wasted your money.-- A clinical pathologist

    Ah yes, another ignorant doctor with delusions of godhood. If you can't figure out how to use the data, it must be worthless, right? How could knowing what's in your dna actually be helpful? What does dna have to do with how your body functions? The FDA says the information isn't helpful, so it must not be.

    If we're tossing credentials around, I got a Ph.D. in EE specializing in statistical inference in causal systems, something of which doctors are worse than ignorant. To paraphrase Ronnie, it's not that doctor's are ignorant, it's that they know so many things which aren't so.

    If it were only that doctors didn't know they elbows from their asses but had delusions of godhood, I wouldn't mind it so much, but they presume to impose their ignorance on me and control my access to medicine by force as deputized government medical agents.

    You run along and keep destroying lives with your primitive statistical methods and decision theory; the grownups are busy trying to do something useful with the little medical knowledge and freedom that hasn't been banned.

  • Neu Mejican||

    I am curious how this EE degree of yours indicates you know something of the practical medical value of these genetic tests. Granted, the skills you learned could be applied to the problem of genetic causality of certain conditions, but that really doesn't speak to the practical value of the information right now, which is what the GAO investigated.

  • DDavis||

    As a matter of principle, I reject the idea that knowledge of your dna sequences is "worse than worthless" and "of no practical use". This actually relates to some foolishness in the machine learning community, where some people mistook the supposed "curse of dimensionality" as recommending that you throw away data before making estimates.

    Different companies using different information come to different risk estimates. Wow. Who woulda thunk it? That GAO bozos could have presented this as if it were a smoking gun of corruption tells me all I need to know about their statistical competence - or honesty.

    I only pointed out my degree as a response to Dr. Dictator's attempt at an argument from authority.

    You ask how my experience in statistical and causal inference relates to decision making in medicine. How should I answer that, refer you to a dictionary?

    Maybe I'll just point out your own mistake in decision theory, in assuming that what I can do "right now" with the information determines what value it has to me today.

    By the way, you do realize that a lot of people do this for genealogy, not health, right? I guess "right now", genealogy has no value, according to you and the GAO.

  • Neu Mejican||

    You ask how my experience in statistical and causal inference relates to decision making in medicine. How should I answer that, refer you to a dictionary?

    You misunderstand my point/question?. I did not ask how your general expertise in the abstract relates to decision making in medicine in the abstract. IMHO, the issue is not (primarily) the decision process, at the moment, it is the knowledge and information that goes into that process. The current problem from a clinical standpoint is that there is no practical change to treatment or lifestyle that can be implemented as a result of the vast majority of these tests. Clinical utility for a medical test is heavily contingent upon the information leading to a change in treatment or outcome.

    Maybe I'll just point out your own mistake in decision theory, in assuming that what I can do "right now" with the information determines what value it has to me today. By the way, you do realize that a lot of people do this for genealogy, not health, right? I guess "right now", genealogy has no value, according to you and the GAO.

    How am I to respond to your conflation of "value today" and "practical value"? Shall I refer you to a dictionary?;^)

    Different companies using different information come to different risk estimates. Wow. Who woulda thunk it? That GAO bozos could have presented this as if it were a smoking gun of corruption tells me all I need to know about their statistical competence - or honesty.

    If you inferred from the report that this particular piece of evidence was being used to argue for a charge of corruption then I think your Ph.D. in statistical inference did not cover inferencing in a plain language context. I think I'll give you the benefit of the doubt and assume that you didn't read the report.

  • ||

    It's his money to 'waste'.

    "have both genetic and environmental risk factors"

    WTF? Do you think he's as stupid as you?

    He clearly understands the difference between genetic and environmental risk. IE the melanoma risk vs Hx of sunburn.

  • Ben||

    They are just trying to corner the market. It is a business now... A really shitaly run business.

  • ||

    This is an excellent article. Note how they (drug companies and alike) attack the other companies through slander and nonsensical "we have to protect you from knowing about yourself" bull? They cannot deny the actual genetic content. You either have a mutation in certain locations of your genes/alleles or you don't. Interpretation of your risk for lets say, heart attack, will vary by companies because as you have stated we are in the apple II phase of genetics. Doesn't mean we shut it down, it means we develop it until we get to macbook pro. there are over 550,000 SNPs with over 55,000 genes that can differ. But with the large amounts of research out there already, we can know things like for example; an MTHFR mutation (tested for women who are pregnant especially) homozygous means you have a high tendency (if not FULL tendency) for higher homocysteine which means higher heart risks. And you cannot use folic acid but need active methylfolate. This is crucial information that can help develop the fetus properly and help people in preventative medicine. Unfortunately, preventative med and nutrition doesn't give drug companies MONEY.

  • Julie||

    What's been missed from this discussion is the regulatory encroachment that happened in DC last week. The FDA declared that they were going to expand their regulatory power to all laboratory testing, including everything from a cholesterol test to whole-genome sequencing. If they get their way, all new clinical innovations in genetics will need to go through pre-market approval. Forget Liberation Biology and the Biotech Revolution -- the FDA declared it illegal last week.

    What will really get your blood boiling is that anyone who's spoken out against this move has been told they must be a "libertarian".

  • ArticlesThatRock||

    Berkeley just followed the lead of the FDA and refuses to allow 1000 students who voluntarily offered their DNA to see the results.

    http://www.insidehighered.com/news/2010/08/13/dna

  • amazon ghd||

    I love your website!

  • blancpain replica watches ||

    i love your website

  • ugg nikeoutlet||

    Dell Laptop Batteries
    tiffany and company store
    swiss replica watches
    nikeoutlet
    ugg factory outlet

  • Scarpe Nike||

    is good

  • ||

    Superior article. I intention certainly share this article with my friends. www.itunes.com/download

GET REASON MAGAZINE

Get Reason's print or digital edition before it’s posted online

  • Progressive Puritans: From e-cigs to sex classifieds, the once transgressive left wants to criminalize fun.
  • Port Authoritarians: Chris Christie’s Bridgegate scandal
  • The Menace of Secret Government: Obama’s proposed intelligence reforms don’t safeguard civil liberties

SUBSCRIBE

advertisement