Misleading Stats From Heritage
US thinktank furthers a political agenda
(Updated June 24th 2022 with a response from Heritage, see below)
In the last few days, the following report from The Heritage Foundation has been released, purporting to link US states which have laws permitting paediatric transition with an increase in suicide.
While some will reflexively seize on this report as proof of the harms of child transition this report is deeply flawed and misleading, and in my opinion should not be taken seriously. More than that though, it does highlight the issue of those exploiting wedge issues to further a wider political agenda.
Here, I will go through some of the key parts of the methodology that I believe undermine its conclusions.
The thrust of the report is: US states which have laws permitting child transition, have a higher rate of suicide in the age group 12-23.
To establish this, it divides US states into two groups: those with laws allowing children to transition without parental oversight, and those without. Then, it uses state-level suicide data from the CDC, and compares the suicide rates in these two groups of states, finding that the group of states which allow child transition without parental oversight have both a higher rate and an increasing rate of suicide.
However, upon reading the report, it is clear that these two groups don’t actually represent states with and without laws permitting paediatric transition.
Because they could not establish which states do allow ready access to medical transition for minors, what they actually do is group states where any routine medical care is conditionally available to minors without parental consent, and assume this to be the same thing.
There is also a lack of precise information on where cross-sex medical interventions are more readily available to adolescents. A reasonable proxy for that data is to identify the states that have a legal provision allowing minors to access routine health care without the consent of their parents or guardian, at least under some circumstances. In states that have those provisions, puberty blockers and cross-sex hormones should be more easily available to teenagers.
This is a much wider criteria that seems it would cover laws allowing minors to access birth control, or for emancipated minors to consent to surgery, which bears no relation to ready access to cross-sex hormones. They would need information on where such cross-sex interventions are available in order to prove if this assumption was correct - but if they had that information, they wouldn’t need the proxy, so it is just a huge assumption, and once you realise it includes conservative states like Alabama it seems a highly dubious basis for the report.
Correlation is not Causation
Having divided the states into these two groups (which are not straightforwardly those which do or do not permit child transition, as claimed), and plotting the difference in suicide rates in the age group 12-23, the report observes that suicide is rising faster in one group than the other. It then asserts that the increase in suicide is due to the claimed difference in legal provision for child transition.
This is a completely unfounded leap that ignores every other possible reason for different suicide rates in the two groups of states in the last ten years, from economic deprivation, to depression over a Trump presidency, to COVID response.
Nor is there any way to link the actual reasons for suicide to the issue of transition, or homophobia, or access to healthcare, or anything.
All suicides, for any reason, are included and blamed on access to paediatric transition.
This should be obviously absurd to anyone, and is illustrated next by examining just one state from the “child transitioning” group.
To check the workings, I downloaded the suicide data from the same source as the report and crudely replicated some of their findings, eg. this is their chart 2:
(A side point, but note that the y-axis on this chart starts at 1, not 0. To the casual observer this manages to gives the visual impression of a higher rise than there is.)
Dividing the states into the two groups as described in the report, with the last ten years of data I was able to replicate the final 8 years of this curve (since it is a 3-year rolling average).
But on looking at the data, one state stands out above all: Alaska.
The suicide rate of this age group in Alaska is far higher than any other state and rising - and this has absolutely nothing to do with paediatric transition, but is down to the shocking prevalence of suicide in Native American men and boys in that state.
During 2019, suicide was the leading overall cause of death for Alaska youths and young adults aged 15 to 24
Replotting this graph, I found that the curve is entirely sensitive to which group of states Alaska is in. If you swap Alaska to the other group, the rise almost entirely disappears. If you move the next-highest state (Wyoming) over, the trend vanishes entirely.
Wyoming has the highest overall suicide rate in the country across all age groups. That this finding is so sensitive to just one or two states, and when the factors behind suicide in those states have nothing to do with laws on child transition, it is being generous to say these findings are not robust.
Looking at the suicide data, another thing that leaps out is that the group most likely to commit suicide - and the group with the largest increase - are males aged 18-23. However the group with the largest increase in trans-identification is younger teenage girls. This again doesn’t add up. If the report is attempting to link suicide rates with juvenile transition, we would expect the demographic most likely to be affected to be most represented in the suicide stats, but the opposite is true. This lack of connection again points to the overall weakness of the report.
There is more analysis in the report, but I won’t go any further because I believe these points already fatally undermine its overconfident conclusions.
However, I think it is important to raise the issue that partisan reports like this are where we see the strategic aim of some sections of the US conservative right to use “trans” as a wedge issue to further a wider political ideology. The Heritage Foundation are a right-wing US think tank who have for decades played an active role in denying climate change and advancing policies in opposition to sex education, birth control and abortion. That such organisations now seek to gain popular support and credibility by exploiting genuine dissent over issues of sex and gender is a serious problem in the current highly polarised political landscape.
At the end of the report you can see a hint of this wider purpose, which is to push their agenda of limiting access to healthcare to minors without parental consent:
This research adds to the well-established wisdom that children are better off if they are not allowed to make major life decisions without their parents’ involvement and permission. In general, parents are better positioned than anyone else, including the children themselves, to understand the needs of their children when making important decisions.
Given Heritage’s politics, it is no great stretch to say that this means birth control and abortion.
A major problem right now - especially in the US - is that the political landscape is so polarised that the mainstream “left” (such as it is) simply cannot talk rationally about this issue, out of fear of enabling “the right”.
And to an extent they do have a point - this report from Heritage is exactly the sort of thing they have feared. The problem is, having cried wolf so many times, people are more inclined to take things like this at face value, be fooled by it, share it, and ignore criticism of its weaknesses, than listen to the aggressive thought police who have been bullying dissenters for years.
The tactic of falsely smearing anybody as a bigot for raising even the slightest question about sex and gender politics, sex-based rights and the harms of medicalising children, has played right into the hands of those seeking to create division. Repeatedly calling completely reasonable, sincere contributions to this debate “hate” has rendered such claims near-worthless. So now, many people who should be natural allies are instead at each others’ throats because some vocal activists think anything other than total capitulation to their worldview is bigotry, and they cannot countenance the existence of any other legitimate, progressive standpoint.
When you are in a position where people who should be on your side are instead inclined to share dubious reports from Heritage, or where Matt Walsh can effortlessly make you look stupid, then your “activism” has gone very very wrong.
After being shown this substack, Jay P. Greene published this response to the issue of sensitivity to Alaska. The argument he makes is that, if Alaska is an outlier, it should be removed from the analysis, not swapped to the opposing group of states, and if that is done, the differential is still significant - indeed that the differential is not sensitive to the dropping of any single state.
I would argue that it all depends on whether you consider Alaska to be an outlier in this way, or a counterfactual, ie, evidence that the selection of state groupings in the first place is weak. Should you delete inconvenient data that doesn’t fit the hypothesis, or should it stack up on the other side and count against the trend?
The core claim is that suicides in under-24s are increasing in states that have minor access provision for healthcare and that therefore this access is the cause. One thing to remember is that - because of the difficulty of determining what access is available - the selection criteria are a proxy for access to paediatric medical transition. As such, the whole report is based on minor access to healthcare generally on the assumption that this translates to minor access to puberty blockers. But how sound is that?
According to the source used by Heritage, Alaska is included because according to statue there:
A minor living apart from his or her parents and who is managing his or her own financial affairs, regardless of the source or extent of income, may give consent for medical or dental services.
Does that sound like a reasonable proxy for access to puberty blockers? By contrast, in Alaska, Planned Parenthood do not prescribe puberty blockers, and the state only started covering medical transition under Medicaid in 2021. Are these indications that access to minor transition is not so straightforward as implied, and does this state actually belong in the alternate group rather than simply discarded as an awkward outlier? Are there, in fact, degrees of access, and the simple binary legislative proxy is inadequate?
And in any event if the cause of the rise is shown to be unrelated to access to puberty blockers in one state, why not others?
As an exercise, I combined the per-capita mental health spending of each state from here with the 2010-2020 data and split the states not by minor access provision, but by average mental health spend. The 17 states with above average per-capita spend were:
Alaska, Arizona, California, Connecticut, Hawaii, Iowa, Maine, Maryland, Minnesota, Montana, New Hampshire, New Jersey, New Mexico, New York, Oregon, Pennsylvania and Vermont
Performing the same simple plot on the raw data splitting the states on these grounds instead produces a differential trend that looks as follows, ie, suicides in this age group are falling in states with above average spend relative to states with below average spend.
Again - this is far too crude to be a serious answer, but the point is that there are so many factors affecting these numbers, from social media access, political landscape, fallout from the 2008 economic crash, the Trump presidency, education, peer support, spending on mental health etc. that to boil it all down to just one crude legislative marker in this way is not even remotely a robust finding. It is also possibly worth noting that half of the top 20 states with the fastest rate of increase in youth suicide in the last 10 years are also in the top 20 states with lowest mental health spend per capita (Alabama, Florida, Georgia, Illinois, Indiana, Louisiana, Oklahoma, South Carolina, Texas, Utah, Virginia).
And if - as with Alaska - the relationship between youth suicide is not to minor access to healthcare, but instead to some combination of all these other factors like mental health spend, do we simply drop those inconvenient outlier states from the analysis too?
How many states can we identify an alternative cause of the rise in suicide for and exclude them from the analysis, before the trend disappears entirely? It becomes a bit circular - minor access provision is the sole factor responsible for the rise in youth suicides except in the cases where it isn’t.
I redid the comparison two more ways with the raw data. The blue line is the original from figure 2. The red line is if we drop Alaska and the 5 states with the lowest per-capita spend on mental health from. The yellow line is, rather than using all states in each group, taking a random selection of 10 from each.
Where states with plausible alternative hypotheses exist are removed, the trend difference goes down, and when looking at a randomly selected, equally sized subset of each group, the trend is reversed. Selecting a different random subset would no doubt produce a different slope, all of which makes it pretty clear to me that this is not a robust analysis, and that the claimed legislative difference does not capture the cause of state differences.
Indiana, for example, is one of the states flagged by Heritage as having minor access provision, but according to this survey, it is also the state with the highest number of Snapchat connections - is that a factor? It hasn’t been ruled out, and Snapchat usage is highest in this age demographic, so would plausibly account for any difference to baseline adult suicide rates.
If you were positing a social contagion hypothesis, it could be argued that a contributing factor might even be the combination of no state provision for medical transition, and high exposure to claims on social media that medical transition is the only alternative to suicide. Given that Instagram launched in 2010, and Facebook knows that Instagram destroys teen mental health, how much of Tennessee’s rise is down to having the highest Instagram usage?
If any of this is a factor, the somewhat arbitrary selection and averaging of states and focus on a singular legislative cause will obscure more than it reveals.
Update 2: 24/06/2022
As of now, in a monstrous setback to women’s rights, Roe vs Wade has been repealed, and I’m sure Heritage are absolutely giddy with excitement.
For no particular reason, below are the states where abortion is now illegal in the US.
Note that every single one is below the average per-capita spend on mental health of $132. Seven of them are in the bottom ten nationally for expenditure.
(*) denotes a state that Heritage claims access to puberty blockers is the reason for any rise in youth suicide since 2010.
*North Dakota ($94.23)
South Dakota ($86.77)