Helmets: Are bad stats leading to bad bills?

California state senator Carol Liu introduced an all-ages bicycle helmet bill yesterday, much to the dismay of many people who know a thing or two about helmet legislation. The early betting has the bill dying quickly in committee or elsewhere, although in a sense it might be productive to have a full and lively hearing about the merits of bicycle helmets. Some of the dismal, even disastrous, examples from places like Australia and New Zealand — where helmet legislation has apparently had no safety effect while also reducing cycling significantly — might kill these ideas for years to come.

Senator Liu’s press release included this statistical “gem” taken from a report by the National Conference of State Legislatures (NCSL): “Ninety-one percent of bicyclists killed in 2009 reportedly were not wearing helmets.” Since she mentioned it, Senator Liu is apparently relying in some part on this number to make her case. Where does it come from? Does it hold up to scrutiny? Does it reflect the California experience?

The underlying NCSL report doesn’t mention sources, but it’s a pretty safe bet that its numbers are coming from FARS, the federal Fatality Analysis Reporting System. FARS, in turn, gets its data directly from state highway data collection systems, which are implemented at the local law enforcement level. In California, the Statewide Integrated Traffic Records System (SWITRS) plays that role, and its data is open to all comers — including me, as it happens.

Over the past few weeks, I’ve been toying with SWITRS data, trying to understand its structure, and looking into what it might reveal about our roads. I’ve only gotten my feet wet with it so far, and I’ve much to learn, but from what I’ve seen, Liu’s “ninety-one percent” number is problematic. Take the year 2009, the year she singles out for mention. FARS shows 628 total cyclists killed in the U.S. that year. SWITRS tells me that 107 California cyclists were killed in the same year; presumably, those 107 are included in the 628 total. How many of those killed California cyclists were wearing helmets? SWITRS tells me there were twenty-two cyclists wearing helmets, sixty-seven who weren’t, and eighteen unknowns. In other words, in California sixty-three percent didn’t wear helmets — not ninety-one percent, as the nationwide number suggests — twenty-one percent wore them, and for seventeen percent of cycling fatalities we have no data.

My point might be small, but it should be in the mix nonetheless: Senator Liu implies that only eight percent of killed cyclists are wearing helmets, but in California that’s just not the case. More than double the cyclists are wearing them, and even more might be, if we knew more about those many unknowns. The hard facts? Lots of Californian cyclists are being killed despite their helmets, just as other studies show. We need to spend time confronting the real factors, like bad road design, high speed limits, and poor cycling infrastructure if we expect to make a real dent in fatalities.

Where does FARS get that huge ninety-one percent number? I’m not sure yet, but it sure looks suspiciously like they are grouping the unknowns in with the nos. That is a big no-no!

Just for completeness, I went back and extracted the numbers for all the years for which we have full SWITRS data (2001-2012):

Year	No Helmet	Helmet	Unknown			No helmet	Helmet	Unknown
2001	76	17	23	116		66%	15%	20%
2002	78	22	15	115		68%	19%	13%
2003	81	27	18	126		64%	21%	14%
2004	74	27	22	123		60%	22%	18%
2005	75	20	37	132		57%	15%	28%
2006	100	34	21	155		65%	22%	14%
2007	76	25	23	124		61%	20%	19%
2008	73	30	27	130		56%	23%	21%
2009	67	22	18	107		63%	21%	17%
2010	67	24	20	111		60%	22%	18%
2011	81	33	26	140		58%	24%	19%
2012	77	49	20	146		53%	34%	14%

As far as I can see, never in the past twelve years of data has any year approached Senator Liu’s number. Whatever comes of Senator Liu’s legistration, we should make sure that incomplete data don’t lead to bad statistics used to justify bad policy enshrined in unhelpful laws.