Rater Hater

Portable People Meter, what have you done? The House Committee for Oversight and Government Reform says the ratings provider Arbitron's new audience-rating technology is hurting minority broadcasters, and has handed out a fishing-trip subpoena to the terrestrial radio industry's ratings accreditation body.

Don't mourn the body, or the industry. The real question is why controversy (from many state attorneys general and both houses of Congress) continues to swirl around the portable people meter (PPM), a metered wearable device that tracks your actual radio-listening habits (through inaudible codes emitted by broadcasters) rather than your claimed radio-listening habits (as was the case with the previous system, a diary you had to turn in once a week). Here's background by me.

Oversight Committee Chairman Edolphus Towns (D-N.Y.) demands thousands of documents related to Arbitron's PPM implementation from the Media Rating Council (MRC), an industry-funded organization that accredits ratings service providers. The charges:

•On multiple occasions, MRC refused to grant accreditation to PPM for use in all markets across the United States except for Houston and Riverside/San Bernardino. MRC denied Arbitron accreditation because of the company's continual failure meet MRC minimum accreditation standards.

•MRC found "persistent problems" with Arbitron's minority sample audiences across the country. For example, New York City 2008 census data indicates African Americans comprised 25 percent and Hispanics comprised 27 percent of the City's population. Meanwhile, the subpoenaed documents show that Arbitron's New York City sample audiences comprised of only 17.7 percent African-American and 21.5 percent Hispanic participants.

•The documents also show that Arbitron's radio ratings almost consistently are based on data they receive from an unacceptably low percentage of their sample audiences. For example, in New York, where there is an average of 5400 sample audience participants, Arbitron uses only the data submitted by 2700 persons or 50% of the sample audience in order to create radio station ratings. Therefore, the radio listening habits of over four million ethnic minorities are represented by only 500 Arbitron recruits. The sample audiences are simply an inadequate representation of the true listening habits of New York's diverse landscape.

So is technology racist? There is a claim here about the sample that should be easy to figure out. But what's the claim against the technology itself? If Arbitron gets a sample that looks like America, or New York, or whatever unit it needs the sample to look like, then PPM just makes a more accurate measure of that sample.

There is one partly credible argument that PPM might skew against minorities in a way the diaries did not: People from different demographics are not equally willing to wear a passive metering device all day.

That split, however, occurs mostly around age and gender: Fighting-age males are less likely to comply with the portable people meter's longterm demands. Women over 35 are most likely to comply, which may help explain the ubiquitous cougar theme that all sampling-based media seem to think America wants to see these days.

Raters have a long policy of providing extra incentives for unlikely-to-comply groups. Audience research companies target compensation packages by age, ethnicity and gender in order to get more generous samplings of particular audience fragments. Towns' statistics are anecdotal evidence that Arbitron's efforts along these lines have failed. (It's notable that most of Towns' case seems to be built out of the MRC's own due diligence with Arbitron—a strong indicator that to whatever degree the system is supposed to work, it's working.)

But maybe there will be strong evidence of Arbitron's negligence in the documents the Committee is ordering MRC to cough up. (Many believe so.)

Or maybe the problem is that people just aren't listening to as much radio as broadcasters and regulators previously believed. The entire medium saw a decline in overall listeners after the switch to PPM-based ratings. Some stations have taken hits that far exceeded the industry average. (And some have benefited, such as Los Angeles' excellent classical station KUSC, which was named the top-ranked public station in the country when PPM measures kicked in.)

What I don't understand—and after a long discussion with a friendly Committee staffer, I understand it even less—is whether Towns is concerned about a new problem or an old one. If his objection is that Arbitron isn't getting the right people in its sample, that's a problem you would have had under the diary system too. And if there is a problem with the PPM technology, well, nobody has been able to explain what that problem is or could be. Undoubtedly Towns has received "many complaints from minority broadcasters."  Less clear is whether they're mad because their PPM ratings are inaccurate, or just because they're lower.