In 1888, a social reformer named Oscar McCulloch delivered a speech in Buffalo titled “The Tribe of Ishmael: A Study in Social Degradation.” Indianapolis, McCulloch declared, had been infected by a “pauper ganglion,” a depraved clan that survived “by stealing, begging, ash-gathering.” In the summer, he said, “they ‘gypsy,’ or travel in wagons east or west.…They have been known to live in hollow trees on the river-bottoms or in empty houses.” They also received “almost unlimited public and private aid,” which merely “encourag[ed] them in this idle, wandering life, and in the propagation of similarly disposed children.”

The speech had lasting implications for both the poor people of Indiana and the budding pseudoscience of eugenics. It also was largely untrue, reports the historian Nathaniel Deutsch in Inventing America’s “Worst” Family (University of California Press), an insightful new study of the Ishmaels and their interpreters. There was no self-conceived “Tribe of Ishmael.” There was an Ishmael family, but McCulloch’s “tribe” often seemed to consist of anyone in the Indianapolis underclass who maintained some folkways of the Upland South.

While the relevant records are spotty, the actual Ishmael family does not seem to have relied much on public charity. Some of them were petty thieves and many of them begged, but they were also attracted to low status sorts of self-employment: odd jobs, recycling, prostitution. (A caveat: It’s not always clear what prostitution meant when McCulloch and his colleagues deployed the word. His assistant James Frank Wright, for example, seemed to use it to describe any woman who had sex out of wedlock.)

McCulloch was an early advocate of both eugenics and the social gospel, a toxic combination that foreshadowed the pending Progressive Era. Many modern readers will find it jarring to watch his economic views move leftward—by the end of his life he was a self-proclaimed socialist—even as he increased his contempt for, and willingness to use the law against, paupers who preferred to remain outside the wage economy. To McCulloch, though, his views were perfectly consistent. He wanted the state to help the deserving poor. To distinguish the deserving from the undeserving, he favored a system of intense surveillance over their lives. And to prevent the undeserving from continuing their ways, he called for coercive measures, including the forcible removal of their children.

But his most influential contention was not that the Ishmaels were a social evil. It was that they were a social evil with a biological basis. Heredity, he felt, had cursed them to coarse and parasitical lives.

Such ideas had consequences. In 1905 Indiana restricted marriages by former inmates “of any county asylum or home for indigent persons.” In 1907, influenced by McCulloch’s studies, the state adopted what may be the world’s first compulsory sterilization law. The eugenicist Harry Laughlin even invoked the Ishmaels (calling them “inferior human stock”) during the crusade that produced the Immigration Act of 1924, a law that sharply curtailed immigration from southern and eastern Europe and barred it entirely from much of Asia. The Ishmaels were a curious family to cite for such purposes, given that they were descended from a Revolutionary War veteran whose ancestors probably hailed from Wales. For the eugenicists, Deutsch explains, “most immigrants were potential Ishmaelites until proven otherwise by intelligence tests and other forms of screening.”

As eugenics lost its luster, the Ishmaels’ infamy faded. But in the 1970s they returned in a radically new guise. Hugo Prosper Leaming, a minister turned historian, offered a strikingly different tale of the tribe in a 1977 paper, “The Ben Ishmael Tribe: A Fugitive ‘Nation’ of the Old Northwest.” Leaming turned McCulloch on his head, presenting the pauper ganglion as a mobile utopia on the margins of society, resisting racism, wage labor, and the state. Leaming was especially interested in the fact that some blacks and Indians had married into the family. In his telling, the Ishmaels were an integrated “triracial” society that preserved African Islamic traditions and adopted Native American ways.

Leaming’s narrative never became as famous as McCulloch’s story had been. But it gained a certain underground currency, especially in the 1990s. Radicals in the ’70s and ’80s often imagined the world as a repressive total system, a place where almost any conceivable act was complicit in a web of social control. (The most extreme critique came from John Zerzan, an anarchist who argued that language, numbers, and even our sense of time were part of the totalitarian megamachine.) A reaction to those feelings of powerlessness and paranoia was brewing. As the ’80s ended, more and more dissidents were searching for signs of “everyday resistance,” “temporary autonomous zones,” and the like.

Leaming’s account of the Ishmaels was tailor-made for such an audience. I first encountered his work in the mid-’90s, when “The Ben Ishmael Tribe” was reprinted in Gone to Croatan: Origins of North American Dropout Culture, a 1993 anthology edited by the radical scholar Ron Sakolsky and the collage artist James Koehnline. Alongside fascinating studies of rural tax rebels, multiracial communities, and colonies of runaway slaves, there was Leaming’s description of the Ishmaels as anti-authoritarian nomads who “participated in the rise of black nationalism, perhaps even contributing memories of African Islam to the new Black Muslim movements.”

Unfortunately, as Deutsch details, Leaming’s account of the Ishmaels was no more reliable than the eugenicists’. He had speculated wildly, ignored inconvenient evidence, and fallen into the same error as those feminists who thought the witch trials had targeted a secret society of goddess worshippers: He disputed McCulloch’s conclusions but never questioned the idea that there was a coherent Tribe of Ishmael in the first place.

In the process, he moved from romanticizing resistance to romanticizing poverty. The eugenicists did get one thing right about the Ishmaels: They lived in desperate circumstances, often unsure where they’d find their next meal. Reading Leaming’s essay, you can lose sight of that fact.

But Leaming got a core truth right, too. The Ishmaels were, by and large, independent Americans, and they weren’t eager to embrace any effort to raise their station that would diminish that autonomy. Indiana greeted the Ishmaels and others like them with institutionalization and sterilization. Some things are worse than poverty.

Managing Editor Jesse Walker is the author of Rebels on the Air: An Alternative History of Radio in America (NYU Press).