That seems to be the conclusion to draw from the U.S. Supreme Court's refusal to hear a case from San Diego, where the D.A.'s office has been sending agents to conduct suspicionless, warrantless searches on the private homes of welfare applicants.
Yes, applicants were free to refuse the searches, though I suspect that refusing a search would itself be (unofficially) enough to trigger further investigation. Refusing a search also means forfeiting welfare benefits.
This is part of a series of incidents across the country over the last few years using administrative or regulatory procedures to conduct warrantless searches for criminal activity (yes, I'm writing an article on it).
I suspect the law-and-order response to the policy in San Diego would be something along the lines of "if they can't prove they're clean, they don't deserve my tax dollars." Of course, if everyone who received any sort of government assistance had to consent to a search of their home, the Fourth Amendment would be pretty much null(er). For example, I'd guess there'd be quite a bit more outrage if these fishing expeditions/searches were being done on the homes of, say, middle class kids applying for government-subsidized student loans instead of low-income people applying for welfare.