City Commissioner Nick Fish has been in crisis management for more than a month since a city-funded study revealed nearly two out of three African-Americans and Latinos face discrimination when they look for a place to rent in Portland.

Fish, whose City Council responsibilities include housing, called the results “appalling” and developed a plan, released June 10, that he says will fight housing discrimination in the city. 

But WW’s review of the audit finds it’s flawed. It’s likely that housing discrimination—while a serious problem in Portland—is not as bad as it appears in the study. The reporting methodology apparently used in the study can greatly exaggerate the true rate of housing discrimination.

Two national experts point out other problems with the study: It’s not statistically sound, and the results are so far out of line with those in other cities they raise doubts about the study’s validity.

The only way to fully check the $13,000 study’s accuracy is to examine and compare the reports filed by testers who visited the various apartment buildings around Portland. 

Fish declined WW’s request to make the records public.

As it turns out, Fish and city officials say they haven’t even seen those detailed records themselves. They have relied on only a selective summary of the findings by the auditor, the Fair Housing Council of Oregon. The actual documents remain locked in the group’s files. Moloy Good, executive director of the Fair Housing Council, also declined WW’s requests to review the reports. Good declined to say why he won’t make the evidence public.

While the Housing Bureau doesn’t hold the records, the city’s contract with Good’s organization does give officials (and, in turn, the public) access to the records.

Fish says the housing council believes releasing the records would jeopardize its ability to do future audits by revealing its methods, and the city isn’t willing to compel their release. He also says the results are clear enough to show that housing discrimination still exists here. 

“We are comfortable going forward on the basis we have now. What we got was what we contracted for,” Fish says. “I’m comfortable based on that data that the city needs to take bold action.”

But accuracy and transparency are important when framing the debate. And the lack of openness about how the data have been reported concerns one expert contacted by WW.

“It’s very unusual to have an audit where everything is buried,” says John Yinger, a professor of public administration and economics at Syracuse University. “It makes the debate about discrimination more difficult.”

In the Fair Housing Council audit, testers posed as prospective renters and were instructed to provide similar information about their background, income and work history. The group sent white testers to 50 apartment buildings. Latino testers were sent to half of those same apartments and black testers sent to the other half. 

The basic test is widely accepted by experts. Such undercover audits are often the only way to discover discrimination. Minorities looking for housing often have no idea, for example, they are being quoted higher rents or move-in fees.

The group’s audit claims the results show African-Americans were treated differently from white testers 15 out of 25 times. The number was 17 of 25 for Latino testers.

To be sure, examples made public by the Fair Housing Council show evidence of outright discrimination. One African-American tester was quoted move-in costs more than two times higher than a white tester. A rental agent asked a Latino tester, “Are you Mexican?”

Yinger says these and other examples show “pretty serious disparate treatment.” But Yinger adds he would have serious doubts about an audit that claimed a 64 percent rate of discrimination—a number he says is so high as to raise questions about the study’s credibility.

“I’ve never seen anything like this,” says Yinger, who has studied housing discrimination nationwide for more than three decades. “This would make me want to look seriously at how this test was done.”

Margery Austin Turner, vice president for research at the Urban Institute in Washington, D.C., who has authored major studies with Yinger, says the Fair Housing Council study’s sample was too small for officials to cite the rate with confidence.

“It’s an important snapshot of practices in this housing market,” Turner says. “I wouldn’t feel the precise number is something to hang a hat on.”

Internal city records show that Portland officials acknowledge the study is not statistically reliable, but they don’t usually offer that caveat when telling the public about the results. And Fish has repeated the 64 percent figure without noting that it’s not statistically sound.

Fish acknowledges problems with relying on the rate.

“I don’t believe this test was intended to be a full picture of housing discrimination in our community,” Fish says. “We do not maintain it is a scientifically complete picture.”

Yinger and Turner said they didn’t know enough about the Portland study to provide an opinion on its validity. But both cautioned that a finding of discrimination should be made only after looking at the testers’ entire experience, and not to assume a single difference in treatment is proof of discrimination. 

“The most important thing is to report the full range of what happened to testers,” Turner says.

Testers are supposed to have a wide-ranging conversation with the rental agent, covering rent, move-in costs, restrictions and amenities, like a pool and fitness room. 

Not all differences are meaningful, however. For example, the Fair Housing Council audit cited the fact a Latino tester was not offered a floor plan and brochure as proof of discrimination.

Housing audits should follow a standardized list of questions and often require testers to fill out questionnaires that help guarantee their reports are consistent and comparable. Representatives of the Fair Housing Council told city officials their testers didn’t follow a script. Good, the housing group’s director, declined to talk with WW about the protocols his group followed.

More importantly, experts say, reporting the results of housing audits the wrong way can make things look worse than they really are.

Audits nationwide have found that black and Latino testers are sometimes given more favorable information than are whites. In other cases, it’s a mixed bag, with both sets of testers given information that could be favorable to one or the other. Auditors, in turn, should weigh the entire experience—not pick and choose details and label them “discrimination.”

“It would be very troubling if you only pulled out selective things from the study,” Yinger says. “We don’t know if that’s what happened here.”

The Fair Housing Council says releasing the records could expose its methodology. But Good and others have already talked in great detail about the methodology at public hearings. Good also told WW that he doesn’t want to release the full report because he wants to protect the testers’ anonymity. WW agreed to accept the reports with the testers’ names removed, but Good still declined to release the report’s details.

The city turned over the audit findings to the state Bureau of Labor and Industries, which is responsible for investigating housing discrimination. But BOLI officials say the information in the audit is too sparse.

“We’ve asked the Fair Housing Council for more details,” says BOLI spokesman Bob Estabrook. “They have not provided us with them.”