That's the story from Diana Olick at CNBC.com, here:
Government-backed mortgages (Fannie Mae, Freddie Mac, Ginnie Mae) accounted for 58 percent of the $10.179 trillion U.S. mortgage market as of the end of March, 2012, according to data compiled by Inside Mortgage Finance.
Private-label mortgage-backed securities (MBS) investors held 10 percent and banks/other financial institutions held 32 percent. It’s that non-government, 42 percent of the market that is having the most trouble refinancing due to poor credit scores and negative equity. Lenders and investors are particularly risk-averse these days.