Higher interest rates could crimp the earnings of U.S. banks with large mortgage banking businesses, says Fitch Ratings in a new report.

The rating agency notes that earnings at banks with large mortgage businesses, such as JPMorgan Chase, Wells Fargo and U.S. Bancorp, “have been boosted over the last several quarters by strong mortgage banking income due largely to the high volume of refinancings amid generationally low mortgage rates.”

Fitch says that it did not expect this level of mortgage banking to persist, and with refinancings dwindling and mortgage rates moving higher it believes bank earnings could be impacted over the next few quarters.

Indeed, it reports that interest rates on fixed 30-year mortgages have risen for nine consecutive weeks to a two-year high. As a result, it expects mortgage volumes will continue to fall, “as many borrowers have already refinanced their homes or are still unable to refinance because of depressed housing values.”

Total mortgage originations are forecast to decline 26.5% in 2014, it notes, a decline largely attributable to a significant drop in refinance originations.

Fitch says a decline in mortgage banking due to higher interest rates and a drop in refinancings could represent a 4% revenue decline for regional banks, adding that this is “a significant drop that we believe would need to be offset to keep revenue at least level.”