We made a cash out refi on a primary residence (single-wide) about 2 months ago. Now same borrowers are requesting a cash out refi on a different property(another single wide) representing that it is their current primary residence (property purchased during refi of first cash out refi). First cash out refi is now being rented. Borrowers also own a larger home in another state and they are stating that this one is their second home. We think the borrowers are borrowering the money to purchase other rentals and by stating that the loan is for their primary residence they get a better rate. We want to turn down their request because we think this is fradulant. Do we have a case?
Also, all single wides are located in the same town.

It does sound a bit like gamesmanship. Have you tried to verify their address? A driver's license or voter registration card might do the trick. Asking for address verification would also communicate to them that you are aware of their many -- and perhaps questionable -- address changes. As long as you play dumb, they'll keep trying. You can also turn them down. ECOA doesn't prohibit this denial. Whether you suspect fraud wouldn't need to come up. You could deny for frequent address changes. Time at address is an old underwriting criterion.

The only caveat on using that as a decline reason is - Make sure you address it somewhere in your policy! Examiners LOVE to take denial reasons and look for the justification in policy. If it's not currently in your policy, then add it to policy as of now. And be sure to consistently apply it to any future applications.

This is most likely a fraudulent request as you said, to get the owner occupied rate. You have every right to demand proof, tax returns etc.of primary address. And if they can't prove that the current address being given is their primary, then either turn them down (they lied) or offer them an investment property loan. Also, what do your mortgage docs say about the prior property being rented out. Docs often address that issue.