Broker licenses and contract requirements are usually governed by state law, I would begin there. But as a side note if I was going to deal with a broker then I would want a contract that specifically spelled out their, and our responsibilities in the transaction.
I'm not aware of any thing preventing you from paying a percentage, that is the common practice, as long as the fee paid can me justified. IOWs, and this is my opinion, what did the broker do different on that $500,000 mortgage to get paid $5000 vs the $1000 they got paid on the $100,000 mortgage?
_________________________
The opinions expressed are mine and they are not to be taken as legal advice.