We have been asked by a broker to do a residential mortgage where
we would not pat any broker fees but they would charge a fee directly to the customer.Is this allowed under Respa?

Sure - but, the broker fee is still a finance charge for Reg. Z disclosure calculations. Of course the fee does have to be earned and the bank cannot charge duplicate fees for services performed by the broker.