When I was a commercial lender at a previous institution, I was asked to be on the Board of one of our commercial customers. I respectfully declined due to potential conflicts of interest. I did inform our Board and they subsequently created a policy prohibiting officers from being on Boards of customers (non-profit agencies were the exception). Our Board also felt it could be a conflict of interest.
I believe it would be a financial institution's policy that would govern this. I would be extremely cautious about it, though. I just felt it could create potential conflicts even if you abstain on certain issues. I would also wonder if you abstain on certain issues, are you providing the most guidance and benefit you can or should to that Board?