Banks have different policies on this. Some allow gifts up to $25 or $50 (or even $100) for example, but they must be reported.
In a situation where it might offend the customer if a valuable gift was to be returned, the gift might be used "in general" by the bank. Perhaps for the use of the whole department, considered bank property and displayed (in the case of some china, or such type of gift), shared (food), and so forth.
Your bank should establish a policy - this would be a good time.