The joint owners have no ability to remove either the joint owner appointing the attorney in fact nor the attorney in fact. All that any of the joint owners can do is to close the account if they do not want an AIF on the account. The AIF can do whatever the joint owner could do: write checks, make deposits, stop payments, close the account (check would not be payable to the AIF). The AIF cannot remove another joint owner or an authorized signer and does not have power to add a POD unless the POA specifically permits this and the other joint owners agree.
You aren't required to notify joint owners that a POA has been appointed and added to a joint account, but neither is the bank prohibited from doing so. In the interest of preventing calls from the other joint owners, I would suggest advising of the appointment of the attorney in fact.