Your bank does not receive and accept deposits at its drive-up stations. It receives deposits at the teller station(s) servicing the drive-up customers. As you suggested, if there is a drive-up window where a teller or tellers receive and process deposits, an FDIC Official Sign should be in that window for customers to see. While the FDIC will not cite you for sticking them at locations where they are not required (except where non-deposit products are offered), sticking the signs on drive-up canisters seems a bit of an overreach and could be a logistic problem given the 3 x 7 inch size of the signs.
If the drive-up is serviced by lobby tellers and there is no dedicated drive-up window, the individual teller stations will have the signs on the lobby side of the teller counter, and that's all you need.