We just handled this question today. Both our state examiners and the Feds handouts won't tell you what to do only that you have to document what you do. We have decided to have 2 individuals take their tokens with them from the Ebanking area and 1 of the admins as well. All others are locked in a secure area overnight. This way if the building blows up we will have the access we need by the 3 having them with them.
Our FDIC exam is coming up shortly so we'll see if it passes. They did want a copy of our token management policy ready when they arrive.