My bank is considering a new office in a rural and underserved county. I want to encourage this, so what are the benefits of being located in here?

CRA doesn't consider underserved counties. It considers underserved tracts. The only way underserved tracts affect CRA performance is via "community development". For small banks below the ISB size standard, there are no CD tests or responsibilities. So a truly small bank gains no advantage from annexing an underserved county even if it has a number of underserved tracts.

I wouldn't approach the issue as you apparently are. I would recommend the bank determine if it wants to expand its market into the new county (assuming it's not already a part of your existing AA's) for the typical non-regulatory purposes (like can the bank make money serving the new area) and then consider the impact on compliance considerations. All too often banks expand into new areas or open new branches without thinking about the regulatory implications until they've already expanded and/or opened the new branch. Then the compliance officer must deal with the consequences. This can have serious and adverse consequences.