Hello all,

I am just nearing the end of my first model validation here at my small community bank as BSA Officer. This may be suited better for the New York section of the website....

Is the expectation of part 504 to have a model validation conducted annually or is it expected to be completed on a schedule as determined by the bank's risk assessment/risk profile?

Thanks!

I don't believe it specifically states that annual model validations are required, however 504.3 (a)(8) and 504.3(b)(5) state the that both the TMP and WLFP be subject to on-going analysis.

FWIW when I was sub-certifying the annual certification I insisted on annual model validations.

Even if not required by part 504, your internal model risk ranking may require annual validations.

Our institution is a small community rank and very low risk. I was already shocked when I got here and they even had a model as a 100 million dollar institution, but even more shocked that a validation had never been performed. I already plan on conducting yearly assessments of alert generation scenarios and a reconcilement to validate the transactional data is properly flowing over to the transaction monitoring program, but as such a small institution, adding verbiage to the policy regarding frequency of independent model validation, I was hoping to do it every other year based on our risk assessment, instead of yearly. I guess I am just unsure if the "unwritten expectation" is that we have one done externally each year.

Thoughts?

How would you submit an annual certification regarding your TMP and WLFP without an annual model validation?

Sign the form? If nothing has changed, why would you need to do a validation again or have second thoughts about re-certifying?

Part 504 requires an annual certification that you're TMP and WLFP are working as intended and covers each of the points detailed in the regulation. How can you certify this on an annual basis if there isn't an annual validation?

Are you a community bank or an FBO?

Community bank with 3 banches ~100m in assets. Can't justify spending $10-15k a year on an independent validation. I would perform a reconciliation ("data validation") and alert parameter review myself on a yearly basis.

When I was DFS regulated we did a full validation one year and a mid-tier validation the next year (Assuming no significant findings). This generally cost us around $5-$8k. This didn't include full back testing but for a small community bank I don't think you need that. If you're interested in chatting with them send me a PM.

You need to revalidate periodically because the population being assessed by the model or the transaction being flagged by the model can change over time. This can cause the model to lose predictiveness. Also, model risk management guidance adopted by the FRB, OCC, and FDIC requires ongoing validation.

If you are using an automated system how else would you know if it is working properly?

I have bee doing this in some fashion for the past 15 years at multiple banks. One only 250 million.

The only part that changed because of 504 (and fuzzy logic) is it went outsourced. and has a a lot more detail.

Even with the greater detail you may be called upon to discuss it in detail with DFS. (You need to understand it)

I have a call scheduled with DFS to discuss this a little further as to what their expectations are of our bank.

I just hung up the phone with our Principal Bank Examiner at DFS. He told me specifically, there is no expectation that an independent model validation be conducted on a yearly basis. Obviously if a major event occurs or large scale tuning, mapping, new accounts, happens they would expect a new validation, otherwise our annual internal audit sampling of transactional data and customer accounts from the core to the model works FOR A BANK OUR SIZE. So I am going to be less specific on a particular timeline in my policy/procedure updates to reflect that information.