Skip to content
GeoDataVision
Thread Options
#1987224 - 01/08/15 02:00 PM Validating the AML Software
Jmalone Offline
New Poster
Joined: Aug 2014
Posts: 16
Ohio
The FFIEC BSA/AML Manual states-
Management should periodically review and test the filtering criteria and thresholds established to ensure that they are still effective. In addition, the monitoring systemís programming methodology and effectiveness should be independently validated to ensure that the models are detecting potentially suspicious activity. The independent validation should also verify the policies in place and that management is complying with those policies.

Is this required ANNUALLY? We outsource our audits since we are so small and our engagement letter has it as optional. BSA Audit with our without validation.

Return to Top
BSA/AML/CIP/OFAC Forum
#1987263 - 01/08/15 03:32 PM Re: Validating the AML Software Jmalone
rlcarey Online
10K Club
rlcarey
Joined: Jul 2001
Posts: 79,618
Galveston, TX
I would suggest making a determination if you need one be done on a annual basis.

Whether you actually need a new one might be based on a number of factors - new products added, significant core system upgrades, significant AML system upgrades, other significant changes in the overall AML risk profile, length of time since the last one, etc.
_________________________
The opinions expressed here should not be construed to be those of my employer: PPDocs.com

Return to Top
#1987367 - 01/08/15 05:57 PM Re: Validating the AML Software Jmalone
J2C Offline
Diamond Poster
Joined: May 2004
Posts: 1,475
Big Brother knows and that's a...
Once you have a validation performed you should periodically conduct a low-level validation of the information going forward. As Randy stated there are a number of factors to consider. If you haven't had any major changes then there is no need to do a full validation, IMO. If you go through a core conversion or another major event, then that is a different story.

All this being said, your policy and what-not should outline/ determine the frequency of these reviews in accordance with the Model Risk Management Guidelines
http://www.occ.gov/news-issuances/bulletins/2011/bulletin-2011-12.html

^^^^from OCC website but applies to ALL financial institutions.
Last edited by jennyfromthebloc; 01/08/15 05:58 PM.
_________________________
My opinion is mine only- not my employer's!


Return to Top
#1987824 - 01/11/15 02:56 PM Re: Validating the AML Software Jmalone
Pat Patriot Act Offline
Gold Star
Pat Patriot Act
Joined: Apr 2009
Posts: 450
Originally Posted By: Jmalone
The FFIEC BSA/AML Manual states-
Management should periodically review and test the filtering criteria and thresholds established to ensure that they are still effective. In addition, the monitoring systemís programming methodology and effectiveness should be independently validated to ensure that the models are detecting potentially suspicious activity. The independent validation should also verify the policies in place and that management is complying with those policies.

Is this required ANNUALLY? We outsource our audits since we are so small and our engagement letter has it as optional. BSA Audit with our without validation.


There's no specific period dictated and there's no definition of how independent the validation must be. Going internal is tough because the folks with the most expertise (BSA Dept) aren't independent enough.

With that being said, I haven't seen or heard of too many mid to small banks getting criticized for this unless there are other systemic issues. However, given the ever increasing focus in this area, I foresee more exam comments on this topic.

Also, you should aim to perform your self-testing at appropriate risk-based intervals. Read the OCC's guidance and identify all of the models you use. This means the ETL process, OFAC/list checks, 314a scans, case management functions, detection parameters, etc. Think of a strategy to self-validate each - it shows that you have a strong model governance program.

I would argue the following:

- AML Software data loads: Daily reconciliation
- OFAC: Validation at SDN update
- 314a: Validate after each run
- Case Management Functions/Processes: Annually
- Detection Parameters:
- Quarterly - Review of select ineffective parameters


- Annual full parameter review/gap analysis:
- Poor scoring rules targeted
_________________________
CFE, CAMS

Return to Top
#1998241 - 02/25/15 09:22 PM Re: Validating the AML Software Jmalone
MT2002 Offline
New Poster
Joined: Oct 2013
Posts: 23

Return to Top
#1998504 - 02/26/15 08:35 PM Re: Validating the AML Software Jmalone
happyauditor Offline
Platinum Poster
happyauditor
Joined: Nov 2004
Posts: 809
NY
This was a hot topic at the last (2013) OCC Midsize Bank BSA Officer Roundtable in Chicago. When I say hot topci, I mean it got heated between the bankers and the regulators. The regulators were outling very technical and advanced requirements for the validation, but when questioned, could not come to a consensus on what is, or is not, considered a model, how often a validation is to be performed, who should perform it, etc. Some of the bankers in the room that had exams and were told they needed to get an external validation done were spending at a minimum $100k for validation and stated there are not many consultants with the appropriate expertise to perform it, as not only do they need to be statisticians and very experienced in model validation, but they also need extensive BSA experience, as per the OCC members presenting.
_________________________
* My opinion is not necessarily that of my employer.

Return to Top
#1999123 - 03/02/15 03:26 PM Re: Validating the AML Software happyauditor
Pat Patriot Act Offline
Gold Star
Pat Patriot Act
Joined: Apr 2009
Posts: 450
Originally Posted By: happyauditor
This was a hot topic at the last (2013) OCC Midsize Bank BSA Officer Roundtable in Chicago. When I say hot topci, I mean it got heated between the bankers and the regulators. The regulators were outling very technical and advanced requirements for the validation, but when questioned, could not come to a consensus on what is, or is not, considered a model, how often a validation is to be performed, who should perform it, etc. Some of the bankers in the room that had exams and were told they needed to get an external validation done were spending at a minimum $100k for validation and stated there are not many consultants with the appropriate expertise to perform it, as not only do they need to be statisticians and very experienced in model validation, but they also need extensive BSA experience, as per the OCC members presenting.


I can see why that got heated!

I tend to agree that the best bet to get the right mix of independence and expertise is to go external, but for them to suggest that you need an individual with a PhD in statistics is a little overboard. BSA experience and the ability to understand systems are really all you need. It does not take a rocket surgeon to run a few "parallel simulations" on the blackbox.
_________________________
CFE, CAMS

Return to Top
#2009813 - 04/23/15 05:17 PM Re: Validating the AML Software Jmalone
Blessed Offline
Diamond Poster
Blessed
Joined: Oct 2007
Posts: 2,389
USA
Does anyone have a Model Validation program they'd be willing to share?

If so please pm me!
Last edited by Blessed; 04/23/15 08:58 PM.
_________________________
Ecclesiastes 10:2 (NIV)

Return to Top

Moderator:  Andy_Z