Thread Options
#1959660 - 09/05/14 03:08 PM Yellow Hammer model validation recommendations
Madawaska Offline
100 Club
Joined: Sep 2008
Posts: 105
VA
Any Yellow Hammer BSA and Fraud users out there that have had a model validation performed, I would love to know what firm you used for the validation and if the validation was deemed adequate by your regulator, and/or if you would recommend the firm.

We've engaged two firms so far to perform the validation and in both cases we found they do not have the expertise of the Yellow Hammer software to complete an adequate validation to pass the scrutiny of our regulator.

Return to Top
BSA/AML/CIP/OFAC Forum
#1959662 - 09/05/14 03:10 PM Re: Yellow Hammer model validation recommendations Madawaska
BSAChick Offline
Member
Joined: Oct 2005
Posts: 94
Illinois
I do. I will message you.
_________________________
CAMS

Return to Top
#1959668 - 09/05/14 03:20 PM Re: Yellow Hammer model validation recommendations Madawaska
osucpa Offline
Diamond Poster
Joined: May 2011
Posts: 1,400
We perform the validation, in house. If you are an OCC bank, the model validation should follow OCC 2011-12 Supervisory Guidance on Model Risk Management.

Return to Top
#1959998 - 09/08/14 03:22 PM Re: Yellow Hammer model validation recommendations Madawaska
komply Offline
Gold Star
Joined: Apr 2008
Posts: 475
NJ
Sent you a message

Return to Top
#1960293 - 09/09/14 01:50 PM Re: Yellow Hammer model validation recommendations Madawaska
Cher Offline
Member
Joined: Oct 2003
Posts: 98
Florida, USA
We do a quick and dirty in house validation on a monthly basis, but we do a full independent validation every 2 years. I'm looking for a new firm as well. Could you please share - thanks. The firm that we used last year, I won't use again so I'll pass that on as well in PM if you're interested. THanks.

Return to Top
#1961910 - 09/15/14 01:48 PM Re: Yellow Hammer model validation recommendations Madawaska
Pat Patriot Act Offline
Gold Star
Pat Patriot Act
Joined: Apr 2009
Posts: 450
Out of curiosity, could you explain what the consultants did and what the examiners thought they lacked? Unfortunately, the firm does not have the chance to defend their service during the exam. I wonder if if some pushback would be effective. It seems borderline inappropriate for them to criticize you if performed adequate due diligence prior to selecting a firm.

Also, are you OCC regulated?
_________________________
CFE, CAMS

Return to Top
#1961916 - 09/15/14 01:58 PM Re: Yellow Hammer model validation recommendations Madawaska
Calmeida12 Offline
100 Club
Joined: Sep 2012
Posts: 249
I just had ours done by a new company and it was very thorough. Would definitely recommend them. Send me a PM and I will get you their information.

Return to Top
#1962446 - 09/17/14 01:00 PM Re: Yellow Hammer model validation recommendations Madawaska
TryingtoComply Offline
Diamond Poster
Joined: Apr 2013
Posts: 1,894
The West
Out here in the west we are experiencing a lot of criticism regarding audit firms as well despite performing considerable due diligence in choosing a firm. We have stopped using two firms now due to FDIC stating they are inadequate. These are firms that have been around forever and they are probably wondering why they aren't getting any engagements.

Madawaska, It would be helpful if you could be more specific about what was considered inadequate.

Cher, perhaps you could list the steps you take to perform your monthly validation so that others can compare.
_________________________
TryingToComply
CRCM

Return to Top
#1962732 - 09/18/14 02:26 AM Re: Yellow Hammer model validation recommendations Madawaska
rlcarey Online
10K Club
rlcarey
Joined: Jul 2001
Posts: 78,971
Galveston, TX
There are several steps to validation of AML/BSA software.

1. Determining that all transactions from your core, wire, ACH systems, etc. are flowing to the software and are categorized correctly. This is the easy part.

2. Validating that your rules/parameters/cut-off scores are working appropriately and efficiently. This is the not so easy part.

Many companies do a good job on 1. Few do a good job on 2.
_________________________
The opinions expressed here should not be construed to be those of my employer: PPDocs.com

Return to Top
#1963084 - 09/18/14 11:52 PM Re: Yellow Hammer model validation recommendations rlcarey
Pat Patriot Act Offline
Gold Star
Pat Patriot Act
Joined: Apr 2009
Posts: 450
In your opinion, what steps are the examiners looking for the firm to conduct relative to validating that the "black box" is set properly?

Are examiners looking for consultants to "confirm" the rules do what they should by performing "parallel simulation" of the client's settings or running their own "high risk scenario" rules and see if the system detected it; or, should they be performing a full-fledged system tuning and gap analysis? Is there anything else that the examiners would be looking for the firm to do?
_________________________
CFE, CAMS

Return to Top
#1963089 - 09/19/14 02:19 AM Re: Yellow Hammer model validation recommendations Madawaska
rlcarey Online
10K Club
rlcarey
Joined: Jul 2001
Posts: 78,971
Galveston, TX
Since most of the BSA examiners have no idea how the various AML software even work or what they are or are not capable of doing, I think you will hear a very wide variety of expectations. Some of which will make sense and many that do not. There are no current standards. We are again left to the whims of individual examiner teams and their wide variations in expectations.
_________________________
The opinions expressed here should not be construed to be those of my employer: PPDocs.com

Return to Top
#1963108 - 09/19/14 12:48 PM Re: Yellow Hammer model validation recommendations rlcarey
Pat Patriot Act Offline
Gold Star
Pat Patriot Act
Joined: Apr 2009
Posts: 450
Originally Posted By: rlcarey
Since most of the BSA examiners have no idea how the various AML software even work or what they are or are not capable of doing, I think you will hear a very wide variety of expectations. Some of which will make sense and many that do not. There are no current standards. We are again left to the whims of individual examiner teams and their wide variations in expectations.


Exactly as I thought, which is why the OP's story makes my blood boil. Perhaps it's unfair for me to think this way, but I'm envisioning a young overzealous examiner that read a couple Big 4 whitepapers on AML validation and went to town on a community bank for choosing a firm that doesn't meet that "standard".
_________________________
CFE, CAMS

Return to Top
#1963123 - 09/19/14 01:21 PM Re: Yellow Hammer model validation recommendations Madawaska
edAudit Online
Power Poster
edAudit
Joined: Jul 2008
Posts: 4,766
You are here
They did not even read a white paper they read this

This bulletin provides guidance to help financial
institutions mitigate potential risks arising from
reliance on computer-based financial models th
at are improperly validated or tested.

http://ithandbook.ffiec.gov/media/resources/3676/occ-bl2000-16_risk_model_validation.pdf


Now IMO BSA models only became financial models due to CMPs and cost to validate them.
_________________________
Opinions can be considered as coming from anywhere but my employer.

CAMS


Return to Top

Moderator:  Andy_Z