Skip to content
BOL Conferences
Thread Options
#1920306 - 05/06/14 01:29 PM audit sampling methods
happyauditor Offline
Platinum Poster
happyauditor
Joined: Nov 2004
Posts: 812
NY
Would anyone be willing to share/describe how they determine their audit sample sizes? Also please include primary regulator and asset size.

Examples of information I am looking for:

Do you rely on each individual auditor's judgement to determine sample size?

Set specifc standards/guidelines such as if the item is high risk and a daily occurrence, review 25 items, medium risk, review 15 items, low risk, process review only.

Statistical sampling methods? If using statistical sampling, what parameters do you use (95% confidence, 5% anticipated error rate, 1% expected error rate?)

Do you use stop and go sampling?

etc.
_________________________
* My opinion is not necessarily that of my employer.

Return to Top
Audit
#1920325 - 05/06/14 02:23 PM Re: audit sampling methods happyauditor
osucpa Offline
Diamond Poster
Joined: May 2011
Posts: 1,406
Reg - OCC
Bank Size - 1.5 billion
Sample size based on level of risk.
We generally use judgmental sampling

Return to Top
#1920387 - 05/06/14 04:03 PM Re: audit sampling methods osucpa
Cornfed Turtle Offline
Diamond Poster
Joined: Mar 2006
Posts: 1,323
"...Somewhere in Middle Americ...
$750, FRB

Judgmental based on risk.

Return to Top
#1920421 - 05/06/14 05:19 PM Re: audit sampling methods happyauditor
happyauditor Offline
Platinum Poster
happyauditor
Joined: Nov 2004
Posts: 812
NY
osucpa and cornfed turtle

Do you give your auditors guidance, for example, minimums and maximums?

For example, if 1,000 consumer mortgage loans were originated in the prior 12 months and you were conducting an annual audit, how many loans would you include in your sample for an underwriting review? For a review of disclosures? let's assume low risk for underwriting and moderate risk for disclosures due to all the recent regulatory changes.

Have you ever been criticized (or questioned) from examiners regarding how you determine the sample size?
_________________________
* My opinion is not necessarily that of my employer.

Return to Top
#1920521 - 05/06/14 08:40 PM Re: audit sampling methods happyauditor
osucpa Offline
Diamond Poster
Joined: May 2011
Posts: 1,406
Happy

Based on your information above, I would select a sample of 25 for disclosure purposes. They would be taken throughout the year. Every now and then an examiner will question me about it but so far I have been able to justify it. My experience is generally a bell shaped curve will naturally play out. During an audit about 4 years ago, an area manager had the same inquiry, and spent hours trying to disprove our sample methodology and ultimately was not able to.

Return to Top
#1920590 - 05/07/14 02:04 AM Re: audit sampling methods happyauditor
rlcarey Online
10K Club
rlcarey
Joined: Jul 2001
Posts: 83,227
Galveston, TX
Disclosures are also usually produced through documentation platforms which equates to the fact that they are usually either all wrong or you might have isolated operator errors. No one gets too excited about those. It is the systemic issues you are looking for.
_________________________
The opinions expressed here should not be construed to be those of my employer: PPDocs.com

Return to Top
#1920637 - 05/07/14 02:04 PM Re: audit sampling methods osucpa
RR Jen Offline
Power Poster
RR Jen
Joined: May 2003
Posts: 3,760
Running and riding everywhere ...
Originally Posted By: osucpa
Reg - OCC
Bank Size - 1.5 billion
Sample size based on level of risk.
We generally use judgmental sampling


$450 million...we have the same methodology. One the loan disclosure example...we have a central doc prep but multiple lenders/loan assistants. I want to be sure my sample includes loans from each location, not just the main office that might make the majority of the loans. The biggest problems I've found have been in the outlying areas. The OCC has not had an issue with this process.
_________________________
I don't need any more negativity in my life...be positive and helpful people or I will kick you in the shins!!!

Return to Top
#1920688 - 05/07/14 02:47 PM Re: audit sampling methods happyauditor
happyauditor Offline
Platinum Poster
happyauditor
Joined: Nov 2004
Posts: 812
NY
Thank you for your responses.

rlcarey, I agree if one is looking at the content of diclosures you would review one (assuming all produced via a system platform). But if performing a disclosure review to ensure proper timing of providing the disclosure to the borrower (based on evidence of date of dislcosure) or accuracy of disclosure where there are manual input items that may be prone to error, I think you would need to select a sample. Do you agree?
_________________________
* My opinion is not necessarily that of my employer.

Return to Top
#1920725 - 05/07/14 03:17 PM Re: audit sampling methods happyauditor
edAudit Offline
Power Poster
edAudit
Joined: Jul 2008
Posts: 4,796
You are here
_________________________
Opinions can be considered as coming from anywhere but my employer.

CAMS


Return to Top
#1920780 - 05/07/14 04:40 PM Re: audit sampling methods happyauditor
happyauditor Offline
Platinum Poster
happyauditor
Joined: Nov 2004
Posts: 812
NY
Thanks EdAudit. I looked at that. We actually have a statistical sampling tool that we use that gives similar results (attribute sampling using statistical variables [percent reliability desired, maximum tolerable error rate, and anticipated error rate], random number generation or nth number generation.) AICPA guidance tables also give similar results. My problem lies in the fact that the sample sizes from these statistical tools are too big for large populations for annual audits. For example, for an annual audit, if I have a population of 5,000 (scope of population is from prior audit to current audit, or prior 12 months), the sample size generated by the aforementioned tool is 192 items (similar with other tools and AICPA guidance based on 90% reliability which is what the aforementioned tool says uses).

So far non one responded they use statistical sampling methodology, although I did not get many responses. I am also asking personal contacts I have and still, no one is using statistical sampling methods (only received 2 responses so far). All are saying judgement based on risk, and I am seeing around 25 as the "max" being used.
Last edited by happyauditor; 05/07/14 04:41 PM.
_________________________
* My opinion is not necessarily that of my employer.

Return to Top
#1921586 - 05/08/14 10:08 PM Re: audit sampling methods RR Jen
QCL Offline
Power Poster
QCL
Joined: May 2002
Posts: 6,259
NW IL
Originally Posted By: RR Jen
Originally Posted By: osucpa
Reg - OCC
Bank Size - 1.5 billion
Sample size based on level of risk.
We generally use judgmental sampling


$450 million...we have the same methodology. One the loan disclosure example...we have a central doc prep but multiple lenders/loan assistants. I want to be sure my sample includes loans from each location, not just the main office that might make the majority of the loans. The biggest problems I've found have been in the outlying areas. The OCC has not had an issue with this process.


Same size as Jen.
Any samples used are auditor judgement.
Except:
1. Confirmations (sample size determined by external auditor)
2. 2nd market loans. We sell majority of loans to Freddie. Sample size is determined by them - 10% of the number of loans originated. We do this monthly to keep up.

Return to Top
#1921622 - 05/09/14 12:43 PM Re: audit sampling methods happyauditor
happyauditor Offline
Platinum Poster
happyauditor
Joined: Nov 2004
Posts: 812
NY
Thanks MamaBear

Can anyone in a mid-size bank (assets of $10B to $50B) or larger weigh in?
_________________________
* My opinion is not necessarily that of my employer.

Return to Top
#1928911 - 06/03/14 04:34 PM Re: audit sampling methods happyauditor
Rosie Offline
New Poster
Joined: Feb 2012
Posts: 3
Illinois
Does anyone have an example of an internal audit report that they can share. Would like to see how everyone is handling the assignment of the audit ratings. I would like to try to keep it simple: Outstanding, Satisfactory, needs improvement and unsatisfactory. Not sure if the rating should be placed in the body of the report?

Return to Top
#1929391 - 06/04/14 04:53 PM Re: audit sampling methods happyauditor
Hobbes Offline
100 Club
Hobbes
Joined: Sep 2007
Posts: 126
On a sled
Our rating used to be right at the top of the report, underneath the audit area, date, and who the report was issued to. We included the current rating and the previous rating.

For whatever reason, we no longer issue ratings, but risk levels, high, medium, or low, with each finding also being risk rated.
_________________________
There's never enough time to do all the nothing you want.

Return to Top
#1929393 - 06/04/14 04:56 PM Re: audit sampling methods happyauditor
rlcarey Online
10K Club
rlcarey
Joined: Jul 2001
Posts: 83,227
Galveston, TX
That is how a lot of audits are going now. The Board and Audit Committee need to focus on the individual findings and the risks they present rather than any arbitrary rating assigned to the overall audit. Many audits are completed across many functional areas at the same time and an "audit report" rating really has no real meaning anymore.
_________________________
The opinions expressed here should not be construed to be those of my employer: PPDocs.com

Return to Top
#1929525 - 06/04/14 08:08 PM Re: audit sampling methods happyauditor
happyauditor Offline
Platinum Poster
happyauditor
Joined: Nov 2004
Posts: 812
NY
We include an overall rating, which is placed in the executive summary of the audit report after an overview of the area and objectives and scope of the audit. FRB recently made us include ratings of individual audit findings as well. These are indicated next to each finding.

(When I say "made us", I literally mean, "made us", in the form of an MRA.)
_________________________
* My opinion is not necessarily that of my employer.

Return to Top
#1936350 - 06/27/14 07:06 PM Re: audit sampling methods happyauditor
NWBankerGal Offline
New Poster
Joined: Feb 2013
Posts: 6
I work for a $22 billion bank and we are regulated by the FDIC. Our sampling methodology is to select 20% or 30 items from our population (usually 6 months), whichever is lower. We always follow this methodology. From there, we typically perform judgmental sampling in picking out those 20% or 30 items for testing. The only time we stray from this is when there is a system control involved; we will test a sample of 1 to ensure that our automated control is working.

Return to Top
#1937926 - 07/03/14 08:37 PM Re: audit sampling methods happyauditor
Philly Fan Offline
Junior Member
Philly Fan
Joined: Nov 2011
Posts: 30
OHIO, USA
I work for a $12 billion bank and we use various methods for selecting our sample (i.e. haphazard, judgemental, statistical, stratified) how the size is formally defined and determined by the type of review (SOX vs. Audit)and the frequency of the control and the level of risk.

For controls that’s are performed daily, multiple times a day, or on an ad-hoc basis the sampling size will be the lesser of 10% of the population with a minimum of 5 and a maximum of 25.

For controls performed on a monthly and quarterly basis the sample size will be 3 and 2 respectively, which will be prorated based on the period of coverage.

For controls performed on a weekly and biweekly basis the sample size will be 6 and 5 respectively, which will be prorated based on the period of coverage.

Our sampling guidance has been reviewed and okayed by the FRB and OCC.

It should be noted that we also have guidance around when,why, and how to expand samples based on exceptions.

Return to Top
#1941549 - 07/16/14 04:00 PM Re: audit sampling methods happyauditor
happyauditor Offline
Platinum Poster
happyauditor
Joined: Nov 2004
Posts: 812
NY
Thanks for the feedback, it is greatly appreciated and helpful.
_________________________
* My opinion is not necessarily that of my employer.

Return to Top

Moderator:  Andy_Z