I am not a techie, but have been asked to give my opinion on a proposed alternative to the annual penetration testing recommended by our examiners. Perhaps, you can all enlighten me. IT plans to contract for a managed IDS in addition to our managed firewall in lieu of penetraiton testing. I think that I have convinced them that the examiners will still require penetration testing; however, they are proposing to use the vulnerability testing services offered by the same vendor. I have concerns about using the same vendor relating to independence. With the managed systems that we have, does it lessen the extent of penetration testing recommended by examiners? Also, does anyone else have an insurer who insists on quarterly penetration testing? Thanks.

It doesn’t matter if the managed firewall and the IDS are outsourced, penetration testing is still required, i.e. the existence of an IDS does not absolve the need for penetration testing. Depending on your exact configuration, your service provider will contract for the penetration testing with an independent third party testing firm and pass the results of the testing to you in the form of a “certification” of “attestation” that their defenses have passed or not. The firm providing the pentration testing for your service provider should be reputable, independent, and qualified – but you may not have control over this.

You should review your contract with your service provider to ensure that periodic qualified penetration testing is included with a periodic reporting requirement. Under no circumstances should you contract for penetration testing without coordinating with your service provider – big obvious problems.

I have not heard of a quarterly penetration testing requirement, however frequent testing of the perimeter security is recommended especially after configuration changes, software upgrades, etc.

-g

Thanks. I was trying to figure out exactly what we should be requiring from our vendor to ensure that the managed systems that we outsource to them were effective. Is there usually a charge to the financial institution for this testing (similar to a SAS-70 charged to all users)? Does this testing fully satisfy examiner concerns, or is there additional testing that we should be considering?

If your provider follows industry best practices, has its own insurance coverage, etc. it should be having penetration testing performed periodically notwithstanding the fact that you (bank) is the customer. They may charge additional for the penetration testing but this should part of the contract terms - review the contract.

Whether of not the testing satisfies the regulators' concerns is impossible to gauge at this moment. The scope of penetration testing can range from basic and unobtrusive to advanced and well, "penetrating". You should meet with your service provider, review the scope of their penetration testing, perform your own risk analysis as to the adequacy of the testing, match that to the results of the testing and document your actions and decisions.

-g

I'm not sure if this will be useful, but we use ITI for our internet banking product and they provide a TruSecure Assessor's Report of Certification (www.trusecure.com) that covers uses such as: 1) Electronic threats and vulnerabilities; 2) Malicious code; 3) Privacy issues; 3) Human factors; 4) Physicial environment, and 5) Downtime issues.

do they charge you for this report?

The report from ITI is useful and shows the examiners that you (the bank) recognizes the need for not only your systems to be tested but so does your vendor. It is always good practice that your vendor for this type of service is servicing you as a customer by having tests done to prove to their customers that they are safe to "compute" with...

It is even more important to conduct the penetration testing either yearly or when changes are made to your network (as stated in previous response by someone else).

You might consider something like www.qualys.com for your penetration testing. One product they offer is on demand penetration testing / scanning. The bank has the ability to both schedule periodic scans and run the scans on demand (via the Internet). The pricing is on their web site and the scans are priced by the ip address or by the number of scans. I believe unlimited (i.e. daily, weekly, monthly, etc.) scanning is only $995 for one ip address. There are quantity discounts for multiple ip addresss. You can also purchase, say, a hundred scans in a package. The product scans for most of the known vulnerabilities.

This approach alleviates much of the problem I see with annual or even quarterly testing. New vulnerabilities are being discovered at such a rapid pace that a scan is only good for a day.

Of course the vendors to whom you outsource your IDS and firewall to will be keeping up with the same vulnerabilities, so this would just be an additional layer of security.

This type of penetration testing would not be as robust as a test which includes manual testing (i.e. a live engineer actually tries to break into your system); however, a number companies I have seen don't use any manual testing methods and just use a product like qualys and "resell" it to you by interpreting the results for you...at a significant markup, I might add.

I have no relation to this company other than we are in the process of implementing their product.

Managed IDS can be a great operational addition to a layered security model but it doesn't do the same job the penetration testing does. I agree that services like Qualys are quite usefull for remote scans but that is only a small piece of what a true penetration test should be. Many of the better audit firms are doing pen testing that is fairly comprehensive. I've also got a handfull of excellent consultants (is that an oxymoron?) that I work with who do a good job in various regions of the country. Drop me a line if you would like some introductions.

While we do both IDS management AND Remote assesment we typically won't do both for the same customer. Too much like the fox watching the hen house.

For what it's worth - the quality of your information security management is the thing that will keep you safe - the pen testing is important but it will only give you a snapshot of that very moment and even then only of the things that are looked at.