Vulnerability assessment and penetration testing guide
You need security testing for an upcoming audit. One provider offers an automated scan, another proposes a manual penetration test, and a third recommends red teaming. The prices and descriptions look very different. Which one do you actually need?
Start with why the testing was requested. You may be supporting a security control, meeting a customer contract, or checking a particular risk before the audit. Get that purpose clear before choosing a service.
What are you buying?
- Automated scanning: software runs configured checks for known vulnerabilities or configuration concerns. It can be useful for repeatable monitoring, but results need review and the tool can only check what it supports and can access.
- Vulnerability validation: someone checks whether reported conditions are supported and relevant. Confirming selected findings does not show whether other weaknesses or workflows were examined.
- Assessor-led penetration testing: a tester uses tools and judgment to examine agreed security boundaries and the impact of selected weaknesses. Ask what human review covers beyond automated results.
- Red teaming: an authorized exercise explores an agreed scenario and how the organization prevents, detects, and responds to it. It can leave many individual vulnerabilities unexamined because it follows a different objective.
“Automated” and “manual” describe how work is performed. Penetration tests commonly use both. The service name alone will not tell you what the provider will do or what evidence you will receive.
What will be acceptable for our SOC audit?
There is no single testing package that is automatically sufficient for every SOC 2 engagement. Ask how the work supports your organization’s controls, risks, and commitments. Discuss the proposed evidence with your auditor before buying it. If a contract or another applicable standard has a specific requirement, check that separately.
“The provider says it’s SOC 2 compliant. Isn’t that enough?” Ask what they mean. Which systems will they examine, what methods will they use, and what evidence will they provide? The auditor evaluates your organization’s controls and evidence. A provider’s marketing description does not settle that evaluation.
For example, a scan can be useful for routine vulnerability management while falling short of a contract that expressly requires an independently performed penetration test. A red-team exercise may provide good evidence about incident response while leaving application permissions outside its scope.
Match the work to the question
- Are we keeping up with known technical weaknesses? Recurring scanning and review may help. Confirm the inventory, access, coverage, and follow-up process.
- Can our application enforce the permissions we promise customers? Discuss an assessment of the relevant user roles, customer separation, APIs, and workflows.
- Will our people and controls detect and respond to a defined incident scenario? Discuss an authorized exercise with clear defensive objectives and agreed limits.
You may need more than one activity. A company could use recurring scans, commission a focused application test after changing its permission model, and separately exercise incident escalation. Keep the results tied to their different purposes.
Compare proposals before signing
Ask each provider for the same basic information so you can compare the offers fairly:
- Scope: systems, applications, locations, user roles, and environments included or excluded.
- Method: automated checks, human analysis, validation, and any relevant methodology.
- People: who performs the work, their relevant experience, and any independence requirements.
- Safety: written permission, operational limits, data handling, emergency contacts, and stop conditions.
- Deliverables: a report that explains coverage, limitations, findings, impact, and recommended corrections.
- Follow-up: how retesting works, what it covers, and whether it is included in the price.
If a provider uses AI, ask where confidential information goes and who checks material conclusions. The same questions apply to services called “automated red teaming.” Find out whether they run a fixed set of checks or support an exercise that evaluates the defensive outcome you need.
What happens when the report arrives?
Read the scope and limitations before the summary of findings. Make sure the report covers the service you intended to assess. A report with no findings is less reassuring if important systems or user roles were excluded.
- Assign an accountable owner, priority, and target date to each confirmed issue.
- Track the correction and the evidence needed to show it worked.
- Keep retest results linked to the original finding and affected scope.
- Document open items, temporary safeguards, and risk decisions.
Give your auditor the report and the follow-up evidence. A technical fix, an improvement to detection, and an accepted residual risk are different outcomes and should be recorded that way. Raise significant issues early so their effect on the engagement can be considered.
Your next step is to write down the purpose of the testing in one sentence, then compare it with the proposed scope. Then allow time for the report, corrections, and retesting before the audit needs the evidence. No purchased test guarantees the audit outcome. What helps is having the right coverage and being able to show how your organization handled the results.
Sources and further reading
See AICPA SOC resources for SOC context, the OWASP Web Security Testing Guide introduction for combining assessment techniques, and NIST SP 800-115 for assessment planning. The NIST-hosted red-team definition describes the exercise perspective.
For payment-card environments, consult the applicable requirements in the PCI SSC document library and the PCI DSS overview. Requirements from that context should not be assumed to apply to every SOC engagement.