Penetration Testing for Regulatory Compliance

What is a penetration test?

A penetration test is a controlled, simulated attack that identifies your organization’s susceptibility to application, network, and operating system breaches. Pen tests can be performed both internally or externally to test vulnerabilities in your IT security; in either case, the aim is to identify security issues that may be exploited by malicious actors.

The National Institute of Standards and Technology (NIST) defines penetration testing as follows:

Penetration testing is a specialized type of assessment conducted on information systems or individual system components to identify vulnerabilities that could be exploited by adversaries. Such testing can be used either to validate vulnerabilities or to determine the degree of resistance organizational information systems have to adversaries within a set of specified constraints (eg. time, resources, and/or skills). Penetration testing attempts to duplicate the actions of adversaries in carrying out hostile cyber attacks against organizations and provides a more in-depth analysis of security-related weaknesses or deficiencies. Organizations can also use the results of vulnerability analyses to support penetration testing activities.

Penetration testing can be conducted on the hardware, software, or firmware components of an information system, and can exercise both physical and technical security controls. A standard method for penetration testing includes, for example, (i) pretest analysis based on full knowledge of the target system; (ii) pretest identification of potential vulnerabilities based on pretest analysis; and (iii) testing designed to determine exploitability of identified vulnerabilities. All parties agree to the rules of engagement before the commencement of penetration testing scenarios. Organizations correlate the penetration testing rules of engagement with the tools, techniques, and procedures that are anticipated to be employed by adversaries carrying out attacks. Organizational risk assessments guide decisions on the level of independence required for personnel conducting penetration testing.

Because new vulnerabilities are discovered all the time, it’s important to conduct penetration tests regularly; this assures that your organization is mitigating real-world risk on an ongoing basis. Testing is especially important when introducing new workplace technologies, moving to the cloud, or outsourcing IT; or when you’ve experienced a breach in the past or aren’t confident about your security.

Any organization that provides an online service or has a network connected to the internet should consider penetration testing.

Who performs a penetration test?

Penetration tests are carried out by trained security experts called penetration testers, also sometimes called ethical hackers. They work with organizations to identify vulnerabilities by imitating the behaviors of their unethical counterparts.

Ethical hackers can help organizations better understand and react to the ever-changing nature of online threats. On behalf of the organization and with its permission, pen testers try to penetrate computer system networks, applications, or other computing resources.

Most businesses are not as prepared for attacks as they believe. By using the techniques that malicious actors might employ to attack their target (that’s you), ethical hackers help determine which security measures are effective, which should be updated, and where vulnerabilities exist that can be exploited.

When hiring a pen tester or ethical hacker, make sure to seek professionals who are certified.

Certifications for ethical hackers

Certified Ethical Hacker (CEH) is a vendor-neutral certification from the EC-Council, a leading certification body for security testing. This certification covers more than 270 attack technologies. Certification prerequisites include attending official training offered by the EC-Council and at least two years of information security-related experience.

Other certifications include Certified Information Systems Auditor (CISA), offered by the Information Systems Audit and Control Association (ISACA); and the Global Information Assurance Certification (GIAC) Security Essentials (GSEC) for security professionals who want to demonstrate they are qualified for IT systems and hands-on security roles.

Experience and other qualifications are also important to consider when selecting a pen test team.

Choosing your pen test team

When considering whom to trust with your penetration test, ask:

• How many years’ experience does the penetration tester have?
• How many years has the organization that employs the pen tester been performing penetration tests?
• Has the pen tester performed assessments for organizations of similar size and scope to yours?
• What pen testing experience has the tester or team had with the technologies in the target environment? (Operating systems, hardware, web applications, highly customized applications, network services, protocols, and so forth.)
• What other skills or qualifications does the pen tester have that will contribute to his or her ability to assess the environment?

Pen testers will often use penetration testing tools that are specific to the type of testing services they provide. Make sure the penetration testing team you choose can competently execute the specific type of penetration test you need to perform.

Once you’ve found a pen testing team that meets your organization’s needs and expectations, you can begin the pen test process. For most organizations, the first step consists of a vulnerability assessment.

Ideally, you will already know what the pen tester will find before they find it. Using third-party tests to verify your own expectations will help you gain a good understanding of your system vulnerabilities.

Penetration tests vs. vulnerability assessments

Vulnerability assessments

Vulnerability assessments, an important part of penetration testing, usually take place before the penetration test begins. They evaluate your system’s weaknesses and then assign severity levels to vulnerabilities.

During a network vulnerability assessment, an external information security consultant will review your corporate environment. The consultant will provide a detailed report of where a hacker might attack and recommend how to fix security vulnerabilities.

Assessments take place in four steps: testing, analysis, assessment, and remediation.

• Testing begins with a comprehensive list of potential vulnerabilities in the system. Security analysts will then scan test applications, servers, or other systems with automated tools or manually.
• Analysis involves identifying the source and root cause of the vulnerabilities identified during the testing phase.
• Assessment assigns a rank or severity score to each vulnerability based on factors including:
  • Which systems are affected;
  • What data is at risk;
  • Which business functions are at risk;
  • The ease of attack or compromise;
  • The severity of an attack;
  • The potential damage the vulnerability could allow.
• During remediation, the security staff, development and operations team work to close security gaps by determining the most effective path for remediation or mitigation for each vulnerability. Remediation steps might include:
  • Introducing new security procedures, measures or tools;
  • Updating operational configuration changes; or
  • Developing and implementing a vulnerability patch

The vulnerability assessment concludes when the final report is submitted.

While a vulnerability assessment is a process that occurs infrequently (say, once a year), a vulnerability scan is an ongoing process that continually assesses your security. Because the risk environment changes over time, your company’s security plan should include regular vulnerability assessments, vulnerability scanning, and penetration testing.

Risk assessments

Once your vulnerabilities are identified, a risk assessment then helps you to understand which vulnerabilities should be addressed first and which ones can be accepted because they’re low-risk. Risk assessments also report on your risk rating and recommend controls to reduce that risk.

Risk assessments and vulnerability assessments help you accomplish several tasks:

1. Catalog your system’s IT assets and resources;
2. Assign quantifiable value and importance to those assets;
3. Identify the vulnerabilities or potential threats to each asset;
4. Mitigate or eliminate the most serious vulnerabilities for the most valuable assets.

Together, vulnerability assessments, vulnerability scans, and risk assessments provide the foundation for risk management.

All that said, vulnerability and risk assessments have their limitations. You still need penetration testing to understand the specific weaknesses that exist in your IT systems, the damage attacks might cause, and the steps necessary to seal up those weak spots.

Penetration testing

A penetration test aims to exploit the vulnerabilities your assessments found, so you can better understand where your security weaknesses are and how to strengthen them. Finding weaknesses is one thing; exploiting them is another. Although a pen test includes an automated vulnerability scan, many skills and much time are needed to manually exploit those vulnerabilities.

The people conducting your pen test understand the nuances of how businesses and IT systems work. Their expertise lets them examine your vulnerabilities from every angle. Unlike automated scanning software, pen testers can ask questions when something doesn’t seem quite right.

A vulnerability assessment is a one-size-fits-all, high-level automated scan that flags common vulnerabilities. Cheaper and quicker than a pen test, vulnerability assessments are often mandatory for regulatory compliance. You should conduct these assessments regularly, and when activating new devices.

Although penetration tests are part of a full security audit, they’re valuable in other ways, too. For instance, they help provide assurance that your organization’s vulnerability assessments are accurate, and that your management processes work as they should.

Vulnerability assessments and penetration tests will both provide you with detailed reports explaining the findings, how critical are your vulnerabilities and why, and what you should do to close the gaps.

Penetration tests vs. security audits

Security audits

A security audit is an evaluation of an IT system or application and its risk level against a set of standards (either mandatory rules or baselines). It measures your ability to achieve the minimal acceptable level of security. An audit helps you to implement security measures consistently, specific to your industry, technologies, and processes.

Organizations most often request security audits for compliance purposes. The auditor will evaluate your enterprise IT infrastructure defenses against an established list of security standards, policies, and procedures.

During a security audit, security professionals measure how well your security protocols comply with a list of established criteria. A typical security audit will assess the following:

1. Bring-your-own-device initiatives;
2. Data- and access-related items;
3. Email;
4. Hardware configurations;
5. Information-handling processes;
6. Network security;
7. Physical configuration of the system and environment;
8. User practices;
9. Internet-of-Things devices;
10. Software configurations and patch management.

Evaluating each of the above against past and future risks, the audit should end with an in-depth report covering the strengths and weaknesses of your current security arrangements. Your security team also should stay current on the latest security trends and the measures taken by other organizations to respond to them.

Penetration testing

Penetration tests go beyond security audits. These attempt to break your system as a hacker would, often using multiple approaches. While security audits can help you identify security gaps in technology, internal processes, or organizational structures, they do not rigorously test the security of those assets.

You’ll often need a penetration test as part of a security audit, for compliance purposes. Security frameworks including the Health Insurance Portability and Accountability Act (HIPAA), the International Organization for Standardization (ISO 27001), SOC 2, General Data Protection Regulation (GDPR), FedRAMP, Cybersecurity Maturity Model Certification (CMMC), and Amazon Web Services (AWS) cloud security all have requirements for penetration testing during compliance audits, which we will examine in further detail below.

Penetration testing vs. Chaos Monkey

Chaos engineering & Chaos Monkey

Considered the next evolution of penetration testing, “chaos engineering” posits that the only way to keep online systems secure is to conduct random experiments to test overall stability. Designed to test systems’ ability to withstand turbulent and unexpected conditions, chaos engineering conducts experiments on software systems while they’re in production to build confidence in their capabilities.

Chaos engineering can help you to stay resilient amid infrastructure, network, or application failure. “Resilience” means you can keep providing mission-critical services even if you’re breached or your systems get disrupted.

Chaos Monkey, a tool invented by Netflix in 2011 to test the resilience of its IT infrastructure, works by intentionally disabling computers in Netflix’s production network to test how remaining systems respond to the outage.

Explained in the book Chaos Monkey by Antonio Garcia Martinez, the name “Chaos Monkey” refers to the idea of a monkey entering a data center, where farms of servers host all the critical functions of online activities:

The monkey randomly rips cables, destroys devices, and returns everything that passes by the hand (i.e. flings excrement). The challenge for IT managers is to design the information system they are responsible for so that it can work despite these monkeys, which no one ever knows when they arrive and what they will destroy.

By randomly selecting a server node within the company’s cloud platform and completely shutting it down, Chaos Monkey simulates the random server failures that happen in real life.

Although chaos experiments can cause some short-term harm, they provide a number of benefits to organizations by identifying larger risks that could be looming. For instance, chaos experiments can show companies how their staff might react in a time of crisis, and can help to identify bottlenecks and problem points in any incident response process.

Penetration testing

Chaos engineering is similar to penetration testing in that it can help better prepare your organization for the unexpected. Chaos experiments, however, take the fundamental principles of penetration testing one step further; they randomize the testing of system vulnerabilities rather than target specific vulnerabilities identified in a vulnerability assessment.

Soon enough, artificial intelligence and machine learning may help to automate pen testing and chaos engineering analysis and recovery. These smart systems may someday even be able to identify threats or discrepancies.

Now let’s review exactly how an ethical hacker or pen tester will conduct a test for your organization, as well as the testing strategies that pen testers may use.

Penetration testing standards

Frameworks and methodologies for conducting penetration tests include:

• The Open Source Security Testing Methodology Manual (OSSTMM)
• The Penetration Testing Execution Standard (PTES)
• NIST Special Publication 800-115
• The Information Systems Security Assessment Framework (ISSAF)
• The OWASP Testing Guide

You can choose which type of penetration test to conduct on your systems.

Penetration testing steps

Ethical hackers understand how threat actors operate, so they typically use the same hacking techniques that malicious actors use to attack an enterprise.

A pen test’s primary goal is to find vulnerabilities that a threat actor could exploit; and then inform the client of those vulnerabilities and recommend mitigation strategies. During a penetration test, a pen tester will typically conduct the following measures to evaluate the security of a system:

• Reconnaissance: the act of gathering information on a target system to be used to better attack the target;
• Scanning: using technical tools to further the attacker’s knowledge of the system;
• Gaining access: using the data gathered in the reconnaissance and scanning phases to exploit the targeted system using a payload;
• Maintaining access: working to stay inside the target environment to gather as much data as possible;
  Covering tracks: clearing any trace of the intrusion, including data and event logs.

Preparing for a penetration test

A typical penetration test will follow this pattern:

• Initial engagement;
• Scoping;
• Testing;
• Reporting; and
• Follow up.

The test’s final report should also include a severity rating for any issues found.

Initial engagement includes evaluating the ethical hacker or penetration tester to assure that they have the relevant qualifications and skills to perform testing on your IT. This stage will also include divulging information to the tester about unusual systems such as mainframes, uncommon networking protocols, or bespoke hardware.

Scoping involves establishing the goals of the test. This will include areas of special concern; the technical boundaries; and which tests will provide a full picture of your vulnerability status.

A current vulnerability assessment should be shared with the testers during the scoping stage, so they can design the test to support a reasonable opinion on the accuracy and completeness of the internal vulnerability assessment.

It’s also important to outline any issues that might affect testing. For example, perhaps you need tests to occur during non-work hours; or maybe certain critical systems need special handling. This is the time to identify those issues.

The scoping exercise should produce a plan of action stating:

• The technical boundaries of the test;
• The type of test expected;
• The timeframe and the amount of effort needed to conduct the testing;
• Scenarios or specific “use cases” to test;
• The penetration testing team’s requirements;
• Compliance or legislative requirements that the testing plan must meet;
• Specific reporting requirements;
• Time constraints on testing or reporting.

Penetration testing steps

Once you’ve outlined your expectations, testing can begin.

Make sure you have technical staff on call and available to the penetration testing team throughout testing. That way, if the team encounters critical issues or problems, your tech people can help resolve them.

Testers should take care to avoid causing undue disruption to your system. It’s impossible to guarantee a problem-free test period, but an experienced pen tester will do his or her best to assure that your systems remain intact.

During the penetration test, the testing team may suggest a change in scope. They’ll do this if they identify systems or components that were outside the initial testing scope, but that could have an impact on your system security.

Changing the scope will most likely increase the testing time and expense. Whether to proceed with the change is up to you. If you decide against it, the testers may recommend that the newly discovered components be recorded as possibly limiting your test’s accuracy.

After the penetration test

After the pen test is completed, the penetration testing team will submit a test report that should include the following:

• All security issues uncovered;
• An assessment of the level of risk that vulnerability poses to your organization;
• A method for resolving each issue found;
• An opinion on the accuracy of your organization’s vulnerability assessment;
• Advice on how to improve your internal vulnerability assessment process.

Prioritizing remediation of severe vulnerabilities

Pen testers often use the Common Vulnerability Scoring System to provide a severity rating. This is a numerical score that identifies the severity of a vulnerability.

CHECK reports can simplify this process, providing risk level assignments of “high,” “medium,” “low,” or “informational.” Most vulnerabilities will fall into one of these categories. Those that don’t will need an explanation from the pen test team as to why.

Taking action

After your test, conduct a follow-up assessment. Your organization’s vulnerability management group should assess the report just as it would the results of an internal vulnerability assessment.

Remember: The penetration testing team’s report tells you how severe your vulnerabilities are and suggests solutions. The risk assessment and decisions on how and whether to fix the problems are your responsibility.

During your assessment, you may find discrepancies. Some issues might be rated higher or lower than you would choose. Perhaps the test team didn’t have all the details about a specific system, or maybe a vulnerability’s potential business impact is higher or lower in your view. When assessing vulnerability levels, you should examine the issues and identify the risks to your organization with objective, serious analysis – not downplay issues.

During follow-up, look especially closely at previously unknown vulnerabilities the test uncovered. How can you detect similar issues in the future?

Finally, choose solutions to address the vulnerabilities found. While the penetration testers’ report will propose solutions, these may not be the only ones available. Do your homework, and choose with care.

Vulnerability risk assessment and mitigation are critical business processes that you shouldn’t leave entirely to the penetration testing team. Your own technical staff and suppliers can best provide alternatives and help choose the optimal solution or solutions for your organization.

Types of penetration testing

Black box testing

In black box testing, pen testers get no advance information about your internal operations. They work in the dark, so to speak, as a real hacker would, to find and probe your system vulnerabilities. This model of penetration testing most accurately models the risks posed by unknown or unaffiliated attackers.

When testing software, pen testers don’t need to have specific knowledge of the application’s code, internal structure, and programming. But they do usually know what the software they are testing is supposed to do.

The downside of balck box testing is this: the scarcity of information shared with the tester can mean that they don’t discover all vulnerabilities during the test.

White box testing

For white box testing, you’ll fully disclose your known software vulnerabilities and system misconfigurations to the testers before they begin. White box tests seek to confirm or correct your internal vulnerability assessment and management controls.

The goal of white box penetration testing is to simulate a malicious attack by someone with insider knowledge of your system or at least the basic credentials needed for access.

One of the primary pen testing methods, white box testing offers several advantages: It’s easy to automate, and provides clear, engineering-based rules for when to stop testing.

The downside of white box tests is that they can miss unimplemented or noncompliant aspects of your system or application. And because these tests can’t possibly allow for every scenario, some may remain untested.

White box testing focuses on the positive: on how well security works in software or a system. That also means this testing may overlook some functions that are missing.

Gray box testing

As the name suggests, a gray box penetration test is a combination of white box and black box testing. Here, the tester starts with limited knowledge of the system or app to be tested. Gray box testing is beneficial because it uses the code-targeted systems of white box testing and the more straightforward techniques of black box testing.

Gray box testing is considered to be better suited for web applications’ distributed networks or systems. White-box doesn’t work well with web applications because there’s no source code or binaries to test.

The benefits of gray box testing include:

• It contains the best of both worlds: advantages from both white box and black box testing models.
• It’s less intrusive than black- and white-box models, being based on functional specification and architectural views as opposed to source code or binaries.
• It’s unbiased, meaning that it maintains a clear boundary between the tester and developer.

External network penetration testing

External network penetration testing checks the security of your network perimeter. It tests the effectiveness of your firewalls, routers, intrusion detection systems (IDS), operating systems, and services available to the internet or untrusted networks.

Conducted from outside the organization, external network penetration testing also includes testing of web applications. The testers act like hackers who are unfamiliar with your internal system. They often work only with the IP address of the target system.

Pen testers simulate attacks on your servers, firewalls, and IDS by searching and scanning public websites to find information about your website hosts, which they then try to compromise.

Internal network penetration testing

Internal network penetration testing looks at the security of your internal networks and systems, and mirrors actual attack scenarios launched from an internal source. In this way, these tests gauge the extent to which an external attacker could roam through your internal networks. The testers can also check the security of your wireless LAN infrastructure.

Carried out from within the organization, internal pen testing can also test Intranet web applications. This type of testing helps to find vulnerabilities inside the corporate firewall.

Because attacks most often occur externally, organizations sometimes neglect to conduct internal pen tests; this is a poor choice. Internal tests can help an organization protect itself from attack by disgruntled employees or contractors who are aware of internal security policies and passwords; social engineering attacks; phishing attack simulators; and attacks using user privileges or abuse of an unlocked terminal.

A number of internal penetration tests can be conducted by accessing the environment without proper credentials, including:

• Proxy server testing;
• Spam email filter testing;
• Network firewall testing;
• Security vulnerabilities testing;
• Credential encryption testing;
• Cookie testing;
• Contact form testing;
• Open ports testing;
• Application login page testing;
• Error message testing;
• HTTP method testing;
• Username and password testing;
• Scanning files;
• SQL injection testing;
• XSS testing;
• Access permission testing;
• Testing user sessions;
• Brute force attack testing;
• Denial-of-Service (DoS) attack testing;
• Browsing directory testing.

Wireless penetration testing

Wireless penetration testing identifies and examines the connections among all devices connected to an organization’s wireless network. These devices include laptops, tablets, smartphones, and internet of things (IoT) devices. Because the penetration tester needs to be in range of the wireless signal to gain access, these types of penetration tests are typically performed on site.

Wireless penetration testing consists of six steps: reconnaissance, identifying wireless networks, vulnerability research, exploitation, reporting, and remediation. Usually, this type of penetration test is conducted due to coding mistakes, specific requirements, or a lack of knowledge in cyber attack vectors.

Penetration test strategies

Social engineering

Social engineering is a term used to describe a range of malicious activities accomplished by using psychological manipulation to dupe users into making security mistakes or giving away sensitive information.

As a penetration testing strategy, social engineering focuses on the vulnerabilities associated with the people and processes within a system. Typically, pen testers conduct a variety of social engineering attacks including phishing, impersonation, or USB drops (explained below) to identify vulnerabilities and how to remedy them.

Phishing occurs via email. The attacker tries to trick the email recipient into giving up sensitive information or opening a malicious file that can infect the user’s machine.

Vishing, which is similar to phishing, occurs via phone calls that attempt to trick the user into giving sensitive information. Smishing occurs via sms text messages, with the same strategy and intent.

Impersonation occurs when the attacker attempts to fool a person into believing the attacker is someone else. For example, the attacker might contact an IT helpdesk, impersonating a legitimate user and requesting a password reset. The goal, as usual, is to gain unauthorized access to a legitimate user’s account. Similarly, a tester may pretend to be delivery personnel and then visit the target’s physical offices, to see whether the tester can gain access to secure areas without question.

USB drops use USBs drives that contain and install malicious software. The tester “drops” the drives in workplace common areas (say, a parking lot or office lobby) in the hope that someone will plug the infected drive into his or her computer.

People are often the weakest link in cybersecurity. Hence it’s important for pen testers to identify who within a company is most susceptible to these types of attacks.

Social engineering pen tests are typically conducted both on-site and off-site. On-site tests test the physical security of a building and usually include impersonation, USB drops, and tailgating (entering an office immediately behind a legitimate employee, perhaps asking him or her to hold the door) as testing methods. Off-site tests test user security awareness and most often include phishing, vishing, and smishing.

Port scanning

Port scanning involves identifying which ports are open on a target computer, and finding out what services are running on these ports. Like any opening that allows entry into a house, ports on a computer function to allow certain services entry to the device.

Port scanners try to locate which network services are available for connection on each device by probing each of the designated network ports or services on the target systems.

When port scanning, penetration testers try to answer three questions:

1. Which ports are open?
2. What services are running on these ports?
3. What versions of those services are running?

Once these questions are answered, penetration testers can use common knowledge about which ports are open to conduct a brute force attack and gain entry into the system.

Most port scanners scan both TCP and UDP ports and can target a specified list of ports. The most popular port scanner is Nmap, which is a free and open source utility for network discovery and security auditing.

SQL injecting

One of the most common threats to data security, SQL injection is an application security weakness that allows attackers to control an application’s database. SQL injection tricks the application into sending unexpected SQL commands; that allows the hacker to access or delete data, or change an application’s data-driven behavior,

A penetration test identifies SQL injection vulnerabilities in web applications. This allows you to prevent attackers from controlling application behavior that’s based on data in the database, altering data in the database without authorization, or gaining access to data without authorization.

Antivirus evasion

Although antivirus software is one of the oldest and most widespread security controls against malicious software (“malware”), attacks have been growing more sophisticated. Hackers often can dodge antivirus software using toolkits, or evade virus signatures altogether. Likewise, penetration testers use a number of techniques to bypass your anti-virus software and gain access into protected systems.

Client-side attacking

Client-side attacks occur when a user downloads malicious content. The flow of data is the opposite of that in server-side attacks. Client-side penetration testing exploits vulnerabilities in client applications such as email clients, web browsers, Macromedia Flash, and Adobe Acrobat.

For compliance audits, penetration testing independently validates your organization’s cybersecurity. It provides evidence that you have fixed vulnerabilities that may have previously exposed your organization to attackers.

Some of the most common compliance frameworks require pen testing as part of an overall audit, especially for larger organizations seeking certain certifications.

Next we will examine pen testing requirements for a variety of compliance frameworks, including PCI-DSS, HIPAA, ISO 27001, SOC 2, GDPR, FedRAMP, CMMC, AWS cloud security, and the CCPA.

PCI-DSS

Organizations that process credit card payments are required to comply with the Payment Card Industry Data Security System (PCI-DSS) to protect cardholder data. The cost and requirements for PCI-DSS compliance depend on the size of your organization, and which level of compliance your business must meet. There are four compliance levels for merchants, two for service providers.

To become PCI-DSS compliant at any level, you must first understand the security framework. That includes:

• The three pillars of PCI-DSS
  • Focused on credit card data
  • Protecting stored data
  • Annual validation
• The six PCI-DSS goals
• The 12 PCI-DSS requirements
• The 281 PCI-DSS directives and sub-requirements
• The four PCI DSS compliance levels

After you’ve determined your organization’s PCI-DSS level and scope, you can begin to take the steps to reach compliance.

Larger organizations typically qualify as Level 1 merchants and service providers. They must meet the most stringent requirements for validating compliance, and therefore can expect the highest cost. Compliance includes passing a yearly on-site audit by a Qualified Security Assessor (QSA). It also includes quarterly vulnerability scans, annual penetration testing, employee security training, and policy development. Altogether, the whole process can take up to two years and can be quite costly.

Level 2 and 3 merchants and service providers can achieve PCI-DSS compliance in less time by foregoing the audit and instead completing a self-assessment questionnaire (SAQ) and filing an Attestation of Compliance (AOC). Many mid-sized and smaller organizations, however, still opt to comply at Level 1 to receive the highest certification.

At any level, a breach that compromises credit card data automatically moves your organization to PCI-DSS Compliance Level 1. For Level 1 compliance, assume that you will need to conduct annual penetration testing and quarterly vulnerability scans in addition to an audit by a QSA.

The requirements for organizations that do need penetration testing include passing a scan every 90 days and submitting to additional testing if there are any changes to the cardholder data environment (CDE).

Specifically, PCI-DSS Requirement 11.3 requires that your organization have a penetration test done. This clause also provides general guidance and guidelines for penetration that includes:

• Penetration testing components
• Qualifications of a penetration tester
• Penetration testing methodologies
• Penetration testing reporting guidelines

PCI-DSS penetration testing requirements

The PCI-DSS standard talks about pen testing across several of its requirements. Among them:

• 6.1 – Identify security vulnerabilities in your internal and external applications by using reputable outside sources for security vulnerability information and assign a risk ranking (e.g. high, medium, or low) to each vulnerability.
• 6.2 – Ensure that all software and system components are protected from known vulnerabilities by installing any applicable security patches. You must install the patches within the first month following their release.
• 6.6 – For public-facing web applications, address new threats and vulnerabilities on an ongoing basis and ensure these applications are protected against known attacks.
• 11.3.1 – Conduct external penetration tests at least once a year and after any significant changes or upgrades to the infrastructure/application. (For example, upgrading the system, adding a subnet, or web server to the environment, etc.)
• 11.3.2 – Conduct internal penetration tests at least once a year and after any changes or upgrade significant infrastructure or the application. (For example, an upgrade of the operating system or adding a subnet or web server in the environment.)
• 11.3.3 – Vulnerabilities found during the penetration tests must be corrected and additional testing performed until the vulnerabilities have been corrected.
• 11.3.4 – If segmentation is used to isolate the CDE from other networks, this requirement mandates a penetration test at least once a year and again after modification of the methods/controls of segmentation to verify that the segmentation methods are operational and effective.

Passing a penetration test indicates that your IT controls are in working order and that the security protections you have in place are satisfying the standards required for your organization. It means that the tester was unable to exploit any aspects of your system. If your organization fails to pass an initial penetration test, you must correct the failure as quickly as possible and conduct another test to prove compliance.

It’s critical that you resolve any security issues that a penetration test might uncover. Non-compliance with PCI-DSS carries serious consequences including the loss of your credit card processing privileges, so it’s important to pass your penetration test every time.

HIPAA

The Health Insurance Portability and Accountability Act (HIPAA) seeks to protect personal health data, where medical records are worth upwards of $250 each. To protect patient health information and your organization’s bottom line, it is increasingly important that covered entities and their business associates become HIPAA compliant.

The U.S. Department of Health and Human Services (HHS) implemented five separate sections covered in HIPAA, including the following:

• The Privacy Rule outlines how healthcare providers can use patient data, what they can disclose without the patient’s permission and to whom, and the “Right to Access.” Essentially, the HIPAA Privacy Rule requires organizations to secure Protected Health Information (PHI).
• The Security Rule outlines how organizations should handle, maintain, and transmit PHI securely. This rule requires healthcare organizations to have three types of data security safeguards in place: administrative, physical, and technical.
• The Omnibus Rule more clearly defines the role of business associates. It introduces new provisions required by the Health Information Technology for Economic and Clinical Health (HITECH) Act to strengthen HIPAA security and privacy protections and to increase the legal and financial liability for non-compliant organizations.
• The Breach Notification Rule requires covered entities and business associates to notify HHS when PHI has been breached, and distinguishes between “minor breaches” and “meaningful breaches.”
• The Enforcement Rule empowers HHS to enforce the Privacy and Security Rules, including the authority to investigate HIPAA complaints, conduct compliance reviews, perform education and outreach, and levy fines of up to $1.5 million.

Within the text of HIPAA, Title II is the section focused on protecting individual medical information. Achieving compliance with HIPAA requires meeting the guidelines in Title II, which includes the Security Rule.

The Security Rule focuses on safeguarding and protecting PHI, especially electronically protected health information (ePHI). It requires that covered entities evaluate risks and vulnerabilities in their environments.

Although HIPAA regulations don’t expressly require penetration testing, they do require the implementation of security controls to address any risks and vulnerabilities.

HIPAA penetration testing requirements

164.308(a)(8) Standard Evaluation requires a covered entity or business associate to perform a periodic technical and nontechnical evaluation. The evaluation helps to establish a process for reviewing and maintaining reasonable, appropriate security measures to comply with the Security Rule.

The initial evaluation must be based on the security standards you’re using to comply with the security rule. You must then perform subsequent periodic evaluations in response to environmental or operational changes that affect the security of ePHI. Periodic evaluations should be performed on a scheduled basis, and the evaluation must include reviews of the technical and non-technical aspects of the security program.

Penetration testing is not explicitly required for periodic evaluations, but “technical testing” is typically defined as performing a vulnerability assessment or a penetration test. Passing these is the easiest way to demonstrate that you’ve properly implemented the technical controls mandated by your policies and procedures in compliance with the HIPAA Security Rule.

To better prepare your organization for a HIPAA audit, take the following steps:

1. Focus on HIPAA training for employees.
2. Create a risk management plan and conduct a risk analysis.
3. Select a security assessment and privacy officer.
4. Review policy implementation.
5. Conduct an internal audit.
6. Create an internal remediation plan.

ISO 27001

ISO 27001 was developed to provide a model for “establishing, implementing, operating, monitoring, reviewing, maintaining and improving an information security management system (ISMS), or a framework of policies and procedures that includes all legal, physical and technical controls involved in an organization’s information risk management processes.”

One of the most requested standards in a business partnership context, ISO 27001 details a specific course of action for organizations to secure their assets. It encompasses 114 controls to implement.

ISO 27001 uses a top down, risk-based approach and is technology-neutral. The specification defines a six-part planning process:

1. Define a security policy.
2. Define the scope of the ISMS.
3. Conduct a risk assessment.
4. Manage identified risks.
5. Select control objectives and controls to be implemented.
6. Prepare a statement of applicability.

Penetration tests are an important part of the risk management process in ISO 27001, and validate that your security controls are working as designed.

ISO 27001 penetration testing requirements

Control A.12.6.1 requires an organization to manage technical vulnerabilities to prevent their being exploited. That includes steps such as:

• Defining and establishing roles and responsibilities related to technical vulnerability management, including vulnerability monitoring, patching, vulnerability risk assessment, and asset tracking.
• Identifying the information resources you need to find relevant technical vulnerabilities and maintain awareness about them.
• Defining a timeline to react to potentially relevant technical vulnerabilities.
• Identifying associated risk and actions to be taken after locating a potential technical vulnerability.
• Carrying out the relevant actions, depending upon how urgently a technical vulnerability needs to be addressed.
• Assessing the risks associated with installing a patch, even when the patch comes from a legitimate source.
• Testing and evaluating patches before they are installed on systems so that they introduce no unintended consequences and are effective.
• Maintaining audit records.
• Monitoring and evaluating the technical vulnerability management process.
• Defining a procedure to address a situation when an identified technical vulnerability has no suitable countermeasures.

Other assessment techniques may be used to satisfy the ISO 27001 requirements, but penetration testing leaves little wiggle room. It allows you to address your identified risks, stay on top of the latest attack methods, and gain a reliable perspective on your cybersecurity risks.

SOC 2

Developed by the American Institute of CPAs, System and Organizational Controls for Service Organizations 2 (SOC 2) defines criteria that a company can use to assess the cybersecurity of its service providers. SOC 2 audits are based on five “trust service principles”: security, availability, processing integrity, confidentiality and privacy. There’s no certification for SOC 2, but rather an auditor’s opinion. Nor is there a defined list of boxes to check to become SOC 2-compliant; each audit depends on the relationship the company wants with its service provider.

Rather than using a defined control set, SOC 2 specifies criteria for which adequate controls must be designed:

1. Security. Information and systems are protected against unauthorized access, unauthorized disclosure of information, and damage to systems that could compromise the availability, integrity, confidentiality, and privacy of information or systems and affect the entity’s ability to meet its objectives.
2. Availability. Information and systems are available for operation and used to meet the entity’s objectives.
3. Processing integrity. System processing is complete, valid, accurate, timely, and authorized to meet the entity’s objectives.
4. Confidentiality. Information designated as confidential is protected to meet the entity’s objectives.
5. Privacy. Personal information is collected, used, retained, disclosed, and disposed to meet the entity’s objectives.

SOC 2 penetration testing requirements

Penetration testing and vulnerability scanning aren’t expressly required to gain SOC 2 compliance. The audits, however, are typically based on the COSO framework for internal control, including Principle 16 of that framework:

“The entity selects, develops, and performs ongoing and/or separate evaluations to ascertain whether the components of internal control are present and functioning.”

Further, Principle 16 includes this “point of focus:”

“Management uses a variety of different types of ongoing and separate evaluations, including penetration testing, independent certification made against established specifications (for example, ISO certifications), and internal audit assessments.”

While this may seem explicit, there are multiple types of evaluations that could be used to satisfy this criteria; penetration testing is one of them among several. For example, your SOC auditor may deem ISO certification enough to satisfy the requirement, and may not call for penetration testing.

That said, consider the risks involved should you choose not to conduct a penetration test. A SOC 2 audit is an industry standard for evaluating infrastructure integrity. Working with audit firms that don’t require vulnerability scans or penetration tests as part of the process can seem questionable to others.

GDPR

The General Data Protection Regulation (GDPR) framework is a law enacted by the European Union (EU) to protect EU citizens’ data from unauthorized use and gives them full control over their privacy.

Any organization operating within the EU, as well as any organizations anywhere in the world that offer goods or services to customers or businesses in the EU, are required to be GDPR compliant.

Penetration testing is expressly required for GDPR compliance. Testing allows an organization serving EU citizens to verify the security of its data processing systems and assures that the organization meets GDPR requirements.

Penetration testing allows organizations to see how their consumers’ data could be compromised. Testers provide practical solutions so you can address these risks actively, in accordance with GDPR requirements.

GDPR penetration testing requirements

Article 32 requires a “process for regularly testing, assessing, and evaluating the effectiveness of technical and organizational measures for ensuring the security of processing.”

Although this requirement is only a small portion of the whole GDPR, it’s an important one for security. Companies that adhere to Article 32 can more easily prevent cybersecurity incidents and avoid large GDPR fines that, in the event of a breach, can climb into the millions of euros.

The GDPR’s 72-hour mandatory breach disclosure provides another incentive for penetration testing. Penetration tests can discover vulnerabilities and even threats before they occur – so you have nothing to disclose.

According to the United Kingdom’s Information Commissioner’s Office, which enforces the GDPR in Britain, organizations must “run regular vulnerability scans and penetration tests to scan your systems for known vulnerabilities – make sure you address any vulnerabilities identified.”

FedRAMP

The Federal Risk and Authorization Management Program (FedRAMP) is a U.S. federal government program that provides a standardized approach to security assessment, authorization, and continuous monitoring for cloud products and services that government agencies might use. Achieving FedRAMP compliance allows a cloud service provider to bid on federal government contracts more efficiently.

For FedRAMP certification, you’ll need to continuously monitor your systems. FedRAMP also requires “offensive testing” – that is, penetration testing – on applications in the cloud.

FedRAMP penetration testing requirements

FedRAMP requires penetration testing as part of the initial authorization assessment for all cloud service providers seeking a “moderate” or “high” FedRAMP authorization.

The FedRAMP Penetration Test Guidance document provides explicit guidelines for penetration testing, and includes the following:

1. Scope
2. Definitions & threats
3. Attack vectors
4. Scoping the penetration test
5. Penetration test methodology and requirements
6. Reporting
7. Testing schedule requirements
8. Third-party assessment organization staffing requirements

To ensure that the penetration testing you undertake is fully acceptable, FedRAMP offers the following recommendations:

• Ensure proper scope. The scope of a penetration test is important and must align with the system’s authorization boundary. If the scope is too small, the testing may be invalid. If the scope is too large, the test team may inadvertently infiltrate a system for which they don’t have authorization. Pen test scope is identified and documented in a Penetration Test Plan (PTP).
• Underlying providers are out of scope. If your SaaS system resides on a FedRAMP-authorized PaaS or IaaS, only your SaaS components are within scope. There’s no need to perform a pen test on the PaaS or IaaS your system sits on top of.
• Some underlying providers have procedures that must be followed. If your SaaS system resides on an underlying PaaS or IaaS, you may need to notify or obtain approval from the underlying provider before performing activities such as penetration testing or scanning. Always check your service provider’s latest policies and procedures. Engage with your PaaS or IaaS provider early in the acquisition phase to assure that you will be able to perform the appropriate penetration testing when the time comes.
• Reporting should be comprehensive. Penetration test assessment activities and results must be organized and compiled into a comprehensive Penetration Test Report (PTR). The PTR includes the actual tests performed and the results of those tests, the access paths utilized, as well as test findings and evidence.
• Be sure to address all FedRAMP-identified attack vectors. FedRAMP-acceptable penetration tests must comprise internal and external threats from trusted and untrusted sources. It should include attempted exploitation of any web applications, mobile applications and the system’s network within the system’s boundary. For multi-tenant systems, penetration testing must include a trusted tenant attempting to cross tenant boundaries.
• Non-production test environments must be configured exactly the same as the production environment. Penetration testing in a productive environment isn’t always feasible or tenable for a variety of reasons. A test environment that is configured to mimic the production environment exactly can be used in these circumstances to facilitate penetration testing. Similar or like environments are not sufficient.

  FedRAMP requires that test and production environments be mirror images to assure that all findings are applicable to the production environment, and to minimize the chance of false-negatives. Tenant-to-tenant testing is a good use case for performing penetration testing in a non-production environment.

• Social engineering must include a phishing test. At a minimum, all CSP staff with logical access to the system must be included in the phishing email distribution. Staff should not be warned of a phishing test.

CMMC

The Cybersecurity Maturity Model Certification (CMMC) framework includes many cybersecurity processes and best practices, drawn from a variety of sources.

The CMMC model includes three maturity levels designated ML 1 through ML 3. Each level has progressively greater compliance requirements based upon NIST 800-171. The requirements related to penetration testing are:

RM.2.142. Scan for vulnerabilities in organizational systems and applications periodically and when new vulnerabilities affecting those systems and applications are identified.

CA.3.162. Employ a security assessment of enterprise software that has been developed internally, for internal use, and that has been organizationally defined as an area of risk.

CA.4.164. Conduct penetration testing periodically using automated scanning tools and ad hoc tests using human experts.

AWS cloud security

Amazon Web Services (AWS) provides more than 90 cloud hosting services, including storage, content delivery, security management, network infrastructure, and physical hosting facilities for tenant organizations. Most commonly, AWS is used for networking, data storage, web application services, and code development.

Cloud services give organizations and individuals the ability to scale their web service needs quickly and effectively on a flexible and reliable platform, and transfers to the cloud provider the maintenance and upfront fixed costs of network-connected hardware.

While your organization’s configuration of the AWS platform and the additional application code or assets in your environment can be pen tested, the AWS platform on which you build your environment cannot be pen tested.

For user-operated services including cloud offerings configured by users, AWS permits organizations to test their AWS EC2 instance fully as long as the tests don’t disrupt the service. For vendor-operated services, or cloud offerings managed and configured by a third party, AWS restricts penetration testing to configuration and implementation of the cloud environment, excluding the underlying infrastructure.

AWS allows penetration testing without approval of the following services:

• Amazon EC2 instances, NAT Gateways, and Elastic Load Balancers
• Amazon RDS
• Amazon Cloudfront
• Amazon Aurora
• Amazon API Gateways
• AWS Lambda and Lambda Edge functions
• Amazon Lightsail resources
• Amazon Elastic Beanstalk environments

You may not conduct penetration testing of:

• DNS zone walking via Amazon Route 53 Hosted Zones
• Denial of Service (DoS), Distributed Denial of Service (DDos), Simulated DoS, Simulated DDoS (These are subjected to the DDoS Simulation Testing policy)
• Port flooding
• Protocol flooding
• Request flooding (login request flooding, API request flooding)

AWS Cloud Services and penetration testing

Because Amazon owns the core infrastructure, pen testing methods of AWS Cloud Services will differ from the tests of traditional security infrastructure. AWS tests must focus on user-owned assets, identity and access management user permissions configurations, and use of AWS APIs.

Performing a pen test within the cloud requires adequate preparation, including:

1. Defining the scope, including the AWS environment and target systems.
2. Running a preliminary test.
3. Determining the type of penetration test you would like to conduct.
4. Outlining the expectations for both internal stakeholders and the penetration testing team or company.
5. Establishing a timeline for the technical assessment to occur, receiving formal reports, and potential remediation and follow-up testing.
6. Developing the protocol and rules of engagement if the penetration test reveals the client may have already been breached or is under an ongoing attack.
7. Obtaining written approval to conduct the test by the client (and other third parties that may be involved). This includes:
   1. Fill out a penetration test request form.
   2. Tell AWS the dates that testing will take place.
   3. Tell AWS the IP Address range the scan or penetration testing will come from.
   4. Tell AWS the IP Address range being tested (scope).

CCPA

The California Consumer Privacy Act (CCPA) is a California law intended to enhance privacy rights and consumer protection for California residents. This includes the rights to:

• Know what personal data is being collected about them;
• Know whether their personal data is sold or disclosed, and to whom;
• Decline the sale of their personal data;
• Access their personal data;
• Request a business to delete any personal information about a consumer collected from that consumer;
• Not be discriminated against for exercising their privacy rights.

The CCPA applies to all businesses that collect personal data of consumers and does business in California, and also satisfies at least one of the following requirements:

• Annual gross revenue of more than $25 million;
• Possesses the personal information of 50,000 or more consumers, households, or devices;
• Earns more than half of its annual revenue by selling consumers’ personal information.

CCPA penetration testing requirements

In Chapter 55, Section 1798.150, CCPA specifies the following:

Any consumer whose nonencrypted or nonredacted personal information … is subject to an unauthorized access and exfiltration, theft, or disclosure as a result of the business’ violation of the duty to implement and maintain reasonable security procedures and practices appropriate to the nature of the information to protect the personal information may institute a civil action for any of the following.

Although the CCPA doesn’t address pen testing explicitly, the phrase “reasonable security practices” in the above section suggests that if your organization needs to comply with the CCPA, it should conduct penetration testing to maintain a reasonable level of security for your technical infrastructure.

If you’re worried about costs: a pen test will likely cost you less than the penalty for CCPA noncompliance, which is up to $750 per consumer whose information gets breached (on top of legal costs and other expenses).

Other Regulations

Other compliance frameworks that require penetration testing include NIST, FFIEC, NYDFS (23 NYCRR 500) and FINRA, to name a few.

Regardless of the specifications laid out to achieve compliance with a certain regulation, penetration testing should be considered a best practice to prevent your organization from falling victim to a cyberattack.

While requirements differ among frameworks, penetration testing consists of much more than simply checking a box. Even if you’re not seeking certification or compliance, a pen test can give you and your customers peace of mind, and can save your organization time and money.

Now that you know the ins and outs of penetration testing, let’s take a closer look at why these tests are important, and whether conducting a penetration test is the right decision for your organization.

Why conduct a penetration test?

Conducting a penetration test can be beneficial to your organization in a number of ways. Typically, a pen test is requested when a system or network has exhausted its investments in security and an organization wants to verify that its bases are covered.

Penetration tests are also useful for prevention, too. They can help to assure that vulnerabilities in your system have been identified and remediated, for tighter security. Like a financial audit, where your finance team tracks expenditure and income day to day, penetration tests help to confirm that your internal security processes are sufficient.

You may plan to use the findings to report on or improve your organization’s internal vulnerability assessment and management process. But penetration testers may find subtle issues that your internal processes might have missed.

How much does a penetration test cost?

A penetration test can cost as little as $4,000 or as much as $100,000. On average, a high quality, professional pen test will cost $10,000 to $30,000.

Factors that can affect the cost include:

• Size. The smaller the organization, the less a penetration test will cost.
• Complexity. The more applications, devices, and systems that need to be tested, the higher the cost. For organizations that have mobile apps, internal and external servers, and more complex computer systems, the cost will be higher. How complex the pen test needs to be depends on the number of networks, applications, IP addresses, users, facilities, and so forth.
• Scope. Set a clear scope before a test begins to assure that the cost does not get out of hand. Identifying specific elements you want to be tested will help to keep the price down.
• Methodology. The tools and practices a pen tester uses can increase the cost of a pen test. That said, a more expensive tool or slower methodology may produce higher quality results. It may be helpful to conduct a more thorough test the first time, then reduce the methodology for following tests.
• Experience. Your tester’s level of experience will help determine the cost of the pen test: the more experience, the higher the price, in most cases.
• External vs. internal testing. Most pen testing is performed off site, but onsite or internal tests may cost more.
• Remediation. The amount of information in the pen test report can also affect your overall costs. Penetration testers who suggest ways to remediate gaps are often more expensive than those whose reports don’t contain those suggestions.

Calculating ROI on penetration testing

To communicate the value of pen testing to your managers, C-suite, and board, you’ll need to offer comparisons of the test’s costs against noncompliance penalties and the harm a breach imposes on your business functions and bottom line. You can also count the money you save remediating vulnerabilities more quickly as a result of the test.

But pen testing provides value beyond compliance and breach prevention, even through a financial lens. You might point this out, as well.

Understanding pen testing’s critical role in improving your security program’s maturity can help you identify issues while they’re developing, rather than spending more to resolve those issues later.

Working toward security maturity means future pen tests can focus on assuring the effectiveness of your existing controls and of the security touchpoints in your development cycle. That’s much more than checking a compliance box.

By identifying metrics rather than measurements, you can avoid misleading data that might cause you to squander time, effort, and budget on the wrong activities. Security teams should focus on vulnerability density trends, pen testing coverage, the ratio of open to remediated vulnerabilities, the costs related to remediation efforts, and the costs of building a secure application.

Using these metrics in collaboration with finance teams will allow you to calculate a tangible ROI for pen testing and broader security programs.

Preemptive cybersecurity activities such as penetration testing have many benefits, and can potentially make your organization more money in the long run by giving your customers peace of mind, preventing attacks, and developing better software overall.

What’s the risk of not penetration testing?

One of the biggest risks of not performing a penetration test lies in regulations and compliance. In addition to a hefty fine (plus legal fees and other internal costs), not complying with regulations could jeopardize your license to operate. In some egregious circumstances it can even lead to jail time.

Data privacy is becoming more important to regulators across the globe. While pen testing may not directly address data privacy, it can help reduce the risk of a data breach by identifying software vulnerabilities.

If a data breach does occur, your organization’s reputation may be at stake. A loss of customer confidence can lead to a drop in revenue and profit, including a potential drop in your organization’s share price.

Should your organization’s data find its way into the hands of rival companies, competition could become a big risk to your organization. While your competitors may not be the ones behind a cyberattack, cybercriminals often publish information gleaned from breaches on public websites or sell that information on the dark web.

While a penetration test cannot completely protect you from these risks, it can help you to mitigate the threats your organization may face.

Penetration Testing and Reciprocity ROAR

Whether your organization is seeking certification from a compliance framework, or simply wants to conduct a penetration test for peace of mind, Reciprocity ROAR makes pen testing less stressful for you and your team.

Our software-as-a-service can prepare you to scrutinize your networks and systems against all the most critical and widely used compliance frameworks. Its color-coded dashboard shows at a glance how to fill compliance gaps, and updates automatically as the framework changes.

Reciprocity ROAR also provides industry standard templates to help you design the necessary controls around your cardholder data environment (CDE).

It performs internal audits as often as you desire, with just a few clicks. It even collects and stores penetration test and audit findings along with your other audit trail information in a “single source of truth” repository; no more hunting and scavenging for documentation.

In addition to providing basic guidelines for implementing and managing your compliance framework, Reciprocity ROAR also provides documented best practices for you and your organization. It even gives you updates and in-the-moment insights regarding the threat landscape so you can respond so you can respond to changes in threats and vulnerabilities.

Our platform can integrate with several vulnerability scanner applications, such as Qualys, to streamline the assessment of controls when you need to show that you have an effective vulnerability management program.

Suddenly, penetration testing just got a lot simpler. With the Reciprocity ROAR platform, you can let the software do the compliance heavy lifting, and turn your attention to keeping your customers and clients happy.