Handle With Care: You Have My VA Report!

Editor's Notes

• #2: Handle With Care: You Have My VA Report!
• #3: I’m Nivedita Murthy. I’m a Cigital Security Consultant with 7+ years of application security and information security experience. My area of expertise is vulnerability remediation.
• #4: Who are you? Organization. Are you an organization looking to buy an application to carry out certain functionalities within your company? Do you know what you should be looking for in an assessment report? How do you deal with these findings? Product Company. Do you belong to an organization that has developed a product which is being used by multiple companies? How can you create a standard report that you can share with all of your clients meeting most of their application security requirements? At the same time ensuring critical, sensitive information does not get shared in the process? Guideline. My aim today is to create a guideline that organizations can follow while reaching a consensus in terms of the application assessments and findings.
• #5: How should the report be structured? Why? This resource provides a great deal of insight.
• #6: Project Objectives. Who did the assessment? You can’t just have any Tom, Dick, and Harry come and say that they have done this assessment and accept that as a valid assessment, right? Details of the assessor help organizations determine the quality of assessment (are they leading vendors in carrying out such assessments?). It also helps in the legitimacy and trustworthiness. How do we know such assessors? Here’s how: Gartner Magic Quadrant- http://techbeacon.com/resources/magic-quadrant-application-security-testing-gartner The other important information that you should be looking out for in this section is what kind of assessment was done. Was it a black box/DAST, white box/SAST, architecture review, threat modeling? Was it just manual or fully automated? This would actually depend on what are you using the application for. What is the data confidentiality of the information it will be handling? Is it something that would be used infrequently or once a quarter (e.g., surveys, polls)? In some scenarios, the report from automated scan should suffice. However, if it is an application that is used enterprise-wide and/or deals with sensitive information, a thorough test using automated tools as well manual testing needs to be considered.
• #7: Project Schedule. When was the last assessment conducted? You really can’t rely on a 5-year-old assessment. It is recommended that the report be no more than a year old considering how frequently we hear of new attack methods and an average release cycle to be six months to one year. However, the answer to this question changes if the product version is significantly higher (2 or more) than the one that was tested. It could be that this product may have had frequent releases (though a rare case). The duration of the assessment also matters. If it is just 2 days, did it cover all functionalities depending on the size of the application? Maybe the assessment was done just to check the top 5 vulnerabilities and not everything (This should ideally been mentioned in the project objectives.) Compare the application size and findings to determine if the assessment duration looks sufficient.
• #8: Targets. What was marked under scope? It needs to list all domains, functionalities, modules, URLs, and port numbers that were part of the assessment. This helps to determine if the modules and functionalities that your organization heavily relies on were included in the assessment or not. Did it include application server and database testing? Or any of the outbound streams or APIs connecting to the application? If you are making use of third-party hosted Web application to enter employee information, you definitely want to know where the data is being stored and if the database is vulnerable or not. The same goes for the server. Such an assessment would definitely go beyond the scope of a pure application assessment. However, it is imperative from an organization’s standpoint as to where their data is being processed and stored. Some applications also have backend reporting capabilities where certain “usage statistics” are sent back to the host company. How is this connection established and what information is sent? Does it include details on who logged into the application and from where (internal IP addresses is not something you want to disclose)?
• #9: Limitations. This traditionally includes modules, domains, and functionalities that weren’t part of the assessment’s scope. Organizations should verify if any of these apply to you. While some modules could have been included as part of testing, due to time constraints, limited application availability, and access issues, some of them could not have been tested. This section of the report should lists these limitations.
• #10: Findings Summary. A summary of critical, high, medium, and low vulnerabilities often gives executives insight into where to focus the security budget. It doesn’t exactly comfort an organization to know the possible ways they may be attacked. What are the good points? What flaws weren’t discovered (e.g., were there no occurrences of SQL Injection and XSS)? Does the critical vulnerability have to do with a hardcoded password or an XSS attack? Where was this found exactly? Is it something that your organization uses heavily? Is it something that can be switched off or disabled by an organization if not being used? Wait a minute. I am from a product company? I can’t share information on where exactly the flaw was found and how… That’s very specific and sensitive information. What do I do? Here is what you should be including in the report: 1) Vulnerability title and description 2) Risk level 3) Which module (without giving details on exact parameters and fields) Two options to hide details: delete it or mask it.
• #11: Appendix. What methodology was used for testing? This section should lists the tools and scripts used for testing. Were any SAST/DAST tools used to run automated checks? What were they? Did the assessor follow any specific checklist? Which attacks were checked? This gives you great deal of information on what kind of assessment was done and helps you determine if the assessment meets the organizations standards. What was the risk rating model used by the assessing team to determine the risk level of the flaws. STRIDE/DREAD/CVSS/NIST 800-30 r1 ? Or a company standard. The matrix should be listed to give an idea on how the risk levels were calculated. Lets say an SQL injection flaw was listed as a medium severity. However, the organization doesn’t agree on this risk level. The model should help them understand how this was calculated.
• #12: Remediation Plan. Now that we have discovered the vulnerabilities, the next step is to fix them. The higher the criticality, the faster they should be resolved. Does the product company plan to fix it? Has it been fixed? Check the version of your software against the one that was tested. The vendor should also give a brief description of how it is going to be fixed. Will they be applying framework-level controls rather than doing a spot fixes? (Ensuring possible future occurrences of flaws can be prevented with these controls.) Is the remediation relevant with current standards?
• #13: This section is more relevant for the vendor than the organization itself.
• #14: Password-protected PDF over secure mail. The sanitized report can be shared with other organizations by sending the password protected PDF via secure mail. The password needs to be sent securely within a separate piece of mail. It would be at the organization’s discretion as to how they would like to share the report internally. This reduces the overhead of distribution from the product company’s hands. On the other hand, the product company will have no visibility as to how many people with whom the report was shared.
• #15: Online hosting. The product company can host the report in a file hosting and storage site and give limited (view only, no download) access to other members of that organization. Here, the product company has full control on who can view this report and can keep logs on when was it accessed, from where, and by whom. However, they have to maintain an externally-facing document sharing portal to ensure availability along with the added tasks on approving access.
• #16: On-site visit. This is the most drastic of all options if you are concerned about the report even after it was sanitized. The only pro in this case is that the actual report will never be available to your clients at their discretion. The disadvantage here is that the product company has to bear out the expenses of getting a representative of the organization visit onsite and read it physically.
• #17: These are side notes that I am listing out when such assessments are shared. This is typically a sub-set of how vendor assessments are carried out. In addition to the report being shared, an organization needs to check on the following points:
• #18: Remediation and re-testing. The vendor may have committed dates by which they aim to resolve the identified vulnerabilities. Re-test needs to be conducted to ensure that these vulnerabilities have in fact been resolved. This requires a follow-up on the part of the organization.
• #19: Secure development practices. The vulnerabilities may get resolved, however the organization needs to determine if there are steps or procedures in place to ensure new issues don’t crop up in future releases. The organization needs to check if the developers of the vendor are being trained in secure development practices. Also, do they have any internal security testing practices to ensure that vulnerabilities are identified and fixed as early as possible.
• #21: For more information, visit us at www.cigital.com.