Retrospective view for false-positive detection

Helping a security team to analyze the configurations of their SAST scans and optimize their results.

Role and Project overview

Role: Solo Product designer.
Time: Early 2018.

This project was a very short-term engagement, done intensively over a few weeks, collaborating with team of two developers, team-lead and the organization's CISO. Being an internal tool, ment to serve a very specific need, it was not a part of larger product, nor intended to be used by external users.

The Problem:

False-positive scan findings (often called “Noise”) are major concern to Application Security teams, creating both significant load of work, as well as shifting attention from potential true-positive threats. The noise filtering work is usually a combination of automated solutions and human handling, amount of each varying according to the case. The client’s security team wanted a clear visibility for the affect of their scans configurations, to optimize their scans and improve their management.

The Goal:

Improve scans configuration (findings predictability).
Improve scans optimization (findings accuracy).
Construct a continuous visual narrative to deepen understanding of causes and effects among the security team in the company.

Guiding principles and technical requirements:

a Single view application.
On premise tool, browser-based. Desktop only.
Prioritize macro insights. include micro as possible.
View whole process at single glance.

Context of use:

Every scan is driven by a unique set of configurations, tailored for the specific application, technologies and threat modeling. Reducing noise in scans without compromising true-positives detection is a continuous process of repeating adjustments. The retrospective view's purpose is provide clear understanding of the factors behind each scan false-positive detections.

The Solution:

As multiple solutions were considered, a Sankey diagram seemed like the best choice to serve the need:

It shows the entire process in single glance
It visualizes quantitative differentiations
It's perfect to present a process of gradual reduction. (See Minard's chart of Napoeon's 1812 Russia campaign)

Early considered options:

The initial options, considered along with the Sankey diagram were a node-based chart and a steps chart. While the node-based structure was much more modular and made for generative trial-and-error, it did not visibly convey quantities, and our general concern was that it will bias users too strongly to a mode of creation rather than observation, missing the point of hte retrospective.

The steps chart, on the other hand, while being strictly more observational by nature- and significantly more affordable to implement- was too rigid and did not allow for the trial-and-error flexibility we did wish to provide. It was also less intuitive to read, giving more emphasis to units rather than the whole flow.

As a Sankey solution was selected, we have set the entire layout on columns, each representing different layer of the filtering process, with affordance allowing editing the filter configurations as well as adding an additional phase(column)

Noise detection retrospective: Layout in rough areas

The Sankey (selected option):

The Sankey diagram provided the best balance between both observational and generative elements, and it conveyed clear, intuitive and readable narrative.

By selecting fractions of the scanned files, it could provide focus state, showing the corelated vulnerabilities and their respective configurations, in relation to the overall results.

Noise detection retrospective: Focus on selected files

Each column in the diagram, representing a stage in the filtering process, could be configured, to simulate the estimated effect of removing or adding attributes to the scan configuration.

Noise detection retrospective: configuration

The retrospective tool initial release was a success. As my part in the project was completed upon shipping, it was further developed and integrated into the client's internal security testing workflow.

Back to portfolio

Contact:

Tel/ Whatsapp: +351-91-3146324
Email: avin@avinvadas.com
Let's talk: Check my Calendly