In this article:
SecurityScorecard’s scoring update involves a monthly recalibration, which is an update to the baselines that we use to calculate our scores in order to reflect changes to companies, the cybersecurity landscape, and the internet. Our scoring algorithm is a measure of the typical number of findings for an organization versus their digital footprint size.
How we calculate the score
The score is based on how many standard deviations an organization is better or worse than the average number of findings for an organization of a particular digital footprint size. We recalibrate to make sure that the average number of findings of issues and factors is up to date.
We update the scoring calibration monthly using two months worth of data. We believe this is the best option for customers because the expected score impact of recalibrations is minimized and this allows for frequent improvements to the platform. The monthly cadence also guarantees our commitment to unbiased and fair scores.
SecurityScorecard provides users a top-level score and factor-level scores for their own company and those that they monitor in their portfolios. A company’s top-level score consists of all the factor scores combined and weighted to capture an accurate representation of the cyber threat landscape, while factor scores consist of different issue types. As we add or remove issues and information with recalibration, certain factor scores may change while others stay the same. Changes in a company’s factor-level score will lead to a change in their top-level score.
The preceding figure the calibration for the open resolver finding (open resolver is a finding where a DNS service is improperly configured in such a way that allows it to be abused for malicious purposes). For both axes we take the logarithm base 10 to accommodate the several orders of magnitude spread in the data. Therefore the 3 on the x axis means 10^3 or 1,000 IPs. The hexagonal bins are color coded based on how many companies fall into that area.
Hexagons that are red have more data points than hexagons that are blue. The red dots are the average values calculated for various ranges of IPs. The pink line represents the average value of findings for an organization of any digital footprint size. The orange lines are the average plus the standard deviation and the average minus the standard deviation respectively. When we calibrate we calculate the pink and orange lines.
Why are scoring updates important?
By keeping the calibration up to date, SecurityScorecard provides customers with scores that continuously reflect the true nature of the cybersecurity landscape. This prevents scores from drifting over time as the number of findings on the internet moves away from the historical baseline.
Additionally, scoring updates allow us to add new issue types, retire old ones, and re-weight the severity of issues which make scores more predictive of negative outcomes such as breaches and malware. We cannot add new issue types without understanding the baseline to score against, which we get from recalibration.
Factor scores are a weighted combination of issues, which are also recalibrated. Each issue is contributing some expectation value to the factor score. If we remove an issue, we need to recalibrate the factor without it.
Expected impacts
When there are no changes to issues and weights, we can expect the score changes to be less significant than when we add in new issues or re-weight the severity of issues.
Portfolio impacts
Since scoring updates affect all Scorecards in the platform, users will also experience score changes within their portfolios. Additionally, the reports, analytics, and other features based on portfolios scores might change.
What to look for in the platform
In platform banner
For users that will experience a change, in platform banners will communicate the potential score impact.
Calibration icon
Select My Scorecard from Scorecards in the top menu. Mouse over of the scoring update calibration icon (wrench) under your score on your Scorecard page provides information on your potential score impact.
The scoring update icon on the History Page and Issue Level Event Log will indicate score changes due to calibration
The score impact in the platform is a reflection of how the recalibration change impacts your score on the day it was computed. Since we run scores daily, the impact may change as we use new data.
Find issues affected by a scoring update
To proactively remediate new measurement changes, download and review the New Findings report for detailed information.
- Select My Scorecard from Scorecards in the top menu.
- Select the calibration icon (wrench) under your score, and click Download Reports.
- Select New Findings to download the .csv file with detailed information.
- Review the finding details to help you start remediation.
FAQ
Why does it take two to four days, or even more, for my score to reflect a change?
Our batch processing method typically takes two to four days: one day to collect the data and one day to process it. The process can last longer when unexpected variables occur. When collecting data we sometimes detect errors in the data; we then reprocess the data to ensure accuracy. Other reasons for delayed updates to your score may be errors in our scoring pipeline.
Why does SecurityScorecard re-attribute IPs to me after Support removed them from my Digital Footprint?
This should not be the case. Once we remove IPs they should not reappear. If they do, an error may have occurred that we can easily fix. Please contact us about reappearing IPs so that we can correct the problem.
How can I stay ahead of possible scoring changes to keep my score high or stable? Specifically, how can I get to high-priority fixes quickly, given that any change takes two to four days to affect my score?
To give you as much time as possible to correct and remediate problems – and thereby keep your scores high – our platform alerts you several weeks before any scoring changes take place. In addition, we highlight any recent changes that may affect your score in our monthly Scoring Update Newsletter. There we explain what changes are significant, and how you can resolve these issues in the platform prior to the scoring update deadline.
How do you show me what changed, so I can start remediation quicker?
We make your score more actionable with our issue-level event log, which shows new and removed findings each day. This insight helps you implement your remediation process faster.
Get help
If you have any questions or comments, contact your Customer Success manager, or submit an inquiry to our Support team.