Scaling up

More scaling up was done during the Holidays, and unfortunately this wasn't always smooth. The software and servers are in much better shape now, and a lot of performance issues and bugs were fixed.

If you do notice trouble, please let me know. One client asked why a specific website wasn't scanned anymore, and I discovered the website and about 200 others were hanging in some sort of crashed queue since the scaling up. All are up and running now thanks to this bug report.

Assets and suggestions overlap checks

The biggest functional improvement this week is that assets and suggestions are cross-checked from now on. If a new asset is added, manually or through discovery, any suggestions about that same assets are automatically removed. The same goes for new suggestions. They are dropped if the suggested asset already exists.

If feels a bit odd that this is only added now and not much earlier. Thing is, it was not a problem in the past. Some accounts have gotten so big with so many related assets on the attack surface that now it had become a problem.

Again, this week's update had more improvements in software detection, and new scanner nodes were added. With the addition of nodes in Brazil, Italy and Sweden the total number of locations we scan from is now 14.

Custom internet standards report

A lot of requests came in about customizing which urls are included in the internet standards report. By default, it only included all pay level domains ("shadowtrackr.com") and not subdomains ("twilightsparkle.shadowtrackr.com").

The reason for this is that daily checking all urls in ShadowTrackr would costs a lot of CPU and memory and is not sustainable at current price levels. I do very much understand that some pay level domains are not that interesting and some subdomains are. So, something needed to change.

As of this week you can edit the internet standards report. You can remove pay level domains in the edit function under reports, and you can add subdomains by clicking on the action menu (three dots, upper right) in the url page. Just pick "add to internet standards report".

Again, the infrastructure was upgraded. There were too much delays occurring in the database cluster so it has been moved to bigger servers and got some performance tweaks.

Lastly, a number of other code changes were moved into production. Besides bug fixes, the software detection has improved. Most notably for Cisco, vBulletin, Jira and Confluence.