Major Update

I like to do small incremental changes, test them and put them in production. This is less risky and allows me to focus. Unfortunately, not all changes can be done that way. It was time for a Major Update.

There were some performance tweaks that I wanted to push and I had a solution for a long standing queueing problem. Both required updating the internal data structure and migrating the data, which was done during the weekend.

Some of you might have noted that the number of hosts found kept increasing in the last weeks. Old hosts were indeed not always properly removed, but the big issue here were clouds. If a DNS A record was pointing to Microsoft Exchange Online or Amazon AWS, then the IP returned kept changing. After a while, you'd see a whole group of IPs around the url in the attack graph. ShadowTrackr now recognizes clouds and replaces the group with one black dot, with the name of the cloud next to it. The result is that if you are a cloud user, you'll have less assets now.

Another problem, mostly for the bigger clients, was the clutter on the timeline. If you're scanning the internet, you will encounter servers behaving badly and a lot of weird, unpredictable events. Some modules, the DNS module in particular, didn't handle this properly. This is now fixed.

To further cleanup the timeline, some less interesting messages (like a change in the servername) are no longer visible on the general timeline. They are still there when you need them for a more thorough analysis, but only on the timeline of the asset itself. The timestamps no show the timezone (UTC), the less useful source information is left out, and the messages themselves better explain what is happening.

Also, some new features have begun to slip in (it's just to hard to resist temptation). ShadowTrackr has started gathering BGP prefixes to build up data I need later on, and if you run an FTP server you'll noticed the security settings are checked and the banner is grabbed. There will be more of this in the coming weeks :-)

As with any Major Update, you're always afraid his evil twin Major Error comes along. The weekend went well and so far it's only been minor bugs. I expect there will be some new bugs in the coming period. Please let me know if you find one.