Suggestions for new assets

The new algorithms for finding your websites and servers work great. Shadowtrackr is finding and monitoring more than ever. A bit too much actually.

Some clients use shared services, and without any restrictions the other websites and servers on the shared infrastructure were automatically added to assets and used for expanding in turn. Without a proper stop condition, this could end up adding most of the internet. One client using a shared Baidu server ended up with 42 unrelated Baidu machines within a couple of hours. Yes, 42.

I've thought long and hard about a proper stop condition, but there isn't any that I can come up with. If machines are not on a dedicated ip (range) for you but on shared servers, there is no way of reliably determining if all urls pointing to it are really yours. You might be able to relate some of them with Whois information or by analysing links on websites, but this does not solve all cases. Whois data is not always available and larger companies tend to have several different whois contacts anyway.

The most user friendly solution I could come up with is offering suggestions. When a new server or domain is found that somehow relates to one of your assets but is not obviously yours, ShadowTrackr will "suggest" it to you and tell you what existing asset it is related to. You then have the option to reject or accept it. Check out the suggestions page in the menu to see yours.

I'm still thinking of ways to minimising the user interaction needed, like tracking known shared hosting and automatically rejecting suggested assets on it. For large organisations the initial amount of rejections needed can build up to dozens or even more than a hundred suggestions. After the initial load that number stays acceptably low though.