Better keyword monitoring and a failed Friday update

As some of you may have noticed, there was an update. On Friday. Late in the afternoon. Despite knowing the jokes about this, I did it anyway and it failed. Throughout the week there are numerous tiny updates that you'll hardly notice. The bigger updates are usually done in the weekend, and only after they have been running successfully in test for a while. This update was running ok in test, but the test was obviously not complete enough. The decision of how much to test is always hard. No one wants to unnecessarily slow down development.

The update
Well, up until now keywords where matched as whole words. This means there are spaces (or tabs, or new lines) around them. If you entered something between quotes, like "@shadowtrackr.com", this would find Shadowtrackr email addresses in passworddumps. But if you just had a surname (without quotes) this would be missed in passworddumps. The surname would likely be prefixed with a '.' (behind the first name) and postfixed with a '@'. No match. That has changed now. Shadowtrackr matches a word between any non-letter and non-number character now.

In some cases you might want to detect a partial match, like sub in subdomain. On the traps page for your keyword you can check the box "match partial keyword" for this.

What went wrong?
When you have multiple keywords in a trap, they should all occur before a notification is send. With this new algorithm, this went wrong. Any of the keywords now was enough. To make things worse, the update set the default state for "match partial keyword" to on. This resulted in way more notifications than you'd be happy with.

The problem only lasted for about half an hour, but you can get an amazing amount of notifications in that time. I myself got about a thousand of them. My sincere apologies to you all, I'll be more careful with updates.

Pastebin.com snippets, updated apps, and more

It's been a busy week with usability improvements, bug fixes and and experimental new feature.

Let's start with the new stuff. ShadowTrackr scans for your keywords in datadumps. Up until now you had to click the link on your timeline to find out the context in which your keyword was mentioned. Only after clicking you could see the full orginal post on the datadump website. Since keyword monitoring is quite popular, this results in a lot of clicking.

To improve this, Shadowtrackr now saves a snippet that includes some lines before and after the keyword as the context. You can see this context on your timeline by clicking on the details link in the right bottom of the post. This might save you some clicks. It also provides you some idea of what the data was like if the original data has been deleted.

There is a new version of the iOS app. An iPad user rightly complained that he had to run it in tiny iPhone mode, and that the links in the menu where so densely packed to it was hard to tap the right one. Both issues are now fixed.

Android users where bothered by an unexpected refresh on the timeline and a not properly functioning back button (the one on the hardware itself). Both these are fixed in the newest version.

There are quite a few other improvements and even some additions that are too small to mention here. I hope these improve the usability as intended. If not and something bothers you, please let me know!

DUHL ip addresses and false positives

In the last few weeks, multiple ip addresses of multiple users have ended up on the SORBS DUHL list. ShadowTrackr picked up on this and dutifully gave a blacklist warning. Now if the ip address is running a mailserver, a proper warning is in place. Unfortunately, in other cases this can be a false alert.

DUHL is short for Dynamic User/Host List. It contains ip addresses that are flagged by ISPs as residential or small business internet lines. These lines are used to browse the internet and should not have any servers running. Considering the SORBS blacklist is mainly used by mailservers looking to filter out SPAM, a DUHL list is quite useful. A mailserver on a home internet line that is sending email likely means the thing is hacked and sending SPAM.

The problem comes when you have your home or branch office internet lines in ShadowTrackr. This is a perfectly good idea since you'll be warned when you have security trouble. We encourage it. But your ISP might have flagged this as a DUHL connection and ShadowTrackr will alert on it. There was even a case where the ISP repurposed the ip range from Dynamic use to a server park and forgot to update the DUHL flag at SORBS. Again, this resulted in a false positive.

In order to prevent these false alerts, ShadowTrackr now uses only the relevant SORBS sublists instead of the main blacklist. The main blacklist included DUHL listings and this way we can avoid the false positives. We shouldn't be scaring you with false positives that can be avoided. Good riddance to this one :-)

You can find more information on the SORBS lists here