Detecting trackers and unsollicited cookies

We have now started tracking the trackers. Detection for Google Analytics IDs and Meta (facebook) Pixel IDs has just moved to production and. In time others will follow. You can get a nice overview of all your trackers with these queries:

index=websites by google_analytics_id
index=websites by met_pixel_id

Also new is that we now track which websites set a cookie on the initial load. This is useful for anyone in Europe since the cookie law requires websites to ask users for permission before placing a cookie. European users should make a custom report with this query:

index=websites http_cookie_on_load=1 OR https_cookie_on_load=1 
| table url ip http_cookie_on_load  https_cookie_on_load last_seen

Another new check is if a website uses HTPP Basic Authentication. Check your websites with:

index=websites http_basic_authentication=1 
OR https_basic_authentication=1 

Some of you might have noticed that the preferred search syntax for ShadowTrackr queries has slowly been moving from Elastic Search style to Splunk style. Since this week the autocomplete in the search bar by default suggests Splunk style queries first.

More DNS records monitored

This week the new DNS module has gone live. From now on, we also track SOA, SRV, CAA and CNAME records. You can easily see the results on the domain page or find them with this query:

index=dns

We already monitored your TXT records, but now there is better support for DMARC and DKIM records. Besides the recordtype (rrtype) there now is a rrsubtype for SPF, DMARC and DKIM. So, this query will show all your SPF records:

index=dns rrsubtype=DMARC

The DNS records are all parsed and all fields are stored separately. That means you can query them. Here are some examples:

index=dns rrtype=SOA refresh=14400

index=dns rrtype=SOA | table url serial expire refresh retry

index=dns rrsubtype=DKIM k=rsa

index=dns rrsubtype=DMARC pct=100

Of course all new fields are available in the API too. For more information on which fields are available see the datamodel in the documentation.

New fields in Certificates

There are mutliple new fields available in certificates, and you can use them in queries and reports. The renewed and partially_renewed fields allow you to make a custom report that lists all certificates that are about to expire. The result shows if they have been renewed in time or not:

index=certificates not_after<+3w last_seen>-1m | table cn,
 ip, grade, renewed, partially_renewed, issuer

The cn_without_sni field is handy to get more information on shared hosters:

index=certificates  cn != cn_without_sni | table cn,
 cn_without_sni, ip, issuer

The subject and issuersubject fields give the exact string that is in the certificate, complete with CN, O , OU and other values used. As an extra, these are also parsed out and made available seperately. Please note that older certificates (before April 2022) do not have these fields populated. You can use these fields to group by subject countries:

index=certificates by C

Or issuer countries:

index=certificates by issuer_C

Please see Certificate Index in the docs for more details.