The HTTP response headers that this site analyses provide huge levels of protection and it's important that sites deploy them. Hopefully, by providing an easy mechanism to assess them, and further information on how to deploy missing headers, we can drive up the usage of security based headers across the web.
How to configure nginx to block certain user agents from accessing a site.
A stackoverflow discussion about how to use python to make an xmlhttprequest. I looked this up because that's how you're supposed to interact with unmark's pseudo API. The accepted answer shows how to do it with scrapy, but I think it could be adapted for use with the requests module. The HTTP method would be POST, and one of the headers would be "X-Requested-With: XMLHttpRequest". It's worth a shot, at any rate.
5004 links, including 377 private