Is Web site alive?

Web site monitoring looks simple and straightforward. However, there still are pitfalls, not so obvious to many webmasters. How would you check that a site is alive and contains expected content?

A human visitor would open site and have a look: visit some pages, perhaps logging in, if it is possible, and doing other checks of this kind. Using a piece of software, we could perform a series of tests, which, taken together, could ensure we are dealing with live and authentic site. Let’s start with basics.

DNS record

Before actually visiting site, let’s make sure it is located at known address. Using DNS monitor allows to make sure the domain name points at expected IP address. If this check fails, either third parties control the domain, or DNS records have been changed at DNS service provider, but current monitor stays out-of-date.

When this check isn’t passed, further checks don’t make much sense. At this stage, one should check both domain registry and DNS services provider used, and make sure the A records are valid.

Ping or TCP response

Simplest availability check, PING monitor is generally used as primary monitoring tool for every device.

If PING response is slow or absent, it’s time to check what’s wrong with connectivity to the site – or with the site itself.

Mind, though, that there are well-known hosting facilities (such as Amazon Lightsail) that do not allow PING traffic. In such a case, TCP monitor (on ports 80 and 443) could do the trick.

Similarly, if PING (or TCP) monitor fails, no need to check further.

HTTP(S) monitor

At this stage we are sure the domain name resolves correctly, and general availability check passed. It’s possibly OK to check the actual HTTP(S) response from the site.

To do that, create HTTP(S) monitor and validate its response: make sure there’s a special string on your site (say, within HTML comment) that can be found via response validation.

That gives extra awareness that site might be authentic and responds as it should. If this check fails, then (apart from inquiring what’s wrong with site) further checks might not be required.

Security checks

Using HTTPS (secure version of HTTP; traffic can’t be efficiently sniffed and/or altered if passing over HTTPS) is nowadays a must. However, X.509 certificates (mostly known as SSL certificates) used by Web server should be up-to-date and reflect actual domain name used by the site.

We have created an article explaining how to check whether SSL certificate approaches its expiration date. Most other checks 9such as domain name match) are performed by browser itself.

If certificate is in error, it’s a good idea to check why. Most modern browsers may refuse opening a site if its certificate is expired or otherwise incorrect.

Emulating human visitor

This step may be required if site expected authentication to reach certain areas. Thus, checking authentication may further improve checks efficiency.

Web Transaction Monitor performs exactly that: it emulates real visitor’s behavior – opening pages, “clicking” links, doing other sorts of human interaction, such as posting forms data.

Please note that WTM is rather a heavy monitor, in terms of resources usage. Only use it when really necessary.

Further checks

There could be several tricks to make site monitoring simpler. For example, a server-based process could do sanity checks and update site’s page accordingly. It could create a small text document (several bytes longs), and HTTP(S) monitor could check it. Serving simple text document is far less resource consuming that serving a page, especially if it is modern site, sending dozens of associated files when a Web page is requested.

Do you know of any other tricks to make site monitoring much simpler? IF you do, feel free to send us a message or comment the article (see the forms below).