NetLay: Up is not the same as working

The gap ping leaves#

A ping proves that a device has power, a network card and a working network stack. It proves nothing whatsoever about the job that device exists to do.

The printer that answers every ping while its print service has hung is the most annoying thing on a status map, because it is green, and green means somebody else's problem, and meanwhile there is a queue of forty documents and a receptionist who has stopped believing your map.

The opposite happens just as often. Plenty of servers are configured to ignore ICMP entirely and are sitting there serving perfectly happily, and NetLay has been drawing them in red since post one.

Both of those are the same bug. The app was measuring the wrong thing and reporting it with confidence.

Service ports.#

Each device can now carry a list of ports it ought to be answering on. Nine thousand one hundred for a printer, three three eight nine for a Windows machine, five five four for a camera.

You do not have to know those numbers. While the field is empty it offers the ones that make sense for the device type, as a single link you click to accept, so a printer suggests raw print, IPP and HTTP and you take them or you do not.

They are checked on every ping cycle, in parallel, using the same timeout. A TCP connection that opens is enough. It does not log in, it does not grab a banner, it just asks whether anything is listening, because anything more would mean credentials and this is a map, not a monitoring suite.

Four outcomes instead of two.#

Ping answers and every checked port answers, it is up.

Ping answers but a port is closed, it is degraded, and it gets a purple dot. That is the printer with the hung service, and it is the whole reason for this post.

Ping does not answer but a port does, it is up, and the inspector says plainly that there was no ping reply but a service port answered. That is the firewalled server, now green and honest about why.

Nothing answers at all, it is down.

One deliberate decision in there: degraded is worked out in the interface and is not stored as a status. The stored state stays up, which means uptime figures and the history from post eight still treat it as up. I did not want a new state quietly rewriting three months of recorded history, and I did not want "the service was broken" and "the box was off" averaged into the same number.

Degraded does outrank slow in the display though, because a working but sluggish device is less interesting than a device that is answering pings while failing at its job.

Where it shows up.#

The purple dot appears on the node, in the sidebar tree, in host cards and group cards. The inspector lists each checked port with a tick or a cross and a friendly name next to the number, so it says raw print rather than leaving you to remember what nine thousand one hundred means.

The map legend gained a line explaining it, because a new colour nobody can decode is just confusion with better styling.

Discovery helps here too. When you place a discovered host, the port list is pre-filled from what was found open, filtered down to ports that are actually services worth watching rather than every open port on the box. Something answering on 9100 gets 9100 watched. Something with an ephemeral port open does not.

What I have not done yet#

Degraded does not raise an alert and does not create an event in the history.

That is deliberate and slightly uncomfortable, because it is obviously the useful bit. The reason is that these checks are new and I do not yet trust them on real equipment. A firewall rule, a slow connection or a device that only accepts one connection at a time could all produce a false cross, and the fastest way to make somebody ignore alerts forever is to send them a few that were wrong.

So it is a colour on a map for now. When it has been right for a few months on kit I did not choose, it gets promoted to something that wakes people up.

Next time, what happened when a server with sixty virtual machines went into a group.

NetLayNetworking