One of the oldest projects of the DNP Technical Committee has been the ongoing effort to write a comprehensive Master Station test procedure. It is a long, difficult thing to do.
Unlike the remote side of a SCADA system, not only does the Master Station has a complex job to communicate correctly with an outstation, but it also has to know all the quirks and extra features for each Outstation, and to be able to adjust polling strategies on the fly to handle new system conditions.
So how does one test all this? Well, we have to standardize how a Master should poll the field. It sounds pretty straightforward, until you realize that DNP is a “Near-Real-Time” protocol. Real Time polling is actually pretty simple. Non-Real Time polling is also pretty simple. However, Near-Real-Time polling is subtle and complex. When you mention poll scheduling, people nod their heads in affirmation and everyone smiles thinking he or she knows exactly what we’re discussing.
Only, like several other features in the history of DNP, we aren’t thinking of exactly the same features. When we start comparing notes, differences emerge out of the woodwork. In this case, it is one thing after another. We think we understand each other, but we do not.
It turns out that there are many tweaks to how a master might poll the outstations in a SCADA system. These tweaks are designed to do something that most IT professionals do not understand at all: save bandwidth. Why are we so obsessed with saving bandwidth? Because we often operate on private circuits to places where there is no reliable internet access. We may not be communicating with great speed, but we do need to communicate with reliability and minimal dependence on other infrastructure. Thus, 1200 BPS circuits are not unheard of even in modern SCADA systems. Radio circuits for licensed applications run at speeds of 9600 BPS, and they often communicate with dozens of outstations.
Many industries have turned to the Internet for their communications infrastructure, and they use VPN wrappers to shield the SCADA system from man in the middle attacks. The only problem with this approach is that we’re dealing with the Internet here, and we really do not know what infrastructure is actually carrying the data, or how it well it may be secured. In many cases, internet traffic is consolidated to just a few very high bandwidth links. If a major disaster were to happen to Northern Virginia, the effect upon the Internet would be very significant.
I prefer knowing where the traffic will be handled. I prefer decentralized traffic management. This will enable a SCADA system to degrade gracefully. It does little good to have backup control centers if they both depend upon similar or even the same infrastructure.
There are many subtleties for designing a good SCADA master. And once it is integrated, the process for validating its performance is also so difficult that many skip this step. In fact, I can recall several conversations in the late 1980s while we were trying to figure out why our SCADA system was taking so long to recover after a communications outage. This happened because we didn’t understand all the quirks and tricks the SCADA master polling software had for managing the bandwidth in the field.
Understanding this situation exposes a significant problem: If the people who use, integrate, and design this stuff are having a hard time understanding what it does routinely, what hope do we have for securing it? Perhaps the first draft of the DNP Master Test Procedures will help resolve this problem.