Monitoring distributed systems

In debugging distributed programs a distinction is made between an observed error and the program fault, or bug, that caused the error. Testing reveals an error; debugging is the process of tracing the error through time and space to the bug that caused it. A program is considered to be in error when some state of computation violates a safety requirement of the program. Expressing safety requirements in such a way that a computation can be monitored for safe behavior is thus a basic preliminary step in the testing-debugging cycle. Safety requirements are usually expressed as predicates. When a state of the computation violates such a safety predicate, that state can be said to be in error. A predicate logic is proposed that permits the specification of relationships between distributed predicates. This increases the scope and precision of situation-specific conditions that can be specified and detected. It also permits the specification of safety primitives such as P unless Q using distributed predicates. Thus a distributed program can be directly monitored for satisfaction and violation of safety requirements. Breakpoint conditions and predicates expressing safety may hold over a number of states of a program. A breakpoint state is meaningful if the causal relationships of events included in the breakpoint are unambiguous. At least two such states exist for each condition: the minimal and the maximal prefix of the computation at which the predicate holds. These states are specifiable as part of a breakpoint definition in the logic presented