Loss in information transmission through two-way channels

A discrete memoryless two-way channel is defined by a set of transmission probabilities {p(y, y¯/x, x¯)}, where x and x¯ are the left and right input signals, and y and y¯ are the left and right received signals, respectively. If the transmission probabilities are restricted so that p(y, y¯/x, x¯) = px¯(y¯/x)p¯x(y/x¯), and if we attach to the two channel terminals independent, finite memory, stationary signal sources which generate channel inputs depending on sequences of past inputs and outputs, then expressions for average information transmission rates in the left-to-right and right-to-left directions can be developed and their sum will be a simple information measure. When mutually independent messages are to be transmitted in opposite directions through the channel, it is desirable that they be encoded into sequences of strategy functions which together with the received signals constitute inputs to a transducer whose outputs are the channel input signals. The message source—encoder—transducer combinations are stochastically equivalent to signal sources whose outputs are governed by appropriate probabilities. We can interpret the transducer—channel combination as a derived two-way channel whose inputs are the strategy functions and whose outputs are the outputs of the underlying channel. Expressions for the information transmission rate through the two directions of the derived channel are developed and are compared to the expressions for the average information about outputs of the equivalent signal sources, transmitted through the underlying two-way channel. The values of the former expressions are found to be less than or equal to the values of the latter, the difference constituting a “coding information loss.” A condition on the transmission probabilities enables us to define a class of lossless channels. Similarly another class is defined having the property that, regardless of the strategy code used, the information transmitted through the derived channel will be strictly less than the information transmitted through the underlying channel. The consequences of the above results on the random selection of message codes are discussed. It is shown that one can obtain the number of variables to be optimized when best random codes for lossy channels are desired, by using the number of variables for lossless channels as an exponent to the product of the size of the input and output signal alphabets. For the lossy channel class a simplified encoding procedure must in practice be applied, but as can be demonstrated, it will not yield optimal codes