Figure: Left-hand panel depicts Model A, a network which encodes inputs as sets of discrete binary features, right-hand panel depicts Model B, a network which encodes all inputs as a single real number.
Model A [see Figure] represents numbers as sets of discrete binary features, while Model B represents numbers as analog values. Model A contains no mechanism for representing a relationship that holds of all possible members of a class. Instead, it can represent the identity relationship only as a set of isolated facts, that the 8 input node should be connected to the 8 output node, that the 4 output node should be connected to the 4 input node, that the 2 input node should be connected to the 2 output node, and the 1 input node should be connected to the 1 input node. A model that was trained using the back-propagation algorithm only on facts involving the numbers 8, 4, and 2 consequently would not generalize the identity function to facts involving the number 1 (). In contrast, model B represents the identity function by simply copying the contents of the input node to the output node; as such it serves an implementation of the simple algebraic rule, f(x) = x, subjecting all inputs to the same operation.