Abstract
This paper highlights a limitation of state space aggregation based model reduction of Markov chains. It is shown that within the set of Markov chains of a given dimension that admit an exact low order representation, the set of Markov chains where this exact low order representation can be extracted by means of aggregation based model reduction is a nowhere dense set.