Representation Dependence in Probabilistic Inference

Title: Representation Dependence in Probabilistic Inference

Main Research Question: How sensitive is probabilistic inference to the way knowledge is represented?

Methodology: The authors investigated this question by examining the maximum entropy approach to non-deductive reasoning. They formalized the notion of representation dependence and showed that it is not a problem specific to maximum entropy. They also demonstrated that any representation-independent probabilistic inference procedure that ignores irrelevant information is essentially entailment, in a precise sense. Furthermore, they showed that representation independence is incompatible with even a weak dependence assumption.

Results: The authors provided examples to illustrate their findings. For instance, they showed that maximum entropy can lead to different answers when representing the same situation in different ways. They also introduced a compromise between representation independence and other desiderata, such as invariance under a restricted class of representation changes, and provided a construction of a family of inference procedures that provides such restricted representation independence using relative entropy.

Implications: The study suggests that representation dependence is a fundamental issue in probabilistic inference. It highlights the importance of considering the way knowledge is represented when making inferences and emphasizes the potential limitations of approaches that ignore this factor. The findings also have implications for the development of more robust and accurate inference procedures in various fields, such as artificial intelligence and statistics.

Link to Article: https://arxiv.org/abs/0312048v1 Authors: arXiv ID: 0312048v1