Convergence of Expected Utility for Universal AI

Date Added: Dec 2009
Format: PDF

The authors consider a sequence of repeated interactions between an agent and an environment. Uncertainty about the environment is captured by a probability distribution over a space of hypotheses, which includes all computable functions. Given a utility function, they can evaluate the expected utility of any computational policy for interaction with the environment. After making some plausible assumptions (and maybe one not-so-plausible assumption), they show that if the utility function is unbounded, then the expected utility of any policy is undefined.