Does Omega a want/need a factor statement?

[zenna]

Many PPLs, such as WebPPL, have a factor statement. When included in a random procedure,factor(x) adds x to the logpdf of the trace.

This is used for all kinds of interesting things That said, the design of it worries me. The main question I'd like to consider is whether Omega needs a factor statement, and if not, what it should have instead.

Example:

var actions = ['italian', 'french'];

var transition = function(state, action) {
  var nextStates = ['bad', 'good', 'spectacular'];
  var nextProbs = (action === 'italian') ? [0.2, 0.6, 0.2] : [0.05, 0.9, 0.05];
  return categorical(nextProbs, nextStates);
};

var utility = function(state) {
  var table = { 
    bad: -10, 
    good: 6, 
    spectacular: 8 
  };
  return table[state];
};

var alpha = 1;

var softMaxAgent = function(state) {
  return Infer({ 
    model() {

      var action = uniformDraw(actions);

      var expectedUtility = function(action) {
        return expectation(Infer({ 
          model() {
            return utility(transition(state, action));
          }
        }));
      };
      
      factor(alpha * expectedUtility(action));
      
      return action;
    }
  });
};

viz(softMaxAgent('initialState'));

Here, factor is used to make actions with high expected utility more probable. It implements a kind of planning as inference.

How should we think about this?

[zenna]

Thinking about this model, the main questions: what are they trying to express?

I think it's something like the agent is an approximately rational agent. If it were the agent were a rational agent, then it's actions would be determinsitic, the arg_max E(utility). But they want to say it is approximately optimal, but then the question is in what way.
There are many ways, we might say it's rational but with some additional chance of it randomly selecting some action uniformly. There are many other variations we can consider.

But what about their particular construction? As described, given the uniform prior, it makes it so that the probability of any action is something like

p(action) = expected_utility(action) / sum(a -> expected_utility(a), actions)

Why don't we just allow Omega to express this directly?

Perhaps if we had the general ability to construct random variables from their densities, we wouldn't need any machinery. This deserves an issue by itself.

There are two advantages that factor has over the expression above:

1. We don't have to waste computation on the normalizing constant
2. There's an explicit way to generate values. It's still a generative model.

[zenna]

For completeness, let me just say what I don't like about the factor interface:

1. It modifies some hidden state. factor(x) is not only mutative, but it mutates something not even expressed in the arguments.
2. It's a non trivial design question of what to put as the absolute values: x in factor(x). Most of the examples use it in a qualitative way, but if I imagine this as part of a much larger model, it seems like it could interact in weird and unpredictable ways. In other words there's a kind of nonlocality to it.