Policies, adjustment, decisions, sharks

Traditionally, the medical research community has focused on treatment effects (and often the causal inference literature does the same). It is worth exploring the decision problem as well— the patient faces a decision problem, and solving it is often a good way to help them. This motivates much of what is written below. On a somewhat related note, having studied causal inference with fresh eyes not too long ago, certain aspects of it were difficult to grasp. For this, it is possible that taking a decision-oriented perspective might help. If so, it might help others better understand causal inference, which would improve medical research, helping patients a lot. 

In this post, I first claim that the policy perspective allows us to derive the adjustment formula using operations from probability alone (I discussed this a bit before, too). Second, I claim that this allows us to think about decision-making in a natural way.

The policy perspective

Let Y be some outcome, X be some intervention, and Z be confounders that mutually cause Y and X. To see that the policy perspective can help us derive the adjustment formula, one can start with the equation found on the top of page 114 of Causality [1],

P(y)|_{P^*(x|z)}=\sum_x\sum_z P(y|\hat{x},z)P^*(x|z)P(z),

where P^* is a distribution that we set ourselves.

From there, one can derive the adjustment formula found on page 80, equation (3.19), i.e.,

P(y|\hat{x})=\sum_z P(y|x,z)p(z),

as the special case where the policy P^*(X=x|z)=P^*(X=x)=1, a degenerate distribution independent of Z. Something to this end is also said on page 113, but the adjustment formula is derived far earlier in the book. I guess my point is that this order can be flipped with good effect.

My main contention is that if we start with policies, i.e., with P(y)|_{P^*(x|z)},  the derivation of the adjustment formula follows from the rules of probability alone; in other words, there is no such thing as a “deletion of a factor” (words taken from page 23), which is not really an operation in probability. This adherence to the rules of probability, I think, makes causal inference easier to understand.  Of course, one still needs to think about DAGs and their implications—one needs a model—but afterwards one can proceed with familiar probability concepts.

Decisions

Now, to see that policies give us a natural way to make decisions, note that we can evaluate any policy, P^*(x|z), using

ER(P(y)|_{P^*(x|z)}),

where R is some reward defined with respect to P(y).

We can even obtain a reward-maxizing policy, 

P^{*opt}(x|z)=\arg\max_{P^*(x|z)} ER(P(y)|_{P^*(x|z)}),

an equation which appears, without the interventional framing, in multiple places, including works in decision theory [2] and operations research / reinforcement learning [3,4].

Shark attacks

As an example, suppose a public health department wants to reduce shark attacks, S\in \{0,1\}. They decide to study them in the context of ice cream intake, I\in\{0,1\}, and ambient heat, H\in\{0,1\}.

The standard approach would be to establish how likely a shark attack would be, given some degree of ice cream intake. I.e., one would target P(s|\hat{i}) as

P(s|\hat{i})=\sum_hP(s|i,h)p(h).

This requires “deletion” of the factor P(i|h).  

However, one can use the policy perspective on page 114 to instead write

P(s)|_{P^*(i|h)} = \sum_h\sum_i P(s|\hat{i},h)P^*(i|h)P(h),

which reduces to (3.19) upon setting P^*(i|h)=P^*(I=i)=1 and noting that H is a confounder, so that P(s|\hat{i},h)=P(s|i,h). Otherwise stated,

P(s)|_{P^*(I=i)} = \sum_hP(s|i,h)\cdot 1\cdot P(h) +\sum_hP(s|i,h)\cdot 0\cdot P(h).

Further, suppose the department wishes to determine whether, in order to minimize the expected number of shark attacks, ES, individuals at the beach should consume ice cream at all. The department’s studies would target

P^{*opt}(i|h)= \arg\min_{P^*(i|h)}ES|_{P^*(i|h)}.

Note that

ES|_{P^*(i|h)}=P(S=1)|_{P^*(i|h)}

\ =\sum_h \sum_i P(S=1|i,h)P^*(i|h)P(h)

\ =\sum_hP(S=1|h)P(h)\sum_i P^*(i|h)

\ =\sum_hP(S=1|h)P(h).

Fortunately, we see that, given heat, ice cream makes no difference. But, if it did, P^{*opt}(i|h) would tell us what to do about it.

References

1. Pearl, Judea. Causality. Cambridge university press, 2009

2. Wald, Abraham. “Statistical decision functions.” Breakthroughs in Statistics: Foundations and Basic Theory. New York, NY: Springer New York, 1950. 342-357.

3. Sutton, Richard S., and Andrew G. Barto. Reinforcement learning: An introduction. Vol. 1. No. 1. Cambridge: MIT press, 1998.

4. Puterman, Martin L. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014.

Leave a comment