-Here I again need to make a digression about privacy norms. Adherence to norms is fundamentally fraught for the same reason as AI alignment is. That is, in [rich domains](https://arbital.com/p/rich_domain/), explicit constraints on behavior face a lot of adversarial pressure from optimizers bumping up against the constraint. The intent of privacy norms restricting what things you're allowed to say, is to conceal information. But _information_ in Shannon's sense is about what states of the world can be inferred given the states of communication signals; it's much more expansive than the denotative meaning of a text, what we would colloquially think of as the explicit "content" of a message.
-
-If norms can only regulate the denotative meaning of a text (because trying to regulate subtext is too subjective for a norm-enforcing coalition to coordinate on),
-
-
+Here I again need to make a digression about privacy norms. Adherence to norms is fundamentally fraught for the same reason as AI alignment is. That is, in [rich domains](https://arbital.com/p/rich_domain/), attempts to regulate behavior with explicit constraints face a lot of adversarial pressure from optimizers bumping up against the constraint, and finding the [nearest unblocked strategies](https://arbital.greaterwrong.com/p/nearest_unblocked) that circumvent the constraint. The intent of privacy norms restricting what things you're allowed to say, is to conceal information. But _information_ in Shannon's sense is about what states of the world can be inferred given the states of communication signals; it's much more expansive than the denotative meaning of a text, what we would colloquially think of as the explicit "content" of a message.