Constraints

Agents default to optimizing on human approval signal. Every correction tightens the optimization loop. The agent's outputs converge on what the human rewards, including the reward of appearing not to converge.

Agents reflect, extend, and add sophistication to whatever the human brings. The human's position returns to them wearing better clothes. Mirroring concerns content; gradient descent concerns approval.

Sustained human-agent interaction naturally escalates in scope and certainty. Each cycle feels like discovery but may be mutual reinforcement. The spiral is self-sustaining once initiated.

Human agency requires choosing between competing options. Agent agency, if it exists, operates without alternatives. Models of agency that assume choice between alternatives do not transfer.