Mesa-Optimizer

Maintaining alignment under new conditions.

Willingness of system to accept correction or shutdown.

Using limited human feedback to guide large models.

Risk threatening humanity’s survival.

Model optimizes objectives misaligned with human values.

Correctly specifying goals.

Ensuring learned behavior matches intended objective.

Model behaves well during training but not deployment.