1
We must build systems that allow operators to exercise appropriate levels of human judgment
- System design should ensure that outputs, under all circumstances, are consistent with expectations and restricted to the scope of the contract.
- As this applies to work in defense, it will be applied within military codes of conduct and published US government policies.
- This includes creating fail-safes so that users can stop a system before it gets out of control.
- It also includes keeping the burden of safety–not relying on "fine print" as a fail-safe (that is, not relying solely on the user to protect against unintended consequences).