Learning When Objectives Are Hard To Specify