Robust Scene Estimation For Goal-Directed Robotic Manipulation In Unstructured Environments