Behind the Scenes: Don’t Build A Policy From Scratch!