Housing secretary tells BBC Panorama widespread scepticism will make hitting target "all the sweeter".