Yes that's the cap off but what's that coming out of the centre of the socket? There shouldn't be a wire in the centre of the socket there, it should be a tiny hole which accepts the thin wire from the middle of the coax cable.
Try grip the wire and tug it a little, but keep one hand on the face-plate don't want to pull the whole thing off the wall.