Because prime lenses only need to be built as a single focal length they can use less of these glass elements - and, the glass elements inside the lens don't have to move in order to zoom. Less glass means less diffraction of light, which usually means sharper images.