The answer to what college has the lowest acceptance rate is the same as what ivy league college has the lowest acceptance rate: Harvard. While all Ivies are known for having some of the lowest college acceptance rates of all colleges, Harvard has generally been the most selective.