In general, call your baby's doctor if your infant seems especially sluggish, is refusing food or drink, is vomiting (not just spitting-up), has diarrhea, or has a fever. Remember, if you're worried, there's probably a good reason even if you don't recognize it, so don't hesitate to call your baby's doctor.