The American Academy of Pediatrics recommends sponge baths until the umbilical cord stump falls off — which might take a week or two. To give your baby a sponge bath, you'll need: A warm place with a flat surface. A bathroom or kitchen counter, changing table, or firm bed will work.