It's very important to keep down the risk of infections in the NICU because preemies and sick babies are so susceptible to them. That's why there will be a sink with antibacterial soap near the entrance to the NICU and probably hand sanitizer dispensers placed around the area. Each time you enter, you'll need to wash your hands. You'll see the members of your baby's NICU team washing their hands constantly before touching him or her.
Depending on what's going on with the baby health-wise, you might also need to wear a sterile gown, a mask or gloves. It might feel strange that you're required to do this -- after all, you're not a stranger -- but remember that it's for the baby's protection. Even if you wash your hands, you could still bring in germs.