Sleepy baby won't nurse

bigtex11

Whenever I try to get my 4 day old to nurse, he latches on and then falls asleep after about a minute.  Nothing will wake him up. We have to put him down and wait until he wakes up and starts whining for food again before starting all over again....it has gone one for an hour before.

I've gotten so frustrated that we are now just giving him pumped milk after trying for about 10 minutes, which he drinks immediately, but I feel like this is furthering the problem.  How would you correct this?