Hi, I have a 4 day old bub and my concern is he...

Hi, I have a 4 day old bub and my concern is he fed this morning at 7.30 then i woke him about 11 to feed him and he was not interested at all. I tried and tried and he would not attach and just got upset and fell back to sleep then again i woke him a few hours later and he had about 4 or 5 sucks then didn't want anymore. I bathed him at 5.30 and he feed for about 5 mins on one side then again he'd had enough and went to sleep. It's now 8pm and he's still asleep. I took him to the doctors today and the doctor isn't too concerned at this point. We was 9 pound 8 born so not exactly a little bub. I also called the breastfeeding assoc and they said to keep an eye on him and continue trying. I've read lots of other forums and books etc that say to wake and feed but I can't exactly make him feed if he doesn't want to. Any advice?? Thanks heaps