you are much better off staying in town-the town is beautiful and you will have no trouble spending six days there - there are lots of things to fill your time.
If you really want, it won't be any trouble arranging a day trip to Hue. it's not ideal and very exhausting, but I meet people every time I'm in Hoi An who have done it.