The manual says:
"By upshifting and downshifting through 1, 2, D3 and D4, you can operate this transmission much like a manual transmission without a clutch pedal".
I do it occasionally and it seems to work very well. I try not to abuse it, though. As much as possible I try to keep the shifts - especially the downshifts- as smooth as I can make 'em.
For some reason, this is one of very few auto transmissions that I've tried to manually shift that return a rewarding experience. The only other one I can think of at the moment is the SportShift in my wife's Celica GT-S.