I have a buddy whos' owned both, & Ive ridden them both lots. The Rancher seemed to me like a trail machine where the Foreman a work machine. I doubt the Foreman is faster top end if that matters. Might even have a rougher ride.
Foreman has more power and is bigger. For someone your size, foreman will be better.
Rancher is more nimble and goes through the woods easier.
I'd say the 450 engines hold up a little better, but both are great engines. I've had to replace cranks on 350's I've worked on. Never had to do a 450 crank.
Both can be upgraded to disc brakes easily using Honda parts, and both can be made selectable 4wd (although as you said, if looking for a 450, the 02-04 would be the way to go because it already has selectable 4wd).
If I had to have one or the other, I'd get the 450, especially at your size.