So does it really matter pushing torment levels? The timer is a backup system in case you get a bad RNG streak. It will gradually push RNG in your favor. So yes, upping T levels does matter.
What torment level should I play Diablo 3?
By the time you complete your Haedrig's Gift set, you should be at Torment V or VI at the least, especially because in order to complete the set you'll have to do a Greater Rift solo at level 20 and kill the four Keywardens and Gnomb and Zoltan Kulle at Torment IV to even get the full set.
What level should I play torment?
You'll unlock Master difficulty when any of your characters completes the game, and Torment difficulty when any of your characters reaches level 60.