To be able to use effectively the bow, the samurai didn't use hand held shields; the lack of defense was compensated by the way the Ousode (大袖) in their armor worked: ... However, the infantry soldiers still relied on shields.

Why did samurai never use shields?

Some of the reasons were: clumsiness. The clumsiness is somehow obvious: with a weapon in one hand you can do some other stuff with the other hand. A shield is not really easy to carry around. A sword isn't either.

Why did Japanese never use shields?

So, the Japanese forces adapted and began fielding horse archers of their own to combat that threat. The samurai's armor adapted accordingly too to this threat. Since you can't use a bow without two hands, personal, hand-held shields became unviable so they instead latched those onto their armor.