Bilbo did go to Erebor (in the book) when he left Bag End and the Ring. He went there and then came back and settled in Rivendell. But then, in the book there were 16 years between his leaving and the beginning of the War of the Ring so he had plenty of time for a leisurely journey and a good long visit.

Did the dwarves visit Bilbo again?

Among the Dwarves of the company, Balin was the second-eldest dwarf, and loosely held the position of deputy leader. ... Of all the Dwarves in the quest, Balin was the only one known to have visited Bilbo at Bag End after their quest for the Lonely Mountain.

How long did it take Bilbo to get to Erebor?

The departure of the quest for Erebor

After 640 km of walking in 40 days, they arrived at Rivendell to meet Elrond. Current statistics: 640 km in 40 days, minus the day of being lost due to the Trolls.