Garp had come to the conclusion of not taking Ace to the Bandits and instead took him to Foosha, claiming Ace to be the child of a long lost child of his who had recently died. When Ace was old enough (coughjuststarted walkingcough), however he met the bandits. Who ended up becoming his babysitters anyways. Then comes Luffy who Garp uses the same cover story as Ace only that Garp’s long lost child turned out didn’t die only faked their death but is genuinely dead this time.

When Sabo comes along Garp realizes that a) this kid probably isn’t a child of a high profile criminal and b) he couldn’t use his cover up story again with more additions. So he claims Sabo to be his long lost sibling’s grandchild whose entire family, Except Garp, Ace, and Luffy of course, died.