Thanks to Mom and Dad and one Grandma and another Grandpa, I heard jazz growing up and dug it, especially the swinging variety. Lots of Miller and Goodman, lots of Gershwin both jazz interpretations and classical, lots of WWI and twenties pop tunes on sheet music courtesy of a pianoplaying and organ dabbling Grandma, a little Ellington, some Brubeck that my Mom dug mainly for Desmond I think, a little dixieland from aforementioned former-banjoist turned parlor-organist Grandpa.
Then when I came back from Africa at 17 I got bit by the electric Miles bug and though I sometimes think that ISN'T jazz it did lead me to really explore the modern stuff, to wallow in fusion for a while, and then to work back to the swing and early jazz.
So before twenty for me.