A few more sets to consider:
Louis Armstrong - Big Band Recordings 1930-32, 2CD (JRT remaster)
Joe King Oliver & His Orchestra - 1929-30, 2CD (JRT remaster)
Henry Red Allen & His New York Orchestra - 1929-30, 2CD (JRT remaster)
Mills Brothers - 1930s Recordings, 5CD
Louis Jordan & His Tympany Five, 5CD
Big Joe Turner - 1938 - '52, 5CD
Joe Venuti & Eddie Lang - 1920/30s, 2CD (JRT remaster. Not sure if this overlaps material on the Mosaic set and if so I would go for the latter, price notwithstanding)
A few more releases I haven't bought such as Django in Rome and the Hoagy box set should be worth exploring. I generally stay away from these sets if the recordings have already released by 'legit' labels such as Mosaic, and I would certainly avoid the outright rip-offs such as the Bunny Berigan set.