Inbreeding becomes less dangerous the more common it is, and people in early tribal groups were a lot more closely related than today. Genetic differences between groups increases and all it takes is a bottleneck event to make the tribe of 44s the new standard number of chromosomes. Speciation through increased diversity between many groups of genetically similar individual, followed by selective or random culling of many of those groups.

That's a viable explanation of how we ended up with 46 instead of the 48 the other great apes have.