I'm not a linguistic prescriptivist, mind you, but I would suggest that if you decide to define "berry" such that most things people call "berries" don't fit your definition, and such that a ton of things people absolutely do not call "berries" do, then your definition is wrong.

I'm sure you've described a perfectly reasonable category of things, but calling the set "berries" is just incorrect naming.