When they come of age to work it will be at a certain time of year where particular work is required. So the time of birth would definitely affect what kind of work they would eventually be exposed to first.

The person you're responding to is trying to rationalize why people thought astronomical bodies affected people's personalities, not that they are exactly correct about it. Assuming people in the past were just nonsensical is bad science and a little arrogant. They had an internal logic, which is interesting to think about.