As I see it, If there's no society there's nothing to prevent someone from simply forcing you to do what they want, If that's acceptable to you ethically than it is acceptable ethically for people as a whole to do it. The state of nature cannot offer freedom and in fact you do exist in that state of nature, If you choose to live according to what authority can be imposed, you can do so.

edit: to be clear this is the very basis of my stance as an Anarchist in that "society is simply forcing people to do things" is the starting point I think we should operate from. And I think a Just society can only be had if we recognize that Authority does not exist other than as the threat of violence. And I think laws/society should be written/structured in a way to maximize the freedom you describe, understanding that if we actually want to maximize justice/freedom/good there must also be economic justice and therefore complex economic structures.