'What, then, is that which makes a man free from hindrance and makes him his own master? For wealth does not do it, nor consulship, nor provincial government, nor royal power; but something else must be discovered. What then is that which, when we write, makes us free from hindrance and unimpeded? "The knowledge of the art of writing." What, then, is it in playing the lute? "The science of playing the lute." Therefore in life also it is the science of life. You have, then, heard in a general way: but examine the thing also in the several parts. Is it possible that he who desires any of the things that depend on others can be free from hindrance? "No". Is it possible for him to be unimpeded? "No." Therefore he cannot be free.'
- Discourses of Epictetus
If we accept the doctrine that we should only worry about the things we control, since it is in the things we control that our emotions are expended the most productively, and that otherwise, we are wasting energy if we cannot influence or change the outcome, it logically follows that we should not worry about those things which are partially or wholly due to circumstance and the judgements of others. The Stoic concept of freedom is not a freedom from external restraints but a freedom from internal distress. Only the latter results in self-mastery. The Stoic freedom is self-mastery: deliberating before you act instead of being conditioned to act on impulse. The freeman uses his/her executive functions to mediate between his/her emotions and external stimuli while the slave, and this is only in an analogous sense, instinctively acts on every emotion that is provoked in him/her.
Side Note: I don't think Epictetus meant that wealth and status should not be desired and pursued at all, only that they should not be pursued as ends in themselves, as if they will bring permeant tranquility.