Religious freedom means the freedom to believe whatever you want, and to talk about and act on your beliefs.  It can be a belief in general, not a just a belief in God.  Atheism is a belief; it is a religion, so just imagine that it's the olden times and there is just one church.  Your right to believe what you want to is gone.  How would you feel being discriminated against?  Don't you want everyone to have that same freedom?  Nobody wants to not be able to believe what they believe.