Atonality I think begins with Schoenberg's twelve-tone system of composition, in which the twelve tones that make up an octave are lined up side-by-side, permuted into any one of the 12! =479,001,600 ways, and the number you need for the duration of the piece are selected and arranged in bars next to each other. Also, you can stack them on top of each other to make chords, add in dynamics and tempo for greater flexibilty, follow these rules strictly or go about it more relaxed, etc. The idea was to completely escape tonality by giving each of these notes an equal focus, and not prioritize any one note, which would lead to a flavouring of one key or the other.
In a way, this is sort of saying that each note is equally important or relevant. In the most conservative version of tonality, the root note being the key that the piece is in is the most important note for that piece. For instance, say that note were an F, and were you to hit an E by mistake in such a piece, you'll get a very jarring effect, and your mistake will be very noticable to all. Notes in importance will be the A, then the C, in decending order of the order of harmonic overtones. All the way at the top you maybe have grace notes that you can even omit entirely without a great loss of quality, for instance Wilhelm Kempff's recording of the Goldberg Variations. In complete atonality, none of that matters - if you're me, you'll think it all sounds equally like noise and you'll have no idea what's a mistake and what's part of the composition. If you appreciate that stuff, you'll probably be more interested in the overall structuring of the piece, still be clueless about the resolution of chords, but you'll say that's a part of the experience, not knowing what comes next - it's exciting.
Whatever. The point is, a piece of music is a bit like a society of notes, where each note has a role and importance in the piece, one is the king, some are the nobility, others upwardly-mobile guild-members, some are expendable foot-soldiers. Much like in the play that is life, the characters enter and leave at different points, but you still get a clear sense of who each person is, and their role in the grand scheme of things. So then after a fashion, atonality is like socialism, where everybody is (theoretically) equal. Given that socialism actively resists the notion of a god and religion, perhaps in a musical sense, atonality is a bit like atheism. On the flip side, in tonality where there is a single note that all others are subordinate to, one might see this as a form of monotheism for music. Bach tried to include a sense of his religious beliefs in his work, like for instance the further and further you got away from the key, the more distant you were from the ideal, and then when you are back in the home key, everything is safe again and in their right place.
A full examination of the rhetoric of tonality and its ability to convey a sense of ultimate direction for all notes, and perhaps atonality's inherent inability to do the same is something I haven't fully thought over, and is a such something I cannot speak on yet. But it is an interesting question that I will pose to the reader and conclude my writeup on.