In my favourite mailing-list, The Faust List, we had a small debate about this topic. It was raised due to experience that especially avantgarde music often makes some people furious. Some of the artist mentioned here was Sun Ra and Captain Beefheart.
This is a question with some complexity attached to it... But here is how I tried to respond to it (and some more which I added just now):
While working with people for many years, I have learned that sometimes it is impossible to predict what provoces a certain individual. This can be both clients and/or professionals I worked with or co-operated with. While working with behaviour problems, I also learned that it is very difficult to predict what triggers anger or rage, unless I was given the opportunity to investigate further. What I found as a common ground, was that most of them had an internal unsafety, sometimes based on disorganized attachement or attachement disorders.
"Attachment disorder refers to an inability to form normal attachments with caregivers during childhood. This may have adverse effects throughout the lifespan. Results of a study showed a positive and strong correlation between the security of the child-mother attachment representation and positiveness of self. It also showed significant and positive correlations between positiveness of self to competence and social acceptance, to behavioral adjustment at school, and to behavioral manifestations of self-esteem"
I am not saying that anger by adults towards avantgarde music co-relates to this (so far). BUT disorganized attachement makes people want to have more control! What I also found was that if someone had low self-esteem, or insecurity in the self and you combine this with a strong need of control, it would often create some form of behavioural problem, depending on how the people around this certain individual would relate to him/her. When these individuals are left in charge, and can do as they please, without being asked questions or in other ways provoced, no problems whatsoever will appear. Other than maybe someone don't like what they do at times.
Say that someone has been exposed to a boss or someone in charge who doesn't want to debate a decision and/or controlling every aspect of someones job. My theory is that this boss has some of these problems I describe. Certainly if they get angry if you try to debate something or ask uncomfortable questions. This need of control can easily adapt to how we relate to most of our world, also art.
It is also easy to understand that how we look upon our selves (self-esteem) affects our feelings, and our feelings in the next step affects our behaviour. And for most people art, and also music is affecting (or as someone pointed out, inspires) feelings as well. This is probably also the order of it; so how we look upon our selves would have a strong affect upon how we respond to art, which again will affect our physical reactions to art, and certainly music.
Music, unlike most art, is difficult to escape, especially when we are exposed to it in closed confinements. Or rooms, work, tv or radio at home...not everybody can slam the door to close it out. A painting or a sculpture that provoces you is easier to shut out. When exposed to something you don't understand and have no control of how you are exposed to it, it surely does something to you - emotionally. Response is often with silence, withdrawal, sadness, or in this case anger.