I can quite understand why you would currently be annoyed (who wouldn't be) but I don't believe banning something because some people use it irresponsibly is the right response.
Organised displays still create noise and much more of it. I have been doing professional displays for the same company for 27yrs now (I was at Danson Park in NW Kent last weekend) and believe me, firing large shells is pretty loud! Many of them are also now accompanied by loud music for hours before and after.
Displays are always more drawn-out when the 5th is mid-week and most concentrated when the 5th is a Saturday - that goes for both public and private displays. We also now have fireworks fairly commonly for New Year and also in some places for Diwali. It wouldn't be unreasonable to reduce availability outside a narrow window around these times (which is how it used to be) which would reasonably contain most displays to within a week or so of the event.
The explosive content of anything available to the public is very low - less than in a shotgun cartridge. They are also quieter than a shotgun. You could reasonably argue that all sport shooting should also be banned if the impact of the noise was the main issue. Shooting might not happen around your house but it does around mine, mainly pheasants and far more times a year than fireworks do. The same argument around disrupting other people's normal course of life could be applied to other activities which people do for pleasure that have an immediate impact on others. Drinking alcohol in large volumes and then being an idiot (drink driving or even just making a lot of noise and mess) is an obvious one on a Saturday night but I believe those who are irresponsible should be targeted, rather than simply imposing a ban. It could (reasonably) be argued that alcohol can be consumed without having an impact on others but there are other activities, such as closing down town centres for carnival parades, going on holiday towing a caravan or even driving a slow classic vehicle (e.g. a Series 2) which cannot be done without having an impact on people around you. These are all likely to cause inconvenience for longer, and on average to more people, than having some fireworks.
Probably easier on balance to tolerate the things which other people enjoy and you don't, unless someone is being a right idiot at which point the authorities could be called (and might even turn up if they want to watch the display!)
Alec
