Well, actually, it's aitch. But why? Here's how I work it out.
I use a method I developed for myself around year 8 or so when I was trying to figure out the H problem (you can't apply the null hypothesis here unfortunately).
I asked myself which would be the most obvious one. Imagine you're a kid again and just learning the alphabet. Your teacher shows you the letter
and says "Pop quiz! You have two options. One is the correct answer and one is false and made up by me: Aitch or Haitch?"
You'd pick haitch, right? It seems to be the obvious choice because it has the letter h in it. Why would you call it something that doesn't sound like the letter?
But in real life, you realise that people all around you use aitch almost as regularly as haitch, and that's not an accident. The two words are similar enough that one is derived from the other. One must be the original word, and one must be the later derivation. The original word may even come from an ancient time when H was pronounced differently or is a corruption of the word taken from another language. These could explain how H could have a name that doesn't sound like its modern pronunciation.
If the original word were haitch, and someone started saying aitch, why would that catch on? It's silly to omit the one letter that makes it make sense in modern pronunciations of the letter and word. However, if the original word is aitch, the pop quiz example shows how alluring it is for us to want to add the h to the beginning in order to make the word more representative of the letter in modern pronunciation.
So I'd conclude that the more likely scenario is that aitch is the original and correct word and that haitch is simply a modern corruption.
And thus incorrect.