"At some point in the future it will reach a level of complexity and intelligence that it can be used as a weapon." - I think Michael's post, and the Israeli missile system makes it clear that we have ALREADY reached that point.

What worries me is that the military will end up meddling in recursively self improving AI without taking the kind of extensive precautions that, say, the SIAI would take. Ever seen/heard of the sorcerer's apprentice?

"The apprentice tires of fetching water for a bath or tank, and enchants a broomstick to do the work for him, using magic he is not yet fully trained in. However, soon the floor is awash with water, and he realises that he cannot stop the broom because he does not know the magic word to make it stop. Despairing, he splits the broom in two with an axe, but each of the pieces takes up a pail and continues fetching water, now faster than ever."

the military and the defense contractors who work for them will meddle in "magic" they don't fully understand, and the end result could well be the destruction of every living thing on the planet.

I don't think that having the military push AI research is, necessarily, a bad thing. Think of it this way: AI will continue to advance, weather the military is involved or not. At some point in the future it will reach a level of complexity and intelligence that it can be used as a weapon. At that point, which is better:

a) Having done a lot of research by the military already on how to use AI and also how to defend against it. Or,
b) Having not invested any time researching the subject and being an easy target.