First of all, I do not think that GUIs are bad ideas whatsoever. There are tasks where they are really useful. If you want to edit images, you want a program with graphic interface. If you want to view a movie, you can't do it in a text mode. Many programs look better when they have a fancy colourful output. What this is about is how the programs are controlled. And this is not at all about games, just about programs to be used as tools for some purpose.
In fact, this is not how a program talks to you. This is about how you talk to the program. So, in fact, if you have a program with graphical output and some reasonable input, it is more than OK.
First, look at a random GUI program you find somewhere. There's bunch of buttons, some menus, options, stuff like that. People say it is intuitive and easy to use. However, it is so, because they are used to such programs. And not only programs. If you look at a office desk, there are some papers on it, some are in a drawer and shelves around. They are where you put them and they just lye there. You can take them and write to them. And you can put them somewhere else. But they are still objects, inanimate entities without any intelligence or soul. It may create feeling of familiarity, if programs (and documents) on a computer behave the same. But it also creates the feeling that programs are inanimate entities, objects.
This, however, is not and doesn't have to be so. If a computer program can be clever enough to control underground trains, aircraft traffic and beat humans in a game of Jeopardy, you can hardly consider them inanimate objects, like stones or papers. This does not only limit the programs that are designed to make the impression of a paper. This limits your thinking. You do not see the full potential of how helpful a computer can be.
Another problem is with communication. First of all, using a mouse is slow and inaccurate. Try hitting an exact pixel with it and look how long it takes. Or, even if you split the screen into 100 pieces, clicking on single one is relatively slow process. Now, you have around 100 keys on your keyboard and if you aren't really slow typist, you can do as many as 120 key strokes per minute, that is, 2 strokes per second. This makes it possible to choose one of 10 000 options in a second by keyboard.
But the more important problem of communication is the nature of it. If you are one of the 99.9% of people in this world, you think in a language. It doesn't really matter which language. And, of course, you can imagine sounds, images, 3D objects, smells, tastes, etc. And, a large part of your thinking is influenced by feelings (a lot larger than the thinking of a computer, at last). But most of the thinking you do is in a language.
If you have a text-based (and I don't mean a GUI drawn by textual characters) interface, you input something very close to a language. It's not completely natural, because, in order for the computer to understand it, it must be too strict in its rules. But it is relatively close to a language. The translation from your thoughts to it is easy, if you happen to know the language and you are able to understand your own thoughts (which, let us admit it, is the harder part).
But if you have a GUI, where you click some buttons, these are separate, independent actions. You don't translate the thoughts. You need to cut the thoughts in pieces and think how to reach some effect by the independent actions. You're wasting more thinking on disassembling the process you want to do into parts and it is not merely a translation of what you do. So, in fact, you stop thinking and just do the clicking after a while. It is very convenient, because thinking is tiring thing and you don't even notice it, because you're not thinking about it. Again, the GUI degrades both your abilities and the amount of work the computer can help with.
Third problem is of the attitude. If you have the buttons, you have some set of choices. The computer tells you „now, you must choose“. It is a rude thing to be forced to something by a computer (more so if you consider it to be an unintelligent thing). On the other hand, if you have a language to talk to the computer, it acts as an servant. You can tell a servant „go to a shop and buy these things“ or „take these numbers and sum them up for me“.
And, of course, there's the thing that you can do more with a language than with clicking on buttons. And it is possible for one program to generate input for another, which is hard with GUI.
So, if GUI is so bad, why does it exist? Well, there are reasons, but they are the wrong reasons in the first place. One of them was already mentioned. Not thinking is comfortable and easy. Who cares if it decreases productivity? People will feel like they know what is happening, so they are more eager to buy computers. The fact that it doesn't help them much doesn't stop them.
Another reason is, designing a language or keyboard interface that is easy to use, powerful and comfortable is hard. You will most probably get it wrong the first few times. But with GUI, you can just „draw“ the buttons, you have a list of features, so there'll be a button for each feature and the worst thing you can do is create minor annoyances, like having two related buttons far away from each other.
And the third one is, if you meet a complete stranger who speaks a different language, you might try to gesticulate to do some communication. That is an awkward way, but it works most of the time. It is the same with GUI. You can gesticulate, what you really want. However, most people prefer to learn languages the people around them use, for everyday life.
And, one of the classics.