from wikipedia:
"Understanding the outcome
It is not possible to understand the conclusion of Lem's Fiasco without prior knowledge of Fred Hoyle's work. The dscovery scenario described at the very end of the novel is exact realization of the same solution/decision which concludes Hoyle-Elliot's "Andromeda Breakthrough", another acclaimed sci-fi novel. (Info from Lem's circle.)
The original, intelligent, mobile biological inhabitants of Quinta consciously transformed themselves in a project, AFTER they developed a technical civilization with a high level of automation and their global politics degraded into an unsolvable, violent global conflict. They became a battery of statically existing, more or less vegetative systems, which are highly resistant to destruction and change - as well as immune to the instincts, which cause all biological civilizations to develop conflicts. The automated industry they had created keeps running and thus, the robotized conflict continues above their heads, while they are vegetating under."
[/b]
.............
If you remember, in the video message sent to the Hermes, there's a scene in which a crowd of *mobile* Quintans enter some kind of vehicle or machine, which then drops beneath a mercury-like field, into the earth, presumably. This is probably the project referred to here..
Seems pretty satisfactory.
so, maybe their civilization is heavily automated with the Quintans themselves involved in a collective meditative huddle with few external stimuli... hence the early attacks are from the automated war-sphere and the Hermes' efforts at communication fall on deaf ears... (if this isn't the case, then the Quintans are either paranoid or insane or otherwise incomprehensible, attacking a more advanced race which looks peaceful is pretty irrational behaviour)
but allow me to speculate -
at what point do the Quintans themselves become aware of the Hermes and the fact that their warsphere has been trying to destroy it? Is there reason to believe that the Quintans EVER become aware of the Hermes? Perhaps the Quintans are truly and completely closed off from external stimuli (but connected amongst themselves), and the Hermes only ever communicates with the automated Quintan technosphere (a notion rejected by DEUS, yes, but DEUS is shown to be fallible)
The auto-technosphere would want to avoid contact at all costs (having no interest in alien races, and its prime directive the protection of the defenseless Quintans), and would try to destroy or at least discourage the Hermes from pursuing contact.
This could explain why the "Quintans" are hostile even after they begin negotiations with the Hermes. The auto-technosphere could reason that by resisting proper contact and suffering minor damage from Herme's reprisals, the Hermes would become fed up and leave... and secret of the Quintans would remain a secret..
The warsphere could even be a defensive system against ETs, rather than the result of a space arms race (in which case the Quintans transformed themselves for some other purpose)..
extremely speculative.. but there are so many variables. Why the hell didn't DEUS figure this out?