Is an electromagnetic wave a wave?
I'll begin by discussing the concept of a medium. There is a history of the idea of 'medium'. Some of the first waves to be studied in classical physics were sound waves in air. Then waves in other 'media': sound in other fluids and solids and strings. In these cases the medium is obvious.
The first electric and magnetic fields to be studied were unchanging or static. Consider the electric charge interaction: there is a force between any two charges. It was found convenient to say the same thing in terms of an electric field. Every charge was associated with an electric field. A charge has an electric field and if you put another charge at a point in the field the electric force on it is the product of the second charge and the strength of the field (a magnetic field has a partially analogous definition). Put this way, the field seems like nothing but a convenience. But the force at a distance and so the field seem to have some reality. However, there is no medium. This seemed weird because most other forces (except gravity) seemed to be contact forces or required an intervening medium--e.g. being pulled by a rope. When it was found that electric and magnetic fields vary with time and that they interact it was only a period of time before Maxwell formulated the classical equations of electromagnetism. These equations predicted waves whose speed was the same as the speed of light and of course it was inferred that light is an electromagnetic wave (this was before the quantum era). It was regarded as weird that these waves did not seem to have a medium. Some physicists postulated a medium called 'aether'.
Relativity seems to eliminate the medium. Einstein's theory of special relativity then demonstrated (or at least seemed to demonstrate) that there is no need for an aether.
But the quantum field or vacuum may be seen as a medium. But today we think that particles and waves may be disturbances in underlying quantum fields (and some people are calling this a medium).
The question of the medium is open--is there one? What is a medium? It is not clear that we have come to the end of the medium or no medium question for fundamental particles and fields.
From a practical perspective focus on the phenomena is important. So it seems to me that a good idea is to focus on the phenomena rather than the question of essence (is there or is there not a medium) not because reality has no importance but because we don't yet know what it is--i.e., what makes it up. Some physicists seem to think that the phenomenon is what's important. But the question of the medium is also important. I happen to think that the idea of reality and what is real is very interesting but we shouldn't let it get in the way of the phenomena. Some people insist on focus only on phenomena; but the history of science shows that focus on the underlying concepts is crucial to progress.
A wave then, as a phenomenon, is a propagating effect. Some kinds of propagating effect. They can be smooth or, as in shock waves in air (sonic boom), have a sharp front. Cerenkov radiation is the light analog of shock waves (they occur when particles travel faster than the speed of light in some medium). They may be extended or travel in groups. A solitary wave is a single hump on the surface of the ocean or a river channel traveling as a wave: tsunamis can be modeled as solitary waves (solitary waves, also called solitons, occur in other situations as well). Some waves have obvious mediums and others don't.
What's really interesting that the variety of waves is enormous. Practically any graduate text in electromagnetism will discuss em (electromagnetic) waves. Two great books on waves are Linear and Nonlinear Waves by GB Whitham and Waves in Fluids by MJ LIghthill (these are dated but I know them from my engineering past). The range of wave phenomena is fascinating and amazing.
An electromagnetic wave is a disturbance in electromagnetic fields. These disturbances can be treated as waves to good effect. Electromagnetic waves aren't classical waves in the sense of having a medium through which they propagate that allows for different relative speeds of the waves depending on the motion of the medium relative to the observer. The speed of light seems to be independent of the motion of the source or of the observer. These waves still have all the other characteristics of classical waves including frequency, wavelength, wave speed and a wave equation.
Before the development of the special theory of relativity, scientist were looking for a classical medium for the propagation of light, and they even had a name for it. It was the aether: Aether (classical element)
Measurements made by Michelson and Morley, Michelson–Morley experiment , failed to detect this aether, and the idea of a classical medium for light was abandoned after Einstein's theory of relativity worked so well with no need for a classical medium.
Yes, electromagnetism does propagate as a wave.
Without going into technical details and high-level physics, the answer is that both definitions are useful. The first definition would be better if it mentioned that the electromagnetic waves were not included. Then the two definitions would not seem to contradict each other.
The electromagnetic wave is a wave of probability amplitude for where the particle (photon) is. So the electromagnetic wave is really both a wave and a particle - as is all matter. Hence wave-particle duality.