The phase of a spectral component does not have an audible effect unless other spectral components of the same frequency are present, i.e., there is interference.
In the case of frequency overlap, the amplitudes will either add or subtract, resulting in a perceived change in the sound.
If the FM spectrum contains frequency components below 0 Hz, they are folded over the 0 Hz axis to their corresponding positive frequencies (akin to aliasing or folding over the Nyquist limit).