The biggest thing I've noticed when listening to other players is their ability to "follow the changes" - not even really in a Jazz sense.
If a track goes from one key to another it's very easy to follow that with a solo/lead playing, but I've found it super hard to follow chords within a key. For example, going from Am to Em in your typical rock song.
To clarify: if I stay in the key of Em when I'm soloing over the Am I'm essentially playing A Dorian, a mode of E Aeolian - so literally nothing is changing. I guess the solution is to play A Aeolian then E Aeolian and just take that major/minor six clash as flavour?