Autonomous vehicles are becoming a reality in places with advanced infrastructure to support their operations. In crowded places, harsh environments, missions that require these vehicles to be aware of the context in which they are operating, and situations requiring continuous coordination with humans such as in disaster relief, advanced-vehicle systems (AVSs) need to be better contextually aware. The vast literature referring to 'context-aware systems' is still sparse, focusing on very limited forms of contextual awareness. It requires a structured approach to bring it together to truly realize contextual awareness in AVSs. This paper uses a human-AVSs (HAVSs) lens to polarize the literature in a coherent form suitable for designing distributed HAVSs. We group the relevant literature into two categories: contextual-awareness related to the vehicle infrastructure itself that enables AVSs to operate, and contextual-awareness related to HAVSs. The former category focuses on the communication backbone for AVSs including ad-hoc networks, services, wireless communication, radio systems, and the cyber security and privacy challenges that arise in these contexts. The latter category covers recommender systems, which are used to coordinate the actions that sit at the interface of the human and AVSs, human-machine interaction issues, and the activity recognition systems as the enabling technology for recommender systems to operate autonomously. The structured analysis of the literature has identified a number of open research questions and opportunities for further research in this area.