According to the World Health Organisation the number one global health issue for young people is their mental health. For students, mental well-being is associated with effective learning, and their ability to navigate through university/college, having the resilience to cope with the challenges and stresses of student life. In Ireland, Computer Science (CS) non-progression rates are alarming, with a large number of students failing to progress each year. Currently non-progression rates are 25% in CS, significantly higher than the national average of 16% across all other fields of study. On top of the normal stressors of transitioning or returning to university, CS students are arguably exposed to a unique set of factors that could further induce anxiety. First, they typically have no formal CS exposure or training to draw on. Second, the number of contact hours and workload are high. Third, CS courses includes programming modules. For some, learning to program is diffcult and many struggle to master the core concepts. Learning typically takes place in a lab environment where inexperienced programmers will begin to type (\code") shortly after being presented with a problem rather than spending time designing a solution. Thus the lab becomes active and busy from the onset, making struggling students cripplingly perceive their peers know more. Further, novice programmers use the compiler to constantly monitor their progress and error messages can be perceived as negative feedback. Such an environment can create or compound anxiety and stress. At our institution a large number of CS students register for counselling services or leave. In this paper we present a systematic literature review on the role of anxiety when learning to program. The work is novel, valuable, and timely. The approach used is systematic, in that a structured search of electronic resources has been conducted and the results are presented and quantitatively analysed. A detailed discussion on the findings is provided and important implications for the teaching and learning of programming are described.