Generative Artificial Intelligence in GP Training and Workplace Based Assessment: Guidance for GP registrars and GP Educators

This guidance builds on the RCGP statement on the use of artificial intelligence (AI) in postgraduate training, examinations and registration – please ensure you are familiar with that statement before reading the guidance on this page.

Generative Artificial Intelligence (AI) refers to a class of AI systems designed to create new content, such as text, images, music, or video, by learning patterns from existing data. A subset of generative AI is Large Language Models (LLMs), which are specifically trained to understand and generate human language. LLMs perform tasks such as text generation, translation, and summarisation. While GPs will mostly use LLMs, the below guidance uses "Generative AI" throughout as there will be occasions when AI outputs are not just text based.

To ensure generative AI¹ is used responsibly, effectively and safely in medical education, ethical AI literacy is crucial. AI literacy involves:

• understanding the capabilities - and limitations - of AI, including tools to enhance health care
• integrating AI tools into teaching
• ensuring inclusion, equity, and ethically responsible use of AI for societal good.²

These skills are necessary not just for GP registrars undertaking their training, but effective, critical, and ethical use of AI technology will be a skill all GPs need.³ One use of generative AI in a GP training context is as a tool to support Workplace Based Assessment (WPBA). The following guidelines seek to provide a resource to help support the use of generative AI by GP registrars and GP Educators specifically within the context of GP Training and WPBA.

GP Educators should expect GP registrars to use generative AI in the ways outlined in this guidance. The RCGP does not require GP registrars to declare the use of generative AI in entries on the Trainee Portfolio, however it is the responsibility of the GP registrar to ensure they are using generative AI in a way that is consistent with this guidance and with their wider professional responsibilities.

There will be other areas where generative AI could be used by GP registrars - such as in transcribing notes, summarising patient records or generating referral letters. However, these uses are outside the scope of this guidance as they are equally applicable to GPs practicing independently as they are to GP registrars. The GP Curriculum capability "Data gathering and interpretation" covers the gathering, interpretation and use of data for clinical judgement and outlines how GP registrars must be able to selectively gather and interpret information from a wide range of sources. Effective use of AI can help develop competence within this capability, but over reliance risks doing the opposite.

GP registrars must be aware of the confidentiality issues that can arise from AI use, and must ensure that they use AI in a way that is consistent with any relevant data protection requirements (or any other requirements - such as those around patient consent, or which programmes, models or resources are approved for use) that may apply - whether those of their employer, deanery or other relevant organisation.

Prompts and basic techniques

AI prompts are specific instructions or questions given to an AI model to generate a desired response. There are many tutorials on how to structure prompts online with examples, and those examples can be adapted and used for specific examples. While these prompts and techniques are largely standardised, users should be aware that there will be variation between AI models.

Some AI models allow users to upload reference material for the AI to retrieve from. In a GP training context this could include RCGP Curriculum topic guides, or WPBA guidance, for example. Uploading reference material may help mitigate the risk of AI ‘hallucination’. Generative AI hallucination (or confabulation) occurs when an AI generates information that is incorrect or nonsensical, despite it sounding plausible. The AI might fabricate details or fill in gaps with invented information, presenting it as if it were true. This happens because generative AI relies on patterns and associations learned from its training data and when faced with incomplete, ambiguous, or unfamiliar inputs, the AI may produce responses that seem coherent but are actually false.

Reference and Information Synthesis

The broadest area where AI can support learning (including within WPBA) is as a "quick reference and information synthesis tool".⁴ It can be used to gather and summarise data and information from many different sources.

GP Educators should be prepared for GP registrars to use generative AI in this way and can use it as an opportunity to emphasise and promote the skills required in assessing evidence, information and data.⁵ This includes ensuring when using evidence, information and data, we are able to trace it back to its origins. This is nothing new - the internet has posed challenges in this area for years, and both patients and healthcare professionals can misunderstand information and data found online due to a lack of health information literacy, or web literacy.⁶ The same risks exist with generative AI - even when generative AI provides references and sources for its findings, these are not always accurate, or complete and can sometimes be completely fictional.⁷ GP registrars should learn how to write prompts that encourage the AI to double-check its own output and include references and website links with each output.

Generative AI can create outputs and answers, but it does not 'understand' these outputs. As such, while generative AI can be used as a reference and information synthesis tool, it must be used in a critical way, understanding its limitations and for users to be prepared to challenge its findings.

Feedback and Teaching

Feedback is a key part of assessment in Medical Education.⁸ While not a substitute for feedback from supervisors, colleagues and experts, generative AI can increase access to feedback and guide learning.⁹ For example, generative AI can review a submission made (e.g. a conclusion made on some data, or a list of topics in a subject area) and identify knowledge gaps, or misunderstandings. As with all feedback, "receiving feedback is not a passive, simple act".¹⁰ Feedback must be reflected upon, and, in the case of feedback received from AI, challenged and fact checked to ensure it is accurate. AI may give a user what it claims to be a comprehensive list of topics, but the user must then check to ensure that list is accurate. GP registrars should be encouraged to critically engage with the AI generated output; question validity, verify sources and discuss discrepancies and differing opinions.

More generally, generative AI can also be used to support teaching and learning as a tool to produce slides, summaries, MCQ questions or as ‘actors’ in case simulations.

Reflection

Generative AI can "complement, not replace, the development of genuine reflective skills".¹¹ Overuse or over reliance on generative AI risks, "undermining the very purpose of reflective practice" as users will not develop reflective skills themselves.

However, if it is used effectively, generative AI can be a useful tool to aid and support reflection. Though it cannot reflect for you, it can provide prompts and suggestions for areas of reflection, that are then used as a starting point to enable personal reflection. These prompts can then be used to guide and assist reflection in a case but not replace it. Potential areas in which AI could help guide reflection could include:

• Using the RCGP website to review the MRCGP Curriculum capabilities and patient groups.
• Suggesting capabilities that may be relevant to the case.
• Suggesting patient groups that may be relevant to this case.

Given that reflection must be about real cases and patients, users must ensure that any prompts are compliant with their employer, deanery or other relevant organisation's data protection policies. Generative AI should not be used to generate reflections without real patient experience.

As with feedback, it is not always possible to access peer support when reflecting and so generative AI can be a useful tool when it may not be possible to immediately access peer support or hold a reflective conversation with a colleague. Again, the limits of generative AI - and its risks - should be recognised. The personal emotions and intricacies "involved in reflective practice may be oversimplified by current AI limitations".¹²

Summary

Development of AI literacy is essential and the capabilities - and limitations - of generative AI must be understood by anyone seeking to use it in a professional context.

It must be used critically - AI-generated references and sources may not always be accurate or complete, and feedback from AI should be reflected upon and fact-checked to ensure accuracy. AI outputs are based on pattern prediction from preceding context (the “most likely next token”) rather than a genuine understanding of facts. Generative AI focusses on answer generation, not generating evidenced truth. Hallucinations and confabulations can occur and the error rates for generative AI are still not well understood. It is the responsibility of the user to check and verify each output.

Generative AI can be a valuable tool for those undertaking WPBA, but as a resource to support the development of skills and competences required for independent General Practice - not to replace them.