Sourced by A/Prof Vanessa Panettieri, Editor (Educational), AFOMP Pulse
It’s not your device, it’s how you use it – Sensitivity and specificity of PSQA devices in the SEAFARER head and neck study
May L, Baeza-Ortega J, Hussein M et al Radiotherapy and Oncology, 2026; 219
DOI: 10.1016/j.radonc.2026.111493
What inspired this study?
PSQA requires a lot of resources and its ability to identify clinical problems has been shown to be limited. (Kry et al 2014) Our earlier work identified large differences in the performance of clinical PSQA systems as well as in the robustness of the plans made by the participants using identical planning guidelines. (Lehmann et al 2022) In this study we centralised the creation of plans taking out this variable. (Baeza-Ortega JA et al 2025) We automated the processes and created a web interface for participants to request the audits and to submit their results. The system also allows centres to compare their results to those of peers, in an anonymized way in real time.
What were the key challenges in this research? We initially anticipated that promoting the study and encouraging participants to take time to perform clinical measurements may be one of the key challenges. However, we were amazed by the strong enthusiasm of the radiotherapy community and the willingness for centres to participate. We even had people asking for additional plans they could deliver. Instead, we found the main key issue to be identifying and adapting our methodology to the wide range of equipment and treatment planning systems used. We also spent a considerable amount of time designing our treatment plans to be representative of practices for most of the community, however with the wide range of clinical practices in place this was not always possible.
What are the key takeaways from this study?
The study had several really interesting findings, but probably the most exciting takeaway is that all of the clinically used PSQA devices included were shown to be capable of achieving a ‘perfect’ result, that is being both sensitive and specific to clinically relevant delivery errors. This means that the performance of PSQA devices is strongly influenced by user procedures and not just inherent characteristics of the device.
Secondly, this study illustrated the importance of sensitivity assessments of PSQA systems. We found that for the cohort tested, 48% of centres passed at least one plan with a clinically significant dose error, using their clinical PSQA system.
How does this research impact the future?
Several clinical centres have informed us that they have changed their PSQA following their participation in the study. In that sense we hope this work has contributed to improvement in patient care.
We are currently able to offer additional centers participation in the study. If you are interested in assessing the sensitivity of your PSQA system for Head and Neck, SABR spine or SRS plans, please sign up here:
https://seafarerapp2-aea6e3d0fdegagez.australiasoutheast-01.azurewebsites.net/request_account
Dr Lauren May
Post-Doctoral Research Fellow
Prof Joerg Lehmann
Principal Medical Physics Specialist
Radiation Oncology Department
Calvary Mater Newcastle (Australia)
References
Kry SF et al. 2014 Institutional Patient-specific IMRT QA Does Not Predict Unacceptable Plan Delivery IJROBP 2014 https://doi.org/10.1016/j.ijrobp.2014.08.334
Lehmann J et al. 2022 SEAFARER – A new concept for validating radiotherapy patient specific QA for clinical trials and clinical practice Radiother Oncol https://doi.org/10.1016/j.radonc.2022.04.019
Baeza-Ortega JA et al. 2025 A proof of concept for improving comparability of dosimetry audits through centralised planning phiRO https://doi.org/10.1016/j.phro.2025.100879