Sounds like they’re talking shit mate
https://www.citizensadvice.org.uk/work/pay/getting-paid-when-you-leave-a-job-rights/
You have a statutory right to any holiday pay and they should automatically pay you for anything outstanding at the end of your contract
I don't even know how I'm going to play it to get the money out. Agency work is the absolute ghetto
How much are you owed?
Put it in writing. This is England, we love writing stuff down, we love written rules and regulations. Use that to your advantage. Include the link to the Citizens Advice Bureau above and reference the Working Time Regulations (1998) that states that all PAYE temporary agency workers are entitled to paid annual leave