Late Payment Penalties in the UK

Late Payment of Commercial Debts

Applicability

The argument is two fold - one for the original invoice, and one for the invoice for late payment penalties.

For the original invoice, it depends on terms agreed, simple. If the service is validly invoiced or not.

Where it gets silly is invoicing for the late payment penalties.

Customer thinks they only have to pay for purchased services, and where a P/O was issued.
This is simply not true - there are many reasons that money may be due, and Statutory Late Payment Penalties are a result of a legal obligation. No P/O is needed for such charges to be due to be paid.
Customer thinks company policy (e.g. "no P/O, no pay") somehow trumps law.
It doesn't, as a judge will be happy to explain.

Obviously you can help your customer by explaining the late payment legislation.
There is no actual need to invoice for late payment penalties. They are due, that is the law. Invoicing is for convenience and clarity.
Once again, if first time, maybe a credit to clear up the misunderstanding, if that would help avoid issues in future.