There may be a much faster way to handle these 1-3 byte SGDMA transfers.
I believe it to be true that there would only be one 1-byte pbuf in a chain, is this true? If it *is* true, you could ensure that all pbufs are allocated with an extra 4 bytes of payload. This would be done for PBUF_POOL (not sure about PBUF_RAM). Take the 1 to 3 bytes of the next pbuf and add it to the end of the payload of the preceding one. Increase that len by 1 to 3 and set the second one to 0. Leave the chain in place but skip pbufs with len==0.
This would be very efficient for the common case of a trailing short pbuf chain.
For PBUF_REF you may have to unwind to a new pbuf. Or ensure 0-copy doesn't use small payloads (probably not hard to manage).