cxiuxfer/copy_to_user_string latency increases with size of reads

[vallerul]

Hello,

I have been trying to debug an issue, where all the data is in kernel pinned memory but the performance differs with the amount of data that is read each time.
The application tries to do fread for data that exists in kernel memory, where it is copied from kernal memory to user space application buffer.

I see that, the latency differs depending on the block size or the amount of data that is being copied from kernel memory to user space buffer.

For example - 16M reads from kernel memory take a lot more time than 4M/1M reads.
Is this expected? and since we are reading from kernel memory - may i know why do i see such a huge latency difference.

This difference is causing a huge drop in GPU efficiency, where i get 20% utilization for 16M reads while i get 75% utilization of 4M reads.

Operating System: CentOS 7.
Kernel : 3.10.0-514.21.2.el7.x86_64

Please do let me know, if i need to provide any more information.

Thank you.

[avij]

I don't know your exact access pattern, but could it be that smaller blocks of memory fit into CPU's cache, but the bigger blocks need to be retrieved from RAM?

[TrevorH]

Kernel : 3.10.0-514.21.2.el7.x86_64

Run yum update as that kernel is ancient and unsupported. It's from 7.3 iirc and the current one is 3.10.0-862.14.4.el7.