Increasing size for Linux kernel module kfifo_alloc() which fails for 128MB

103 Views Asked by At

We have an embedded product with quad Cortex-A53 cores, 4 GB RAM and runs Linux Kernel v5.4 aarch64. Whole Linux currently consumes only 200MB to work, therefore I have plenty of RAM available to use.

I made a kernel module which reads 20MB/sec data from an external device through PCIe bus with chunks of 1MB. I want to push this 1MB data in the interrupt context to a FIFO works in the kernel module to be transferred later to a user space.

I looked for available fifo patterns in the Linux kernel and saw that kfifo_alloc() and bunch of APIs to put and get data already provided. This suits my purpose perfectly. As I understand it supports single producer and single consumer approach without any locking.

Required FIFO size for my application should be around 128MB. I already have 4GB RAM, therefore I thought initially that allocating such memory should have no issue.

But following code returns ENOMEM error unexpectedly

ret = kfifo_alloc(&io_dev.streaming_dma_fifo, 128 * 1024 * 1024, GFP_KERNEL);
if (ret) {
    dev_err(&pci_dev->dev, "%s:error kfifo_alloc, ret=%d\n", __func__, ret);
    return ret;    
}

Then I searched the stackoverflow and saw people use kmalloc() complain about same thing. I think kfifo_alloc() function internally also uses kmalloc().

Then I executed following code as suggested again in this forum:

dev_info(&pci_dev->dev, "KMALLOC_SHILFT_LOW:%d, KMALLOC_SHILFT_HIGH:%d, KMALLOC_MIN_SIZE:%d, KMALLOC_MAX_SIZE:%lu\n", KMALLOC_SHIFT_LOW, KMALLOC_SHIFT_HIGH, KMALLOC_MIN_SIZE, KMALLOC_MAX_SIZE);

And got these outputs for my system:

KMALLOC_SHILFT_LOW: 7
KMALLOC_SHILFT_HIGH: 13
KMALLOC_MIN_SIZE: 128
KMALLOC_MAX_SIZE: 33554432

What I understand from this output is that max size that I can allocate with kfifo_alloc() is limited to 32MB.

If I try to reduce the size requested for kfifo_alloc() from 128MB to 32MB, it returns no error.

Questions:

  1. My FIFO RAM doesn't have any special requirements such as it should be contiguous, or atomic or DMAable or something else. I didn't understand why kfifo_alloc() fails with 128MB while I can allocate same amount in the user space easily with malloc(). Is there a way to ask kfifo_alloc to use malloc or kernel equivalent of it, instead of special kmalloc?

  2. Is there a way to increase KMALLOC_MAX_SIZE defined as 32MB for my system?

  3. What are the consequences of setting KMALLOC_MAX_SIZE as 256MB for example?

  4. What is the most suggested way if my driver requires a FIFO larger than 32MB?

0

There are 0 best solutions below