Friday, February 8, 2013

GFP_ATOMIC Allocation Failures

I see a good number of bug reports or complaints about GFP_ATOMIC allocations failing.  The general consensus seems to be that if there is an allocation failure in the kernel, something is wrong with the kernel (or specifically the VM).

GFP_ATOMIC allocations are special.  If you call kmalloc(GFP_KERNEL), or alloc_page(GFP_KERNEL), you actually implicitly pass a couple of other flags in:

 #define GFP_KERNEL (__GFP_WAIT | __GFP_IO | __GFP_FS)
The most important of those if __GFP_WAIT.  If you call in to the allocator with that flag, it tells the VM that it is OK for you to sleep for a bit.  The VM will actually put the allocating process to sleep in order to go off and free up other memory.

But, GFP_ATOMIC does not have the __GFP_WAIT bit set.  It effectively says to the kernel, "give me memory NOW and don't put me to sleep."  But, the implication here is that the kernel will not be given a chance to go off and free other memory.

Let's say you walk in to a fast food joint because you want french fries.  You walk up to the counter and order your fries, but they are out.  If you can wait (__GFP_WAIT), you'll get your fries in a moment once they cook another batch.  But, if you are in a hurry and can not wait for another batch to be cooked (GFP_ATOMIC) you are going to have to walk away empty handed.  You can complain that there should have been more fries cooked in advance (increase min_free_kbytes), but that only helps keep them from running out less often, not from ever running out.

Inherently, being in a hurry (GFP_ATOMIC) exposes you to the possibility of failure.  If you do not want failures, then do not use GFP_ATOMIC.