archive/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2025-08-16 06:31:34 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

mm: mempool: fix wake-up edge case bug for zero-minimum pools

Browse source 
The mempool wake-up path has a edge case bug that affects pools created
with min_nr=0.  When a thread blocks waiting for memory from an empty pool
(curr_nr == 0), subsequent mempool_free() calls fail to wake the waiting
thread because the condition "curr_nr < min_nr" evaluates to "0 < 0" which
is false, this can cause threads to sleep indefinitely according to the
code logic.

There is at least 2 places where the mempool created with min_nr=0:

1. lib/btree.c:191: mempool_create(0, btree_alloc, btree_free, NULL)
2. drivers/md/dm-verity-fec.c:791:
 mempool_init_slab_pool(&f->extra_pool, 0, f->cache)

Add an explicit check in mempool_free() to handle the min_nr=0 case: when
the pool has zero minimum reserves, is currently empty, and has active
waiters, allocate the element then wake up the sleeper.

Link: https://lkml.kernel.org/r/f28a81ba-615c-481e-86fb-c0bf4115ec89@suse.com
Signed-off-by: Yadan Fan <ydfan@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
This commit is contained in:
Yadan Fan 2025-07-19 06:06:51 +08:00 • committed by Andrew Morton
parent 5631da56c9
commit a5867a218d
1 changed files with 33 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

32
mm/mempool.c
View file

@ -540,11 +540,43 @@ void mempool_free(void *element, mempool_t *pool)
if (likely(pool->curr_nr < pool->min_nr)) { if (likely(pool->curr_nr < pool->min_nr)) {
add_element(pool, element); add_element(pool, element);
spin_unlock_irqrestore(&pool->lock, flags); spin_unlock_irqrestore(&pool->lock, flags);
if (wq_has_sleeper(&pool->wait))
wake_up(&pool->wait); wake_up(&pool->wait);
return; return;
} }
spin_unlock_irqrestore(&pool->lock, flags); spin_unlock_irqrestore(&pool->lock, flags);
} }
/*
* Handle the min_nr = 0 edge case:
*
* For zero-minimum pools, curr_nr < min_nr (0 < 0) never succeeds,
* so waiters sleeping on pool->wait would never be woken by the
* wake-up path of previous test. This explicit check ensures the
* allocation of element when both min_nr and curr_nr are 0, and
* any active waiters are properly awakened.
*
* Inline the same logic as previous test, add_element() cannot be
* directly used here since it has BUG_ON to deny if min_nr equals
* curr_nr, so here picked rest of add_element() to use without
* BUG_ON check.
*/
if (unlikely(pool->min_nr == 0 &&
READ_ONCE(pool->curr_nr) == 0)) {
spin_lock_irqsave(&pool->lock, flags);
if (likely(pool->curr_nr == 0)) {
/* Inline the logic of add_element() */
poison_element(pool, element);
if (kasan_poison_element(pool, element))
pool->elements[pool->curr_nr++] = element;
spin_unlock_irqrestore(&pool->lock, flags);
if (wq_has_sleeper(&pool->wait))
wake_up(&pool->wait);
return;
}
spin_unlock_irqrestore(&pool->lock, flags);
}
pool->free(element, pool->pool_data); pool->free(element, pool->pool_data);
} }
EXPORT_SYMBOL(mempool_free); EXPORT_SYMBOL(mempool_free);