| CVE |
Vendors |
Products |
Updated |
CVSS v3.1 |
| In the Linux kernel, the following vulnerability has been resolved:
phonet/pep: disable BH around forwarded sk_receive_skb()
The networking receive path is usually run from softirq context, but
protocols that take the socket lock may have packets stored in the
backlog and processed later from process context. In that case
release_sock() -> __release_sock() drops the slock with spin_unlock_bh()
and then calls sk->sk_backlog_rcv() with bottom halves enabled.
Typical sk_backlog_rcv handlers process the socket whose backlog is
being drained, so the BH state at entry is irrelevant for the slocks
they touch. pep_do_rcv() is different: when the inbound skb targets an
existing PEP pipe, it forwards the skb to a different *child* socket
via sk_receive_skb(). That helper takes the child slock with
bh_lock_sock_nested(), which is just spin_lock_nested() and assumes BH
is already off. The same child slock therefore ends up acquired with
BH on (process path) and with BH off (softirq path):
process context softirq context
--------------- ---------------
release_sock(listener) __netif_receive_skb()
__release_sock() phonet_rcv()
spin_unlock_bh() __sk_receive_skb(listener)
[BH now ENABLED] [BH already disabled]
sk_backlog_rcv: sk_backlog_rcv:
pep_do_rcv() pep_do_rcv()
sk_receive_skb(child) sk_receive_skb(child)
bh_lock_sock_nested(child) bh_lock_sock_nested(child)
=> SOFTIRQ-ON-W => IN-SOFTIRQ-W
Lockdep flags this as inconsistent lock state, and it can become a real
self-deadlock if a softirq on the same CPU tries to receive to the same
child socket while its slock is held in the BH-enabled path:
WARNING: inconsistent lock state
inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
(slock-AF_PHONET/1){+.?.}-{3:3}, at: __sk_receive_skb+0x1cf/0x900
__sk_receive_skb net/core/sock.c:563
sk_receive_skb include/net/sock.h:2022 [inline]
pep_do_rcv net/phonet/pep.c:675
sk_backlog_rcv include/net/sock.h:1190
__release_sock net/core/sock.c:3216
release_sock net/core/sock.c:3815
pep_sock_accept net/phonet/pep.c:879
Wrap the forwarded sk_receive_skb() in local_bh_disable() /
local_bh_enable() so the child slock is always acquired with BH off.
local_bh_disable() nests safely on the softirq path.
Discovered via in-house syzkaller fuzzing; the same root cause also
on the linux-6.1.y syzbot dashboard as extid 44f0626dd6284f02663c.
Reproduced under KASAN + LOCKDEP + PROVE_LOCKING, reproducer:
https://pastebin.com/A3t8xzCR |
| In the Linux kernel, the following vulnerability has been resolved:
bpf: Do not allow deleting local storage in NMI
Currently, local storage may deadlock when deferring freeing selem or
local storage through kfree_rcu(), call_rcu() or call_rcu_tasks_trace()
in NMI or reentrant. Since deleting selem in NMI is an unlikely use
case, partially mitigate it by returning error when calling from
bpf_xxx_storage_delete() helpers in NMI. Note that, it is still possible
to deadlock through reentrant. A full mitigation requires returning
error when irqs_disabled() is true, which, however is too heavy-handed
for bpf_xxx_storage_delete().
The long-term solution requires _nolock versions of call_rcu. Another
possible solution is to defer the free through irq_work [0], but it
would grow the size of selem, which is non-ideal.
The check is only needed in bpf_selem_unlink(), which is used by helpers
and syscalls. bpf_selem_unlink_nofail() is fine as it is called during
map and owner tear down that never run in NMI or reentrant.
[0] https://lore.kernel.org/bpf/20260205190233.912-1-alexei.starovoitov@gmail.com/ |
| In the Linux kernel, the following vulnerability has been resolved:
gpio: shared: fix deadlock on shared proxy's parent removal
Commit 710abda58055 ("gpio: shared: call gpio_chip::of_xlate() if set")
used the mutex embedded in struct gpio_shared_entry to protect the
offset field which now can be modified after assignment. The critical
section however is too wide and introduced a potential deadlock on the
removal of the shared GPIO proxy's parent.
Make the critical section shorter - only protect the offset when it's
being read.
While at it: mention the fact that the entry lock is now also used to
protect against concurrent access to the offset field in the structure's
documentation. |
| In the Linux kernel, the following vulnerability has been resolved:
bpf, sockmap: Fix af_unix iter deadlock
bpf_iter_unix_seq_show() may deadlock when lock_sock_fast() takes the fast
path and the iter prog attempts to update a sockmap. Which ends up spinning
at sock_map_update_elem()'s bh_lock_sock():
WARNING: possible recursive locking detected
test_progs/1393 is trying to acquire lock:
ffff88811ec25f58 (slock-AF_UNIX){+...}-{3:3}, at: sock_map_update_elem+0xdb/0x1f0
but task is already holding lock:
ffff88811ec25f58 (slock-AF_UNIX){+...}-{3:3}, at: __lock_sock_fast+0x37/0xe0
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0
----
lock(slock-AF_UNIX);
lock(slock-AF_UNIX);
*** DEADLOCK ***
May be due to missing lock nesting notation
4 locks held by test_progs/1393:
#0: ffff88814b59c790 (&p->lock){+.+.}-{4:4}, at: bpf_seq_read+0x59/0x10d0
#1: ffff88811ec25fd8 (sk_lock-AF_UNIX){+.+.}-{0:0}, at: bpf_seq_read+0x42c/0x10d0
#2: ffff88811ec25f58 (slock-AF_UNIX){+...}-{3:3}, at: __lock_sock_fast+0x37/0xe0
#3: ffffffff85a6a7c0 (rcu_read_lock){....}-{1:3}, at: bpf_iter_run_prog+0x51d/0xb00
Call Trace:
dump_stack_lvl+0x5d/0x80
print_deadlock_bug.cold+0xc0/0xce
__lock_acquire+0x130f/0x2590
lock_acquire+0x14e/0x2b0
_raw_spin_lock+0x30/0x40
sock_map_update_elem+0xdb/0x1f0
bpf_prog_2d0075e5d9b721cd_dump_unix+0x55/0x4f4
bpf_iter_run_prog+0x5b9/0xb00
bpf_iter_unix_seq_show+0x1f7/0x2e0
bpf_seq_read+0x42c/0x10d0
vfs_read+0x171/0xb20
ksys_read+0xff/0x200
do_syscall_64+0x6b/0x3a0
entry_SYSCALL_64_after_hwframe+0x76/0x7e |
| In the Linux kernel, the following vulnerability has been resolved:
drm/msm: Fix shrinker deadlock
With PROVE_LOCKING on an Snapdragon X1 and VM reclaim pressure, we see:
======================================================
WARNING: possible circular locking dependency detected
7.0.0-debug+ #43 Tainted: G W
------------------------------------------------------
kswapd0/82 is trying to acquire lock:
ffff800080ec3870 (reservation_ww_class_acquire){+.+.}-{0:0}, at: msm_gem_shrinker_scan+0x17c/0x400 [msm]
but task is already holding lock:
ffffc31709b263b8 (fs_reclaim){+.+.}-{0:0}, at: balance_pgdat+0x88/0x988
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #2 (fs_reclaim){+.+.}-{0:0}:
__lock_acquire+0x4d0/0xad0
lock_acquire.part.0+0xc4/0x248
lock_acquire+0x8c/0x248
fs_reclaim_acquire+0xd0/0xf0
dma_resv_lockdep+0x224/0x348
do_one_initcall+0x84/0x5d0
do_initcalls+0x194/0x1d8
kernel_init_freeable+0x128/0x180
kernel_init+0x2c/0x160
ret_from_fork+0x10/0x20
-> #1 (reservation_ww_class_mutex){+.+.}-{4:4}:
__lock_acquire+0x4d0/0xad0
lock_acquire.part.0+0xc4/0x248
lock_acquire+0x8c/0x248
dma_resv_lockdep+0x1a8/0x348
do_one_initcall+0x84/0x5d0
do_initcalls+0x194/0x1d8
kernel_init_freeable+0x128/0x180
kernel_init+0x2c/0x160
ret_from_fork+0x10/0x20
-> #0 (reservation_ww_class_acquire){+.+.}-{0:0}:
check_prev_add+0x114/0x790
validate_chain+0x594/0x6f0
__lock_acquire+0x4d0/0xad0
lock_acquire.part.0+0xc4/0x248
lock_acquire+0x8c/0x248
drm_gem_lru_scan+0x1ac/0x440
msm_gem_shrinker_scan+0x17c/0x400 [msm]
do_shrink_slab+0x150/0x4a0
shrink_slab+0x144/0x460
shrink_one+0x9c/0x1b0
shrink_many+0x27c/0x5c0
shrink_node+0x344/0x550
balance_pgdat+0x2c0/0x988
kswapd+0x11c/0x318
kthread+0x10c/0x128
ret_from_fork+0x10/0x20
other info that might help us debug this:
Chain exists of:
reservation_ww_class_acquire --> reservation_ww_class_mutex --> fs_reclaim
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(fs_reclaim);
lock(reservation_ww_class_mutex);
lock(fs_reclaim);
lock(reservation_ww_class_acquire);
*** DEADLOCK ***
1 lock held by kswapd0/82:
#0: ffffc31709b263b8 (fs_reclaim){+.+.}-{0:0}, at: balance_pgdat+0x88/0x988
stack backtrace:
CPU: 4 UID: 0 PID: 82 Comm: kswapd0 Tainted: G W 7.0.0-debug+ #43 PREEMPT(full)
Tainted: [W]=WARN
Hardware name: LENOVO 21BX0016US/21BX0016US, BIOS N3HET94W (1.66 ) 09/15/2025
Call trace:
show_stack+0x20/0x40 (C)
dump_stack_lvl+0x9c/0xd0
dump_stack+0x18/0x30
print_circular_bug+0x114/0x120
check_noncircular+0x178/0x198
check_prev_add+0x114/0x790
validate_chain+0x594/0x6f0
__lock_acquire+0x4d0/0xad0
lock_acquire.part.0+0xc4/0x248
lock_acquire+0x8c/0x248
drm_gem_lru_scan+0x1ac/0x440
msm_gem_shrinker_scan+0x17c/0x400 [msm]
do_shrink_slab+0x150/0x4a0
shrink_slab+0x144/0x460
shrink_one+0x9c/0x1b0
shrink_many+0x27c/0x5c0
shrink_node+0x344/0x550
balance_pgdat+0x2c0/0x988
kswapd+0x11c/0x318
kthread+0x10c/0x128
ret_from_fork+0x10/0x20
kswapd0 holding fs_reclaim calls the MSM shrinker, which calls
dma_resv_lock. This in turn acquires fs_reclaim.
Fix this deadlock by using dma_resv_trylock() instead, dropping the
subsequently unused passed wait-wound lock 'ticket'.
Patchwork: https://patchwork.freedesktop.org/patch/723564/
[rob: fixup compile errors, replace lockdep splat with somethin
---truncated--- |
| In the Linux kernel, the following vulnerability has been resolved:
tty: serial: samsung: Remove redundant port lock acquisition in rx helpers
Sashiko identified a deadlock when the console flow is engaged [1].
When console flow control is enabled (UPF_CONS_FLOW),
s3c24xx_serial_stop_tx() calls s3c24xx_serial_rx_enable() and
s3c24xx_serial_start_tx() calls s3c24xx_serial_rx_disable().
The serial core framework invokes the .stop_tx() and .start_tx()
callbacks with the port->lock spinlock already held. Furthermore, all
internal driver paths that invoke stop_tx (such as the DMA TX
completion handler s3c24xx_serial_tx_dma_complete() or the PIO TX IRQ
handler s3c24xx_serial_tx_irq()) also acquire port->lock prior to
calling it. (Note that s3c24xx_serial_start_tx() is only invoked by the
serial core).
However, s3c24xx_serial_rx_enable() and s3c24xx_serial_rx_disable()
unconditionally attempt to acquire port->lock again using
uart_port_lock_irqsave(). Since spinlocks are not recursive, this
causes a deadlock on the same CPU when console flow control is engaged.
Remove the redundant lock acquisition from both rx helper functions. |
| In the Linux kernel, the following vulnerability has been resolved:
net/handshake: Use spin_lock_bh for hn_lock
nvmet_tcp_state_change(), a socket callback that runs in BH context,
can reach handshake_req_cancel() via nvmet_tcp_schedule_release_queue()
and tls_handshake_cancel(). handshake_req_cancel() acquires
hn->hn_lock with plain spin_lock(). If a process-context thread on
the same CPU holds hn->hn_lock when a softirq invokes the cancel path,
the lock attempt deadlocks. This is the only caller that invokes
tls_handshake_cancel() from BH context; every other consumer calls it
from process context.
Deferring the cancel to process context in the NVMe target is not
straightforward: nvmet_tcp_schedule_release_queue() must call
tls_handshake_cancel() atomically with its state transition to
DISCONNECTING. If the cancel were deferred, the handshake completion
callback could fire in the window before the cancel runs, observe the
unexpected state, and return without dropping its kref on the queue.
Reworking that interlock is considerably more invasive than hardening
the handshake lock. Convert all hn->hn_lock acquisitions from
spin_lock/spin_unlock to spin_lock_bh/spin_unlock_bh so the lock is
never taken with softirqs enabled. |
| In the Linux kernel, the following vulnerability has been resolved:
netfs: Fix potential deadlock in write-through mode
Fix netfs_advance_writethrough() to always unlock the supplied folio and to
mark it dirty if it isn't yet written to the end. Unfortunately, it can't
be marked for writeback until the folio is done with as that may cause a
deadlock against mmapped reads and writes.
Even though it has been marked dirty, premature writeback can't occur as
the caller is holding both inode->i_rwsem (which will prevent concurrent
truncation, fallocation, DIO and other writes) and ictx->wb_lock (which
will cause flushing to wait and writeback to skip or wait).
Note that this may be easier to deal with once the queuing of folios is
split from the generation of subrequests. |
| In the Linux kernel, the following vulnerability has been resolved:
wifi: mt76: mt7921: fix potential deadlock in mt7921_roc_abort_sync
roc_abort_sync() can deadlock with roc_work(). roc_work() holds
dev->mt76.mutex, while cancel_work_sync() waits for roc_work()
to finish. If the caller already owns the same mutex, both
sides block and no progress is possible.
This deadlock can occur during station removal when
mt76_sta_state() -> mt76_sta_remove() -> mt7921_mac_sta_remove() ->
mt7921_roc_abort_sync() invokes cancel_work_sync() while
roc_work() is still running and holding dev->mt76.mutex.
This avoids the mutex deadlock and preserves exactly-once
work ownership. |
| In the Linux kernel, the following vulnerability has been resolved:
drm/gma500/oaktrail_lvds: fix hang on init failure
The LVDS init code looks up an I2C adapter using i2c_get_adapter() and
tries to read the EDID before falling back to allocating and registering
its own adapter.
The error handling does not separate these cases so on a late init
failure it will try to deregister and free also an adapter that had
previously been registered. Since i2c_get_adapter() takes another
reference to the adapter, deregistration hangs indefinitely while
waiting for the reference to be released.
Fix this by only destroying adapters allocated during LVDS init on
errors. |
| In the Linux kernel, the following vulnerability has been resolved:
drm/amdgpu: fix AMDGPU_INFO_READ_MMR_REG
There were multiple issues in that code.
First of all the order between the reset semaphore and the mm_lock was
wrong (e.g. copy_to_user) was called while holding the lock.
Then we allocated memory while holding the reset semaphore which is also
a pretty big bug and can deadlock.
Then we used down_read_trylock() instead of waiting for the reset to
finish.
(cherry picked from commit 361b6e6b303d4b691f6c5974d3eaab67ca6dd90e) |
| In the Linux kernel, the following vulnerability has been resolved:
md: wake raid456 reshape waiters before suspend
During raid456 reshape, direct IO across the reshape position can sleep
in raid5_make_request() waiting for reshape progress while still
holding an active_io reference. If userspace then freezes reshape and
writes md/suspend_lo or md/suspend_hi, mddev_suspend() kills active_io
and waits for all in-flight IO to drain.
This can deadlock: the IO needs reshape progress to continue, but the
reshape thread is already frozen, so the active_io reference is never
dropped and suspend never completes.
raid5_prepare_suspend() already wakes wait_for_reshape for dm-raid. Do
the same for normal md suspend when reshape is already interrupted, so
waiting raid456 IO can abort, drop its reference, and let suspend
finish.
The mdadm test tests/25raid456-reshape-deadlock reproduces the hang. |
| In the Linux kernel, the following vulnerability has been resolved:
bpf: return VMA snapshot from task_vma iterator
Holding the per-VMA lock across the BPF program body creates a lock
ordering problem when helpers acquire locks that depend on mmap_lock:
vm_lock -> i_rwsem -> mmap_lock -> vm_lock
Snapshot the VMA under the per-VMA lock in _next() via memcpy(), then
drop the lock before returning. The BPF program accesses only the
snapshot.
The verifier only trusts vm_mm and vm_file pointers (see
BTF_TYPE_SAFE_TRUSTED_OR_NULL in verifier.c). vm_file is reference-
counted with get_file() under the lock and released via fput() on the
next iteration or in _destroy(). vm_mm is already correct because
lock_vma_under_rcu() verifies vma->vm_mm == mm. All other pointers
are left as-is by memcpy() since the verifier treats them as untrusted. |
| In the Linux kernel, the following vulnerability has been resolved:
wifi: mt76: mt7925: fix potential deadlock in mt7925_roc_abort_sync
roc_abort_sync() can deadlock with roc_work(). roc_work() holds
dev->mt76.mutex, while cancel_work_sync() waits for roc_work()
to finish. If the caller already owns the same mutex, both
sides block and no progress is possible.
This deadlock can occur during station removal when
mt76_sta_state() -> mt76_sta_remove() ->
mt7925_mac_sta_remove_link() -> mt7925_mac_link_sta_remove() ->
mt7925_roc_abort_sync() invokes cancel_work_sync() while
roc_work() is still running and holding dev->mt76.mutex.
This avoids the mutex deadlock and preserves exactly-once
work ownership. |
| In the Linux kernel, the following vulnerability has been resolved:
wifi: mt76: fix deadlock in remain-on-channel
mt76_remain_on_channel() and mt76_roc_complete() call mt76_set_channel()
while already holding dev->mutex. Since mt76_set_channel() also acquires
dev->mutex, this results in a deadlock.
Use __mt76_set_channel() instead of mt76_set_channel().
Add cancel_delayed_work_sync() for mac_work before acquiring the mutex
in mt76_remain_on_channel() to prevent a secondary deadlock with the
mac_work workqueue. |
| In the Linux kernel, the following vulnerability has been resolved:
HID: usbhid: fix deadlock in hid_post_reset()
You can build a USB device that includes a HID component
and a storage or UAS component. The components can be reset
only together. That means that hid_pre_reset() and hid_post_reset()
are in the block IO error handling. Hence no memory allocation
used in them may do block IO because the IO can deadlock
on the mutex held while resetting a device and calling the
interface drivers.
Use GFP_NOIO for all allocations in them. |
| In the Linux kernel, the following vulnerability has been resolved:
greybus: gb-beagleplay: fix sleep in atomic context in hdlc_tx_frames()
hdlc_append() calls usleep_range() to wait for circular buffer space,
but it is called with tx_producer_lock (a spinlock) held via
hdlc_tx_frames() -> hdlc_append_tx_frame()/hdlc_append_tx_u8()/etc.
Sleeping while holding a spinlock is illegal and can trigger
"BUG: scheduling while atomic".
Fix this by moving the buffer-space wait out of hdlc_append() and into
hdlc_tx_frames(), before the spinlock is acquired. The new flow:
1. Pre-calculate the worst-case encoded frame length.
2. Wait (with sleep) outside the lock until enough space is available,
kicking the TX consumer work to drain the buffer.
3. Acquire the spinlock, re-verify space, and write the entire frame
atomically.
This ensures that sleeping only happens without any lock held, and
that frames are either fully enqueued or not written at all.
This bug is found by CodeQL static analysis tool (interprocedural
sleep-in-atomic query) and my code review. |
| In the Linux kernel, the following vulnerability has been resolved:
jbd2: fix deadlock in jbd2_journal_cancel_revoke()
Commit f76d4c28a46a ("fs/jbd2: use sleeping version of
__find_get_block()") changed jbd2_journal_cancel_revoke() to use
__find_get_block_nonatomic() which holds the folio lock instead of
i_private_lock. This breaks the lock ordering (folio -> buffer) and
causes an ABBA deadlock when the filesystem blocksize < pagesize:
T1 T2
ext4_mkdir()
ext4_init_new_dir()
ext4_append()
ext4_getblk()
lock_buffer() <- A
sync_blockdev()
blkdev_writepages()
writeback_iter()
writeback_get_folio()
folio_lock() <- B
ext4_journal_get_create_access()
jbd2_journal_cancel_revoke()
__find_get_block_nonatomic()
folio_lock() <- B
block_write_full_folio()
lock_buffer() <- A
This can occasionally cause generic/013 to hang.
Fix by only calling __find_get_block_nonatomic() when the passed
buffer_head doesn't belong to the bdev, which is the only case that we
need to look up its bdev alias. Otherwise, the lookup is redundant since
the found buffer_head is equal to the one we passed in. |
| In the Linux kernel, the following vulnerability has been resolved:
nvmet: avoid recursive nvmet-wq flush in nvmet_ctrl_free
nvmet_tcp_release_queue_work() runs on nvmet-wq and can drop the
final controller reference through nvmet_cq_put(). If that triggers
nvmet_ctrl_free(), the teardown path flushes ctrl->async_event_work on
the same nvmet-wq.
Call chain:
nvmet_tcp_schedule_release_queue()
kref_put(&queue->kref, nvmet_tcp_release_queue)
nvmet_tcp_release_queue()
queue_work(nvmet_wq, &queue->release_work) <--- nvmet_wq
process_one_work()
nvmet_tcp_release_queue_work()
nvmet_cq_put(&queue->nvme_cq)
nvmet_cq_destroy()
nvmet_ctrl_put(cq->ctrl)
nvmet_ctrl_free()
flush_work(&ctrl->async_event_work) <--- nvmet_wq
Previously Scheduled by :-
nvmet_add_async_event
queue_work(nvmet_wq, &ctrl->async_event_work);
This trips lockdep with a possible recursive locking warning.
[ 5223.015876] run blktests nvme/003 at 2026-04-07 20:53:55
[ 5223.061801] loop0: detected capacity change from 0 to 2097152
[ 5223.072206] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[ 5223.088368] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[ 5223.126086] nvmet: Created discovery controller 1 for subsystem nqn.2014-08.org.nvmexpress.discovery for NQN nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349.
[ 5223.128453] nvme nvme1: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery", addr 127.0.0.1:4420, hostnqn: nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349
[ 5233.199447] nvme nvme1: Removing ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery"
[ 5233.227718] ============================================
[ 5233.231283] WARNING: possible recursive locking detected
[ 5233.234696] 7.0.0-rc3nvme+ #20 Tainted: G O N
[ 5233.238434] --------------------------------------------
[ 5233.241852] kworker/u192:6/2413 is trying to acquire lock:
[ 5233.245429] ffff888111632548 ((wq_completion)nvmet-wq){+.+.}-{0:0}, at: touch_wq_lockdep_map+0x26/0x90
[ 5233.251438]
but task is already holding lock:
[ 5233.255254] ffff888111632548 ((wq_completion)nvmet-wq){+.+.}-{0:0}, at: process_one_work+0x5cc/0x6e0
[ 5233.261125]
other info that might help us debug this:
[ 5233.265333] Possible unsafe locking scenario:
[ 5233.269217] CPU0
[ 5233.270795] ----
[ 5233.272436] lock((wq_completion)nvmet-wq);
[ 5233.275241] lock((wq_completion)nvmet-wq);
[ 5233.278020]
*** DEADLOCK ***
[ 5233.281793] May be due to missing lock nesting notation
[ 5233.286195] 3 locks held by kworker/u192:6/2413:
[ 5233.289192] #0: ffff888111632548 ((wq_completion)nvmet-wq){+.+.}-{0:0}, at: process_one_work+0x5cc/0x6e0
[ 5233.294569] #1: ffffc9000e2a7e40 ((work_completion)(&queue->release_work)){+.+.}-{0:0}, at: process_one_work+0x1c5/0x6e0
[ 5233.300128] #2: ffffffff82d7dc40 (rcu_read_lock){....}-{1:3}, at: __flush_work+0x62/0x530
[ 5233.304290]
stack backtrace:
[ 5233.306520] CPU: 4 UID: 0 PID: 2413 Comm: kworker/u192:6 Tainted: G O N 7.0.0-rc3nvme+ #20 PREEMPT(full)
[ 5233.306524] Tainted: [O]=OOT_MODULE, [N]=TEST
[ 5233.306525] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014
[ 5233.306527] Workqueue: nvmet-wq nvmet_tcp_release_queue_work [nvmet_tcp]
[ 5233.306532] Call Trace:
[ 5233.306534] <TASK>
[ 5233.306536] dump_stack_lvl+0x73/0xb0
[ 5233.306552] print_deadlock_bug+0x225/0x2f0
[ 5233.306556] __lock_acquire+0x13f0/0x2290
[ 5233.306563] lock_acquire+0xd0/0x300
[ 5233.306565] ? touch_wq_lockdep_map+0x26/0x90
[ 5233.306571] ? __flush_work+0x20b/0x530
[ 5233.306573] ? touch_wq_lockdep_map+0x26/0x90
[ 5233.306577] touch_wq_lockdep_map+0x3b/0x90
[ 5233.306580] ? touch_wq_lockdep_map+0x26/0x90
[ 52
---truncated--- |
| In the Linux kernel, the following vulnerability has been resolved:
cgroup: Defer css percpu_ref kill on rmdir until cgroup is depopulated
A chain of commits going back to v7.0 reworked rmdir to satisfy the
controller invariant that a subsystem's ->css_offline() must not run while
tasks are still doing kernel-side work in the cgroup.
[1] d245698d727a ("cgroup: Defer task cgroup unlink until after the task is done switching out")
[2] a72f73c4dd9b ("cgroup: Don't expose dead tasks in cgroup")
[3] 1b164b876c36 ("cgroup: Wait for dying tasks to leave on rmdir")
[4] 4c56a8ac6869 ("cgroup: Fix cgroup_drain_dying() testing the wrong condition")
[5] 13e786b64bd3 ("cgroup: Increment nr_dying_subsys_* from rmdir context")
[1] moved task cset unlink from do_exit() to finish_task_switch() so a
task's cset link drops only after the task has fully stopped scheduling.
That made tasks past exit_signals() linger on cset->tasks until their final
context switch, which led to a series of problems as what userspace expected
to see after rmdir diverged from what the kernel needs to wait for. [2]-[5]
tried to bridge that divergence: [2] filtered the exiting tasks from
cgroup.procs; [3] had rmdir(2) sleep in TASK_UNINTERRUPTIBLE for them; [4]
fixed the wait's condition; [5] made nr_dying_subsys_* visible
synchronously.
The cgroup_drain_dying() wait in [3] turned out to be a dead end. When the
rmdir caller is also the reaper of a zombie that pins a pidns teardown (e.g.
host PID 1 systemd reaping orphan pids that were re-parented to it during
the same teardown), rmdir blocks in TASK_UNINTERRUPTIBLE waiting for those
pids to free, the pids can't free because PID 1 is the reaper and it's stuck
in rmdir, and the system A-A deadlocks. No internal lock ordering breaks
this; the wait itself is the bug.
The css killing side that drove the original reorder, however, can be made
cleanly asynchronous: ->css_offline() is already async, run from
css_killed_work_fn() driven by percpu_ref_kill_and_confirm(). The fix is to
make that chain start only after all tasks have left the cgroup. rmdir's
user-visible side then returns as soon as cgroup.procs and friends are
empty, while ->css_offline() still runs only after the cgroup is fully
drained.
Verified by the original reproducer (pidns teardown + zombie reaper, runs
under vng) which hangs vanilla and succeeds here, and by per-commit
deterministic repros for [2], [3], [4], [5] with a boot parameter that
widens the post-exit_signals() window so each state is reliably reachable.
Some stress tests on top of that.
cgroup_apply_control_disable() has the same shape of pre-existing race:
when a controller is disabled via subtree_control, kill_css() ran
synchronously while tasks past exit_signals() could still be linked to
the cgroup's csets, and ->css_offline() could fire before they drained.
This patch preserves the existing synchronous behavior at that call site
(kill_css_sync() + kill_css_finish() back-to-back) and a follow-up patch
will defer kill_css_finish() there using a per-css trigger.
This seems like the right approach and I don't see problems with it. The
changes are somewhat invasive but not excessively so, so backporting to
-stable should be okay. If something does turn out to be wrong, the fallback
is to revert the entire chain ([1]-[5]) and rework in the development branch
instead.
v2: Pin cgrp across the deferred destroy work with explicit
cgroup_get()/cgroup_put() around queue_work() and the work_fn. v1
wasn't actually broken (ordered cgroup_offline_wq + queue_work order
in cgroup_task_dead() saved it) but the explicit ref removes the
dependency on those non-obvious invariants. Also note the
pre-existing cgroup_apply_control_disable() race in the description;
a follow-up will defer kill_css_finish() there. |
