loop4: detected capacity change from 0 to 1352 loop5: detected capacity change from 0 to 1294 ======================================================== WARNING: possible irq lock inversion dependency detected 6.4.0-rc7-next-20230622 #1 Not tainted -------------------------------------------------------- syz-executor.2/132346 just changed the state of lock: ffffffff85814c18 (blkg_stat_lock){+.-.}-{2:2}, at: __blkcg_rstat_flush.isra.0+0x11f/0x4e0 but this lock was taken by another, HARDIRQ-safe lock in the past: (per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)){-.-.}-{2:2} and interrupts could create inverse lock ordering between them. other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(blkg_stat_lock); local_irq_disable(); lock(per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)); lock(blkg_stat_lock); lock(per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)); *** DEADLOCK *** 4 locks held by syz-executor.2/132346: #0: ffff888015ba0ae0 (&mm->mmap_lock){++++}-{3:3}, at: exit_mmap+0x1ae/0x8d0 #1: ffffffff8560b6c0 (rcu_read_lock){....}-{1:2}, at: __is_insn_slot_addr+0x4/0x290 #2: ffffffff8560b5a0 (rcu_callback){....}-{0:0}, at: rcu_core+0x83e/0x2860 #3: ffffffff8560b6c0 (rcu_read_lock){....}-{1:2}, at: __blkcg_rstat_flush.isra.0+0x93/0x4e0 the shortest dependencies between 2nd lock and 1st lock: -> (per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)){-.-.}-{2:2} { IN-HARDIRQ-W at: lock_acquire+0x19a/0x4c0 _raw_spin_lock_irqsave+0x3a/0x60 cgroup_rstat_updated+0xcb/0x2e0 __cgroup_account_cputime_field+0xa0/0x120 account_system_index_time+0x199/0x2c0 update_process_times+0x26/0x150 tick_sched_handle+0x8e/0x170 tick_sched_timer+0xe6/0x110 __hrtimer_run_queues+0x17f/0xb60 hrtimer_interrupt+0x2ef/0x750 __sysvec_apic_timer_interrupt+0xff/0x380 sysvec_apic_timer_interrupt+0x69/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 lock_acquire+0x1c7/0x4c0 __flush_workqueue+0x10d/0x1110 cgroup_procs_write_finish+0xeb/0x170 __cgroup_procs_write+0x371/0x6b0 cgroup_procs_write+0x26/0x60 cgroup_file_write+0x1e5/0x7c0 kernfs_fop_write_iter+0x3f7/0x610 vfs_write+0x97d/0xda0 ksys_write+0x122/0x250 do_syscall_64+0x3f/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 IN-SOFTIRQ-W at: lock_acquire+0x19a/0x4c0 _raw_spin_lock_irqsave+0x3a/0x60 cgroup_rstat_updated+0xcb/0x2e0 __cgroup_account_cputime+0x75/0xc0 update_curr+0x350/0x6d0 enqueue_task_fair+0x6a9/0x1ca0 activate_task+0xe7/0x250 ttwu_do_activate+0x10d/0x7a0 try_to_wake_up+0x619/0x1b20 wake_page_function+0x22a/0x430 __wake_up_common+0x14c/0x4c0 folio_wake_bit+0xfe/0x230 folio_unlock+0x83/0xb0 __read_end_io+0x10c/0x3e0 mpage_end_io+0x7a/0xe0 bio_endio+0x51a/0x620 blk_update_request+0x397/0x12e0 scsi_end_request+0x7a/0xa20 scsi_io_completion+0xc0/0x1640 scsi_complete+0x114/0x380 blk_complete_reqs+0xb8/0xf0 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_call_function_single+0x6e/0x90 asm_sysvec_call_function_single+0x1a/0x20 _raw_spin_unlock_irqrestore+0x34/0x50 kmemleak_free+0x18a/0x200 kmem_cache_free+0xc1/0x4d0 mas_destroy+0x1fe/0x8b0 mas_store_prealloc+0xf4/0x160 vma_complete+0xa71/0xfc0 __split_vma+0x65e/0x770 do_vmi_align_munmap+0x3ef/0x12e0 do_vmi_munmap+0x1f1/0x2b0 mmap_region+0x1fe/0x26c0 do_mmap+0x833/0xea0 vm_mmap_pgoff+0x1a8/0x3b0 ksys_mmap_pgoff+0x3af/0x500 do_syscall_64+0x3f/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 INITIAL USE at: lock_acquire+0x19a/0x4c0 _raw_spin_lock_irqsave+0x3a/0x60 cgroup_rstat_flush_locked+0x131/0xd80 cgroup_rstat_flush+0x37/0x50 do_flush_stats+0x97/0xf0 flush_memcg_stats_dwork+0x9/0x50 process_one_work+0xabf/0x1770 worker_thread+0x64f/0x12a0 kthread+0x33f/0x440 ret_from_fork+0x2c/0x50 } ... key at: [] __key.0+0x0/0x40 ... acquired at: _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 blkcg_rstat_flush+0x87/0xb0 cgroup_rstat_flush_locked+0x706/0xd80 cgroup_rstat_flush+0x37/0x50 do_flush_stats+0x97/0xf0 mem_cgroup_wb_stats+0x3d8/0x4b0 wb_over_bg_thresh+0x30e/0x6a0 wb_workfn+0x486/0xed0 process_one_work+0xabf/0x1770 worker_thread+0x64f/0x12a0 kthread+0x33f/0x440 ret_from_fork+0x2c/0x50 -> (blkg_stat_lock){+.-.}-{2:2} { HARDIRQ-ON-W at: lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 __blkg_release+0xfa/0x3b0 rcu_core+0x8c8/0x2860 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_apic_timer_interrupt+0x6e/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 strlen+0x58/0xa0 perf_trace_lock+0xc1/0x480 lock_release+0x4a9/0x680 __is_insn_slot_addr+0x147/0x290 kernel_text_address+0x48/0xc0 __kernel_text_address+0xd/0x40 unwind_get_return_address+0x59/0xa0 arch_stack_walk+0x9d/0xf0 stack_trace_save+0x90/0xd0 kasan_save_stack+0x22/0x50 __kasan_record_aux_stack+0x8e/0xa0 __call_rcu_common.constprop.0+0x6a/0xb90 kmem_cache_free+0xc1/0x4d0 ___pte_free_tlb+0x1d/0x110 free_pgd_range+0x7be/0x1330 free_pgtables+0x68a/0x930 exit_mmap+0x291/0x8d0 mmput+0xd5/0x390 do_exit+0x99f/0x2780 do_group_exit+0xd4/0x2a0 get_signal+0x263f/0x26c0 arch_do_signal_or_restart+0x79/0x590 exit_to_user_mode_prepare+0xeb/0x180 syscall_exit_to_user_mode+0x1d/0x50 do_syscall_64+0x4c/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 IN-SOFTIRQ-W at: lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 __blkg_release+0xfa/0x3b0 rcu_core+0x8c8/0x2860 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_apic_timer_interrupt+0x6e/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 strlen+0x58/0xa0 perf_trace_lock+0xc1/0x480 lock_release+0x4a9/0x680 __is_insn_slot_addr+0x147/0x290 kernel_text_address+0x48/0xc0 __kernel_text_address+0xd/0x40 unwind_get_return_address+0x59/0xa0 arch_stack_walk+0x9d/0xf0 stack_trace_save+0x90/0xd0 kasan_save_stack+0x22/0x50 __kasan_record_aux_stack+0x8e/0xa0 __call_rcu_common.constprop.0+0x6a/0xb90 kmem_cache_free+0xc1/0x4d0 ___pte_free_tlb+0x1d/0x110 free_pgd_range+0x7be/0x1330 free_pgtables+0x68a/0x930 exit_mmap+0x291/0x8d0 mmput+0xd5/0x390 do_exit+0x99f/0x2780 do_group_exit+0xd4/0x2a0 get_signal+0x263f/0x26c0 arch_do_signal_or_restart+0x79/0x590 exit_to_user_mode_prepare+0xeb/0x180 syscall_exit_to_user_mode+0x1d/0x50 do_syscall_64+0x4c/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 INITIAL USE at: lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 blkcg_rstat_flush+0x87/0xb0 cgroup_rstat_flush_locked+0x706/0xd80 cgroup_rstat_flush+0x37/0x50 do_flush_stats+0x97/0xf0 mem_cgroup_wb_stats+0x3d8/0x4b0 wb_over_bg_thresh+0x30e/0x6a0 wb_workfn+0x486/0xed0 process_one_work+0xabf/0x1770 worker_thread+0x64f/0x12a0 kthread+0x33f/0x440 ret_from_fork+0x2c/0x50 } ... key at: [] blkg_stat_lock+0x18/0x60 ... acquired at: __lock_acquire+0x8b8/0x6340 lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 __blkg_release+0xfa/0x3b0 rcu_core+0x8c8/0x2860 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_apic_timer_interrupt+0x6e/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 strlen+0x58/0xa0 perf_trace_lock+0xc1/0x480 lock_release+0x4a9/0x680 __is_insn_slot_addr+0x147/0x290 kernel_text_address+0x48/0xc0 __kernel_text_address+0xd/0x40 unwind_get_return_address+0x59/0xa0 arch_stack_walk+0x9d/0xf0 stack_trace_save+0x90/0xd0 kasan_save_stack+0x22/0x50 __kasan_record_aux_stack+0x8e/0xa0 __call_rcu_common.constprop.0+0x6a/0xb90 kmem_cache_free+0xc1/0x4d0 ___pte_free_tlb+0x1d/0x110 free_pgd_range+0x7be/0x1330 free_pgtables+0x68a/0x930 exit_mmap+0x291/0x8d0 mmput+0xd5/0x390 do_exit+0x99f/0x2780 do_group_exit+0xd4/0x2a0 get_signal+0x263f/0x26c0 arch_do_signal_or_restart+0x79/0x590 exit_to_user_mode_prepare+0xeb/0x180 syscall_exit_to_user_mode+0x1d/0x50 do_syscall_64+0x4c/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 stack backtrace: CPU: 1 PID: 132346 Comm: syz-executor.2 Not tainted 6.4.0-rc7-next-20230622 #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 Call Trace: dump_stack_lvl+0x91/0xf0 print_irq_inversion_bug.part.0+0x3d5/0x570 mark_lock.part.0+0x900/0x2f50 __lock_acquire+0x8b8/0x6340 lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 __blkg_release+0xfa/0x3b0 rcu_core+0x8c8/0x2860 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_apic_timer_interrupt+0x6e/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 RIP: 0010:strlen+0x58/0xa0 Code: 75 50 80 7d 00 00 74 3d 48 bb 00 00 00 00 00 fc ff df 48 89 e8 48 83 c0 01 48 89 c2 48 89 c1 48 c1 ea 03 83 e1 07 0f b6 14 1a <38> ca 7f 04 84 d2 75 27 80 38 00 75 de 48 83 c4 08 48 29 e8 5b 5d RSP: 0018:ffff888049dd71d0 EFLAGS: 00000202 RAX: ffffffff84884cca RBX: dffffc0000000000 RCX: 0000000000000002 RDX: 0000000000000006 RSI: ffffffff8560b6c0 RDI: ffffffff84884cc0 RBP: ffffffff84884cc0 R08: 0000000000000000 R09: fffffbfff0ba7c1a R10: ffffffff85d3e0d7 R11: 0000000000000001 R12: 1ffff110093bae44 R13: ffff888049dd72a0 R14: ffffffff8550ee00 R15: 0000000000000000 perf_trace_lock+0xc1/0x480 lock_release+0x4a9/0x680 __is_insn_slot_addr+0x147/0x290 kernel_text_address+0x48/0xc0 __kernel_text_address+0xd/0x40 unwind_get_return_address+0x59/0xa0 arch_stack_walk+0x9d/0xf0 stack_trace_save+0x90/0xd0 kasan_save_stack+0x22/0x50 __kasan_record_aux_stack+0x8e/0xa0 __call_rcu_common.constprop.0+0x6a/0xb90 kmem_cache_free+0xc1/0x4d0 ___pte_free_tlb+0x1d/0x110 free_pgd_range+0x7be/0x1330 free_pgtables+0x68a/0x930 exit_mmap+0x291/0x8d0 mmput+0xd5/0x390 do_exit+0x99f/0x2780 do_group_exit+0xd4/0x2a0 get_signal+0x263f/0x26c0 arch_do_signal_or_restart+0x79/0x590 exit_to_user_mode_prepare+0xeb/0x180 syscall_exit_to_user_mode+0x1d/0x50 do_syscall_64+0x4c/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 RIP: 0033:0x7f45b49c8b19 Code: Unable to access opcode bytes at 0x7f45b49c8aef. RSP: 002b:00007f45b1f3e218 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca RAX: 0000000000000001 RBX: 00007f45b4adbf68 RCX: 00007f45b49c8b19 RDX: 00000000000f4240 RSI: 0000000000000081 RDI: 00007f45b4adbf6c RBP: 00007f45b4adbf60 R08: 0000000000000010 R09: 0000000000000000 R10: ffffffffffffffff R11: 0000000000000246 R12: 00007f45b4adbf6c R13: 00007ffcc97ebcef R14: 00007f45b1f3e300 R15: 0000000000022000 loop4: detected capacity change from 0 to 1352 loop6: detected capacity change from 0 to 5392 loop5: detected capacity change from 0 to 1294 loop4: detected capacity change from 0 to 1352 loop6: detected capacity change from 0 to 5392 loop5: detected capacity change from 0 to 1294 loop4: detected capacity change from 0 to 1352 loop5: detected capacity change from 0 to 1294 loop6: detected capacity change from 0 to 5392 loop5: detected capacity change from 0 to 1294 loop5: detected capacity change from 0 to 1294 loop4: detected capacity change from 0 to 1352 loop6: detected capacity change from 0 to 5392 loop5: detected capacity change from 0 to 1294 loop6: detected capacity change from 0 to 5392 ---------------- Code disassembly (best guess): 0: 75 50 jne 0x52 2: 80 7d 00 00 cmpb $0x0,0x0(%rbp) 6: 74 3d je 0x45 8: 48 bb 00 00 00 00 00 movabs $0xdffffc0000000000,%rbx f: fc ff df 12: 48 89 e8 mov %rbp,%rax 15: 48 83 c0 01 add $0x1,%rax 19: 48 89 c2 mov %rax,%rdx 1c: 48 89 c1 mov %rax,%rcx 1f: 48 c1 ea 03 shr $0x3,%rdx 23: 83 e1 07 and $0x7,%ecx 26: 0f b6 14 1a movzbl (%rdx,%rbx,1),%edx * 2a: 38 ca cmp %cl,%dl <-- trapping instruction 2c: 7f 04 jg 0x32 2e: 84 d2 test %dl,%dl 30: 75 27 jne 0x59 32: 80 38 00 cmpb $0x0,(%rax) 35: 75 de jne 0x15 37: 48 83 c4 08 add $0x8,%rsp 3b: 48 29 e8 sub %rbp,%rax 3e: 5b pop %rbx 3f: 5d pop %rbp