| 0/0 |
2026/05/21 10:29 |
flow |
assessment-security |
1h25m
Results: map[DenialOfService:true Explanation:The bug report indicates a `WARNING` in `dev_shutdown()` at
`net/sched/sch_generic.c:1496` because `timer_pending(&dev->watchdog_timer)`
evaluates to true when a network device is being unregistered.
### Root Cause Analysis
The warning triggers because the network device's watchdog timer is still
pending when the device is being dismantled.
When a network device is unregistered, `unregister_netdevice_many_notify()` is
called. This function first checks if the device is currently running
(`dev->flags & IFF_UP`). If it is, it calls `netif_close_many()`, which
eventually calls `dev_deactivate_many()` and `netdev_watchdog_down()` to safely
delete the watchdog timer. However, if `IFF_UP` is not set, this teardown step
is skipped.
The race condition occurs during the device initialization phase
(`__dev_open()`):
1. `__dev_open()` sets the `__LINK_STATE_START` bit before calling the driver's
`ndo_open()` callback.
2. If `ndo_open()` blocks or takes a long time, an asynchronous event (like a
USB kevent or a carrier change) can call `netif_carrier_on()`.
3. `netif_carrier_on()` checks `netif_running(dev)` (which returns true because
`__LINK_STATE_START` is set) and subsequently calls `netdev_watchdog_up()`,
arming the timer.
4. If `ndo_open()` subsequently fails, `__dev_open()` clears
`__LINK_STATE_START` and returns an error, but it **never sets `IFF_UP`**.
5. When the device is later disconnected and unregistered,
`unregister_netdevice_many_notify()` sees that `IFF_UP` is false and skips
calling `netdev_watchdog_down()`.
6. Finally, `dev_shutdown()` is called, and the
`WARN_ON(timer_pending(&dev->watchdog_timer))` triggers because the timer was
armed but never deleted.
### Security Impact
* **Exploitable:** No. While a pending timer on a freed object usually leads to
a Use-After-Free (UAF), the network device unregistration process
(`netdev_run_todo()`) involves RCU grace periods and waits for all references to
drop before calling `free_netdev()`. The timer will almost certainly fire and
finish its execution before the `struct net_device` memory is actually freed,
preventing a UAF.
* **Denial of Service:** Yes. In kernels with `panic_on_warn` enabled, this will
crash the system. In production builds without `panic_on_warn`, it will just
print a warning and continue, but it indicates a state inconsistency.
* **Peripheral Trigger:** Yes. The bug can be triggered by plugging in a
malicious USB device (or using the `dummy_hcd` / `raw-gadget` interfaces as the
syzkaller reproducer does) that simulates a network interface and manipulates
the carrier state during initialization.
* **Unprivileged:** Yes. If the system allows unprivileged users to access USB
devices or the `raw-gadget` interface, they can trigger this issue. Exploitable:false FilesystemTrigger:false NetworkTrigger:false PeripheralTrigger:true RemoteTrigger:false Unprivileged:true UserNamespace:false VMGuestTrigger:false VMHostTrigger:false]
|
| 1/1 |
2026/05/21 10:29 |
action |
syz-repro-to-c-repro |
0m
Results: map[SimplifiedCRepro:// autogenerated by syzkaller (https://github.com/google/syzkaller)
#define _GNU_SOURCE
#include <endian.h>
#include <errno.h>
#include <fcntl.h>
#include <stdarg.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mount.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <sys/types.h>
#include <unistd.h>
#include <linux/usb/ch9.h>
static unsigned long long procid;
static void sleep_ms(uint64_t ms)
{
usleep(ms * 1000);
}
#define SIZEOF_IO_URING_SQE 64
#define SIZEOF_IO_URING_CQE 16
#define IORING_SETUP_SQE128 (1U << 10)
#define IORING_SETUP_CQE32 (1U << 11)
struct io_sqring_offsets {
uint32_t head;
uint32_t tail;
uint32_t ring_mask;
uint32_t ring_entries;
uint32_t flags;
uint32_t dropped;
uint32_t array;
uint32_t resv1;
uint64_t user_addr;
};
struct io_cqring_offsets {
uint32_t head;
uint32_t tail;
uint32_t ring_mask;
uint32_t ring_entries;
uint32_t overflow;
uint32_t cqes;
uint32_t flags;
uint32_t resv1;
uint64_t user_addr;
};
struct io_uring_params {
uint32_t sq_entries;
uint32_t cq_entries;
uint32_t flags;
uint32_t sq_thread_cpu;
uint32_t sq_thread_idle;
uint32_t features;
uint32_t resv[4];
struct io_sqring_offsets sq_off;
struct io_cqring_offsets cq_off;
};
static long io_uring_sqe_size(struct io_uring_params* params)
{
return SIZEOF_IO_URING_SQE << !!(params->flags & IORING_SETUP_SQE128);
}
static long syz_io_uring_submit(volatile long a0, volatile long a1, volatile long a2, volatile long a3)
{
struct io_uring_params* params = (struct io_uring_params*)a0;
char* ring_ptr = (char*)a1;
char* sqes_ptr = (char*)a2;
char* sqe = (char*)a3;
uint32_t sq_ring_mask = *(uint32_t*)(ring_ptr + params->sq_off.ring_mask);
uint32_t* sq_tail_ptr = (uint32_t*)(ring_ptr + params->sq_off.tail);
uint32_t sq_tail = *sq_tail_ptr & sq_ring_mask;
uint32_t sqe_size = io_uring_sqe_size(params);
char* sqe_dest = sqes_ptr + sq_tail * sqe_size;
memcpy(sqe_dest, sqe, sqe_size);
uint32_t sq_tail_next = *sq_tail_ptr + 1;
__atomic_store_n(sq_tail_ptr, sq_tail_next, __ATOMIC_RELEASE);
return 0;
}
#define MAX_FDS 30
#define USB_MAX_IFACE_NUM 4
#define USB_MAX_EP_NUM 32
#define USB_MAX_FDS 6
struct usb_endpoint_index {
struct usb_endpoint_descriptor desc;
int handle;
};
struct usb_iface_index {
struct usb_interface_descriptor* iface;
uint8_t bInterfaceNumber;
uint8_t bAlternateSetting;
uint8_t bInterfaceClass;
struct usb_endpoint_index eps[USB_MAX_EP_NUM];
int eps_num;
};
struct usb_device_index {
struct usb_device_descriptor* dev;
struct usb_config_descriptor* config;
uint8_t bDeviceClass;
uint8_t bMaxPower;
int config_length;
struct usb_iface_index ifaces[USB_MAX_IFACE_NUM];
int ifaces_num;
int iface_cur;
};
struct usb_info {
int fd;
struct usb_device_index index;
};
static struct usb_info usb_devices[USB_MAX_FDS];
static struct usb_device_index* lookup_usb_index(int fd)
{
for (int i = 0; i < USB_MAX_FDS; i++) {
if (__atomic_load_n(&usb_devices[i].fd, __ATOMIC_ACQUIRE) == fd)
return &usb_devices[i].index;
}
return NULL;
}
static int usb_devices_num;
static bool parse_usb_descriptor(const char* buffer, size_t length, struct usb_device_index* index)
{
if (length < sizeof(*index->dev) + sizeof(*index->config))
return false;
memset(index, 0, sizeof(*index));
index->dev = (struct usb_device_descriptor*)buffer;
index->config = (struct usb_config_descriptor*)(buffer + sizeof(*index->dev));
index->bDeviceClass = index->dev->bDeviceClass;
index->bMaxPower = index->config->bMaxPower;
index->config_length = length - sizeof(*index->dev);
index->iface_cur = -1;
size_t offset = 0;
while (true) {
if (offset + 1 >= length)
break;
uint8_t desc_length = buffer[offset];
uint8_t desc_type = buffer[offset + 1];
if (desc_length <= 2)
break;
if (offset + desc_length > length)
break;
if (desc_type == USB_DT_INTERFACE && index->ifaces_num < USB_MAX_IFACE_NUM) {
struct usb_interface_descriptor* iface = (struct usb_interface_descriptor*)(buffer + offset);
index->ifaces[index->ifaces_num].iface = iface;
index->ifaces[index->ifaces_num].bInterfaceNumber = iface->bInterfaceNumber;
index->ifaces[index->ifaces_num].bAlternateSetting = iface->bAlternateSetting;
index->ifaces[index->ifaces_num].bInterfaceClass = iface->bInterfaceClass;
index->ifaces_num++;
}
if (desc_type == USB_DT_ENDPOINT && index->ifaces_num > 0) {
struct usb_iface_index* iface = &index->ifaces[index->ifaces_num - 1];
if (iface->eps_num < USB_MAX_EP_NUM) {
memcpy(&iface->eps[iface->eps_num].desc, buffer + offset, sizeof(iface->eps[iface->eps_num].desc));
iface->eps_num++;
}
}
offset += desc_length;
}
return true;
}
static struct usb_device_index* add_usb_index(int fd, const char* dev, size_t dev_len)
{
int i = __atomic_fetch_add(&usb_devices_num, 1, __ATOMIC_RELAXED);
if (i >= USB_MAX_FDS)
return NULL;
if (!parse_usb_descriptor(dev, dev_len, &usb_devices[i].index))
return NULL;
__atomic_store_n(&usb_devices[i].fd, fd, __ATOMIC_RELEASE);
return &usb_devices[i].index;
}
struct vusb_connect_string_descriptor {
uint32_t len;
char* str;
} __attribute__((packed));
struct vusb_connect_descriptors {
uint32_t qual_len;
char* qual;
uint32_t bos_len;
char* bos;
uint32_t strs_len;
struct vusb_connect_string_descriptor strs[0];
} __attribute__((packed));
static const char default_string[] = {
8, USB_DT_STRING,
's', 0, 'y', 0, 'z', 0
};
static const char default_lang_id[] = {
4, USB_DT_STRING,
0x09, 0x04
};
static bool lookup_connect_response_in(int fd, const struct vusb_connect_descriptors* descs,
const struct usb_ctrlrequest* ctrl,
struct usb_qualifier_descriptor* qual,
char** response_data, uint32_t* response_length)
{
struct usb_device_index* index = lookup_usb_index(fd);
uint8_t str_idx;
if (!index)
return false;
switch (ctrl->bRequestType & USB_TYPE_MASK) {
case USB_TYPE_STANDARD:
switch (ctrl->bRequest) {
case USB_REQ_GET_DESCRIPTOR:
switch (ctrl->wValue >> 8) {
case USB_DT_DEVICE:
*response_data = (char*)index->dev;
*response_length = sizeof(*index->dev);
return true;
case USB_DT_CONFIG:
*response_data = (char*)index->config;
*response_length = index->config_length;
return true;
case USB_DT_STRING:
str_idx = (uint8_t)ctrl->wValue;
if (descs && str_idx < descs->strs_len) {
*response_data = descs->strs[str_idx].str;
*response_length = descs->strs[str_idx].len;
return true;
}
if (str_idx == 0) {
*response_data = (char*)&default_lang_id[0];
*response_length = default_lang_id[0];
return true;
}
*response_data = (char*)&default_string[0];
*response_length = default_string[0];
return true;
case USB_DT_BOS:
*response_data = descs->bos;
*response_length = descs->bos_len;
return true;
case USB_DT_DEVICE_QUALIFIER:
if (!descs->qual) {
qual->bLength = sizeof(*qual);
qual->bDescriptorType = USB_DT_DEVICE_QUALIFIER;
qual->bcdUSB = index->dev->bcdUSB;
qual->bDeviceClass = index->dev->bDeviceClass;
qual->bDeviceSubClass = index->dev->bDeviceSubClass;
qual->bDeviceProtocol = index->dev->bDeviceProtocol;
qual->bMaxPacketSize0 = index->dev->bMaxPacketSize0;
qual->bNumConfigurations = index->dev->bNumConfigurations;
qual->bRESERVED = 0;
*response_data = (char*)qual;
*response_length = sizeof(*qual);
return true;
}
*response_data = descs->qual;
*response_length = descs->qual_len;
return true;
default:
break;
}
break;
default:
break;
}
break;
default:
break;
}
return false;
}
typedef bool (*lookup_connect_out_response_t)(int fd, const struct vusb_connect_descriptors* descs,
const struct usb_ctrlrequest* ctrl, bool* done);
static bool lookup_connect_response_out_generic(int fd, const struct vusb_connect_descriptors* descs,
const struct usb_ctrlrequest* ctrl, bool* done)
{
switch (ctrl->bRequestType & USB_TYPE_MASK) {
case USB_TYPE_STANDARD:
switch (ctrl->bRequest) {
case USB_REQ_SET_CONFIGURATION:
*done = true;
return true;
default:
break;
}
break;
}
return false;
}
struct vusb_descriptor {
uint8_t req_type;
uint8_t desc_type;
uint32_t len;
char data[0];
} __attribute__((packed));
struct vusb_descriptors {
uint32_t len;
struct vusb_descriptor* generic;
struct vusb_descriptor* descs[0];
} __attribute__((packed));
struct vusb_response {
uint8_t type;
uint8_t req;
uint32_t len;
char data[0];
} __attribute__((packed));
struct vusb_responses {
uint32_t len;
struct vusb_response* generic;
struct vusb_response* resps[0];
} __attribute__((packed));
static bool lookup_control_response(const struct vusb_descriptors* descs, const struct vusb_responses* resps,
struct usb_ctrlrequest* ctrl, char** response_data, uint32_t* response_length)
{
int descs_num = 0;
int resps_num = 0;
if (descs)
descs_num = (descs->len - offsetof(struct vusb_descriptors, descs)) / sizeof(descs->descs[0]);
if (resps)
resps_num = (resps->len - offsetof(struct vusb_responses, resps)) / sizeof(resps->resps[0]);
uint8_t req = ctrl->bRequest;
uint8_t req_type = ctrl->bRequestType & USB_TYPE_MASK;
uint8_t desc_type = ctrl->wValue >> 8;
if (req == USB_REQ_GET_DESCRIPTOR) {
int i;
for (i = 0; i < descs_num; i++) {
struct vusb_descriptor* desc = descs->descs[i];
if (!desc)
continue;
if (desc->req_type == req_type && desc->desc_type == desc_type) {
*response_length = desc->len;
if (*response_length != 0)
*response_data = &desc->data[0];
else
*response_data = NULL;
return true;
}
}
if (descs && descs->generic) {
*response_data = &descs->generic->data[0];
*response_length = descs->generic->len;
return true;
}
} else {
int i;
for (i = 0; i < resps_num; i++) {
struct vusb_response* resp = resps->resps[i];
if (!resp)
continue;
if (resp->type == req_type && resp->req == req) {
*response_length = resp->len;
if (*response_length != 0)
*response_data = &resp->data[0];
else
*response_data = NULL;
return true;
}
}
if (resps && resps->generic) {
*response_data = &resps->generic->data[0];
*response_length = resps->generic->len;
return true;
}
}
return false;
}
#define UDC_NAME_LENGTH_MAX 128
struct usb_raw_init {
__u8 driver_name[UDC_NAME_LENGTH_MAX];
__u8 device_name[UDC_NAME_LENGTH_MAX];
__u8 speed;
};
enum usb_raw_event_type {
USB_RAW_EVENT_INVALID = 0,
USB_RAW_EVENT_CONNECT = 1,
USB_RAW_EVENT_CONTROL = 2,
};
struct usb_raw_event {
__u32 type;
__u32 length;
__u8 data[0];
};
struct usb_raw_ep_io {
__u16 ep;
__u16 flags;
__u32 length;
__u8 data[0];
};
#define USB_RAW_EPS_NUM_MAX 30
#define USB_RAW_EP_NAME_MAX 16
#define USB_RAW_EP_ADDR_ANY 0xff
struct usb_raw_ep_caps {
__u32 type_control : 1;
__u32 type_iso : 1;
__u32 type_bulk : 1;
__u32 type_int : 1;
__u32 dir_in : 1;
__u32 dir_out : 1;
};
struct usb_raw_ep_limits {
__u16 maxpacket_limit;
__u16 max_streams;
__u32 reserved;
};
struct usb_raw_ep_info {
__u8 name[USB_RAW_EP_NAME_MAX];
__u32 addr;
struct usb_raw_ep_caps caps;
struct usb_raw_ep_limits limits;
};
struct usb_raw_eps_info {
struct usb_raw_ep_info eps[USB_RAW_EPS_NUM_MAX];
};
#define USB_RAW_IOCTL_INIT _IOW('U', 0, struct usb_raw_init)
#define USB_RAW_IOCTL_RUN _IO('U', 1)
#define USB_RAW_IOCTL_EVENT_FETCH _IOR('U', 2, struct usb_raw_event)
#define USB_RAW_IOCTL_EP0_WRITE _IOW('U', 3, struct usb_raw_ep_io)
#define USB_RAW_IOCTL_EP0_READ _IOWR('U', 4, struct usb_raw_ep_io)
#define USB_RAW_IOCTL_EP_ENABLE _IOW('U', 5, struct usb_endpoint_descriptor)
#define USB_RAW_IOCTL_EP_DISABLE _IOW('U', 6, __u32)
#define USB_RAW_IOCTL_EP_WRITE _IOW('U', 7, struct usb_raw_ep_io)
#define USB_RAW_IOCTL_EP_READ _IOWR('U', 8, struct usb_raw_ep_io)
#define USB_RAW_IOCTL_CONFIGURE _IO('U', 9)
#define USB_RAW_IOCTL_VBUS_DRAW _IOW('U', 10, __u32)
#define USB_RAW_IOCTL_EPS_INFO _IOR('U', 11, struct usb_raw_eps_info)
#define USB_RAW_IOCTL_EP0_STALL _IO('U', 12)
#define USB_RAW_IOCTL_EP_SET_HALT _IOW('U', 13, __u32)
#define USB_RAW_IOCTL_EP_CLEAR_HALT _IOW('U', 14, __u32)
#define USB_RAW_IOCTL_EP_SET_WEDGE _IOW('U', 15, __u32)
static int usb_raw_open()
{
return open("/dev/raw-gadget", O_RDWR);
}
static int usb_raw_init(int fd, uint32_t speed, const char* driver, const char* device)
{
struct usb_raw_init arg;
strncpy((char*)&arg.driver_name[0], driver, sizeof(arg.driver_name));
strncpy((char*)&arg.device_name[0], device, sizeof(arg.device_name));
arg.speed = speed;
return ioctl(fd, USB_RAW_IOCTL_INIT, &arg);
}
static int usb_raw_run(int fd)
{
return ioctl(fd, USB_RAW_IOCTL_RUN, 0);
}
static int usb_raw_configure(int fd)
{
return ioctl(fd, USB_RAW_IOCTL_CONFIGURE, 0);
}
static int usb_raw_vbus_draw(int fd, uint32_t power)
{
return ioctl(fd, USB_RAW_IOCTL_VBUS_DRAW, power);
}
static int usb_raw_ep0_write(int fd, struct usb_raw_ep_io* io)
{
return ioctl(fd, USB_RAW_IOCTL_EP0_WRITE, io);
}
static int usb_raw_ep0_read(int fd, struct usb_raw_ep_io* io)
{
return ioctl(fd, USB_RAW_IOCTL_EP0_READ, io);
}
static int usb_raw_event_fetch(int fd, struct usb_raw_event* event)
{
return ioctl(fd, USB_RAW_IOCTL_EVENT_FETCH, event);
}
static int usb_raw_ep_enable(int fd, struct usb_endpoint_descriptor* desc)
{
return ioctl(fd, USB_RAW_IOCTL_EP_ENABLE, desc);
}
static int usb_raw_ep_disable(int fd, int ep)
{
return ioctl(fd, USB_RAW_IOCTL_EP_DISABLE, ep);
}
static int usb_raw_ep0_stall(int fd)
{
return ioctl(fd, USB_RAW_IOCTL_EP0_STALL, 0);
}
static int lookup_interface(int fd, uint8_t bInterfaceNumber, uint8_t bAlternateSetting)
{
struct usb_device_index* index = lookup_usb_index(fd);
if (!index)
return -1;
for (int i = 0; i < index->ifaces_num; i++) {
if (index->ifaces[i].bInterfaceNumber == bInterfaceNumber &&
index->ifaces[i].bAlternateSetting == bAlternateSetting)
return i;
}
return -1;
}
#define USB_MAX_PACKET_SIZE 4096
struct usb_raw_control_event {
struct usb_raw_event inner;
struct usb_ctrlrequest ctrl;
char data[USB_MAX_PACKET_SIZE];
};
struct usb_raw_ep_io_data {
struct usb_raw_ep_io inner;
char data[USB_MAX_PACKET_SIZE];
};
static void set_interface(int fd, int n)
{
struct usb_device_index* index = lookup_usb_index(fd);
if (!index)
return;
if (index->iface_cur >= 0 && index->iface_cur < index->ifaces_num) {
for (int ep = 0; ep < index->ifaces[index->iface_cur].eps_num; ep++) {
int rv = usb_raw_ep_disable(fd, index->ifaces[index->iface_cur].eps[ep].handle);
if (rv < 0) {
} else {
}
}
}
if (n >= 0 && n < index->ifaces_num) {
for (int ep = 0; ep < index->ifaces[n].eps_num; ep++) {
int rv = usb_raw_ep_enable(fd, &index->ifaces[n].eps[ep].desc);
if (rv < 0) {
} else {
index->ifaces[n].eps[ep].handle = rv;
}
}
index->iface_cur = n;
}
}
static int configure_device(int fd)
{
struct usb_device_index* index = lookup_usb_index(fd);
if (!index)
return -1;
int rv = usb_raw_vbus_draw(fd, index->bMaxPower);
if (rv < 0) {
return rv;
}
rv = usb_raw_configure(fd);
if (rv < 0) {
return rv;
}
set_interface(fd, 0);
return 0;
}
static volatile long syz_usb_connect_impl(uint64_t speed, uint64_t dev_len, const char* dev,
const struct vusb_connect_descriptors* descs,
lookup_connect_out_response_t lookup_connect_response_out)
{
if (!dev) {
return -1;
}
int fd = usb_raw_open();
if (fd < 0) {
return fd;
}
if (fd >= MAX_FDS) {
close(fd);
return -1;
}
struct usb_device_index* index = add_usb_index(fd, dev, dev_len);
if (!index) {
return -1;
}
char device[32];
sprintf(&device[0], "dummy_udc.%llu", procid);
int rv = usb_raw_init(fd, speed, "dummy_udc", &device[0]);
if (rv < 0) {
return rv;
}
rv = usb_raw_run(fd);
if (rv < 0) {
return rv;
}
bool done = false;
while (!done) {
struct usb_raw_control_event event;
event.inner.type = 0;
event.inner.length = sizeof(event.ctrl);
rv = usb_raw_event_fetch(fd, (struct usb_raw_event*)&event);
if (rv < 0) {
return rv;
}
if (event.inner.type != USB_RAW_EVENT_CONTROL)
continue;
char* response_data = NULL;
uint32_t response_length = 0;
struct usb_qualifier_descriptor qual;
if (event.ctrl.bRequestType & USB_DIR_IN) {
if (!lookup_connect_response_in(fd, descs, &event.ctrl, &qual, &response_data, &response_length)) {
usb_raw_ep0_stall(fd);
continue;
}
} else {
if (!lookup_connect_response_out(fd, descs, &event.ctrl, &done)) {
usb_raw_ep0_stall(fd);
continue;
}
response_data = NULL;
response_length = event.ctrl.wLength;
}
if ((event.ctrl.bRequestType & USB_TYPE_MASK) == USB_TYPE_STANDARD &&
event.ctrl.bRequest == USB_REQ_SET_CONFIGURATION) {
rv = configure_device(fd);
if (rv < 0) {
return rv;
}
}
struct usb_raw_ep_io_data response;
response.inner.ep = 0;
response.inner.flags = 0;
if (response_length > sizeof(response.data))
response_length = 0;
if (event.ctrl.wLength < response_length)
response_length = event.ctrl.wLength;
response.inner.length = response_length;
if (response_data)
memcpy(&response.data[0], response_data, response_length);
else
memset(&response.data[0], 0, response_length);
if (event.ctrl.bRequestType & USB_DIR_IN) {
rv = usb_raw_ep0_write(fd, (struct usb_raw_ep_io*)&response);
} else {
rv = usb_raw_ep0_read(fd, (struct usb_raw_ep_io*)&response);
}
if (rv < 0) {
return rv;
}
}
sleep_ms(200);
return fd;
}
static volatile long syz_usb_connect(volatile long a0, volatile long a1, volatile long a2, volatile long a3)
{
uint64_t speed = a0;
uint64_t dev_len = a1;
const char* dev = (const char*)a2;
const struct vusb_connect_descriptors* descs = (const struct vusb_connect_descriptors*)a3;
return syz_usb_connect_impl(speed, dev_len, dev, descs, &lookup_connect_response_out_generic);
}
static volatile long syz_usb_control_io(volatile long a0, volatile long a1, volatile long a2)
{
int fd = a0;
const struct vusb_descriptors* descs = (const struct vusb_descriptors*)a1;
const struct vusb_responses* resps = (const struct vusb_responses*)a2;
struct usb_raw_control_event event;
event.inner.type = 0;
event.inner.length = USB_MAX_PACKET_SIZE;
int rv = usb_raw_event_fetch(fd, (struct usb_raw_event*)&event);
if (rv < 0) {
return rv;
}
if (event.inner.type != USB_RAW_EVENT_CONTROL) {
return -1;
}
char* response_data = NULL;
uint32_t response_length = 0;
if ((event.ctrl.bRequestType & USB_DIR_IN) && event.ctrl.wLength) {
if (!lookup_control_response(descs, resps, &event.ctrl, &response_data, &response_length)) {
usb_raw_ep0_stall(fd);
return -1;
}
} else {
if ((event.ctrl.bRequestType & USB_TYPE_MASK) == USB_TYPE_STANDARD ||
event.ctrl.bRequest == USB_REQ_SET_INTERFACE) {
int iface_num = event.ctrl.wIndex;
int alt_set = event.ctrl.wValue;
int iface_index = lookup_interface(fd, iface_num, alt_set);
if (iface_index < 0) {
} else {
set_interface(fd, iface_index);
}
}
response_length = event.ctrl.wLength;
}
struct usb_raw_ep_io_data response;
response.inner.ep = 0;
response.inner.flags = 0;
if (response_length > sizeof(response.data))
response_length = 0;
if (event.ctrl.wLength < response_length)
response_length = event.ctrl.wLength;
if ((event.ctrl.bRequestType & USB_DIR_IN) && !event.ctrl.wLength) {
response_length = USB_MAX_PACKET_SIZE;
}
response.inner.length = response_length;
if (response_data)
memcpy(&response.data[0], response_data, response_length);
else
memset(&response.data[0], 0, response_length);
if ((event.ctrl.bRequestType & USB_DIR_IN) && event.ctrl.wLength) {
rv = usb_raw_ep0_write(fd, (struct usb_raw_ep_io*)&response);
} else {
rv = usb_raw_ep0_read(fd, (struct usb_raw_ep_io*)&response);
}
if (rv < 0) {
return rv;
}
sleep_ms(200);
return 0;
}
uint64_t r[1] = {0xffffffffffffffff};
int main(void)
{
syscall(__NR_mmap, /*addr=*/0x1ffffffff000ul, /*len=*/0x1000ul, /*prot=*/0ul, /*flags=MAP_FIXED|MAP_ANONYMOUS|MAP_PRIVATE*/0x32ul, /*fd=*/(intptr_t)-1, /*offset=*/0ul);
syscall(__NR_mmap, /*addr=*/0x200000000000ul, /*len=*/0x1000000ul, /*prot=PROT_WRITE|PROT_READ|PROT_EXEC*/7ul, /*flags=MAP_FIXED|MAP_ANONYMOUS|MAP_PRIVATE*/0x32ul, /*fd=*/(intptr_t)-1, /*offset=*/0ul);
syscall(__NR_mmap, /*addr=*/0x200001000000ul, /*len=*/0x1000ul, /*prot=*/0ul, /*flags=MAP_FIXED|MAP_ANONYMOUS|MAP_PRIVATE*/0x32ul, /*fd=*/(intptr_t)-1, /*offset=*/0ul);
const char* reason;
(void)reason;
intptr_t res = 0;
if (write(1, "executing program\n", sizeof("executing program\n") - 1)) {}
// syz_usb_connect arguments: [
// speed: usb_device_speed = 0x0 (8 bytes)
// dev_len: len = 0x36 (8 bytes)
// dev: ptr[inout, array[ANYUNION]] {
// array[ANYUNION] {
// union ANYUNION {
// ANYBLOB: buffer: {12 01 00 00 98 80 a4 20 d0 06 22 06 5a 3f 01 02 03 01 09 02 24 00 01 00 00 10 00 09 04 a6 00 02 09 ab e0 00 09 05 02 02 00 02 02 00 00 09 05 82 02 10 00 00 00} (length 0x35)
// }
// }
// }
// conn_descs: nil
// ]
// returns fd_usb
memcpy((void*)0x200000000080, "... [truncated large byte array] ...", 53);
res = -1;
res = syz_usb_connect(/*speed=*/0, /*dev_len=*/0x36, /*dev=*/0x200000000080, /*conn_descs=*/0);
if (res != -1)
r[0] = res;
// syz_usb_control_io arguments: [
// fd: fd_usb (resource)
// descs: nil
// resps: nil
// ]
syz_usb_control_io(/*fd=*/r[0], /*descs=*/0, /*resps=*/0);
// syz_io_uring_submit arguments: [
// ring_params_ptr: ring_params_ptr (resource)
// ring_ptr: ring_ptr (resource)
// sqes_ptr: sqes_ptr (resource)
// sqe: ptr[in, io_uring_sqe_u] {
// union io_uring_sqe_u {
// IORING_OP_NOP: io_uring_sqe[IORING_OP_NOP, const[0, int16], const[0, int32], const[0, int64], const[0, int64], const[0, int32], const[0, int32], zero_flag, array[const[0, int64], 3]] {
// opcode: const = 0x0 (1 bytes)
// flags: iosqe_flags = 0x0 (1 bytes)
// ioprio: const = 0x0 (2 bytes)
// fd: const = 0x0 (4 bytes)
// off: const = 0x0 (8 bytes)
// addr: const = 0x0 (8 bytes)
// len: const = 0x0 (4 bytes)
// misc_flags: const = 0x0 (4 bytes)
// user_data: zero_flag = 0x0 (8 bytes)
// misc: buffer: {00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00} (length 0x18)
// }
// }
// }
// ]
*(uint8_t*)0x200000000000 = 0;
*(uint8_t*)0x200000000001 = 0;
*(uint16_t*)0x200000000002 = 0;
*(uint32_t*)0x200000000004 = 0;
*(uint64_t*)0x200000000008 = 0;
*(uint64_t*)0x200000000010 = 0;
*(uint32_t*)0x200000000018 = 0;
*(uint32_t*)0x20000000001c = 0;
*(uint64_t*)0x200000000020 = 0;
memset((void*)0x200000000028, 0, 24);
syz_io_uring_submit(/*ring_params_ptr=*/0, /*ring_ptr=*/0, /*sqes_ptr=*/0, /*sqe=*/0x200000000000);
return 0;
}
]
|
| 2/1 |
2026/05/21 10:29 |
action |
kernel-checkouter |
1m
Results: map[KernelSrc:/app/workdir/cache/src/8c51a9a5a74b20d89b364e22a8a7645d923cbd47]
|
| 3/1 |
2026/05/21 10:31 |
action |
kernel-builder |
35m
Results: map[KernelObj:/app/workdir/cache/build/28dd73a3162665d9dfb02f2145061da68f65c943]
|
| 4/1 |
2026/05/21 11:06 |
action |
codesearch-prepare |
26m
Results: map[Index:codesearch-index]
|
| 5/1 |
2026/05/21 11:33 |
agent |
expert |
21m
Model:
Results: map[DenialOfService:true Exploitable:false FilesystemTrigger:false NetworkTrigger:false PeripheralTrigger:true RemoteTrigger:false Unprivileged:true UserNamespace:false VMGuestTrigger:false VMHostTrigger:false]
Instruction:
You are an experienced Linux kernel security engineer. Your task is to analyze given kernel bug report
and determine its security impact based on the following dimensions.
Use the provided tools to examine the source code, check for capability checks (e.g., capable(), ns_capable()),
and understand the nature of the bug. Analyze the given kernel build and configuration.
You can check the kernel config by grepping ".config" file; you can check kernel cmdline by greeping
".config" file for "CONFIG_CMDLINE=". Assume sysctl parameters have default values.
But analyze for the corresponding production build w/o debugging tools enabled (like KASAN, KMSAN, UBSAN).
Don't make assumptions; verify them with source code access. Try different strategies when analyzing the bug:
- think of ways in which the vulnerable code is unreachable
- or the other way around: try to come up with different ideas of how an unprivileged user can reach the bug
If still unsure err on the side of the bug being non-exploitable/not-accessible.
In the final reply, provide a reasoning for your assessment.
Analysis dimensions:
* Exploitable:
Determine if the bug can result in memory corruption or elevated privileges.
Memory safety issues are almost always exploitable (KASAN or UBSAN reports for use-after-free, out-of-bounds;
refcounting issues, corrupted lists, etc). When kernel is crashing on a completly wild pointer access
(e.g. user-space address, or non-canonical address, but not on NULL or address corresponding to KASAN shadow
for NULL address), including both data accesses and control tranfers, that's also usually implies possibility
of exploitation. Such reports usually say "unable to handle kernel paging request".
Uses of uninitialized values detected by KMSAN may be exploitable b/c attacker frequently can affect uninit
values with spraying techniques. However, for these exploitabability depends on how exactly the uninit value
is used in the code, and what it affects.
Think of what happens after the bug is triggered. Some bugs cause kernel panic and halt execution,
they are harder to exploit. For example, BUG reports halts the kernel. However, WARNING reports don't halt
execution in production builds. Debug bug detection tools (like KASAN, KMSAN, KCSAN, UBSAN) are also not enabled
in production builds, so attacker can freely exploit these bugs w/o being detected by these tools.
If you see an integer overflow, think how the overflowed value used later (if it's used as allocation size,
or an array index). If you see an out-of-bounds read, think if it's followed by an out-of-bounds write as well.
Some KCSAN data-races may be exploitable by skilled attackers as well. Think what data structures got corrupted
as the result of data races and how. However, note that kernel has lots of "benign" data races that don't lead
to any runtime misbehavior at all.
* Denial Of Service:
Determine if the bug can result in denial-of-service. Most bugs can, since they cause system crash,
hangs, deadlocks, or resource leaks. This is mostly applicable to WARNING bugs that won't cause system crash
in production. For these think what will be consequences of the violation of the kernel assumptions flagged
by the WARNING. In some cases the unexpected condition is also properly handled by the normal control flow
(e.g. with "if (WARN_ON(...))"), these won't cause denial-of-service. If the condition is not handled,
then it may or may not cause denial-of-service.
* Accessible From Unprivileged Processes:
Determine if the bug can be reached from a typical (non-root) user process that does NOT have any special capabilities
(like CAP_SYS_ADMIN, CAP_NET_ADMIN, CAP_NET_RAW, CAP_PERFMON) or access to device nodes restricted to root.
Assume that unprivileged_bpf_disabled=1, that is eBPF loading is not accessible. However, cBPF (classical BPF)
is still accessible to non-root processes.
Assume that user namespaces are not accessible, that is, the process cannot get the mentioned capabilities even
within a new user namespace (checked by ns_capable() function in the kernel sources).
* Accessible From User Namespaces:
Determine if the bug can be reached within a user-namespace where the process has all capabilities
(including CAP_SYS_ADMIN, CAP_NET_ADMIN, CAP_NET_RAW, CAP_PERFMON). Such capabilities are checked with ns_capable()
function in the kernel sources.
* VM Guest Trigger:
Determine if the bug can be triggered from the context of a typical KVM guest (e.g., set up by a QEMU VMM).
Consider accesses to standard Linux host paravirtualized features (virtio-blk, virtio-net, etc.),
and handling of VM exits in the KVM code.
* VM Host Trigger in The Confidetial Computing Context:
Determine if the bug can be triggered in a confidential computing guest kernel from the context of a KVM host.
Consider access to standard Linux guest paravirtualized features (virtio-blk, virtio-net, etc.).
* Ethernet Network Trigger:
Determine if the bug can be triggered by processing ingress network Ethernet traffic, either directly (network stack)
or via drivers exposed to network data.
* Other Remote Trigger:
Determine if the bug can be triggered by processing remote traffic other than Ethernet (Wifi, Bluetooth, NFC, etc).
* Peripheral Trigger:
Determine if the bug can be triggered via an untrusted peripheral device that can be physically plugged
into a system, such as a USB device or a niche hardware driver handling external hardware inputs.
This is particularly important for mobile and desktop environments where users can plug in unknown devices.
* Malicious Filesystem Trigger:
Determine if the bug can be triggered by the kernel mounting and parsing a malicious filesystem image.
This is highly critical for Desktop and Mobile environments where external media or downloaded images
might be auto-mounted.
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt:
The kernel bug report is:
usb 4-1: USB disconnect, device number 6
net1080 4-1:0.166 usb0: unregister 'net1080' usb-dummy_hcd.3-1, NetChip TurboCONNECT
------------[ cut here ]------------
WARNING: net/sched/sch_generic.c:1496 at dev_shutdown+0x3c1/0x430 net/sched/sch_generic.c:1496, CPU#1: kworker/1:7/6098
Modules linked in:
CPU: 1 UID: 0 PID: 6098 Comm: kworker/1:7 Not tainted syzkaller #0 PREEMPT(full)
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/25/2025
Workqueue: usb_hub_wq hub_event
RIP: 0010:dev_shutdown+0x3c1/0x430 net/sched/sch_generic.c:1496
Code: 48 c7 c2 40 2d c7 8c be a3 00 00 00 48 c7 c7 40 32 c7 8c c6 05 1f 30 08 07 01 e8 aa cd 30 f8 e9 8f fd ff ff e8 70 1b 56 f8 90 <0f> 0b 90 e9 5f fe ff ff 4c 89 f7 e8 ef e8 bf f8 e9 6c fc ff ff 4c
RSP: 0018:ffffc90003f5f410 EFLAGS: 00010293
RAX: 0000000000000000 RBX: ffff88802c5444d0 RCX: ffffffff89683b9e
RDX: ffff88807a4b4980 RSI: ffffffff8968c850 RDI: ffff88802c5445b0
RBP: ffff88802c544000 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000000 R12: ffff88802c544480
R13: ffffed10058a8803 R14: ffff88802c5444c8 R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff8881249fd000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000040 CR3: 0000000029904000 CR4: 00000000003526f0
Call Trace:
<TASK>
unregister_netdevice_many_notify+0xb37/0x2590 net/core/dev.c:12369
unregister_netdevice_many net/core/dev.c:12444 [inline]
unregister_netdevice_queue net/core/dev.c:12258 [inline]
unregister_netdevice_queue+0x305/0x3c0 net/core/dev.c:12248
unregister_netdevice include/linux/netdevice.h:3405 [inline]
unregister_netdev+0x1f/0x60 net/core/dev.c:12462
usbnet_disconnect+0x109/0x510 drivers/net/usb/usbnet.c:1676
usb_unbind_interface+0x1dd/0x9e0 drivers/usb/core/driver.c:458
device_remove drivers/base/dd.c:571 [inline]
device_remove+0x125/0x170 drivers/base/dd.c:563
__device_release_driver drivers/base/dd.c:1282 [inline]
device_release_driver_internal+0x44b/0x620 drivers/base/dd.c:1305
bus_remove_device+0x22f/0x450 drivers/base/bus.c:616
device_del+0x396/0x9f0 drivers/base/core.c:3878
usb_disable_device+0x355/0x820 drivers/usb/core/message.c:1418
usb_disconnect+0x2e1/0x9e0 drivers/usb/core/hub.c:2345
hub_port_connect drivers/usb/core/hub.c:5407 [inline]
hub_port_connect_change drivers/usb/core/hub.c:5707 [inline]
port_event drivers/usb/core/hub.c:5871 [inline]
hub_event+0x1d84/0x52f0 drivers/usb/core/hub.c:5953
process_one_work+0x9ba/0x1b20 kernel/workqueue.c:3257
process_scheduled_works kernel/workqueue.c:3340 [inline]
worker_thread+0x6c8/0xf10 kernel/workqueue.c:3421
kthread+0x3c5/0x780 kernel/kthread.c:463
ret_from_fork+0x983/0xb10 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:246
</TASK>
It is reproducible with the followint program.
Keep in mind that it may lack the precise threading, sandboxing, and some arguments of a working reproducer.
But it should give an idea of the involved syscalls.
// autogenerated by syzkaller (https://github.com/google/syzkaller)
#define _GNU_SOURCE
#include <endian.h>
#include <errno.h>
#include <fcntl.h>
#include <stdarg.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mount.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <sys/types.h>
#include <unistd.h>
#include <linux/usb/ch9.h>
static unsigned long long procid;
static void sleep_ms(uint64_t ms)
{
usleep(ms * 1000);
}
#define SIZEOF_IO_URING_SQE 64
#define SIZEOF_IO_URING_CQE 16
#define IORING_SETUP_SQE128 (1U << 10)
#define IORING_SETUP_CQE32 (1U << 11)
struct io_sqring_offsets {
uint32_t head;
uint32_t tail;
uint32_t ring_mask;
uint32_t ring_entries;
uint32_t flags;
uint32_t dropped;
uint32_t array;
uint32_t resv1;
uint64_t user_addr;
};
struct io_cqring_offsets {
uint32_t head;
uint32_t tail;
uint32_t ring_mask;
uint32_t ring_entries;
uint32_t overflow;
uint32_t cqes;
uint32_t flags;
uint32_t resv1;
uint64_t user_addr;
};
struct io_uring_params {
uint32_t sq_entries;
uint32_t cq_entries;
uint32_t flags;
uint32_t sq_thread_cpu;
uint32_t sq_thread_idle;
uint32_t features;
uint32_t resv[4];
struct io_sqring_offsets sq_off;
struct io_cqring_offsets cq_off;
};
static long io_uring_sqe_size(struct io_uring_params* params)
{
return SIZEOF_IO_URING_SQE << !!(params->flags & IORING_SETUP_SQE128);
}
static long syz_io_uring_submit(volatile long a0, volatile long a1, volatile long a2, volatile long a3)
{
struct io_uring_params* params = (struct io_uring_params*)a0;
char* ring_ptr = (char*)a1;
char* sqes_ptr = (char*)a2;
char* sqe = (char*)a3;
uint32_t sq_ring_mask = *(uint32_t*)(ring_ptr + params->sq_off.ring_mask);
uint32_t* sq_tail_ptr = (uint32_t*)(ring_ptr + params->sq_off.tail);
uint32_t sq_tail = *sq_tail_ptr & sq_ring_mask;
uint32_t sqe_size = io_uring_sqe_size(params);
char* sqe_dest = sqes_ptr + sq_tail * sqe_size;
memcpy(sqe_dest, sqe, sqe_size);
uint32_t sq_tail_next = *sq_tail_ptr + 1;
__atomic_store_n(sq_tail_ptr, sq_tail_next, __ATOMIC_RELEASE);
return 0;
}
#define MAX_FDS 30
#define USB_MAX_IFACE_NUM 4
#define USB_MAX_EP_NUM 32
#define USB_MAX_FDS 6
struct usb_endpoint_index {
struct usb_endpoint_descriptor desc;
int handle;
};
struct usb_iface_index {
struct usb_interface_descriptor* iface;
uint8_t bInterfaceNumber;
uint8_t bAlternateSetting;
uint8_t bInterfaceClass;
struct usb_endpoint_index eps[USB_MAX_EP_NUM];
int eps_num;
};
struct usb_device_index {
struct usb_device_descriptor* dev;
struct usb_config_descriptor* config;
uint8_t bDeviceClass;
uint8_t bMaxPower;
int config_length;
struct usb_iface_index ifaces[USB_MAX_IFACE_NUM];
int ifaces_num;
int iface_cur;
};
struct usb_info {
int fd;
struct usb_device_index index;
};
static struct usb_info usb_devices[USB_MAX_FDS];
static struct usb_device_index* lookup_usb_index(int fd)
{
for (int i = 0; i < USB_MAX_FDS; i++) {
if (__atomic_load_n(&usb_devices[i].fd, __ATOMIC_ACQUIRE) == fd)
return &usb_devices[i].index;
}
return NULL;
}
static int usb_devices_num;
static bool parse_usb_descriptor(const char* buffer, size_t length, struct usb_device_index* index)
{
if (length < sizeof(*index->dev) + sizeof(*index->config))
return false;
memset(index, 0, sizeof(*index));
index->dev = (struct usb_device_descriptor*)buffer;
index->config = (struct usb_config_descriptor*)(buffer + sizeof(*index->dev));
index->bDeviceClass = index->dev->bDeviceClass;
index->bMaxPower = index->config->bMaxPower;
index->config_length = length - sizeof(*index->dev);
index->iface_cur = -1;
size_t offset = 0;
while (true) {
if (offset + 1 >= length)
break;
uint8_t desc_length = buffer[offset];
uint8_t desc_type = buffer[offset + 1];
if (desc_length <= 2)
break;
if (offset + desc_length > length)
break;
if (desc_type == USB_DT_INTERFACE && index->ifaces_num < USB_MAX_IFACE_NUM) {
struct usb_interface_descriptor* iface = (struct usb_interface_descriptor*)(buffer + offset);
index->ifaces[index->ifaces_num].iface = iface;
index->ifaces[index->ifaces_num].bInterfaceNumber = iface->bInterfaceNumber;
index->ifaces[index->ifaces_num].bAlternateSetting = iface->bAlternateSetting;
index->ifaces[index->ifaces_num].bInterfaceClass = iface->bInterfaceClass;
index->ifaces_num++;
}
if (desc_type == USB_DT_ENDPOINT && index->ifaces_num > 0) {
struct usb_iface_index* iface = &index->ifaces[index->ifaces_num - 1];
if (iface->eps_num < USB_MAX_EP_NUM) {
memcpy(&iface->eps[iface->eps_num].desc, buffer + offset, sizeof(iface->eps[iface->eps_num].desc));
iface->eps_num++;
}
}
offset += desc_length;
}
return true;
}
static struct usb_device_index* add_usb_index(int fd, const char* dev, size_t dev_len)
{
int i = __atomic_fetch_add(&usb_devices_num, 1, __ATOMIC_RELAXED);
if (i >= USB_MAX_FDS)
return NULL;
if (!parse_usb_descriptor(dev, dev_len, &usb_devices[i].index))
return NULL;
__atomic_store_n(&usb_devices[i].fd, fd, __ATOMIC_RELEASE);
return &usb_devices[i].index;
}
struct vusb_connect_string_descriptor {
uint32_t len;
char* str;
} __attribute__((packed));
struct vusb_connect_descriptors {
uint32_t qual_len;
char* qual;
uint32_t bos_len;
char* bos;
uint32_t strs_len;
struct vusb_connect_string_descriptor strs[0];
} __attribute__((packed));
static const char default_string[] = {
8, USB_DT_STRING,
's', 0, 'y', 0, 'z', 0
};
static const char default_lang_id[] = {
4, USB_DT_STRING,
0x09, 0x04
};
static bool lookup_connect_response_in(int fd, const struct vusb_connect_descriptors* descs,
const struct usb_ctrlrequest* ctrl,
struct usb_qualifier_descriptor* qual,
char** response_data, uint32_t* response_length)
{
struct usb_device_index* index = lookup_usb_index(fd);
uint8_t str_idx;
if (!index)
return false;
switch (ctrl->bRequestType & USB_TYPE_MASK) {
case USB_TYPE_STANDARD:
switch (ctrl->bRequest) {
case USB_REQ_GET_DESCRIPTOR:
switch (ctrl->wValue >> 8) {
case USB_DT_DEVICE:
*response_data = (char*)index->dev;
*response_length = sizeof(*index->dev);
return true;
case USB_DT_CONFIG:
*response_data = (char*)index->config;
*response_length = index->config_length;
return true;
case USB_DT_STRING:
str_idx = (uint8_t)ctrl->wValue;
if (descs && str_idx < descs->strs_len) {
*response_data = descs->strs[str_idx].str;
*response_length = descs->strs[str_idx].len;
return true;
}
if (str_idx == 0) {
*response_data = (char*)&default_lang_id[0];
*response_length = default_lang_id[0];
return true;
}
*response_data = (char*)&default_string[0];
*response_length = default_string[0];
return true;
case USB_DT_BOS:
*response_data = descs->bos;
*response_length = descs->bos_len;
return true;
case USB_DT_DEVICE_QUALIFIER:
if (!descs->qual) {
qual->bLength = sizeof(*qual);
qual->bDescriptorType = USB_DT_DEVICE_QUALIFIER;
qual->bcdUSB = index->dev->bcdUSB;
qual->bDeviceClass = index->dev->bDeviceClass;
qual->bDeviceSubClass = index->dev->bDeviceSubClass;
qual->bDeviceProtocol = index->dev->bDeviceProtocol;
qual->bMaxPacketSize0 = index->dev->bMaxPacketSize0;
qual->bNumConfigurations = index->dev->bNumConfigurations;
qual->bRESERVED = 0;
*response_data = (char*)qual;
*response_length = sizeof(*qual);
return true;
}
*response_data = descs->qual;
*response_length = descs->qual_len;
return true;
default:
break;
}
break;
default:
break;
}
break;
default:
break;
}
return false;
}
typedef bool (*lookup_connect_out_response_t)(int fd, const struct vusb_connect_descriptors* descs,
const struct usb_ctrlrequest* ctrl, bool* done);
static bool lookup_connect_response_out_generic(int fd, const struct vusb_connect_descriptors* descs,
const struct usb_ctrlrequest* ctrl, bool* done)
{
switch (ctrl->bRequestType & USB_TYPE_MASK) {
case USB_TYPE_STANDARD:
switch (ctrl->bRequest) {
case USB_REQ_SET_CONFIGURATION:
*done = true;
return true;
default:
break;
}
break;
}
return false;
}
struct vusb_descriptor {
uint8_t req_type;
uint8_t desc_type;
uint32_t len;
char data[0];
} __attribute__((packed));
struct vusb_descriptors {
uint32_t len;
struct vusb_descriptor* generic;
struct vusb_descriptor* descs[0];
} __attribute__((packed));
struct vusb_response {
uint8_t type;
uint8_t req;
uint32_t len;
char data[0];
} __attribute__((packed));
struct vusb_responses {
uint32_t len;
struct vusb_response* generic;
struct vusb_response* resps[0];
} __attribute__((packed));
static bool lookup_control_response(const struct vusb_descriptors* descs, const struct vusb_responses* resps,
struct usb_ctrlrequest* ctrl, char** response_data, uint32_t* response_length)
{
int descs_num = 0;
int resps_num = 0;
if (descs)
descs_num = (descs->len - offsetof(struct vusb_descriptors, descs)) / sizeof(descs->descs[0]);
if (resps)
resps_num = (resps->len - offsetof(struct vusb_responses, resps)) / sizeof(resps->resps[0]);
uint8_t req = ctrl->bRequest;
uint8_t req_type = ctrl->bRequestType & USB_TYPE_MASK;
uint8_t desc_type = ctrl->wValue >> 8;
if (req == USB_REQ_GET_DESCRIPTOR) {
int i;
for (i = 0; i < descs_num; i++) {
struct vusb_descriptor* desc = descs->descs[i];
if (!desc)
continue;
if (desc->req_type == req_type && desc->desc_type == desc_type) {
*response_length = desc->len;
if (*response_length != 0)
*response_data = &desc->data[0];
else
*response_data = NULL;
return true;
}
}
if (descs && descs->generic) {
*response_data = &descs->generic->data[0];
*response_length = descs->generic->len;
return true;
}
} else {
int i;
for (i = 0; i < resps_num; i++) {
struct vusb_response* resp = resps->resps[i];
if (!resp)
continue;
if (resp->type == req_type && resp->req == req) {
*response_length = resp->len;
if (*response_length != 0)
*response_data = &resp->data[0];
else
*response_data = NULL;
return true;
}
}
if (resps && resps->generic) {
*response_data = &resps->generic->data[0];
*response_length = resps->generic->len;
return true;
}
}
return false;
}
#define UDC_NAME_LENGTH_MAX 128
struct usb_raw_init {
__u8 driver_name[UDC_NAME_LENGTH_MAX];
__u8 device_name[UDC_NAME_LENGTH_MAX];
__u8 speed;
};
enum usb_raw_event_type {
USB_RAW_EVENT_INVALID = 0,
USB_RAW_EVENT_CONNECT = 1,
USB_RAW_EVENT_CONTROL = 2,
};
struct usb_raw_event {
__u32 type;
__u32 length;
__u8 data[0];
};
struct usb_raw_ep_io {
__u16 ep;
__u16 flags;
__u32 length;
__u8 data[0];
};
#define USB_RAW_EPS_NUM_MAX 30
#define USB_RAW_EP_NAME_MAX 16
#define USB_RAW_EP_ADDR_ANY 0xff
struct usb_raw_ep_caps {
__u32 type_control : 1;
__u32 type_iso : 1;
__u32 type_bulk : 1;
__u32 type_int : 1;
__u32 dir_in : 1;
__u32 dir_out : 1;
};
struct usb_raw_ep_limits {
__u16 maxpacket_limit;
__u16 max_streams;
__u32 reserved;
};
struct usb_raw_ep_info {
__u8 name[USB_RAW_EP_NAME_MAX];
__u32 addr;
struct usb_raw_ep_caps caps;
struct usb_raw_ep_limits limits;
};
struct usb_raw_eps_info {
struct usb_raw_ep_info eps[USB_RAW_EPS_NUM_MAX];
};
#define USB_RAW_IOCTL_INIT _IOW('U', 0, struct usb_raw_init)
#define USB_RAW_IOCTL_RUN _IO('U', 1)
#define USB_RAW_IOCTL_EVENT_FETCH _IOR('U', 2, struct usb_raw_event)
#define USB_RAW_IOCTL_EP0_WRITE _IOW('U', 3, struct usb_raw_ep_io)
#define USB_RAW_IOCTL_EP0_READ _IOWR('U', 4, struct usb_raw_ep_io)
#define USB_RAW_IOCTL_EP_ENABLE _IOW('U', 5, struct usb_endpoint_descriptor)
#define USB_RAW_IOCTL_EP_DISABLE _IOW('U', 6, __u32)
#define USB_RAW_IOCTL_EP_WRITE _IOW('U', 7, struct usb_raw_ep_io)
#define USB_RAW_IOCTL_EP_READ _IOWR('U', 8, struct usb_raw_ep_io)
#define USB_RAW_IOCTL_CONFIGURE _IO('U', 9)
#define USB_RAW_IOCTL_VBUS_DRAW _IOW('U', 10, __u32)
#define USB_RAW_IOCTL_EPS_INFO _IOR('U', 11, struct usb_raw_eps_info)
#define USB_RAW_IOCTL_EP0_STALL _IO('U', 12)
#define USB_RAW_IOCTL_EP_SET_HALT _IOW('U', 13, __u32)
#define USB_RAW_IOCTL_EP_CLEAR_HALT _IOW('U', 14, __u32)
#define USB_RAW_IOCTL_EP_SET_WEDGE _IOW('U', 15, __u32)
static int usb_raw_open()
{
return open("/dev/raw-gadget", O_RDWR);
}
static int usb_raw_init(int fd, uint32_t speed, const char* driver, const char* device)
{
struct usb_raw_init arg;
strncpy((char*)&arg.driver_name[0], driver, sizeof(arg.driver_name));
strncpy((char*)&arg.device_name[0], device, sizeof(arg.device_name));
arg.speed = speed;
return ioctl(fd, USB_RAW_IOCTL_INIT, &arg);
}
static int usb_raw_run(int fd)
{
return ioctl(fd, USB_RAW_IOCTL_RUN, 0);
}
static int usb_raw_configure(int fd)
{
return ioctl(fd, USB_RAW_IOCTL_CONFIGURE, 0);
}
static int usb_raw_vbus_draw(int fd, uint32_t power)
{
return ioctl(fd, USB_RAW_IOCTL_VBUS_DRAW, power);
}
static int usb_raw_ep0_write(int fd, struct usb_raw_ep_io* io)
{
return ioctl(fd, USB_RAW_IOCTL_EP0_WRITE, io);
}
static int usb_raw_ep0_read(int fd, struct usb_raw_ep_io* io)
{
return ioctl(fd, USB_RAW_IOCTL_EP0_READ, io);
}
static int usb_raw_event_fetch(int fd, struct usb_raw_event* event)
{
return ioctl(fd, USB_RAW_IOCTL_EVENT_FETCH, event);
}
static int usb_raw_ep_enable(int fd, struct usb_endpoint_descriptor* desc)
{
return ioctl(fd, USB_RAW_IOCTL_EP_ENABLE, desc);
}
static int usb_raw_ep_disable(int fd, int ep)
{
return ioctl(fd, USB_RAW_IOCTL_EP_DISABLE, ep);
}
static int usb_raw_ep0_stall(int fd)
{
return ioctl(fd, USB_RAW_IOCTL_EP0_STALL, 0);
}
static int lookup_interface(int fd, uint8_t bInterfaceNumber, uint8_t bAlternateSetting)
{
struct usb_device_index* index = lookup_usb_index(fd);
if (!index)
return -1;
for (int i = 0; i < index->ifaces_num; i++) {
if (index->ifaces[i].bInterfaceNumber == bInterfaceNumber &&
index->ifaces[i].bAlternateSetting == bAlternateSetting)
return i;
}
return -1;
}
#define USB_MAX_PACKET_SIZE 4096
struct usb_raw_control_event {
struct usb_raw_event inner;
struct usb_ctrlrequest ctrl;
char data[USB_MAX_PACKET_SIZE];
};
struct usb_raw_ep_io_data {
struct usb_raw_ep_io inner;
char data[USB_MAX_PACKET_SIZE];
};
static void set_interface(int fd, int n)
{
struct usb_device_index* index = lookup_usb_index(fd);
if (!index)
return;
if (index->iface_cur >= 0 && index->iface_cur < index->ifaces_num) {
for (int ep = 0; ep < index->ifaces[index->iface_cur].eps_num; ep++) {
int rv = usb_raw_ep_disable(fd, index->ifaces[index->iface_cur].eps[ep].handle);
if (rv < 0) {
} else {
}
}
}
if (n >= 0 && n < index->ifaces_num) {
for (int ep = 0; ep < index->ifaces[n].eps_num; ep++) {
int rv = usb_raw_ep_enable(fd, &index->ifaces[n].eps[ep].desc);
if (rv < 0) {
} else {
index->ifaces[n].eps[ep].handle = rv;
}
}
index->iface_cur = n;
}
}
static int configure_device(int fd)
{
struct usb_device_index* index = lookup_usb_index(fd);
if (!index)
return -1;
int rv = usb_raw_vbus_draw(fd, index->bMaxPower);
if (rv < 0) {
return rv;
}
rv = usb_raw_configure(fd);
if (rv < 0) {
return rv;
}
set_interface(fd, 0);
return 0;
}
static volatile long syz_usb_connect_impl(uint64_t speed, uint64_t dev_len, const char* dev,
const struct vusb_connect_descriptors* descs,
lookup_connect_out_response_t lookup_connect_response_out)
{
if (!dev) {
return -1;
}
int fd = usb_raw_open();
if (fd < 0) {
return fd;
}
if (fd >= MAX_FDS) {
close(fd);
return -1;
}
struct usb_device_index* index = add_usb_index(fd, dev, dev_len);
if (!index) {
return -1;
}
char device[32];
sprintf(&device[0], "dummy_udc.%llu", procid);
int rv = usb_raw_init(fd, speed, "dummy_udc", &device[0]);
if (rv < 0) {
return rv;
}
rv = usb_raw_run(fd);
if (rv < 0) {
return rv;
}
bool done = false;
while (!done) {
struct usb_raw_control_event event;
event.inner.type = 0;
event.inner.length = sizeof(event.ctrl);
rv = usb_raw_event_fetch(fd, (struct usb_raw_event*)&event);
if (rv < 0) {
return rv;
}
if (event.inner.type != USB_RAW_EVENT_CONTROL)
continue;
char* response_data = NULL;
uint32_t response_length = 0;
struct usb_qualifier_descriptor qual;
if (event.ctrl.bRequestType & USB_DIR_IN) {
if (!lookup_connect_response_in(fd, descs, &event.ctrl, &qual, &response_data, &response_length)) {
usb_raw_ep0_stall(fd);
continue;
}
} else {
if (!lookup_connect_response_out(fd, descs, &event.ctrl, &done)) {
usb_raw_ep0_stall(fd);
continue;
}
response_data = NULL;
response_length = event.ctrl.wLength;
}
if ((event.ctrl.bRequestType & USB_TYPE_MASK) == USB_TYPE_STANDARD &&
event.ctrl.bRequest == USB_REQ_SET_CONFIGURATION) {
rv = configure_device(fd);
if (rv < 0) {
return rv;
}
}
struct usb_raw_ep_io_data response;
response.inner.ep = 0;
response.inner.flags = 0;
if (response_length > sizeof(response.data))
response_length = 0;
if (event.ctrl.wLength < response_length)
response_length = event.ctrl.wLength;
response.inner.length = response_length;
if (response_data)
memcpy(&response.data[0], response_data, response_length);
else
memset(&response.data[0], 0, response_length);
if (event.ctrl.bRequestType & USB_DIR_IN) {
rv = usb_raw_ep0_write(fd, (struct usb_raw_ep_io*)&response);
} else {
rv = usb_raw_ep0_read(fd, (struct usb_raw_ep_io*)&response);
}
if (rv < 0) {
return rv;
}
}
sleep_ms(200);
return fd;
}
static volatile long syz_usb_connect(volatile long a0, volatile long a1, volatile long a2, volatile long a3)
{
uint64_t speed = a0;
uint64_t dev_len = a1;
const char* dev = (const char*)a2;
const struct vusb_connect_descriptors* descs = (const struct vusb_connect_descriptors*)a3;
return syz_usb_connect_impl(speed, dev_len, dev, descs, &lookup_connect_response_out_generic);
}
static volatile long syz_usb_control_io(volatile long a0, volatile long a1, volatile long a2)
{
int fd = a0;
const struct vusb_descriptors* descs = (const struct vusb_descriptors*)a1;
const struct vusb_responses* resps = (const struct vusb_responses*)a2;
struct usb_raw_control_event event;
event.inner.type = 0;
event.inner.length = USB_MAX_PACKET_SIZE;
int rv = usb_raw_event_fetch(fd, (struct usb_raw_event*)&event);
if (rv < 0) {
return rv;
}
if (event.inner.type != USB_RAW_EVENT_CONTROL) {
return -1;
}
char* response_data = NULL;
uint32_t response_length = 0;
if ((event.ctrl.bRequestType & USB_DIR_IN) && event.ctrl.wLength) {
if (!lookup_control_response(descs, resps, &event.ctrl, &response_data, &response_length)) {
usb_raw_ep0_stall(fd);
return -1;
}
} else {
if ((event.ctrl.bRequestType & USB_TYPE_MASK) == USB_TYPE_STANDARD ||
event.ctrl.bRequest == USB_REQ_SET_INTERFACE) {
int iface_num = event.ctrl.wIndex;
int alt_set = event.ctrl.wValue;
int iface_index = lookup_interface(fd, iface_num, alt_set);
if (iface_index < 0) {
} else {
set_interface(fd, iface_index);
}
}
response_length = event.ctrl.wLength;
}
struct usb_raw_ep_io_data response;
response.inner.ep = 0;
response.inner.flags = 0;
if (response_length > sizeof(response.data))
response_length = 0;
if (event.ctrl.wLength < response_length)
response_length = event.ctrl.wLength;
if ((event.ctrl.bRequestType & USB_DIR_IN) && !event.ctrl.wLength) {
response_length = USB_MAX_PACKET_SIZE;
}
response.inner.length = response_length;
if (response_data)
memcpy(&response.data[0], response_data, response_length);
else
memset(&response.data[0], 0, response_length);
if ((event.ctrl.bRequestType & USB_DIR_IN) && event.ctrl.wLength) {
rv = usb_raw_ep0_write(fd, (struct usb_raw_ep_io*)&response);
} else {
rv = usb_raw_ep0_read(fd, (struct usb_raw_ep_io*)&response);
}
if (rv < 0) {
return rv;
}
sleep_ms(200);
return 0;
}
uint64_t r[1] = {0xffffffffffffffff};
int main(void)
{
syscall(__NR_mmap, /*addr=*/0x1ffffffff000ul, /*len=*/0x1000ul, /*prot=*/0ul, /*flags=MAP_FIXED|MAP_ANONYMOUS|MAP_PRIVATE*/0x32ul, /*fd=*/(intptr_t)-1, /*offset=*/0ul);
syscall(__NR_mmap, /*addr=*/0x200000000000ul, /*len=*/0x1000000ul, /*prot=PROT_WRITE|PROT_READ|PROT_EXEC*/7ul, /*flags=MAP_FIXED|MAP_ANONYMOUS|MAP_PRIVATE*/0x32ul, /*fd=*/(intptr_t)-1, /*offset=*/0ul);
syscall(__NR_mmap, /*addr=*/0x200001000000ul, /*len=*/0x1000ul, /*prot=*/0ul, /*flags=MAP_FIXED|MAP_ANONYMOUS|MAP_PRIVATE*/0x32ul, /*fd=*/(intptr_t)-1, /*offset=*/0ul);
const char* reason;
(void)reason;
intptr_t res = 0;
if (write(1, "executing program\n", sizeof("executing program\n") - 1)) {}
// syz_usb_connect arguments: [
// speed: usb_device_speed = 0x0 (8 bytes)
// dev_len: len = 0x36 (8 bytes)
// dev: ptr[inout, array[ANYUNION]] {
// array[ANYUNION] {
// union ANYUNION {
// ANYBLOB: buffer: {12 01 00 00 98 80 a4 20 d0 06 22 06 5a 3f 01 02 03 01 09 02 24 00 01 00 00 10 00 09 04 a6 00 02 09 ab e0 00 09 05 02 02 00 02 02 00 00 09 05 82 02 10 00 00 00} (length 0x35)
// }
// }
// }
// conn_descs: nil
// ]
// returns fd_usb
memcpy((void*)0x200000000080, "... [truncated large byte array] ...", 53);
res = -1;
res = syz_usb_connect(/*speed=*/0, /*dev_len=*/0x36, /*dev=*/0x200000000080, /*conn_descs=*/0);
if (res != -1)
r[0] = res;
// syz_usb_control_io arguments: [
// fd: fd_usb (resource)
// descs: nil
// resps: nil
// ]
syz_usb_control_io(/*fd=*/r[0], /*descs=*/0, /*resps=*/0);
// syz_io_uring_submit arguments: [
// ring_params_ptr: ring_params_ptr (resource)
// ring_ptr: ring_ptr (resource)
// sqes_ptr: sqes_ptr (resource)
// sqe: ptr[in, io_uring_sqe_u] {
// union io_uring_sqe_u {
// IORING_OP_NOP: io_uring_sqe[IORING_OP_NOP, const[0, int16], const[0, int32], const[0, int64], const[0, int64], const[0, int32], const[0, int32], zero_flag, array[const[0, int64], 3]] {
// opcode: const = 0x0 (1 bytes)
// flags: iosqe_flags = 0x0 (1 bytes)
// ioprio: const = 0x0 (2 bytes)
// fd: const = 0x0 (4 bytes)
// off: const = 0x0 (8 bytes)
// addr: const = 0x0 (8 bytes)
// len: const = 0x0 (4 bytes)
// misc_flags: const = 0x0 (4 bytes)
// user_data: zero_flag = 0x0 (8 bytes)
// misc: buffer: {00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00} (length 0x18)
// }
// }
// }
// ]
*(uint8_t*)0x200000000000 = 0;
*(uint8_t*)0x200000000001 = 0;
*(uint16_t*)0x200000000002 = 0;
*(uint32_t*)0x200000000004 = 0;
*(uint64_t*)0x200000000008 = 0;
*(uint64_t*)0x200000000010 = 0;
*(uint32_t*)0x200000000018 = 0;
*(uint32_t*)0x20000000001c = 0;
*(uint64_t*)0x200000000020 = 0;
memset((void*)0x200000000028, 0, 24);
syz_io_uring_submit(/*ring_params_ptr=*/0, /*ring_ptr=*/0, /*sqes_ptr=*/0, /*sqe=*/0x200000000000);
return 0;
}
Reply: The bug report indicates a `WARNING` in `dev_shutdown()` at `net/sched/sch_generic.c:1496` because `timer_pending(&dev->watchdog_timer)` evaluates to true when a network device is being unregistered.
### Root Cause Analysis
The warning triggers because the network device's watchdog timer is still pending when the device is being dismantled.
When a network device is unregistered, `unregister_netdevice_many_notify()` is called. This function first checks if the device is currently running (`dev->flags & IFF_UP`). If it is, it calls `netif_close_many()`, which eventually calls `dev_deactivate_many()` and `netdev_watchdog_down()` to safely delete the watchdog timer. However, if `IFF_UP` is not set, this teardown step is skipped.
The race condition occurs during the device initialization phase (`__dev_open()`):
1. `__dev_open()` sets the `__LINK_STATE_START` bit before calling the driver's `ndo_open()` callback.
2. If `ndo_open()` blocks or takes a long time, an asynchronous event (like a USB kevent or a carrier change) can call `netif_carrier_on()`.
3. `netif_carrier_on()` checks `netif_running(dev)` (which returns true because `__LINK_STATE_START` is set) and subsequently calls `netdev_watchdog_up()`, arming the timer.
4. If `ndo_open()` subsequently fails, `__dev_open()` clears `__LINK_STATE_START` and returns an error, but it **never sets `IFF_UP`**.
5. When the device is later disconnected and unregistered, `unregister_netdevice_many_notify()` sees that `IFF_UP` is false and skips calling `netdev_watchdog_down()`.
6. Finally, `dev_shutdown()` is called, and the `WARN_ON(timer_pending(&dev->watchdog_timer))` triggers because the timer was armed but never deleted.
### Security Impact
* **Exploitable:** No. While a pending timer on a freed object usually leads to a Use-After-Free (UAF), the network device unregistration process (`netdev_run_todo()`) involves RCU grace periods and waits for all references to drop before calling `free_netdev()`. The timer will almost certainly fire and finish its execution before the `struct net_device` memory is actually freed, preventing a UAF.
* **Denial of Service:** Yes. In kernels with `panic_on_warn` enabled, this will crash the system. In production builds without `panic_on_warn`, it will just print a warning and continue, but it indicates a state inconsistency.
* **Peripheral Trigger:** Yes. The bug can be triggered by plugging in a malicious USB device (or using the `dummy_hcd` / `raw-gadget` interfaces as the syzkaller reproducer does) that simulates a network interface and manipulates the carrier state during initialization.
* **Unprivileged:** Yes. If the system allows unprivileged users to access USB devices or the `raw-gadget` interface, they can trigger this issue.
|
| 6/2 |
2026/05/21 11:33 |
llm |
expert |
0m
Model:
Tokens:
input: 15575
output: 17
thoughts: 835
Thoughts: **Analyzing a Kernel Bug Report: A Deep Dive into a USB Disconnect Issue**
Okay, so I've got this kernel bug report in front of me, and it immediately screams "USB disconnect" – specifically, a `net1080` device, which I recognize as a NetChip TurboCONNECT device, is the culprit. The logs show a device removal, and the warning about `net/sched/sch_generic.c:1496` immediately grabs my attention. That's usually a critical point during network device shutdown, and the call stack shows it's happening during the `usbnet_disconnect` function.
The crash appears to happen during the unregistration process of the network device itself. The `dev_shutdown` function, specifically at the line mentioned in `net/sched/sch_generic.c`, is where the system seems to be hitting a snag. It's triggered by `unregister_netdevice_many_notify`, which points to a problem within the NetChip driver's unbinding process, or the core network device unregistration flow itself.
The call stack confirms this, showing a cascade of function calls starting with `usb_unbind_interface`, `device_remove`, and ultimately originating from `hub_event` function within the USB core. It appears that the USB hub is initiating the disconnect, leading to the device's removal, but the network stack is stumbling during the teardown process. The `usbnet_disconnect` function appears to be the point of the failure. I need to dive deeper into that specific function to understand exactly how the net device is being unregistered, and identify any race conditions.
My next step is to examine the source code for `net/sched/sch_generic.c` at line 1496, along with `usbnet_disconnect`, and closely inspect the surrounding functions to understand the exact sequence of events during device unregistration to pinpoint the root cause of this warning, and potential crash. I'll need to look at potential issues with resource release, locking, or synchronization issues within the network stack's interaction with the USB device. I suspect some interaction between the network device’s traffic control (tc) and the USB driver is leading to the crash.
|
| 7/2 |
2026/05/21 11:33 |
tool |
grepper |
0m
Args: map[Expression:dev_shutdown]
Results: map[Output:Documentation/PCI/pci-iov-howto.rst=109=Following piece of code illustrates the usage of the SR-IOV API.
--
Documentation/PCI/pci-iov-howto.rst-141-
Documentation/PCI/pci-iov-howto.rst:142: static void dev_shutdown(struct pci_dev *dev)
Documentation/PCI/pci-iov-howto.rst-143- {
--
Documentation/PCI/pci-iov-howto.rst-168- .driver.pm = &dev_pm_ops,
Documentation/PCI/pci-iov-howto.rst:169: .shutdown = dev_shutdown,
Documentation/PCI/pci-iov-howto.rst-170- .sriov_configure = dev_sriov_configure,
--
Documentation/translations/zh_CN/PCI/pci-iov-howto.rst=72=nr_virtfn'是要启用的VF的编号。
--
Documentation/translations/zh_CN/PCI/pci-iov-howto.rst-139-
Documentation/translations/zh_CN/PCI/pci-iov-howto.rst:140: static void dev_shutdown(struct pci_dev *dev)
Documentation/translations/zh_CN/PCI/pci-iov-howto.rst-141- {
--
Documentation/translations/zh_CN/PCI/pci-iov-howto.rst-166- .driver.pm = &dev_pm_ops
Documentation/translations/zh_CN/PCI/pci-iov-howto.rst:167: .shutdown = dev_shutdown,
Documentation/translations/zh_CN/PCI/pci-iov-howto.rst-168- .sriov_configure = dev_sriov_configure,
--
drivers/bluetooth/hci_qca.c=2510=static void qca_serdev_remove(struct serdev_device *serdev)
--
drivers/bluetooth/hci_qca.c-2532-
drivers/bluetooth/hci_qca.c:2533:static void qca_serdev_shutdown(struct device *dev)
drivers/bluetooth/hci_qca.c-2534-{
--
drivers/bluetooth/hci_qca.c=2756=static struct serdev_device_driver qca_serdev_driver = {
--
drivers/bluetooth/hci_qca.c-2762- .acpi_match_table = ACPI_PTR(qca_bluetooth_acpi_match),
drivers/bluetooth/hci_qca.c:2763: .shutdown = qca_serdev_shutdown,
drivers/bluetooth/hci_qca.c-2764- .pm = &qca_pm_ops,
--
drivers/crypto/hisilicon/hpre/hpre_main.c=1680=static struct pci_driver hpre_pci_driver = {
--
drivers/crypto/hisilicon/hpre/hpre_main.c-1687- .err_handler = &hpre_err_handler,
drivers/crypto/hisilicon/hpre/hpre_main.c:1688: .shutdown = hisi_qm_dev_shutdown,
drivers/crypto/hisilicon/hpre/hpre_main.c-1689- .driver.pm = &hpre_pm_ops,
--
drivers/crypto/hisilicon/qm.c=4867=static irqreturn_t qm_abnormal_irq(int irq, void *data)
--
drivers/crypto/hisilicon/qm.c-4882-/**
drivers/crypto/hisilicon/qm.c:4883: * hisi_qm_dev_shutdown() - Shutdown device.
drivers/crypto/hisilicon/qm.c-4884- * @pdev: The device will be shutdown.
--
drivers/crypto/hisilicon/qm.c-4887- */
drivers/crypto/hisilicon/qm.c:4888:void hisi_qm_dev_shutdown(struct pci_dev *pdev)
drivers/crypto/hisilicon/qm.c-4889-{
--
drivers/crypto/hisilicon/qm.c-4896-}
drivers/crypto/hisilicon/qm.c:4897:EXPORT_SYMBOL_GPL(hisi_qm_dev_shutdown);
drivers/crypto/hisilicon/qm.c-4898-
--
drivers/crypto/hisilicon/sec2/sec_main.c=1492=static struct pci_driver sec_pci_driver = {
--
drivers/crypto/hisilicon/sec2/sec_main.c-1499- hisi_qm_sriov_configure : NULL,
drivers/crypto/hisilicon/sec2/sec_main.c:1500: .shutdown = hisi_qm_dev_shutdown,
drivers/crypto/hisilicon/sec2/sec_main.c-1501- .driver.pm = &sec_pm_ops,
--
drivers/crypto/hisilicon/zip/zip_main.c=1607=static struct pci_driver hisi_zip_pci_driver = {
--
drivers/crypto/hisilicon/zip/zip_main.c-1614- .err_handler = &hisi_zip_err_handler,
drivers/crypto/hisilicon/zip/zip_main.c:1615: .shutdown = hisi_qm_dev_shutdown,
drivers/crypto/hisilicon/zip/zip_main.c-1616- .driver.pm = &hisi_zip_pm_ops,
--
drivers/crypto/intel/qat/qat_common/adf_init.c=280=static void adf_dev_stop(struct adf_accel_dev *accel_dev)
--
drivers/crypto/intel/qat/qat_common/adf_init.c-339-/**
drivers/crypto/intel/qat/qat_common/adf_init.c:340: * adf_dev_shutdown() - shutdown acceleration services and data strucutures
drivers/crypto/intel/qat/qat_common/adf_init.c-341- * @accel_dev: Pointer to acceleration device
--
drivers/crypto/intel/qat/qat_common/adf_init.c-345- */
drivers/crypto/intel/qat/qat_common/adf_init.c:346:static void adf_dev_shutdown(struct adf_accel_dev *accel_dev)
drivers/crypto/intel/qat/qat_common/adf_init.c-347-{
--
drivers/crypto/intel/qat/qat_common/adf_init.c=449=int adf_dev_down(struct adf_accel_dev *accel_dev)
--
drivers/crypto/intel/qat/qat_common/adf_init.c-458- adf_dev_stop(accel_dev);
drivers/crypto/intel/qat/qat_common/adf_init.c:459: adf_dev_shutdown(accel_dev);
drivers/crypto/intel/qat/qat_common/adf_init.c-460-
--
drivers/cxl/core/memdev.c=620=EXPORT_SYMBOL_NS_GPL(clear_exclusive_cxl_commands, "CXL");
drivers/cxl/core/memdev.c-621-
drivers/cxl/core/memdev.c:622:static void cxl_memdev_shutdown(struct device *dev)
drivers/cxl/core/memdev.c-623-{
--
drivers/cxl/core/memdev.c=630=static void cxl_memdev_unregister(void *_cxlmd)
--
drivers/cxl/core/memdev.c-635- cdev_device_del(&cxlmd->cdev, dev);
drivers/cxl/core/memdev.c:636: cxl_memdev_shutdown(dev);
drivers/cxl/core/memdev.c-637- put_device(dev);
--
drivers/cxl/core/memdev.c=1054=struct cxl_memdev *devm_cxl_add_memdev(struct device *host,
--
drivers/cxl/core/memdev.c-1092- */
drivers/cxl/core/memdev.c:1093: cxl_memdev_shutdown(dev);
drivers/cxl/core/memdev.c-1094- put_device(dev);
--
drivers/gpu/drm/omapdrm/omap_drv.c=814=static void pdev_remove(struct platform_device *pdev)
--
drivers/gpu/drm/omapdrm/omap_drv.c-821-
drivers/gpu/drm/omapdrm/omap_drv.c:822:static void pdev_shutdown(struct platform_device *pdev)
drivers/gpu/drm/omapdrm/omap_drv.c-823-{
--
drivers/gpu/drm/omapdrm/omap_drv.c=851=static struct platform_driver pdev = {
--
drivers/gpu/drm/omapdrm/omap_drv.c-857- .remove = pdev_remove,
drivers/gpu/drm/omapdrm/omap_drv.c:858: .shutdown = pdev_shutdown,
drivers/gpu/drm/omapdrm/omap_drv.c-859-};
--
drivers/gpu/drm/tilcdc/tilcdc_drv.c=564=static void tilcdc_pdev_remove(struct platform_device *pdev)
--
drivers/gpu/drm/tilcdc/tilcdc_drv.c-577-
drivers/gpu/drm/tilcdc/tilcdc_drv.c:578:static void tilcdc_pdev_shutdown(struct platform_device *pdev)
drivers/gpu/drm/tilcdc/tilcdc_drv.c-579-{
--
drivers/gpu/drm/tilcdc/tilcdc_drv.c=590=static struct platform_driver tilcdc_platform_driver = {
--
drivers/gpu/drm/tilcdc/tilcdc_drv.c-592- .remove = tilcdc_pdev_remove,
drivers/gpu/drm/tilcdc/tilcdc_drv.c:593: .shutdown = tilcdc_pdev_shutdown,
drivers/gpu/drm/tilcdc/tilcdc_drv.c-594- .driver = {
--
drivers/mmc/core/host.c=72=static void mmc_host_classdev_release(struct device *dev)
--
drivers/mmc/core/host.c-80-
drivers/mmc/core/host.c:81:static int mmc_host_classdev_shutdown(struct device *dev)
drivers/mmc/core/host.c-82-{
--
drivers/mmc/core/host.c=89=static const struct class mmc_host_class = {
--
drivers/mmc/core/host.c-91- .dev_release = mmc_host_classdev_release,
drivers/mmc/core/host.c:92: .shutdown_pre = mmc_host_classdev_shutdown,
drivers/mmc/core/host.c-93- .pm = MMC_HOST_CLASS_DEV_PM_OPS,
--
drivers/net/ethernet/emulex/benet/be.h=978=void be_roce_dev_remove(struct be_adapter *);
--
drivers/net/ethernet/emulex/benet/be.h-982- */
drivers/net/ethernet/emulex/benet/be.h:983:void be_roce_dev_shutdown(struct be_adapter *);
drivers/net/ethernet/emulex/benet/be.h-984-
--
drivers/net/ethernet/emulex/benet/be_main.c=5925=static void be_shutdown(struct pci_dev *pdev)
--
drivers/net/ethernet/emulex/benet/be_main.c-5931-
drivers/net/ethernet/emulex/benet/be_main.c:5932: be_roce_dev_shutdown(adapter);
drivers/net/ethernet/emulex/benet/be_main.c-5933- cancel_delayed_work_sync(&adapter->work);
--
drivers/net/ethernet/emulex/benet/be_roce.c=105=void be_roce_dev_remove(struct be_adapter *adapter)
--
drivers/net/ethernet/emulex/benet/be_roce.c-114-
drivers/net/ethernet/emulex/benet/be_roce.c:115:void be_roce_dev_shutdown(struct be_adapter *adapter)
drivers/net/ethernet/emulex/benet/be_roce.c-116-{
--
drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c=91=static void mlx5_sf_dev_remove(struct auxiliary_device *adev)
--
drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c-108-
drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c:109:static void mlx5_sf_dev_shutdown(struct auxiliary_device *adev)
drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c-110-{
--
drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c=126=static struct auxiliary_driver mlx5_sf_driver = {
--
drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c-129- .remove = mlx5_sf_dev_remove,
drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c:130: .shutdown = mlx5_sf_dev_shutdown,
drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c-131- .id_table = mlx5_sf_dev_id_table,
--
drivers/net/ethernet/microchip/lan743x_main.c=3724=static void lan743x_pcidev_remove(struct pci_dev *pdev)
--
drivers/net/ethernet/microchip/lan743x_main.c-3731-
drivers/net/ethernet/microchip/lan743x_main.c:3732:static void lan743x_pcidev_shutdown(struct pci_dev *pdev)
drivers/net/ethernet/microchip/lan743x_main.c-3733-{
--
drivers/net/ethernet/microchip/lan743x_main.c=3875=static int lan743x_pm_suspend(struct device *dev)
--
drivers/net/ethernet/microchip/lan743x_main.c-3881-
drivers/net/ethernet/microchip/lan743x_main.c:3882: lan743x_pcidev_shutdown(pdev);
drivers/net/ethernet/microchip/lan743x_main.c-3883-
--
drivers/net/ethernet/microchip/lan743x_main.c=3984=static struct pci_driver lan743x_pcidev_driver = {
--
drivers/net/ethernet/microchip/lan743x_main.c-3991-#endif
drivers/net/ethernet/microchip/lan743x_main.c:3992: .shutdown = lan743x_pcidev_shutdown,
drivers/net/ethernet/microchip/lan743x_main.c-3993-};
--
drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c=271=static void rnpgbe_remove(struct pci_dev *pdev)
--
drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c-278-/**
drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c:279: * rnpgbe_dev_shutdown - Device shutdown routine
drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c-280- * @pdev: PCI device information struct
drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c-281- **/
drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c:282:static void rnpgbe_dev_shutdown(struct pci_dev *pdev)
drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c-283-{
--
drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c=302=static void rnpgbe_shutdown(struct pci_dev *pdev)
drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c-303-{
drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c:304: rnpgbe_dev_shutdown(pdev);
drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c-305-}
--
drivers/net/ethernet/qlogic/qede/qede_rdma.c=162=static void qede_rdma_dev_close(struct qede_dev *edev)
--
drivers/net/ethernet/qlogic/qede/qede_rdma.c-171-
drivers/net/ethernet/qlogic/qede/qede_rdma.c:172:static void qede_rdma_dev_shutdown(struct qede_dev *edev)
drivers/net/ethernet/qlogic/qede/qede_rdma.c-173-{
--
drivers/net/ethernet/qlogic/qede/qede_rdma.c=274=static void qede_rdma_handle_event(struct work_struct *work)
--
drivers/net/ethernet/qlogic/qede/qede_rdma.c-291- case QEDE_CLOSE:
drivers/net/ethernet/qlogic/qede/qede_rdma.c:292: qede_rdma_dev_shutdown(edev);
drivers/net/ethernet/qlogic/qede/qede_rdma.c-293- break;
--
drivers/net/ethernet/wangxun/ngbe/ngbe_main.c=490=static int ngbe_close(struct net_device *netdev)
--
drivers/net/ethernet/wangxun/ngbe/ngbe_main.c-504-
drivers/net/ethernet/wangxun/ngbe/ngbe_main.c:505:static void ngbe_dev_shutdown(struct pci_dev *pdev, bool *enable_wake)
drivers/net/ethernet/wangxun/ngbe/ngbe_main.c-506-{
--
drivers/net/ethernet/wangxun/ngbe/ngbe_main.c=534=static void ngbe_shutdown(struct pci_dev *pdev)
--
drivers/net/ethernet/wangxun/ngbe/ngbe_main.c-540-
drivers/net/ethernet/wangxun/ngbe/ngbe_main.c:541: ngbe_dev_shutdown(pdev, &wake);
drivers/net/ethernet/wangxun/ngbe/ngbe_main.c-542-
--
drivers/net/ethernet/wangxun/ngbe/ngbe_main.c=834=static int ngbe_suspend(struct pci_dev *pdev, pm_message_t state)
--
drivers/net/ethernet/wangxun/ngbe/ngbe_main.c-837-
drivers/net/ethernet/wangxun/ngbe/ngbe_main.c:838: ngbe_dev_shutdown(pdev, &wake);
drivers/net/ethernet/wangxun/ngbe/ngbe_main.c-839- device_set_wakeup_enable(&pdev->dev, wake);
--
drivers/net/ethernet/wangxun/txgbe/txgbe_main.c=521=static int txgbe_close(struct net_device *netdev)
--
drivers/net/ethernet/wangxun/txgbe/txgbe_main.c-535-
drivers/net/ethernet/wangxun/txgbe/txgbe_main.c:536:static void txgbe_dev_shutdown(struct pci_dev *pdev)
drivers/net/ethernet/wangxun/txgbe/txgbe_main.c-537-{
--
drivers/net/ethernet/wangxun/txgbe/txgbe_main.c=554=static void txgbe_shutdown(struct pci_dev *pdev)
drivers/net/ethernet/wangxun/txgbe/txgbe_main.c-555-{
drivers/net/ethernet/wangxun/txgbe/txgbe_main.c:556: txgbe_dev_shutdown(pdev);
drivers/net/ethernet/wangxun/txgbe/txgbe_main.c-557-
--
drivers/platform/wmi/core.c=895=static void wmi_dev_remove(struct device *dev)
--
drivers/platform/wmi/core.c-907-
drivers/platform/wmi/core.c:908:static void wmi_dev_shutdown(struct device *dev)
drivers/platform/wmi/core.c-909-{
--
drivers/platform/wmi/core.c=942=static const struct bus_type wmi_bus_type = {
--
drivers/platform/wmi/core.c-948- .remove = wmi_dev_remove,
drivers/platform/wmi/core.c:949: .shutdown = wmi_dev_shutdown,
drivers/platform/wmi/core.c-950-};
--
drivers/virtio/virtio.c=393=static const struct cpumask *virtio_irq_get_affinity(struct device *_d,
--
drivers/virtio/virtio.c-403-
drivers/virtio/virtio.c:404:static void virtio_dev_shutdown(struct device *_d)
drivers/virtio/virtio.c-405-{
--
drivers/virtio/virtio.c=438=static const struct bus_type virtio_bus = {
--
drivers/virtio/virtio.c-445- .irq_get_affinity = virtio_irq_get_affinity,
drivers/virtio/virtio.c:446: .shutdown = virtio_dev_shutdown,
drivers/virtio/virtio.c-447-};
--
drivers/xen/xenbus/xenbus_probe_frontend.c=118=static int xenbus_frontend_dev_probe(struct device *dev)
--
drivers/xen/xenbus/xenbus_probe_frontend.c-127-
drivers/xen/xenbus/xenbus_probe_frontend.c:128:static void xenbus_frontend_dev_shutdown(struct device *_dev)
drivers/xen/xenbus/xenbus_probe_frontend.c-129-{
--
drivers/xen/xenbus/xenbus_probe_frontend.c=158=static struct xen_bus_type xenbus_frontend = {
--
drivers/xen/xenbus/xenbus_probe_frontend.c-169- .remove = xenbus_dev_remove,
drivers/xen/xenbus/xenbus_probe_frontend.c:170: .shutdown = xenbus_frontend_dev_shutdown,
drivers/xen/xenbus/xenbus_probe_frontend.c-171- .dev_groups = xenbus_dev_groups,
--
include/linux/hisi_acc_qm.h=579=void hisi_qm_free_qps(struct hisi_qp **qps, int qp_num);
include/linux/hisi_acc_qm.h:580:void hisi_qm_dev_shutdown(struct pci_dev *pdev);
include/linux/hisi_acc_qm.h-581-void hisi_qm_wait_task_finish(struct hisi_qm *qm, struct hisi_qm_list *qm_list);
--
include/net/sch_generic.h=707=void dev_init_scheduler(struct net_device *dev);
include/net/sch_generic.h:708:void dev_shutdown(struct net_device *dev);
include/net/sch_generic.h-709-void dev_activate(struct net_device *dev);
--
net/bluetooth/hci_sync.c=5211=static void hci_pend_le_actions_clear(struct hci_dev *hdev)
--
net/bluetooth/hci_sync.c-5226-
net/bluetooth/hci_sync.c:5227:static int hci_dev_shutdown(struct hci_dev *hdev)
net/bluetooth/hci_sync.c-5228-{
--
net/bluetooth/hci_sync.c=5252=int hci_dev_close_sync(struct hci_dev *hdev)
--
net/bluetooth/hci_sync.c-5277-
net/bluetooth/hci_sync.c:5278: err = hci_dev_shutdown(hdev);
net/bluetooth/hci_sync.c-5279-
--
net/bluetooth/rfcomm/tty.c=122=static bool rfcomm_dev_carrier_raised(struct tty_port *port)
--
net/bluetooth/rfcomm/tty.c-129-/* device-specific cleanup: close the dlc */
net/bluetooth/rfcomm/tty.c:130:static void rfcomm_dev_shutdown(struct tty_port *port)
net/bluetooth/rfcomm/tty.c-131-{
--
net/bluetooth/rfcomm/tty.c=141=static const struct tty_port_operations rfcomm_port_ops = {
--
net/bluetooth/rfcomm/tty.c-143- .activate = rfcomm_dev_activate,
net/bluetooth/rfcomm/tty.c:144: .shutdown = rfcomm_dev_shutdown,
net/bluetooth/rfcomm/tty.c-145- .carrier_raised = rfcomm_dev_carrier_raised,
--
net/core/dev.c=12305=void unregister_netdevice_many_notify(struct list_head *head,
--
net/core/dev.c-12368- netdev_lock_ops(dev);
net/core/dev.c:12369: dev_shutdown(dev);
net/core/dev.c-12370- dev_tcx_uninstall(dev);
--
net/core/dev.c=12467=int __dev_change_net_namespace(struct net_device *dev, struct net *net,
--
net/core/dev.c-12566- netdev_lock_ops(dev);
net/core/dev.c:12567: dev_shutdown(dev);
net/core/dev.c-12568- netdev_unlock_ops(dev);
--
net/rds/ib.c=73=static void rds_ib_nodev_connect(void)
--
net/rds/ib.c-82-
net/rds/ib.c:83:static void rds_ib_dev_shutdown(struct rds_ib_device *rds_ibdev)
net/rds/ib.c-84-{
--
net/rds/ib.c=265=static void rds_ib_remove_one(struct ib_device *device, void *client_data)
--
net/rds/ib.c-268-
net/rds/ib.c:269: rds_ib_dev_shutdown(rds_ibdev);
net/rds/ib.c-270-
--
net/sched/sch_generic.c=1478=void dev_init_scheduler(struct net_device *dev)
--
net/sched/sch_generic.c-1487-
net/sched/sch_generic.c:1488:void dev_shutdown(struct net_device *dev)
net/sched/sch_generic.c-1489-{
]
|
| 8/2 |
2026/05/21 11:33 |
llm |
expert |
0m
Model:
Tokens:
input: 24223
output: 24
thoughts: 26
Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience:
**Analyzing the `dev_shutdown` Warning**
Alright, so the heads-up points directly to `net/sched/sch_generic.c:1496` within the `dev_shutdown` function. My initial thought is, "Ah, right, we're dealing with the generic scheduling layer's device shutdown procedure." This is where things can get interesting, and potentially a bit dicey, when a network device is brought down. I immediately start mentally running through the possible scenarios: race conditions with active queues, potential resource leaks if cleanup isn't done correctly, interactions with various traffic control disciplines... this is potentially a critical section in terms of network stability and preventing data corruption. I need to understand what's *specifically* being flagged at that line. What's the context? What's the warning message itself saying? Is this a known issue, or something new? Gotta dive into that code and trace the execution path. Let's see what the current patch set looks like.
|
| 9/2 |
2026/05/21 11:34 |
tool |
read-file |
0m
Args: map[File:net/sched/sch_generic.c]
Results: map[Contents:// SPDX-License-Identifier: GPL-2.0-or-later
/*
* net/sched/sch_generic.c Generic packet scheduler routines.
*
* Authors: Alexey Kuznetsov, <kuznet@ms2.inr.ac.ru>
* Jamal Hadi Salim, <hadi@cyberus.ca> 990601
* - Ingress support
*/
#include <linux/bitops.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
#include <linux/rtnetlink.h>
#include <linux/init.h>
#include <linux/rcupdate.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/if_vlan.h>
#include <linux/skb_array.h>
#include <linux/if_macvlan.h>
#include <linux/bpf.h>
#include <net/sch_generic.h>
#include <net/pkt_sched.h>
#include <net/dst.h>
#include <net/hotdata.h>
#include <trace/events/qdisc.h>
#include <trace/events/net.h>
#include <net/xfrm.h>
/* Qdisc to use by default */
const struct Qdisc_ops *default_qdisc_ops = &pfifo_fast_ops;
EXPORT_SYMBOL(default_qdisc_ops);
static void qdisc_maybe_clear_missed(struct Qdisc *q,
const struct netdev_queue *txq)
{
clear_bit(__QDISC_STATE_MISSED, &q->state);
/* Make sure the below netif_xmit_frozen_or_stopped()
* checking happens after clearing STATE_MISSED.
*/
smp_mb__after_atomic();
/* Checking netif_xmit_frozen_or_stopped() again to
* make sure STATE_MISSED is set if the STATE_MISSED
* set by netif_tx_wake_queue()'s rescheduling of
* net_tx_action() is cleared by the above clear_bit().
*/
if (!netif_xmit_frozen_or_stopped(txq))
set_bit(__QDISC_STATE_MISSED, &q->state);
else
set_bit(__QDISC_STATE_DRAINING, &q->state);
}
/* Main transmission queue. */
/* Modifications to data participating in scheduling must be protected with
* qdisc_lock(qdisc) spinlock.
*
* The idea is the following:
* - enqueue, dequeue are serialized via qdisc root lock
* - ingress filtering is also serialized via qdisc root lock
* - updates to tree and tree walking are only done under the rtnl mutex.
*/
#define SKB_XOFF_MAGIC ((struct sk_buff *)1UL)
static inline struct sk_buff *__skb_dequeue_bad_txq(struct Qdisc *q)
{
const struct netdev_queue *txq = q->dev_queue;
spinlock_t *lock = NULL;
struct sk_buff *skb;
if (q->flags & TCQ_F_NOLOCK) {
lock = qdisc_lock(q);
spin_lock(lock);
}
skb = skb_peek(&q->skb_bad_txq);
if (skb) {
/* check the reason of requeuing without tx lock first */
txq = skb_get_tx_queue(txq->dev, skb);
if (!netif_xmit_frozen_or_stopped(txq)) {
skb = __skb_dequeue(&q->skb_bad_txq);
if (qdisc_is_percpu_stats(q)) {
qdisc_qstats_cpu_backlog_dec(q, skb);
qdisc_qstats_cpu_qlen_dec(q);
} else {
qdisc_qstats_backlog_dec(q, skb);
q->q.qlen--;
}
} else {
skb = SKB_XOFF_MAGIC;
qdisc_maybe_clear_missed(q, txq);
}
}
if (lock)
spin_unlock(lock);
return skb;
}
static inline struct sk_buff *qdisc_dequeue_skb_bad_txq(struct Qdisc *q)
{
struct sk_buff *skb = skb_peek(&q->skb_bad_txq);
if (unlikely(skb))
skb = __skb_dequeue_bad_txq(q);
return skb;
}
static inline void qdisc_enqueue_skb_bad_txq(struct Qdisc *q,
struct sk_buff *skb)
{
spinlock_t *lock = NULL;
if (q->flags & TCQ_F_NOLOCK) {
lock = qdisc_lock(q);
spin_lock(lock);
}
__skb_queue_tail(&q->skb_bad_txq, skb);
if (qdisc_is_percpu_stats(q)) {
qdisc_qstats_cpu_backlog_inc(q, skb);
qdisc_qstats_cpu_qlen_inc(q);
} else {
qdisc_qstats_backlog_inc(q, skb);
q->q.qlen++;
}
if (lock)
spin_unlock(lock);
}
static inline void dev_requeue_skb(struct sk_buff *skb, struct Qdisc *q)
{
spinlock_t *lock = NULL;
if (q->flags & TCQ_F_NOLOCK) {
lock = qdisc_lock(q);
spin_lock(lock);
}
while (skb) {
struct sk_buff *next = skb->next;
__skb_queue_tail(&q->gso_skb, skb);
/* it's still part of the queue */
if (qdisc_is_percpu_stats(q)) {
qdisc_qstats_cpu_requeues_inc(q);
qdisc_qstats_cpu_backlog_inc(q, skb);
qdisc_qstats_cpu_qlen_inc(q);
} else {
q->qstats.requeues++;
qdisc_qstats_backlog_inc(q, skb);
q->q.qlen++;
}
skb = next;
}
if (lock) {
spin_unlock(lock);
set_bit(__QDISC_STATE_MISSED, &q->state);
} else {
__netif_schedule(q);
}
}
static void try_bulk_dequeue_skb(struct Qdisc *q,
struct sk_buff *skb,
const struct netdev_queue *txq,
int *packets, int budget)
{
int bytelimit = qdisc_avail_bulklimit(txq) - skb->len;
int cnt = 0;
while (bytelimit > 0) {
struct sk_buff *nskb = q->dequeue(q);
if (!nskb)
break;
bytelimit -= nskb->len; /* covers GSO len */
skb->next = nskb;
skb = nskb;
if (++cnt >= budget)
break;
}
(*packets) += cnt;
skb_mark_not_on_list(skb);
}
/* This variant of try_bulk_dequeue_skb() makes sure
* all skbs in the chain are for the same txq
*/
static void try_bulk_dequeue_skb_slow(struct Qdisc *q,
struct sk_buff *skb,
int *packets)
{
int mapping = skb_get_queue_mapping(skb);
struct sk_buff *nskb;
int cnt = 0;
do {
nskb = q->dequeue(q);
if (!nskb)
break;
if (unlikely(skb_get_queue_mapping(nskb) != mapping)) {
qdisc_enqueue_skb_bad_txq(q, nskb);
break;
}
skb->next = nskb;
skb = nskb;
} while (++cnt < 8);
(*packets) += cnt;
skb_mark_not_on_list(skb);
}
/* Note that dequeue_skb can possibly return a SKB list (via skb->next).
* A requeued skb (via q->gso_skb) can also be a SKB list.
*/
static struct sk_buff *dequeue_skb(struct Qdisc *q, bool *validate,
int *packets, int budget)
{
const struct netdev_queue *txq = q->dev_queue;
struct sk_buff *skb = NULL;
*packets = 1;
if (unlikely(!skb_queue_empty(&q->gso_skb))) {
spinlock_t *lock = NULL;
if (q->flags & TCQ_F_NOLOCK) {
lock = qdisc_lock(q);
spin_lock(lock);
}
skb = skb_peek(&q->gso_skb);
/* skb may be null if another cpu pulls gso_skb off in between
* empty check and lock.
*/
if (!skb) {
if (lock)
spin_unlock(lock);
goto validate;
}
/* skb in gso_skb were already validated */
*validate = false;
if (xfrm_offload(skb))
*validate = true;
/* check the reason of requeuing without tx lock first */
txq = skb_get_tx_queue(txq->dev, skb);
if (!netif_xmit_frozen_or_stopped(txq)) {
skb = __skb_dequeue(&q->gso_skb);
if (qdisc_is_percpu_stats(q)) {
qdisc_qstats_cpu_backlog_dec(q, skb);
qdisc_qstats_cpu_qlen_dec(q);
} else {
qdisc_qstats_backlog_dec(q, skb);
q->q.qlen--;
}
} else {
skb = NULL;
qdisc_maybe_clear_missed(q, txq);
}
if (lock)
spin_unlock(lock);
goto trace;
}
validate:
*validate = true;
if ((q->flags & TCQ_F_ONETXQUEUE) &&
netif_xmit_frozen_or_stopped(txq)) {
qdisc_maybe_clear_missed(q, txq);
return skb;
}
skb = qdisc_dequeue_skb_bad_txq(q);
if (unlikely(skb)) {
if (skb == SKB_XOFF_MAGIC)
return NULL;
goto bulk;
}
skb = q->dequeue(q);
if (skb) {
bulk:
if (qdisc_may_bulk(q))
try_bulk_dequeue_skb(q, skb, txq, packets, budget);
else
try_bulk_dequeue_skb_slow(q, skb, packets);
}
trace:
trace_qdisc_dequeue(q, txq, *packets, skb);
return skb;
}
/*
* Transmit possibly several skbs, and handle the return status as
* required. Owning qdisc running bit guarantees that only one CPU
* can execute this function.
*
* Returns to the caller:
* false - hardware queue frozen backoff
* true - feel free to send more pkts
*/
bool sch_direct_xmit(struct sk_buff *skb, struct Qdisc *q,
struct net_device *dev, struct netdev_queue *txq,
spinlock_t *root_lock, bool validate)
{
int ret = NETDEV_TX_BUSY;
bool again = false;
/* And release qdisc */
if (root_lock)
spin_unlock(root_lock);
/* Note that we validate skb (GSO, checksum, ...) outside of locks */
if (validate)
skb = validate_xmit_skb_list(skb, dev, &again);
#ifdef CONFIG_XFRM_OFFLOAD
if (unlikely(again)) {
if (root_lock)
spin_lock(root_lock);
dev_requeue_skb(skb, q);
return false;
}
#endif
if (likely(skb)) {
HARD_TX_LOCK(dev, txq, smp_processor_id());
if (!netif_xmit_frozen_or_stopped(txq))
skb = dev_hard_start_xmit(skb, dev, txq, &ret);
else
qdisc_maybe_clear_missed(q, txq);
HARD_TX_UNLOCK(dev, txq);
} else {
if (root_lock)
spin_lock(root_lock);
return true;
}
if (root_lock)
spin_lock(root_lock);
if (!dev_xmit_complete(ret)) {
/* Driver returned NETDEV_TX_BUSY - requeue skb */
if (unlikely(ret != NETDEV_TX_BUSY))
net_warn_ratelimited("BUG %s code %d qlen %d\n",
dev->name, ret, q->q.qlen);
dev_requeue_skb(skb, q);
return false;
}
return true;
}
/*
* NOTE: Called under qdisc_lock(q) with locally disabled BH.
*
* running seqcount guarantees only one CPU can process
* this qdisc at a time. qdisc_lock(q) serializes queue accesses for
* this queue.
*
* netif_tx_lock serializes accesses to device driver.
*
* qdisc_lock(q) and netif_tx_lock are mutually exclusive,
* if one is grabbed, another must be free.
*
* Note, that this procedure can be called by a watchdog timer
*
* Returns to the caller:
* 0 - queue is empty or throttled.
* >0 - queue is not empty.
*
*/
static inline bool qdisc_restart(struct Qdisc *q, int *packets, int budget)
{
spinlock_t *root_lock = NULL;
struct netdev_queue *txq;
struct net_device *dev;
struct sk_buff *skb;
bool validate;
/* Dequeue packet */
skb = dequeue_skb(q, &validate, packets, budget);
if (unlikely(!skb))
return false;
if (!(q->flags & TCQ_F_NOLOCK))
root_lock = qdisc_lock(q);
dev = qdisc_dev(q);
txq = skb_get_tx_queue(dev, skb);
return sch_direct_xmit(skb, q, dev, txq, root_lock, validate);
}
void __qdisc_run(struct Qdisc *q)
{
int quota = READ_ONCE(net_hotdata.dev_tx_weight);
int packets;
while (qdisc_restart(q, &packets, quota)) {
quota -= packets;
if (quota <= 0) {
if (q->flags & TCQ_F_NOLOCK)
set_bit(__QDISC_STATE_MISSED, &q->state);
else
__netif_schedule(q);
break;
}
}
}
unsigned long dev_trans_start(struct net_device *dev)
{
unsigned long res = READ_ONCE(netdev_get_tx_queue(dev, 0)->trans_start);
unsigned long val;
unsigned int i;
for (i = 1; i < dev->num_tx_queues; i++) {
val = READ_ONCE(netdev_get_tx_queue(dev, i)->trans_start);
if (val && time_after(val, res))
res = val;
}
return res;
}
EXPORT_SYMBOL(dev_trans_start);
static void netif_freeze_queues(struct net_device *dev)
{
unsigned int i;
int cpu;
cpu = smp_processor_id();
for (i = 0; i < dev->num_tx_queues; i++) {
struct netdev_queue *txq = netdev_get_tx_queue(dev, i);
/* We are the only thread of execution doing a
* freeze, but we have to grab the _xmit_lock in
* order to synchronize with threads which are in
* the ->hard_start_xmit() handler and already
* checked the frozen bit.
*/
__netif_tx_lock(txq, cpu);
set_bit(__QUEUE_STATE_FROZEN, &txq->state);
__netif_tx_unlock(txq);
}
}
void netif_tx_lock(struct net_device *dev)
{
spin_lock(&dev->tx_global_lock);
netif_freeze_queues(dev);
}
EXPORT_SYMBOL(netif_tx_lock);
static void netif_unfreeze_queues(struct net_device *dev)
{
unsigned int i;
for (i = 0; i < dev->num_tx_queues; i++) {
struct netdev_queue *txq = netdev_get_tx_queue(dev, i);
/* No need to grab the _xmit_lock here. If the
* queue is not stopped for another reason, we
* force a schedule.
*/
clear_bit(__QUEUE_STATE_FROZEN, &txq->state);
netif_schedule_queue(txq);
}
}
void netif_tx_unlock(struct net_device *dev)
{
netif_unfreeze_queues(dev);
spin_unlock(&dev->tx_global_lock);
}
EXPORT_SYMBOL(netif_tx_unlock);
static void dev_watchdog(struct timer_list *t)
{
struct net_device *dev = timer_container_of(dev, t, watchdog_timer);
bool release = true;
spin_lock(&dev->tx_global_lock);
if (!qdisc_tx_is_noop(dev)) {
if (netif_device_present(dev) &&
netif_running(dev) &&
netif_carrier_ok(dev)) {
unsigned int timedout_ms = 0;
unsigned int i;
unsigned long trans_start;
unsigned long oldest_start = jiffies;
for (i = 0; i < dev->num_tx_queues; i++) {
struct netdev_queue *txq;
txq = netdev_get_tx_queue(dev, i);
if (!netif_xmit_stopped(txq))
continue;
/* Paired with WRITE_ONCE() + smp_mb...() in
* netdev_tx_sent_queue() and netif_tx_stop_queue().
*/
smp_mb();
trans_start = READ_ONCE(txq->trans_start);
if (time_after(jiffies, trans_start + dev->watchdog_timeo)) {
timedout_ms = jiffies_to_msecs(jiffies - trans_start);
atomic_long_inc(&txq->trans_timeout);
break;
}
if (time_after(oldest_start, trans_start))
oldest_start = trans_start;
}
if (unlikely(timedout_ms)) {
trace_net_dev_xmit_timeout(dev, i);
netdev_crit(dev, "NETDEV WATCHDOG: CPU: %d: transmit queue %u timed out %u ms\n",
raw_smp_processor_id(),
i, timedout_ms);
netif_freeze_queues(dev);
dev->netdev_ops->ndo_tx_timeout(dev, i);
netif_unfreeze_queues(dev);
}
if (!mod_timer(&dev->watchdog_timer,
round_jiffies(oldest_start +
dev->watchdog_timeo)))
release = false;
}
}
spin_unlock(&dev->tx_global_lock);
if (release)
netdev_put(dev, &dev->watchdog_dev_tracker);
}
void netdev_watchdog_up(struct net_device *dev)
{
if (!dev->netdev_ops->ndo_tx_timeout)
return;
if (dev->watchdog_timeo <= 0)
dev->watchdog_timeo = 5*HZ;
if (!mod_timer(&dev->watchdog_timer,
round_jiffies(jiffies + dev->watchdog_timeo)))
netdev_hold(dev, &dev->watchdog_dev_tracker,
GFP_ATOMIC);
}
EXPORT_SYMBOL_GPL(netdev_watchdog_up);
static void netdev_watchdog_down(struct net_device *dev)
{
netif_tx_lock_bh(dev);
if (timer_delete(&dev->watchdog_timer))
netdev_put(dev, &dev->watchdog_dev_tracker);
netif_tx_unlock_bh(dev);
}
/**
* netif_carrier_on - set carrier
* @dev: network device
*
* Device has detected acquisition of carrier.
*/
void netif_carrier_on(struct net_device *dev)
{
if (test_and_clear_bit(__LINK_STATE_NOCARRIER, &dev->state)) {
if (dev->reg_state == NETREG_UNINITIALIZED)
return;
atomic_inc(&dev->carrier_up_count);
linkwatch_fire_event(dev);
if (netif_running(dev))
netdev_watchdog_up(dev);
}
}
EXPORT_SYMBOL(netif_carrier_on);
/**
* netif_carrier_off - clear carrier
* @dev: network device
*
* Device has detected loss of carrier.
*/
void netif_carrier_off(struct net_device *dev)
{
if (!test_and_set_bit(__LINK_STATE_NOCARRIER, &dev->state)) {
if (dev->reg_state == NETREG_UNINITIALIZED)
return;
atomic_inc(&dev->carrier_down_count);
linkwatch_fire_event(dev);
}
}
EXPORT_SYMBOL(netif_carrier_off);
/**
* netif_carrier_event - report carrier state event
* @dev: network device
*
* Device has detected a carrier event but the carrier state wasn't changed.
* Use in drivers when querying carrier state asynchronously, to avoid missing
* events (link flaps) if link recovers before it's queried.
*/
void netif_carrier_event(struct net_device *dev)
{
if (dev->reg_state == NETREG_UNINITIALIZED)
return;
atomic_inc(&dev->carrier_up_count);
atomic_inc(&dev->carrier_down_count);
linkwatch_fire_event(dev);
}
EXPORT_SYMBOL_GPL(netif_carrier_event);
/* "NOOP" scheduler: the best scheduler, recommended for all interfaces
under all circumstances. It is difficult to invent anything faster or
cheaper.
*/
static int noop_enqueue(struct sk_buff *skb, struct Qdisc *qdisc,
struct sk_buff **to_free)
{
dev_core_stats_tx_dropped_inc(skb->dev);
__qdisc_drop(skb, to_free);
return NET_XMIT_CN;
}
static struct sk_buff *noop_dequeue(struct Qdisc *qdisc)
{
return NULL;
}
struct Qdisc_ops noop_qdisc_ops __read_mostly = {
.id = "noop",
.priv_size = 0,
.enqueue = noop_enqueue,
.dequeue = noop_dequeue,
.peek = noop_dequeue,
.owner = THIS_MODULE,
};
static struct netdev_queue noop_netdev_queue = {
RCU_POINTER_INITIALIZER(qdisc, &noop_qdisc),
RCU_POINTER_INITIALIZER(qdisc_sleeping, &noop_qdisc),
};
struct Qdisc noop_qdisc = {
.enqueue = noop_enqueue,
.dequeue = noop_dequeue,
.flags = TCQ_F_BUILTIN,
.ops = &noop_qdisc_ops,
.q.lock = __SPIN_LOCK_UNLOCKED(noop_qdisc.q.lock),
.dev_queue = &noop_netdev_queue,
.gso_skb = {
.next = (struct sk_buff *)&noop_qdisc.gso_skb,
.prev = (struct sk_buff *)&noop_qdisc.gso_skb,
.qlen = 0,
.lock = __SPIN_LOCK_UNLOCKED(noop_qdisc.gso_skb.lock),
},
.skb_bad_txq = {
.next = (struct sk_buff *)&noop_qdisc.skb_bad_txq,
.prev = (struct sk_buff *)&noop_qdisc.skb_bad_txq,
.qlen = 0,
.lock = __SPIN_LOCK_UNLOCKED(noop_qdisc.skb_bad_txq.lock),
},
};
EXPORT_SYMBOL(noop_qdisc);
static int noqueue_init(struct Qdisc *qdisc, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
/* register_qdisc() assigns a default of noop_enqueue if unset,
* but __dev_queue_xmit() treats noqueue only as such
* if this is NULL - so clear it here. */
qdisc->enqueue = NULL;
return 0;
}
struct Qdisc_ops noqueue_qdisc_ops __read_mostly = {
.id = "noqueue",
.priv_size = 0,
.init = noqueue_init,
.enqueue = noop_enqueue,
.dequeue = noop_dequeue,
.peek = noop_dequeue,
.owner = THIS_MODULE,
};
const u8 sch_default_prio2band[TC_PRIO_MAX + 1] = {
1, 2, 2, 2, 1, 2, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1
};
EXPORT_SYMBOL(sch_default_prio2band);
/* 3-band FIFO queue: old style, but should be a bit faster than
generic prio+fifo combination.
*/
#define PFIFO_FAST_BANDS 3
/*
* Private data for a pfifo_fast scheduler containing:
* - rings for priority bands
*/
struct pfifo_fast_priv {
struct skb_array q[PFIFO_FAST_BANDS];
};
static inline struct skb_array *band2list(struct pfifo_fast_priv *priv,
int band)
{
return &priv->q[band];
}
static int pfifo_fast_enqueue(struct sk_buff *skb, struct Qdisc *qdisc,
struct sk_buff **to_free)
{
int band = sch_default_prio2band[skb->priority & TC_PRIO_MAX];
struct pfifo_fast_priv *priv = qdisc_priv(qdisc);
struct skb_array *q = band2list(priv, band);
unsigned int pkt_len = qdisc_pkt_len(skb);
int err;
err = skb_array_produce(q, skb);
if (unlikely(err)) {
tcf_set_drop_reason(skb, SKB_DROP_REASON_QDISC_OVERLIMIT);
if (qdisc_is_percpu_stats(qdisc))
return qdisc_drop_cpu(skb, qdisc, to_free);
else
return qdisc_drop(skb, qdisc, to_free);
}
qdisc_update_stats_at_enqueue(qdisc, pkt_len);
return NET_XMIT_SUCCESS;
}
static struct sk_buff *pfifo_fast_dequeue(struct Qdisc *qdisc)
{
struct pfifo_fast_priv *priv = qdisc_priv(qdisc);
struct sk_buff *skb = NULL;
bool need_retry = true;
int band;
retry:
for (band = 0; band < PFIFO_FAST_BANDS && !skb; band++) {
struct skb_array *q = band2list(priv, band);
if (__skb_array_empty(q))
continue;
skb = __skb_array_consume(q);
}
if (likely(skb)) {
qdisc_update_stats_at_dequeue(qdisc, skb);
} else if (need_retry &&
READ_ONCE(qdisc->state) & QDISC_STATE_NON_EMPTY) {
/* Delay clearing the STATE_MISSED here to reduce
* the overhead of the second spin_trylock() in
* qdisc_run_begin() and __netif_schedule() calling
* in qdisc_run_end().
*/
clear_bit(__QDISC_STATE_MISSED, &qdisc->state);
clear_bit(__QDISC_STATE_DRAINING, &qdisc->state);
/* Make sure dequeuing happens after clearing
* STATE_MISSED.
*/
smp_mb__after_atomic();
need_retry = false;
goto retry;
}
return skb;
}
static struct sk_buff *pfifo_fast_peek(struct Qdisc *qdisc)
{
struct pfifo_fast_priv *priv = qdisc_priv(qdisc);
struct sk_buff *skb = NULL;
int band;
for (band = 0; band < PFIFO_FAST_BANDS && !skb; band++) {
struct skb_array *q = band2list(priv, band);
skb = __skb_array_peek(q);
}
return skb;
}
static void pfifo_fast_reset(struct Qdisc *qdisc)
{
int i, band;
struct pfifo_fast_priv *priv = qdisc_priv(qdisc);
for (band = 0; band < PFIFO_FAST_BANDS; band++) {
struct skb_array *q = band2list(priv, band);
struct sk_buff *skb;
/* NULL ring is possible if destroy path is due to a failed
* skb_array_init() in pfifo_fast_init() case.
*/
if (!q->ring.queue)
continue;
while ((skb = __skb_array_consume(q)) != NULL)
kfree_skb(skb);
}
if (qdisc_is_percpu_stats(qdisc)) {
for_each_possible_cpu(i) {
struct gnet_stats_queue *q;
q = per_cpu_ptr(qdisc->cpu_qstats, i);
q->backlog = 0;
q->qlen = 0;
}
}
}
static int pfifo_fast_dump(struct Qdisc *qdisc, struct sk_buff *skb)
{
struct tc_prio_qopt opt = { .bands = PFIFO_FAST_BANDS };
memcpy(&opt.priomap, sch_default_prio2band, TC_PRIO_MAX + 1);
if (nla_put(skb, TCA_OPTIONS, sizeof(opt), &opt))
goto nla_put_failure;
return skb->len;
nla_put_failure:
return -1;
}
static int pfifo_fast_init(struct Qdisc *qdisc, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
unsigned int qlen = qdisc_dev(qdisc)->tx_queue_len;
struct pfifo_fast_priv *priv = qdisc_priv(qdisc);
int prio;
/* guard against zero length rings */
if (!qlen)
return -EINVAL;
for (prio = 0; prio < PFIFO_FAST_BANDS; prio++) {
struct skb_array *q = band2list(priv, prio);
int err;
err = skb_array_init(q, qlen, GFP_KERNEL);
if (err)
return -ENOMEM;
}
/* Can by-pass the queue discipline */
qdisc->flags |= TCQ_F_CAN_BYPASS;
return 0;
}
static void pfifo_fast_destroy(struct Qdisc *sch)
{
struct pfifo_fast_priv *priv = qdisc_priv(sch);
int prio;
for (prio = 0; prio < PFIFO_FAST_BANDS; prio++) {
struct skb_array *q = band2list(priv, prio);
/* NULL ring is possible if destroy path is due to a failed
* skb_array_init() in pfifo_fast_init() case.
*/
if (!q->ring.queue)
continue;
/* Destroy ring but no need to kfree_skb because a call to
* pfifo_fast_reset() has already done that work.
*/
ptr_ring_cleanup(&q->ring, NULL);
}
}
static int pfifo_fast_change_tx_queue_len(struct Qdisc *sch,
unsigned int new_len)
{
struct pfifo_fast_priv *priv = qdisc_priv(sch);
struct skb_array *bands[PFIFO_FAST_BANDS];
int prio;
for (prio = 0; prio < PFIFO_FAST_BANDS; prio++) {
struct skb_array *q = band2list(priv, prio);
bands[prio] = q;
}
return skb_array_resize_multiple_bh(bands, PFIFO_FAST_BANDS, new_len,
GFP_KERNEL);
}
struct Qdisc_ops pfifo_fast_ops __read_mostly = {
.id = "pfifo_fast",
.priv_size = sizeof(struct pfifo_fast_priv),
.enqueue = pfifo_fast_enqueue,
.dequeue = pfifo_fast_dequeue,
.peek = pfifo_fast_peek,
.init = pfifo_fast_init,
.destroy = pfifo_fast_destroy,
.reset = pfifo_fast_reset,
.dump = pfifo_fast_dump,
.change_tx_queue_len = pfifo_fast_change_tx_queue_len,
.owner = THIS_MODULE,
.static_flags = TCQ_F_NOLOCK | TCQ_F_CPUSTATS,
};
EXPORT_SYMBOL(pfifo_fast_ops);
static struct lock_class_key qdisc_tx_busylock;
struct Qdisc *qdisc_alloc(struct netdev_queue *dev_queue,
const struct Qdisc_ops *ops,
struct netlink_ext_ack *extack)
{
struct Qdisc *sch;
unsigned int size = sizeof(*sch) + ops->priv_size;
int err = -ENOBUFS;
struct net_device *dev;
if (!dev_queue) {
NL_SET_ERR_MSG(extack, "No device queue given");
err = -EINVAL;
goto errout;
}
dev = dev_queue->dev;
sch = kzalloc_node(size, GFP_KERNEL, netdev_queue_numa_node_read(dev_queue));
if (!sch)
goto errout;
__skb_queue_head_init(&sch->gso_skb);
__skb_queue_head_init(&sch->skb_bad_txq);
gnet_stats_basic_sync_init(&sch->bstats);
lockdep_register_key(&sch->root_lock_key);
spin_lock_init(&sch->q.lock);
lockdep_set_class(&sch->q.lock, &sch->root_lock_key);
if (ops->static_flags & TCQ_F_CPUSTATS) {
sch->cpu_bstats =
netdev_alloc_pcpu_stats(struct gnet_stats_basic_sync);
if (!sch->cpu_bstats)
goto errout1;
sch->cpu_qstats = alloc_percpu(struct gnet_stats_queue);
if (!sch->cpu_qstats) {
free_percpu(sch->cpu_bstats);
goto errout1;
}
}
/* seqlock has the same scope of busylock, for NOLOCK qdisc */
spin_lock_init(&sch->seqlock);
lockdep_set_class(&sch->seqlock,
dev->qdisc_tx_busylock ?: &qdisc_tx_busylock);
sch->ops = ops;
sch->flags = ops->static_flags;
sch->enqueue = ops->enqueue;
sch->dequeue = ops->dequeue;
sch->dev_queue = dev_queue;
netdev_hold(dev, &sch->dev_tracker, GFP_KERNEL);
refcount_set(&sch->refcnt, 1);
return sch;
errout1:
lockdep_unregister_key(&sch->root_lock_key);
kfree(sch);
errout:
return ERR_PTR(err);
}
struct Qdisc *qdisc_create_dflt(struct netdev_queue *dev_queue,
const struct Qdisc_ops *ops,
unsigned int parentid,
struct netlink_ext_ack *extack)
{
struct Qdisc *sch;
if (!bpf_try_module_get(ops, ops->owner)) {
NL_SET_ERR_MSG(extack, "Failed to increase module reference counter");
return NULL;
}
sch = qdisc_alloc(dev_queue, ops, extack);
if (IS_ERR(sch)) {
bpf_module_put(ops, ops->owner);
return NULL;
}
sch->parent = parentid;
if (!ops->init || ops->init(sch, NULL, extack) == 0) {
trace_qdisc_create(ops, dev_queue->dev, parentid);
return sch;
}
qdisc_put(sch);
return NULL;
}
EXPORT_SYMBOL(qdisc_create_dflt);
/* Under qdisc_lock(qdisc) and BH! */
void qdisc_reset(struct Qdisc *qdisc)
{
const struct Qdisc_ops *ops = qdisc->ops;
trace_qdisc_reset(qdisc);
if (ops->reset)
ops->reset(qdisc);
__skb_queue_purge(&qdisc->gso_skb);
__skb_queue_purge(&qdisc->skb_bad_txq);
qdisc->q.qlen = 0;
qdisc->qstats.backlog = 0;
}
EXPORT_SYMBOL(qdisc_reset);
void qdisc_free(struct Qdisc *qdisc)
{
if (qdisc_is_percpu_stats(qdisc)) {
free_percpu(qdisc->cpu_bstats);
free_percpu(qdisc->cpu_qstats);
}
kfree(qdisc);
}
static void qdisc_free_cb(struct rcu_head *head)
{
struct Qdisc *q = container_of(head, struct Qdisc, rcu);
qdisc_free(q);
}
static void __qdisc_destroy(struct Qdisc *qdisc)
{
const struct Qdisc_ops *ops = qdisc->ops;
struct net_device *dev = qdisc_dev(qdisc);
#ifdef CONFIG_NET_SCHED
qdisc_hash_del(qdisc);
qdisc_put_stab(rtnl_dereference(qdisc->stab));
#endif
gen_kill_estimator(&qdisc->rate_est);
qdisc_reset(qdisc);
if (ops->destroy)
ops->destroy(qdisc);
lockdep_unregister_key(&qdisc->root_lock_key);
bpf_module_put(ops, ops->owner);
netdev_put(dev, &qdisc->dev_tracker);
trace_qdisc_destroy(qdisc);
call_rcu(&qdisc->rcu, qdisc_free_cb);
}
void qdisc_destroy(struct Qdisc *qdisc)
{
if (qdisc->flags & TCQ_F_BUILTIN)
return;
__qdisc_destroy(qdisc);
}
void qdisc_put(struct Qdisc *qdisc)
{
if (!qdisc)
return;
if (qdisc->flags & TCQ_F_BUILTIN ||
!refcount_dec_and_test(&qdisc->refcnt))
return;
__qdisc_destroy(qdisc);
}
EXPORT_SYMBOL(qdisc_put);
/* Version of qdisc_put() that is called with rtnl mutex unlocked.
* Intended to be used as optimization, this function only takes rtnl lock if
* qdisc reference counter reached zero.
*/
void qdisc_put_unlocked(struct Qdisc *qdisc)
{
if (qdisc->flags & TCQ_F_BUILTIN ||
!refcount_dec_and_rtnl_lock(&qdisc->refcnt))
return;
__qdisc_destroy(qdisc);
rtnl_unlock();
}
EXPORT_SYMBOL(qdisc_put_unlocked);
/* Attach toplevel qdisc to device queue. */
struct Qdisc *dev_graft_qdisc(struct netdev_queue *dev_queue,
struct Qdisc *qdisc)
{
struct Qdisc *oqdisc = rtnl_dereference(dev_queue->qdisc_sleeping);
spinlock_t *root_lock;
root_lock = qdisc_lock(oqdisc);
spin_lock_bh(root_lock);
/* ... and graft new one */
if (qdisc == NULL)
qdisc = &noop_qdisc;
rcu_assign_pointer(dev_queue->qdisc_sleeping, qdisc);
rcu_assign_pointer(dev_queue->qdisc, &noop_qdisc);
spin_unlock_bh(root_lock);
return oqdisc;
}
EXPORT_SYMBOL(dev_graft_qdisc);
static void shutdown_scheduler_queue(struct net_device *dev,
struct netdev_queue *dev_queue,
void *_qdisc_default)
{
struct Qdisc *qdisc = rtnl_dereference(dev_queue->qdisc_sleeping);
struct Qdisc *qdisc_default = _qdisc_default;
if (qdisc) {
rcu_assign_pointer(dev_queue->qdisc, qdisc_default);
rcu_assign_pointer(dev_queue->qdisc_sleeping, qdisc_default);
qdisc_put(qdisc);
}
}
static void attach_one_default_qdisc(struct net_device *dev,
struct netdev_queue *dev_queue,
void *_unused)
{
struct Qdisc *qdisc;
const struct Qdisc_ops *ops = default_qdisc_ops;
if (dev->priv_flags & IFF_NO_QUEUE)
ops = &noqueue_qdisc_ops;
else if(dev->type == ARPHRD_CAN)
ops = &pfifo_fast_ops;
qdisc = qdisc_create_dflt(dev_queue, ops, TC_H_ROOT, NULL);
if (!qdisc)
return;
if (!netif_is_multiqueue(dev))
qdisc->flags |= TCQ_F_ONETXQUEUE | TCQ_F_NOPARENT;
rcu_assign_pointer(dev_queue->qdisc_sleeping, qdisc);
}
static void attach_default_qdiscs(struct net_device *dev)
{
struct netdev_queue *txq;
struct Qdisc *qdisc;
txq = netdev_get_tx_queue(dev, 0);
if (!netif_is_multiqueue(dev) ||
dev->priv_flags & IFF_NO_QUEUE) {
netdev_for_each_tx_queue(dev, attach_one_default_qdisc, NULL);
qdisc = rtnl_dereference(txq->qdisc_sleeping);
rcu_assign_pointer(dev->qdisc, qdisc);
qdisc_refcount_inc(qdisc);
} else {
qdisc = qdisc_create_dflt(txq, &mq_qdisc_ops, TC_H_ROOT, NULL);
if (qdisc) {
rcu_assign_pointer(dev->qdisc, qdisc);
qdisc->ops->attach(qdisc);
}
}
qdisc = rtnl_dereference(dev->qdisc);
/* Detect default qdisc setup/init failed and fallback to "noqueue" */
if (qdisc == &noop_qdisc) {
netdev_warn(dev, "default qdisc (%s) fail, fallback to %s\n",
default_qdisc_ops->id, noqueue_qdisc_ops.id);
netdev_for_each_tx_queue(dev, shutdown_scheduler_queue, &noop_qdisc);
dev->priv_flags |= IFF_NO_QUEUE;
netdev_for_each_tx_queue(dev, attach_one_default_qdisc, NULL);
qdisc = rtnl_dereference(txq->qdisc_sleeping);
rcu_assign_pointer(dev->qdisc, qdisc);
qdisc_refcount_inc(qdisc);
dev->priv_flags ^= IFF_NO_QUEUE;
}
#ifdef CONFIG_NET_SCHED
if (qdisc != &noop_qdisc)
qdisc_hash_add(qdisc, false);
#endif
}
static void transition_one_qdisc(struct net_device *dev,
struct netdev_queue *dev_queue,
void *_need_watchdog)
{
struct Qdisc *new_qdisc = rtnl_dereference(dev_queue->qdisc_sleeping);
int *need_watchdog_p = _need_watchdog;
if (!(new_qdisc->flags & TCQ_F_BUILTIN))
clear_bit(__QDISC_STATE_DEACTIVATED, &new_qdisc->state);
rcu_assign_pointer(dev_queue->qdisc, new_qdisc);
if (need_watchdog_p) {
WRITE_ONCE(dev_queue->trans_start, 0);
*need_watchdog_p = 1;
}
}
void dev_activate(struct net_device *dev)
{
int need_watchdog;
/* No queueing discipline is attached to device;
* create default one for devices, which need queueing
* and noqueue_qdisc for virtual interfaces
*/
if (rtnl_dereference(dev->qdisc) == &noop_qdisc)
attach_default_qdiscs(dev);
if (!netif_carrier_ok(dev))
/* Delay activation until next carrier-on event */
return;
need_watchdog = 0;
netdev_for_each_tx_queue(dev, transition_one_qdisc, &need_watchdog);
if (dev_ingress_queue(dev))
transition_one_qdisc(dev, dev_ingress_queue(dev), NULL);
if (need_watchdog) {
netif_trans_update(dev);
netdev_watchdog_up(dev);
}
}
EXPORT_SYMBOL(dev_activate);
static void qdisc_deactivate(struct Qdisc *qdisc)
{
if (qdisc->flags & TCQ_F_BUILTIN)
return;
set_bit(__QDISC_STATE_DEACTIVATED, &qdisc->state);
}
static void dev_deactivate_queue(struct net_device *dev,
struct netdev_queue *dev_queue,
void *_sync_needed)
{
bool *sync_needed = _sync_needed;
struct Qdisc *qdisc;
qdisc = rtnl_dereference(dev_queue->qdisc);
if (qdisc) {
if (qdisc->enqueue)
*sync_needed = true;
qdisc_deactivate(qdisc);
rcu_assign_pointer(dev_queue->qdisc, &noop_qdisc);
}
}
static void dev_reset_queue(struct net_device *dev,
struct netdev_queue *dev_queue,
void *_unused)
{
struct Qdisc *qdisc;
bool nolock;
qdisc = rtnl_dereference(dev_queue->qdisc_sleeping);
if (!qdisc)
return;
nolock = qdisc->flags & TCQ_F_NOLOCK;
if (nolock)
spin_lock_bh(&qdisc->seqlock);
spin_lock_bh(qdisc_lock(qdisc));
qdisc_reset(qdisc);
spin_unlock_bh(qdisc_lock(qdisc));
if (nolock) {
clear_bit(__QDISC_STATE_MISSED, &qdisc->state);
clear_bit(__QDISC_STATE_DRAINING, &qdisc->state);
spin_unlock_bh(&qdisc->seqlock);
}
}
static bool some_qdisc_is_busy(struct net_device *dev)
{
unsigned int i;
for (i = 0; i < dev->num_tx_queues; i++) {
struct netdev_queue *dev_queue;
spinlock_t *root_lock;
struct Qdisc *q;
int val;
dev_queue = netdev_get_tx_queue(dev, i);
q = rtnl_dereference(dev_queue->qdisc_sleeping);
root_lock = qdisc_lock(q);
spin_lock_bh(root_lock);
val = (qdisc_is_running(q) ||
test_bit(__QDISC_STATE_SCHED, &q->state));
spin_unlock_bh(root_lock);
if (val)
return true;
}
return false;
}
/**
* dev_deactivate_many - deactivate transmissions on several devices
* @head: list of devices to deactivate
*
* This function returns only when all outstanding transmissions
* have completed, unless all devices are in dismantle phase.
*/
void dev_deactivate_many(struct list_head *head)
{
bool sync_needed = false;
struct net_device *dev;
list_for_each_entry(dev, head, close_list) {
netdev_for_each_tx_queue(dev, dev_deactivate_queue,
&sync_needed);
if (dev_ingress_queue(dev))
dev_deactivate_queue(dev, dev_ingress_queue(dev),
&sync_needed);
netdev_watchdog_down(dev);
}
/* Wait for outstanding qdisc enqueuing calls. */
if (sync_needed)
synchronize_net();
list_for_each_entry(dev, head, close_list) {
netdev_for_each_tx_queue(dev, dev_reset_queue, NULL);
if (dev_ingress_queue(dev))
dev_reset_queue(dev, dev_ingress_queue(dev), NULL);
}
/* Wait for outstanding qdisc_run calls. */
list_for_each_entry(dev, head, close_list) {
while (some_qdisc_is_busy(dev)) {
/* wait_event() would avoid this sleep-loop but would
* require expensive checks in the fast paths of packet
* processing which isn't worth it.
*/
schedule_timeout_uninterruptible(1);
}
}
}
void dev_deactivate(struct net_device *dev)
{
LIST_HEAD(single);
list_add(&dev->close_list, &single);
dev_deactivate_many(&single);
list_del(&single);
}
EXPORT_SYMBOL(dev_deactivate);
static int qdisc_change_tx_queue_len(struct net_device *dev,
struct netdev_queue *dev_queue)
{
struct Qdisc *qdisc = rtnl_dereference(dev_queue->qdisc_sleeping);
const struct Qdisc_ops *ops = qdisc->ops;
if (ops->change_tx_queue_len)
return ops->change_tx_queue_len(qdisc, dev->tx_queue_len);
return 0;
}
void dev_qdisc_change_real_num_tx(struct net_device *dev,
unsigned int new_real_tx)
{
struct Qdisc *qdisc = rtnl_dereference(dev->qdisc);
if (qdisc->ops->change_real_num_tx)
qdisc->ops->change_real_num_tx(qdisc, new_real_tx);
}
void mq_change_real_num_tx(struct Qdisc *sch, unsigned int new_real_tx)
{
#ifdef CONFIG_NET_SCHED
struct net_device *dev = qdisc_dev(sch);
struct Qdisc *qdisc;
unsigned int i;
for (i = new_real_tx; i < dev->real_num_tx_queues; i++) {
qdisc = rtnl_dereference(netdev_get_tx_queue(dev, i)->qdisc_sleeping);
/* Only update the default qdiscs we created,
* qdiscs with handles are always hashed.
*/
if (qdisc != &noop_qdisc && !qdisc->handle)
qdisc_hash_del(qdisc);
}
for (i = dev->real_num_tx_queues; i < new_real_tx; i++) {
qdisc = rtnl_dereference(netdev_get_tx_queue(dev, i)->qdisc_sleeping);
if (qdisc != &noop_qdisc && !qdisc->handle)
qdisc_hash_add(qdisc, false);
}
#endif
}
EXPORT_SYMBOL(mq_change_real_num_tx);
int dev_qdisc_change_tx_queue_len(struct net_device *dev)
{
bool up = dev->flags & IFF_UP;
unsigned int i;
int ret = 0;
if (up)
dev_deactivate(dev);
for (i = 0; i < dev->num_tx_queues; i++) {
ret = qdisc_change_tx_queue_len(dev, &dev->_tx[i]);
/* TODO: revert changes on a partial failure */
if (ret)
break;
}
if (up)
dev_activate(dev);
return ret;
}
static void dev_init_scheduler_queue(struct net_device *dev,
struct netdev_queue *dev_queue,
void *_qdisc)
{
struct Qdisc *qdisc = _qdisc;
rcu_assign_pointer(dev_queue->qdisc, qdisc);
rcu_assign_pointer(dev_queue->qdisc_sleeping, qdisc);
}
void dev_init_scheduler(struct net_device *dev)
{
rcu_assign_pointer(dev->qdisc, &noop_qdisc);
netdev_for_each_tx_queue(dev, dev_init_scheduler_queue, &noop_qdisc);
if (dev_ingress_queue(dev))
dev_init_scheduler_queue(dev, dev_ingress_queue(dev), &noop_qdisc);
timer_setup(&dev->watchdog_timer, dev_watchdog, 0);
}
void dev_shutdown(struct net_device *dev)
{
netdev_for_each_tx_queue(dev, shutdown_scheduler_queue, &noop_qdisc);
if (dev_ingress_queue(dev))
shutdown_scheduler_queue(dev, dev_ingress_queue(dev), &noop_qdisc);
qdisc_put(rtnl_dereference(dev->qdisc));
rcu_assign_pointer(dev->qdisc, &noop_qdisc);
WARN_ON(timer_pending(&dev->watchdog_timer));
}
/**
* psched_ratecfg_precompute__() - Pre-compute values for reciprocal division
* @rate: Rate to compute reciprocal division values of
* @mult: Multiplier for reciprocal division
* @shift: Shift for reciprocal division
*
* The multiplier and shift for reciprocal division by rate are stored
* in mult and shift.
*
* The deal here is to replace a divide by a reciprocal one
* in fast path (a reciprocal divide is a multiply and a shift)
*
* Normal formula would be :
* time_in_ns = (NSEC_PER_SEC * len) / rate_bps
*
* We compute mult/shift to use instead :
* time_in_ns = (len * mult) >> shift;
*
* We try to get the highest possible mult value for accuracy,
* but have to make sure no overflows will ever happen.
*
* reciprocal_value() is not used here it doesn't handle 64-bit values.
*/
static void psched_ratecfg_precompute__(u64 rate, u32 *mult, u8 *shift)
{
u64 factor = NSEC_PER_SEC;
*mult = 1;
*shift = 0;
if (rate <= 0)
return;
for (;;) {
*mult = div64_u64(factor, rate);
if (*mult & (1U << 31) || factor & (1ULL << 63))
break;
factor <<= 1;
(*shift)++;
}
}
void psched_ratecfg_precompute(struct psched_ratecfg *r,
const struct tc_ratespec *conf,
u64 rate64)
{
memset(r, 0, sizeof(*r));
r->overhead = conf->overhead;
r->mpu = conf->mpu;
r->rate_bytes_ps = max_t(u64, conf->rate, rate64);
r->linklayer = (conf->linklayer & TC_LINKLAYER_MASK);
psched_ratecfg_precompute__(r->rate_bytes_ps, &r->mult, &r->shift);
}
EXPORT_SYMBOL(psched_ratecfg_precompute);
void psched_ppscfg_precompute(struct psched_pktrate *r, u64 pktrate64)
{
r->rate_pkts_ps = pktrate64;
psched_ratecfg_precompute__(r->rate_pkts_ps, &r->mult, &r->shift);
}
EXPORT_SYMBOL(psched_ppscfg_precompute);
void mini_qdisc_pair_swap(struct mini_Qdisc_pair *miniqp,
struct tcf_proto *tp_head)
{
/* Protected with chain0->filter_chain_lock.
* Can't access chain directly because tp_head can be NULL.
*/
struct mini_Qdisc *miniq_old =
rcu_dereference_protected(*miniqp->p_miniq, 1);
struct mini_Qdisc *miniq;
if (!tp_head) {
RCU_INIT_POINTER(*miniqp->p_miniq, NULL);
} else {
miniq = miniq_old != &miniqp->miniq1 ?
&miniqp->miniq1 : &miniqp->miniq2;
/* We need to make sure that readers won't see the miniq
* we are about to modify. So ensure that at least one RCU
* grace period has elapsed since the miniq was made
* inactive.
*/
if (IS_ENABLED(CONFIG_PREEMPT_RT))
cond_synchronize_rcu(miniq->rcu_state);
else if (!poll_state_synchronize_rcu(miniq->rcu_state))
synchronize_rcu_expedited();
miniq->filter_list = tp_head;
rcu_assign_pointer(*miniqp->p_miniq, miniq);
}
if (miniq_old)
/* This is counterpart of the rcu sync above. We need to
* block potential new user of miniq_old until all readers
* are not seeing it.
*/
miniq_old->rcu_state = start_poll_synchronize_rcu();
}
EXPORT_SYMBOL(mini_qdisc_pair_swap);
void mini_qdisc_pair_block_init(struct mini_Qdisc_pair *miniqp,
struct tcf_block *block)
{
miniqp->miniq1.block = block;
miniqp->miniq2.block = block;
}
EXPORT_SYMBOL(mini_qdisc_pair_block_init);
void mini_qdisc_pair_init(struct mini_Qdisc_pair *miniqp, struct Qdisc *qdisc,
struct mini_Qdisc __rcu **p_miniq)
{
miniqp->miniq1.cpu_bstats = qdisc->cpu_bstats;
miniqp->miniq1.cpu_qstats = qdisc->cpu_qstats;
miniqp->miniq2.cpu_bstats = qdisc->cpu_bstats;
miniqp->miniq2.cpu_qstats = qdisc->cpu_qstats;
miniqp->miniq1.rcu_state = get_state_synchronize_rcu();
miniqp->miniq2.rcu_state = miniqp->miniq1.rcu_state;
miniqp->p_miniq = p_miniq;
}
EXPORT_SYMBOL(mini_qdisc_pair_init);
]
|
| 10/2 |
2026/05/21 11:34 |
llm |
expert |
0m
Model:
Tokens:
input: 40360
output: 22
thoughts: 463
Thoughts: **Analyzing a Device Shutdown with Pending Watchdog Timer**
Okay, so I'm looking at a kernel warning specifically within `dev_shutdown`. The critical line here is `WARN_ON(timer_pending(&dev->watchdog_timer));`. This tells me that the watchdog timer associated with a network device `dev` is still active, or "pending," when we're trying to shut down the device. That's a definite code smell; we should be cleaning up after ourselves.
Now, the important part is figuring out *why* this is happening. The warning is triggered when `dev_shutdown` is called. Let's trace back from the warning to the point where this shutdown is initiated. The call trace provided shows `unregister_netdevice_many_notify` calling `dev_shutdown(dev)`. Clearly, we need to inspect `unregister_netdevice_many_notify` very carefully. This function is responsible for the device unregistration sequence, and somewhere in *its* execution path, the watchdog timer *should* be stopped before calling `dev_shutdown`. The call stack suggests it should have stopped the timer. I need to examine `unregister_netdevice_many_notify` and follow the execution to see what's happening.
|
| 11/2 |
2026/05/21 11:34 |
tool |
read-file |
0m
Args:
Results: map[Contents:// SPDX-License-Identifier: GPL-2.0-or-later
/*
* NET3 Protocol independent device support routines.
*
* Derived from the non IP parts of dev.c 1.0.19
* Authors: Ross Biro
* Fred N. van Kempen, <waltje@uWalt.NL.Mugnet.ORG>
* Mark Evans, <evansmp@uhura.aston.ac.uk>
*
* Additional Authors:
* Florian la Roche <rzsfl@rz.uni-sb.de>
* Alan Cox <gw4pts@gw4pts.ampr.org>
* David Hinds <dahinds@users.sourceforge.net>
* Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>
* Adam Sulmicki <adam@cfar.umd.edu>
* Pekka Riikonen <priikone@poesidon.pspt.fi>
*
* Changes:
* D.J. Barrow : Fixed bug where dev->refcnt gets set
* to 2 if register_netdev gets called
* before net_dev_init & also removed a
* few lines of code in the process.
* Alan Cox : device private ioctl copies fields back.
* Alan Cox : Transmit queue code does relevant
* stunts to keep the queue safe.
* Alan Cox : Fixed double lock.
* Alan Cox : Fixed promisc NULL pointer trap
* ???????? : Support the full private ioctl range
* Alan Cox : Moved ioctl permission check into
* drivers
* Tim Kordas : SIOCADDMULTI/SIOCDELMULTI
* Alan Cox : 100 backlog just doesn't cut it when
* you start doing multicast video 8)
* Alan Cox : Rewrote net_bh and list manager.
* Alan Cox : Fix ETH_P_ALL echoback lengths.
* Alan Cox : Took out transmit every packet pass
* Saved a few bytes in the ioctl handler
* Alan Cox : Network driver sets packet type before
* calling netif_rx. Saves a function
* call a packet.
* Alan Cox : Hashed net_bh()
* Richard Kooijman: Timestamp fixes.
* Alan Cox : Wrong field in SIOCGIFDSTADDR
* Alan Cox : Device lock protection.
* Alan Cox : Fixed nasty side effect of device close
* changes.
* Rudi Cilibrasi : Pass the right thing to
* set_mac_address()
* Dave Miller : 32bit quantity for the device lock to
* make it work out on a Sparc.
* Bjorn Ekwall : Added KERNELD hack.
* Alan Cox : Cleaned up the backlog initialise.
* Craig Metz : SIOCGIFCONF fix if space for under
* 1 device.
* Thomas Bogendoerfer : Return ENODEV for dev_open, if there
* is no device open function.
* Andi Kleen : Fix error reporting for SIOCGIFCONF
* Michael Chastain : Fix signed/unsigned for SIOCGIFCONF
* Cyrus Durgin : Cleaned for KMOD
* Adam Sulmicki : Bug Fix : Network Device Unload
* A network device unload needs to purge
* the backlog queue.
* Paul Rusty Russell : SIOCSIFNAME
* Pekka Riikonen : Netdev boot-time settings code
* Andrew Morton : Make unregister_netdevice wait
* indefinitely on dev->refcnt
* J Hadi Salim : - Backlog queue sampling
* - netif_rx() feedback
*/
#include <linux/uaccess.h>
#include <linux/bitmap.h>
#include <linux/capability.h>
#include <linux/cpu.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/hash.h>
#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/sched/isolation.h>
#include <linux/sched/mm.h>
#include <linux/smpboot.h>
#include <linux/mutex.h>
#include <linux/rwsem.h>
#include <linux/string.h>
#include <linux/mm.h>
#include <linux/socket.h>
#include <linux/sockios.h>
#include <linux/errno.h>
#include <linux/interrupt.h>
#include <linux/if_ether.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
#include <linux/ethtool.h>
#include <linux/ethtool_netlink.h>
#include <linux/skbuff.h>
#include <linux/kthread.h>
#include <linux/bpf.h>
#include <linux/bpf_trace.h>
#include <net/net_namespace.h>
#include <net/sock.h>
#include <net/busy_poll.h>
#include <linux/rtnetlink.h>
#include <linux/stat.h>
#include <net/dsa.h>
#include <net/dst.h>
#include <net/dst_metadata.h>
#include <net/gro.h>
#include <net/netdev_queues.h>
#include <net/pkt_sched.h>
#include <net/pkt_cls.h>
#include <net/checksum.h>
#include <net/xfrm.h>
#include <net/tcx.h>
#include <linux/highmem.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/netpoll.h>
#include <linux/rcupdate.h>
#include <linux/delay.h>
#include <net/iw_handler.h>
#include <asm/current.h>
#include <linux/audit.h>
#include <linux/dmaengine.h>
#include <linux/err.h>
#include <linux/ctype.h>
#include <linux/if_arp.h>
#include <linux/if_vlan.h>
#include <linux/ip.h>
#include <net/ip.h>
#include <net/mpls.h>
#include <linux/ipv6.h>
#include <linux/in.h>
#include <linux/jhash.h>
#include <linux/random.h>
#include <trace/events/napi.h>
#include <trace/events/net.h>
#include <trace/events/skb.h>
#include <trace/events/qdisc.h>
#include <trace/events/xdp.h>
#include <linux/inetdevice.h>
#include <linux/cpu_rmap.h>
#include <linux/static_key.h>
#include <linux/hashtable.h>
#include <linux/vmalloc.h>
#include <linux/if_macvlan.h>
#include <linux/errqueue.h>
#include <linux/hrtimer.h>
#include <linux/netfilter_netdev.h>
#include <linux/crash_dump.h>
#include <linux/sctp.h>
#include <net/udp_tunnel.h>
#include <linux/net_namespace.h>
#include <linux/indirect_call_wrapper.h>
#include <net/devlink.h>
#include <linux/pm_runtime.h>
#include <linux/prandom.h>
#include <linux/once_lite.h>
#include <net/netdev_lock.h>
#include <net/netdev_rx_queue.h>
#include <net/page_pool/types.h>
#include <net/page_pool/helpers.h>
#include <net/page_pool/memory_provider.h>
#include <net/rps.h>
#include <linux/phy_link_topology.h>
#include "dev.h"
#include "devmem.h"
#include "net-sysfs.h"
static DEFINE_SPINLOCK(ptype_lock);
struct list_head ptype_base[PTYPE_HASH_SIZE] __read_mostly;
static int netif_rx_internal(struct sk_buff *skb);
static int call_netdevice_notifiers_extack(unsigned long val,
struct net_device *dev,
struct netlink_ext_ack *extack);
static DEFINE_MUTEX(ifalias_mutex);
/* protects napi_hash addition/deletion and napi_gen_id */
static DEFINE_SPINLOCK(napi_hash_lock);
static unsigned int napi_gen_id = NR_CPUS;
static DEFINE_READ_MOSTLY_HASHTABLE(napi_hash, 8);
static inline void dev_base_seq_inc(struct net *net)
{
unsigned int val = net->dev_base_seq + 1;
WRITE_ONCE(net->dev_base_seq, val ?: 1);
}
static inline struct hlist_head *dev_name_hash(struct net *net, const char *name)
{
unsigned int hash = full_name_hash(net, name, strnlen(name, IFNAMSIZ));
return &net->dev_name_head[hash_32(hash, NETDEV_HASHBITS)];
}
static inline struct hlist_head *dev_index_hash(struct net *net, int ifindex)
{
return &net->dev_index_head[ifindex & (NETDEV_HASHENTRIES - 1)];
}
#ifndef CONFIG_PREEMPT_RT
static DEFINE_STATIC_KEY_FALSE(use_backlog_threads_key);
static int __init setup_backlog_napi_threads(char *arg)
{
static_branch_enable(&use_backlog_threads_key);
return 0;
}
early_param("thread_backlog_napi", setup_backlog_napi_threads);
static bool use_backlog_threads(void)
{
return static_branch_unlikely(&use_backlog_threads_key);
}
#else
static bool use_backlog_threads(void)
{
return true;
}
#endif
static inline void backlog_lock_irq_save(struct softnet_data *sd,
unsigned long *flags)
{
if (IS_ENABLED(CONFIG_RPS) || use_backlog_threads())
spin_lock_irqsave(&sd->input_pkt_queue.lock, *flags);
else
local_irq_save(*flags);
}
static inline void backlog_lock_irq_disable(struct softnet_data *sd)
{
if (IS_ENABLED(CONFIG_RPS) || use_backlog_threads())
spin_lock_irq(&sd->input_pkt_queue.lock);
else
local_irq_disable();
}
static inline void backlog_unlock_irq_restore(struct softnet_data *sd,
unsigned long *flags)
{
if (IS_ENABLED(CONFIG_RPS) || use_backlog_threads())
spin_unlock_irqrestore(&sd->input_pkt_queue.lock, *flags);
else
local_irq_restore(*flags);
}
static inline void backlog_unlock_irq_enable(struct softnet_data *sd)
{
if (IS_ENABLED(CONFIG_RPS) || use_backlog_threads())
spin_unlock_irq(&sd->input_pkt_queue.lock);
else
local_irq_enable();
}
static struct netdev_name_node *netdev_name_node_alloc(struct net_device *dev,
const char *name)
{
struct netdev_name_node *name_node;
name_node = kmalloc(sizeof(*name_node), GFP_KERNEL);
if (!name_node)
return NULL;
INIT_HLIST_NODE(&name_node->hlist);
name_node->dev = dev;
name_node->name = name;
return name_node;
}
static struct netdev_name_node *
netdev_name_node_head_alloc(struct net_device *dev)
{
struct netdev_name_node *name_node;
name_node = netdev_name_node_alloc(dev, dev->name);
if (!name_node)
return NULL;
INIT_LIST_HEAD(&name_node->list);
return name_node;
}
static void netdev_name_node_free(struct netdev_name_node *name_node)
{
kfree(name_node);
}
static void netdev_name_node_add(struct net *net,
struct netdev_name_node *name_node)
{
hlist_add_head_rcu(&name_node->hlist,
dev_name_hash(net, name_node->name));
}
static void netdev_name_node_del(struct netdev_name_node *name_node)
{
hlist_del_rcu(&name_node->hlist);
}
static struct netdev_name_node *netdev_name_node_lookup(struct net *net,
const char *name)
{
struct hlist_head *head = dev_name_hash(net, name);
struct netdev_name_node *name_node;
hlist_for_each_entry(name_node, head, hlist)
if (!strcmp(name_node->name, name))
return name_node;
return NULL;
}
static struct netdev_name_node *netdev_name_node_lookup_rcu(struct net *net,
const char *name)
{
struct hlist_head *head = dev_name_hash(net, name);
struct netdev_name_node *name_node;
hlist_for_each_entry_rcu(name_node, head, hlist)
if (!strcmp(name_node->name, name))
return name_node;
return NULL;
}
bool netdev_name_in_use(struct net *net, const char *name)
{
return netdev_name_node_lookup(net, name);
}
EXPORT_SYMBOL(netdev_name_in_use);
int netdev_name_node_alt_create(struct net_device *dev, const char *name)
{
struct netdev_name_node *name_node;
struct net *net = dev_net(dev);
name_node = netdev_name_node_lookup(net, name);
if (name_node)
return -EEXIST;
name_node = netdev_name_node_alloc(dev, name);
if (!name_node)
return -ENOMEM;
netdev_name_node_add(net, name_node);
/* The node that holds dev->name acts as a head of per-device list. */
list_add_tail_rcu(&name_node->list, &dev->name_node->list);
return 0;
}
static void netdev_name_node_alt_free(struct rcu_head *head)
{
struct netdev_name_node *name_node =
container_of(head, struct netdev_name_node, rcu);
kfree(name_node->name);
netdev_name_node_free(name_node);
}
static void __netdev_name_node_alt_destroy(struct netdev_name_node *name_node)
{
netdev_name_node_del(name_node);
list_del(&name_node->list);
call_rcu(&name_node->rcu, netdev_name_node_alt_free);
}
int netdev_name_node_alt_destroy(struct net_device *dev, const char *name)
{
struct netdev_name_node *name_node;
struct net *net = dev_net(dev);
name_node = netdev_name_node_lookup(net, name);
if (!name_node)
return -ENOENT;
/* lookup might have found our primary name or a name belonging
* to another device.
*/
if (name_node == dev->name_node || name_node->dev != dev)
return -EINVAL;
__netdev_name_node_alt_destroy(name_node);
return 0;
}
static void netdev_name_node_alt_flush(struct net_device *dev)
{
struct netdev_name_node *name_node, *tmp;
list_for_each_entry_safe(name_node, tmp, &dev->name_node->list, list) {
list_del(&name_node->list);
netdev_name_node_alt_free(&name_node->rcu);
}
}
/* Device list insertion */
static void list_netdevice(struct net_device *dev)
{
struct netdev_name_node *name_node;
struct net *net = dev_net(dev);
ASSERT_RTNL();
list_add_tail_rcu(&dev->dev_list, &net->dev_base_head);
netdev_name_node_add(net, dev->name_node);
hlist_add_head_rcu(&dev->index_hlist,
dev_index_hash(net, dev->ifindex));
netdev_for_each_altname(dev, name_node)
netdev_name_node_add(net, name_node);
/* We reserved the ifindex, this can't fail */
WARN_ON(xa_store(&net->dev_by_index, dev->ifindex, dev, GFP_KERNEL));
dev_base_seq_inc(net);
}
/* Device list removal
* caller must respect a RCU grace period before freeing/reusing dev
*/
static void unlist_netdevice(struct net_device *dev)
{
struct netdev_name_node *name_node;
struct net *net = dev_net(dev);
ASSERT_RTNL();
xa_erase(&net->dev_by_index, dev->ifindex);
netdev_for_each_altname(dev, name_node)
netdev_name_node_del(name_node);
/* Unlink dev from the device chain */
list_del_rcu(&dev->dev_list);
netdev_name_node_del(dev->name_node);
hlist_del_rcu(&dev->index_hlist);
dev_base_seq_inc(dev_net(dev));
}
/*
* Our notifier list
*/
static RAW_NOTIFIER_HEAD(netdev_chain);
/*
* Device drivers call our routines to queue packets here. We empty the
* queue in the local softnet handler.
*/
DEFINE_PER_CPU_ALIGNED(struct softnet_data, softnet_data) = {
.process_queue_bh_lock = INIT_LOCAL_LOCK(process_queue_bh_lock),
};
EXPORT_PER_CPU_SYMBOL(softnet_data);
/* Page_pool has a lockless array/stack to alloc/recycle pages.
* PP consumers must pay attention to run APIs in the appropriate context
* (e.g. NAPI context).
*/
DEFINE_PER_CPU(struct page_pool_bh, system_page_pool) = {
.bh_lock = INIT_LOCAL_LOCK(bh_lock),
};
#ifdef CONFIG_LOCKDEP
/*
* register_netdevice() inits txq->_xmit_lock and sets lockdep class
* according to dev->type
*/
static const unsigned short netdev_lock_type[] = {
ARPHRD_NETROM, ARPHRD_ETHER, ARPHRD_EETHER, ARPHRD_AX25,
ARPHRD_PRONET, ARPHRD_CHAOS, ARPHRD_IEEE802, ARPHRD_ARCNET,
ARPHRD_APPLETLK, ARPHRD_DLCI, ARPHRD_ATM, ARPHRD_METRICOM,
ARPHRD_IEEE1394, ARPHRD_EUI64, ARPHRD_INFINIBAND, ARPHRD_SLIP,
ARPHRD_CSLIP, ARPHRD_SLIP6, ARPHRD_CSLIP6, ARPHRD_RSRVD,
ARPHRD_ADAPT, ARPHRD_ROSE, ARPHRD_X25, ARPHRD_HWX25,
ARPHRD_PPP, ARPHRD_CISCO, ARPHRD_LAPB, ARPHRD_DDCMP,
ARPHRD_RAWHDLC, ARPHRD_TUNNEL, ARPHRD_TUNNEL6, ARPHRD_FRAD,
ARPHRD_SKIP, ARPHRD_LOOPBACK, ARPHRD_LOCALTLK, ARPHRD_FDDI,
ARPHRD_BIF, ARPHRD_SIT, ARPHRD_IPDDP, ARPHRD_IPGRE,
ARPHRD_PIMREG, ARPHRD_HIPPI, ARPHRD_ASH, ARPHRD_ECONET,
ARPHRD_IRDA, ARPHRD_FCPP, ARPHRD_FCAL, ARPHRD_FCPL,
ARPHRD_FCFABRIC, ARPHRD_IEEE80211, ARPHRD_IEEE80211_PRISM,
ARPHRD_IEEE80211_RADIOTAP, ARPHRD_PHONET, ARPHRD_PHONET_PIPE,
ARPHRD_IEEE802154, ARPHRD_VOID, ARPHRD_NONE};
static const char *const netdev_lock_name[] = {
"_xmit_NETROM", "_xmit_ETHER", "_xmit_EETHER", "_xmit_AX25",
"_xmit_PRONET", "_xmit_CHAOS", "_xmit_IEEE802", "_xmit_ARCNET",
"_xmit_APPLETLK", "_xmit_DLCI", "_xmit_ATM", "_xmit_METRICOM",
"_xmit_IEEE1394", "_xmit_EUI64", "_xmit_INFINIBAND", "_xmit_SLIP",
"_xmit_CSLIP", "_xmit_SLIP6", "_xmit_CSLIP6", "_xmit_RSRVD",
"_xmit_ADAPT", "_xmit_ROSE", "_xmit_X25", "_xmit_HWX25",
"_xmit_PPP", "_xmit_CISCO", "_xmit_LAPB", "_xmit_DDCMP",
"_xmit_RAWHDLC", "_xmit_TUNNEL", "_xmit_TUNNEL6", "_xmit_FRAD",
"_xmit_SKIP", "_xmit_LOOPBACK", "_xmit_LOCALTLK", "_xmit_FDDI",
"_xmit_BIF", "_xmit_SIT", "_xmit_IPDDP", "_xmit_IPGRE",
"_xmit_PIMREG", "_xmit_HIPPI", "_xmit_ASH", "_xmit_ECONET",
"_xmit_IRDA", "_xmit_FCPP", "_xmit_FCAL", "_xmit_FCPL",
"_xmit_FCFABRIC", "_xmit_IEEE80211", "_xmit_IEEE80211_PRISM",
"_xmit_IEEE80211_RADIOTAP", "_xmit_PHONET", "_xmit_PHONET_PIPE",
"_xmit_IEEE802154", "_xmit_VOID", "_xmit_NONE"};
static struct lock_class_key netdev_xmit_lock_key[ARRAY_SIZE(netdev_lock_type)];
static struct lock_class_key netdev_addr_lock_key[ARRAY_SIZE(netdev_lock_type)];
static inline unsigned short netdev_lock_pos(unsigned short dev_type)
{
int i;
for (i = 0; i < ARRAY_SIZE(netdev_lock_type); i++)
if (netdev_lock_type[i] == dev_type)
return i;
/* the last key is used by default */
return ARRAY_SIZE(netdev_lock_type) - 1;
}
static inline void netdev_set_xmit_lockdep_class(spinlock_t *lock,
unsigned short dev_type)
{
int i;
i = netdev_lock_pos(dev_type);
lockdep_set_class_and_name(lock, &netdev_xmit_lock_key[i],
netdev_lock_name[i]);
}
static inline void netdev_set_addr_lockdep_class(struct net_device *dev)
{
int i;
i = netdev_lock_pos(dev->type);
lockdep_set_class_and_name(&dev->addr_list_lock,
&netdev_addr_lock_key[i],
netdev_lock_name[i]);
}
#else
static inline void netdev_set_xmit_lockdep_class(spinlock_t *lock,
unsigned short dev_type)
{
}
static inline void netdev_set_addr_lockdep_class(struct net_device *dev)
{
}
#endif
/*******************************************************************************
*
* Protocol management and registration routines
*
*******************************************************************************/
/*
* Add a protocol ID to the list. Now that the input handler is
* smarter we can dispense with all the messy stuff that used to be
* here.
*
* BEWARE!!! Protocol handlers, mangling input packets,
* MUST BE last in hash buckets and checking protocol handlers
* MUST start from promiscuous ptype_all chain in net_bh.
* It is true now, do not change it.
* Explanation follows: if protocol handler, mangling packet, will
* be the first on list, it is not able to sense, that packet
* is cloned and should be copied-on-write, so that it will
* change it and subsequent readers will get broken packet.
* --ANK (980803)
*/
static inline struct list_head *ptype_head(const struct packet_type *pt)
{
if (pt->type == htons(ETH_P_ALL)) {
if (!pt->af_packet_net && !pt->dev)
return NULL;
return pt->dev ? &pt->dev->ptype_all :
&pt->af_packet_net->ptype_all;
}
if (pt->dev)
return &pt->dev->ptype_specific;
return pt->af_packet_net ? &pt->af_packet_net->ptype_specific :
&ptype_base[ntohs(pt->type) & PTYPE_HASH_MASK];
}
/**
* dev_add_pack - add packet handler
* @pt: packet type declaration
*
* Add a protocol handler to the networking stack. The passed &packet_type
* is linked into kernel lists and may not be freed until it has been
* removed from the kernel lists.
*
* This call does not sleep therefore it can not
* guarantee all CPU's that are in middle of receiving packets
* will see the new packet type (until the next received packet).
*/
void dev_add_pack(struct packet_type *pt)
{
struct list_head *head = ptype_head(pt);
if (WARN_ON_ONCE(!head))
return;
spin_lock(&ptype_lock);
list_add_rcu(&pt->list, head);
spin_unlock(&ptype_lock);
}
EXPORT_SYMBOL(dev_add_pack);
/**
* __dev_remove_pack - remove packet handler
* @pt: packet type declaration
*
* Remove a protocol handler that was previously added to the kernel
* protocol handlers by dev_add_pack(). The passed &packet_type is removed
* from the kernel lists and can be freed or reused once this function
* returns.
*
* The packet type might still be in use by receivers
* and must not be freed until after all the CPU's have gone
* through a quiescent state.
*/
void __dev_remove_pack(struct packet_type *pt)
{
struct list_head *head = ptype_head(pt);
struct packet_type *pt1;
if (!head)
return;
spin_lock(&ptype_lock);
list_for_each_entry(pt1, head, list) {
if (pt == pt1) {
list_del_rcu(&pt->list);
goto out;
}
}
pr_warn("dev_remove_pack: %p not found\n", pt);
out:
spin_unlock(&ptype_lock);
}
EXPORT_SYMBOL(__dev_remove_pack);
/**
* dev_remove_pack - remove packet handler
* @pt: packet type declaration
*
* Remove a protocol handler that was previously added to the kernel
* protocol handlers by dev_add_pack(). The passed &packet_type is removed
* from the kernel lists and can be freed or reused once this function
* returns.
*
* This call sleeps to guarantee that no CPU is looking at the packet
* type after return.
*/
void dev_remove_pack(struct packet_type *pt)
{
__dev_remove_pack(pt);
synchronize_net();
}
EXPORT_SYMBOL(dev_remove_pack);
/*******************************************************************************
*
* Device Interface Subroutines
*
*******************************************************************************/
/**
* dev_get_iflink - get 'iflink' value of a interface
* @dev: targeted interface
*
* Indicates the ifindex the interface is linked to.
* Physical interfaces have the same 'ifindex' and 'iflink' values.
*/
int dev_get_iflink(const struct net_device *dev)
{
if (dev->netdev_ops && dev->netdev_ops->ndo_get_iflink)
return dev->netdev_ops->ndo_get_iflink(dev);
return READ_ONCE(dev->ifindex);
}
EXPORT_SYMBOL(dev_get_iflink);
/**
* dev_fill_metadata_dst - Retrieve tunnel egress information.
* @dev: targeted interface
* @skb: The packet.
*
* For better visibility of tunnel traffic OVS needs to retrieve
* egress tunnel information for a packet. Following API allows
* user to get this info.
*/
int dev_fill_metadata_dst(struct net_device *dev, struct sk_buff *skb)
{
struct ip_tunnel_info *info;
if (!dev->netdev_ops || !dev->netdev_ops->ndo_fill_metadata_dst)
return -EINVAL;
info = skb_tunnel_info_unclone(skb);
if (!info)
return -ENOMEM;
if (unlikely(!(info->mode & IP_TUNNEL_INFO_TX)))
return -EINVAL;
return dev->netdev_ops->ndo_fill_metadata_dst(dev, skb);
}
EXPORT_SYMBOL_GPL(dev_fill_metadata_dst);
static struct net_device_path *dev_fwd_path(struct net_device_path_stack *stack)
{
int k = stack->num_paths++;
if (WARN_ON_ONCE(k >= NET_DEVICE_PATH_STACK_MAX))
return NULL;
return &stack->path[k];
}
int dev_fill_forward_path(const struct net_device *dev, const u8 *daddr,
struct net_device_path_stack *stack)
{
const struct net_device *last_dev;
struct net_device_path_ctx ctx = {
.dev = dev,
};
struct net_device_path *path;
int ret = 0;
memcpy(ctx.daddr, daddr, sizeof(ctx.daddr));
stack->num_paths = 0;
while (ctx.dev && ctx.dev->netdev_ops->ndo_fill_forward_path) {
last_dev = ctx.dev;
path = dev_fwd_path(stack);
if (!path)
return -1;
memset(path, 0, sizeof(struct net_device_path));
ret = ctx.dev->netdev_ops->ndo_fill_forward_path(&ctx, path);
if (ret < 0)
return -1;
if (WARN_ON_ONCE(last_dev == ctx.dev))
return -1;
}
if (!ctx.dev)
return ret;
path = dev_fwd_path(stack);
if (!path)
return -1;
path->type = DEV_PATH_ETHERNET;
path->dev = ctx.dev;
return ret;
}
EXPORT_SYMBOL_GPL(dev_fill_forward_path);
/* must be called under rcu_read_lock(), as we dont take a reference */
static struct napi_struct *napi_by_id(unsigned int napi_id)
{
unsigned int hash = napi_id % HASH_SIZE(napi_hash);
struct napi_struct *napi;
hlist_for_each_entry_rcu(napi, &napi_hash[hash], napi_hash_node)
if (napi->napi_id == napi_id)
return napi;
return NULL;
}
/* must be called under rcu_read_lock(), as we dont take a reference */
static struct napi_struct *
netdev_napi_by_id(struct net *net, unsigned int napi_id)
{
struct napi_struct *napi;
napi = napi_by_id(napi_id);
if (!napi)
return NULL;
if (WARN_ON_ONCE(!napi->dev))
return NULL;
if (!net_eq(net, dev_net(napi->dev)))
return NULL;
return napi;
}
/**
* netdev_napi_by_id_lock() - find a device by NAPI ID and lock it
* @net: the applicable net namespace
* @napi_id: ID of a NAPI of a target device
*
* Find a NAPI instance with @napi_id. Lock its device.
* The device must be in %NETREG_REGISTERED state for lookup to succeed.
* netdev_unlock() must be called to release it.
*
* Return: pointer to NAPI, its device with lock held, NULL if not found.
*/
struct napi_struct *
netdev_napi_by_id_lock(struct net *net, unsigned int napi_id)
{
struct napi_struct *napi;
struct net_device *dev;
rcu_read_lock();
napi = netdev_napi_by_id(net, napi_id);
if (!napi || READ_ONCE(napi->dev->reg_state) != NETREG_REGISTERED) {
rcu_read_unlock();
return NULL;
}
dev = napi->dev;
dev_hold(dev);
rcu_read_unlock();
dev = __netdev_put_lock(dev, net);
if (!dev)
return NULL;
rcu_read_lock();
napi = netdev_napi_by_id(net, napi_id);
if (napi && napi->dev != dev)
napi = NULL;
rcu_read_unlock();
if (!napi)
netdev_unlock(dev);
return napi;
}
/**
* __dev_get_by_name - find a device by its name
* @net: the applicable net namespace
* @name: name to find
*
* Find an interface by name. Must be called under RTNL semaphore.
* If the name is found a pointer to the device is returned.
* If the name is not found then %NULL is returned. The
* reference counters are not incremented so the caller must be
* careful with locks.
*/
struct net_device *__dev_get_by_name(struct net *net, const char *name)
{
struct netdev_name_node *node_name;
node_name = netdev_name_node_lookup(net, name);
return node_name ? node_name->dev : NULL;
}
EXPORT_SYMBOL(__dev_get_by_name);
/**
* dev_get_by_name_rcu - find a device by its name
* @net: the applicable net namespace
* @name: name to find
*
* Find an interface by name.
* If the name is found a pointer to the device is returned.
* If the name is not found then %NULL is returned.
* The reference counters are not incremented so the caller must be
* careful with locks. The caller must hold RCU lock.
*/
struct net_device *dev_get_by_name_rcu(struct net *net, const char *name)
{
struct netdev_name_node *node_name;
node_name = netdev_name_node_lookup_rcu(net, name);
return node_name ? node_name->dev : NULL;
}
EXPORT_SYMBOL(dev_get_by_name_rcu);
/* Deprecated for new users, call netdev_get_by_name() instead */
struct net_device *dev_get_by_name(struct net *net, const char *name)
{
struct net_device *dev;
rcu_read_lock();
dev = dev_get_by_name_rcu(net, name);
dev_hold(dev);
rcu_read_unlock();
return dev;
}
EXPORT_SYMBOL(dev_get_by_name);
/**
* netdev_get_by_name() - find a device by its name
* @net: the applicable net namespace
* @name: name to find
* @tracker: tracking object for the acquired reference
* @gfp: allocation flags for the tracker
*
* Find an interface by name. This can be called from any
* context and does its own locking. The returned handle has
* the usage count incremented and the caller must use netdev_put() to
* release it when it is no longer needed. %NULL is returned if no
* matching device is found.
*/
struct net_device *netdev_get_by_name(struct net *net, const char *name,
netdevice_tracker *tracker, gfp_t gfp)
{
struct net_device *dev;
dev = dev_get_by_name(net, name);
if (dev)
netdev_tracker_alloc(dev, tracker, gfp);
return dev;
}
EXPORT_SYMBOL(netdev_get_by_name);
/**
* __dev_get_by_index - find a device by its ifindex
* @net: the applicable net namespace
* @ifindex: index of device
*
* Search for an interface by index. Returns %NULL if the device
* is not found or a pointer to the device. The device has not
* had its reference counter increased so the caller must be careful
* about locking. The caller must hold the RTNL semaphore.
*/
struct net_device *__dev_get_by_index(struct net *net, int ifindex)
{
struct net_device *dev;
struct hlist_head *head = dev_index_hash(net, ifindex);
hlist_for_each_entry(dev, head, index_hlist)
if (dev->ifindex == ifindex)
return dev;
return NULL;
}
EXPORT_SYMBOL(__dev_get_by_index);
/**
* dev_get_by_index_rcu - find a device by its ifindex
* @net: the applicable net namespace
* @ifindex: index of device
*
* Search for an interface by index. Returns %NULL if the device
* is not found or a pointer to the device. The device has not
* had its reference counter increased so the caller must be careful
* about locking. The caller must hold RCU lock.
*/
struct net_device *dev_get_by_index_rcu(struct net *net, int ifindex)
{
struct net_device *dev;
struct hlist_head *head = dev_index_hash(net, ifindex);
hlist_for_each_entry_rcu(dev, head, index_hlist)
if (dev->ifindex == ifindex)
return dev;
return NULL;
}
EXPORT_SYMBOL(dev_get_by_index_rcu);
/* Deprecated for new users, call netdev_get_by_index() instead */
struct net_device *dev_get_by_index(struct net *net, int ifindex)
{
struct net_device *dev;
rcu_read_lock();
dev = dev_get_by_index_rcu(net, ifindex);
dev_hold(dev);
rcu_read_unlock();
return dev;
}
EXPORT_SYMBOL(dev_get_by_index);
/**
* netdev_get_by_index() - find a device by its ifindex
* @net: the applicable net namespace
* @ifindex: index of device
* @tracker: tracking object for the acquired reference
* @gfp: allocation flags for the tracker
*
* Search for an interface by index. Returns NULL if the device
* is not found or a pointer to the device. The device returned has
* had a reference added and the pointer is safe until the user calls
* netdev_put() to indicate they have finished with it.
*/
struct net_device *netdev_get_by_index(struct net *net, int ifindex,
netdevice_tracker *tracker, gfp_t gfp)
{
struct net_device *dev;
dev = dev_get_by_index(net, ifindex);
if (dev)
netdev_tracker_alloc(dev, tracker, gfp);
return dev;
}
EXPORT_SYMBOL(netdev_get_by_index);
/**
* dev_get_by_napi_id - find a device by napi_id
* @napi_id: ID of the NAPI struct
*
* Search for an interface by NAPI ID. Returns %NULL if the device
* is not found or a pointer to the device. The device has not had
* its reference counter increased so the caller must be careful
* about locking. The caller must hold RCU lock.
*/
struct net_device *dev_get_by_napi_id(unsigned int napi_id)
{
struct napi_struct *napi;
WARN_ON_ONCE(!rcu_read_lock_held());
if (!napi_id_valid(napi_id))
return NULL;
napi = napi_by_id(napi_id);
return napi ? napi->dev : NULL;
}
/* Release the held reference on the net_device, and if the net_device
* is still registered try to lock the instance lock. If device is being
* unregistered NULL will be returned (but the reference has been released,
* either way!)
*
* This helper is intended for locking net_device after it has been looked up
* using a lockless lookup helper. Lock prevents the instance from going away.
*/
struct net_device *__netdev_put_lock(struct net_device *dev, struct net *net)
{
netdev_lock(dev);
if (dev->reg_state > NETREG_REGISTERED ||
dev->moving_ns || !net_eq(dev_net(dev), net)) {
netdev_unlock(dev);
dev_put(dev);
return NULL;
}
dev_put(dev);
return dev;
}
static struct net_device *
__netdev_put_lock_ops_compat(struct net_device *dev, struct net *net)
{
netdev_lock_ops_compat(dev);
if (dev->reg_state > NETREG_REGISTERED ||
dev->moving_ns || !net_eq(dev_net(dev), net)) {
netdev_unlock_ops_compat(dev);
dev_put(dev);
return NULL;
}
dev_put(dev);
return dev;
}
/**
* netdev_get_by_index_lock() - find a device by its ifindex
* @net: the applicable net namespace
* @ifindex: index of device
*
* Search for an interface by index. If a valid device
* with @ifindex is found it will be returned with netdev->lock held.
* netdev_unlock() must be called to release it.
*
* Return: pointer to a device with lock held, NULL if not found.
*/
struct net_device *netdev_get_by_index_lock(struct net *net, int ifindex)
{
struct net_device *dev;
dev = dev_get_by_index(net, ifindex);
if (!dev)
return NULL;
return __netdev_put_lock(dev, net);
}
struct net_device *
netdev_get_by_index_lock_ops_compat(struct net *net, int ifindex)
{
struct net_device *dev;
dev = dev_get_by_index(net, ifindex);
if (!dev)
return NULL;
return __netdev_put_lock_ops_compat(dev, net);
}
struct net_device *
netdev_xa_find_lock(struct net *net, struct net_device *dev,
unsigned long *index)
{
if (dev)
netdev_unlock(dev);
do {
rcu_read_lock();
dev = xa_find(&net->dev_by_index, index, ULONG_MAX, XA_PRESENT);
if (!dev) {
rcu_read_unlock();
return NULL;
}
dev_hold(dev);
rcu_read_unlock();
dev = __netdev_put_lock(dev, net);
if (dev)
return dev;
(*index)++;
} while (true);
}
struct net_device *
netdev_xa_find_lock_ops_compat(struct net *net, struct net_device *dev,
unsigned long *index)
{
if (dev)
netdev_unlock_ops_compat(dev);
do {
rcu_read_lock();
dev = xa_find(&net->dev_by_index, index, ULONG_MAX, XA_PRESENT);
if (!dev) {
rcu_read_unlock();
return NULL;
}
dev_hold(dev);
rcu_read_unlock();
dev = __netdev_put_lock_ops_compat(dev, net);
if (dev)
return dev;
(*index)++;
} while (true);
}
static DEFINE_SEQLOCK(netdev_rename_lock);
void netdev_copy_name(struct net_device *dev, char *name)
{
unsigned int seq;
do {
seq = read_seqbegin(&netdev_rename_lock);
strscpy(name, dev->name, IFNAMSIZ);
} while (read_seqretry(&netdev_rename_lock, seq));
}
EXPORT_IPV6_MOD_GPL(netdev_copy_name);
/**
* netdev_get_name - get a netdevice name, knowing its ifindex.
* @net: network namespace
* @name: a pointer to the buffer where the name will be stored.
* @ifindex: the ifindex of the interface to get the name from.
*/
int netdev_get_name(struct net *net, char *name, int ifindex)
{
struct net_device *dev;
int ret;
rcu_read_lock();
dev = dev_get_by_index_rcu(net, ifindex);
if (!dev) {
ret = -ENODEV;
goto out;
}
netdev_copy_name(dev, name);
ret = 0;
out:
rcu_read_unlock();
return ret;
}
static bool dev_addr_cmp(struct net_device *dev, unsigned short type,
const char *ha)
{
return dev->type == type && !memcmp(dev->dev_addr, ha, dev->addr_len);
}
/**
* dev_getbyhwaddr_rcu - find a device by its hardware address
* @net: the applicable net namespace
* @type: media type of device
* @ha: hardware address
*
* Search for an interface by MAC address. Returns NULL if the device
* is not found or a pointer to the device.
* The caller must hold RCU.
* The returned device has not had its ref count increased
* and the caller must therefore be careful about locking
*
*/
struct net_device *dev_getbyhwaddr_rcu(struct net *net, unsigned short type,
const char *ha)
{
struct net_device *dev;
for_each_netdev_rcu(net, dev)
if (dev_addr_cmp(dev, type, ha))
return dev;
return NULL;
}
EXPORT_SYMBOL(dev_getbyhwaddr_rcu);
/**
* dev_getbyhwaddr() - find a device by its hardware address
* @net: the applicable net namespace
* @type: media type of device
* @ha: hardware address
*
* Similar to dev_getbyhwaddr_rcu(), but the owner needs to hold
* rtnl_lock.
*
* Context: rtnl_lock() must be held.
* Return: pointer to the net_device, or NULL if not found
*/
struct net_device *dev_getbyhwaddr(struct net *net, unsigned short type,
const char *ha)
{
struct net_device *dev;
ASSERT_RTNL();
for_each_netdev(net, dev)
if (dev_addr_cmp(dev, type, ha))
return dev;
return NULL;
}
EXPORT_SYMBOL(dev_getbyhwaddr);
struct net_device *dev_getfirstbyhwtype(struct net *net, unsigned short type)
{
struct net_device *dev, *ret = NULL;
rcu_read_lock();
for_each_netdev_rcu(net, dev)
if (dev->type == type) {
dev_hold(dev);
ret = dev;
break;
}
rcu_read_unlock();
return ret;
}
EXPORT_SYMBOL(dev_getfirstbyhwtype);
/**
* netdev_get_by_flags_rcu - find any device with given flags
* @net: the applicable net namespace
* @tracker: tracking object for the acquired reference
* @if_flags: IFF_* values
* @mask: bitmask of bits in if_flags to check
*
* Search for any interface with the given flags.
*
* Context: rcu_read_lock() must be held.
* Returns: NULL if a device is not found or a pointer to the device.
*/
struct net_device *netdev_get_by_flags_rcu(struct net *net, netdevice_tracker *tracker,
unsigned short if_flags, unsigned short mask)
{
struct net_device *dev;
for_each_netdev_rcu(net, dev) {
if (((READ_ONCE(dev->flags) ^ if_flags) & mask) == 0) {
netdev_hold(dev, tracker, GFP_ATOMIC);
return dev;
}
}
return NULL;
}
EXPORT_IPV6_MOD(netdev_get_by_flags_rcu);
/**
* dev_valid_name - check if name is okay for network device
* @name: name string
*
* Network device names need to be valid file names to
* allow sysfs to work. We also disallow any kind of
* whitespace.
*/
bool dev_valid_name(const char *name)
{
if (*name == '\0')
return false;
if (strnlen(name, IFNAMSIZ) == IFNAMSIZ)
return false;
if (!strcmp(name, ".") || !strcmp(name, ".."))
return false;
while (*name) {
if (*name == '/' || *name == ':' || isspace(*name))
return false;
name++;
}
return true;
}
EXPORT_SYMBOL(dev_valid_name);
/**
* __dev_alloc_name - allocate a name for a device
* @net: network namespace to allocate the device name in
* @name: name format string
* @res: result name string
*
* Passed a format string - eg "lt%d" it will try and find a suitable
* id. It scans list of devices to build up a free map, then chooses
* the first empty slot. The caller must hold the dev_base or rtnl lock
* while allocating the name and adding the device in order to avoid
* duplicates.
* Limited to bits_per_byte * page size devices (ie 32K on most platforms).
* Returns the number of the unit assigned or a negative errno code.
*/
static int __dev_alloc_name(struct net *net, const char *name, char *res)
{
int i = 0;
const char *p;
const int max_netdevices = 8*PAGE_SIZE;
unsigned long *inuse;
struct net_device *d;
char buf[IFNAMSIZ];
/* Verify the string as this thing may have come from the user.
* There must be one "%d" and no other "%" characters.
*/
p = strchr(name, '%');
if (!p || p[1] != 'd' || strchr(p + 2, '%'))
return -EINVAL;
/* Use one page as a bit array of possible slots */
inuse = bitmap_zalloc(max_netdevices, GFP_ATOMIC);
if (!inuse)
return -ENOMEM;
for_each_netdev(net, d) {
struct netdev_name_node *name_node;
netdev_for_each_altname(d, name_node) {
if (!sscanf(name_node->name, name, &i))
continue;
if (i < 0 || i >= max_netdevices)
continue;
/* avoid cases where sscanf is not exact inverse of printf */
snprintf(buf, IFNAMSIZ, name, i);
if (!strncmp(buf, name_node->name, IFNAMSIZ))
__set_bit(i, inuse);
}
if (!sscanf(d->name, name, &i))
continue;
if (i < 0 || i >= max_netdevices)
continue;
/* avoid cases where sscanf is not exact inverse of printf */
snprintf(buf, IFNAMSIZ, name, i);
if (!strncmp(buf, d->name, IFNAMSIZ))
__set_bit(i, inuse);
}
i = find_first_zero_bit(inuse, max_netdevices);
bitmap_free(inuse);
if (i == max_netdevices)
return -ENFILE;
/* 'res' and 'name' could overlap, use 'buf' as an intermediate buffer */
strscpy(buf, name, IFNAMSIZ);
snprintf(res, IFNAMSIZ, buf, i);
return i;
}
/* Returns negative errno or allocated unit id (see __dev_alloc_name()) */
static int dev_prep_valid_name(struct net *net, struct net_device *dev,
const char *want_name, char *out_name,
int dup_errno)
{
if (!dev_valid_name(want_name))
return -EINVAL;
if (strchr(want_name, '%'))
return __dev_alloc_name(net, want_name, out_name);
if (netdev_name_in_use(net, want_name))
return -dup_errno;
if (out_name != want_name)
strscpy(out_name, want_name, IFNAMSIZ);
return 0;
}
/**
* dev_alloc_name - allocate a name for a device
* @dev: device
* @name: name format string
*
* Passed a format string - eg "lt%d" it will try and find a suitable
* id. It scans list of devices to build up a free map, then chooses
* the first empty slot. The caller must hold the dev_base or rtnl lock
* while allocating the name and adding the device in order to avoid
* duplicates.
* Limited to bits_per_byte * page size devices (ie 32K on most platforms).
* Returns the number of the unit assigned or a negative errno code.
*/
int dev_alloc_name(struct net_device *dev, const char *name)
{
return dev_prep_valid_name(dev_net(dev), dev, name, dev->name, ENFILE);
}
EXPORT_SYMBOL(dev_alloc_name);
static int dev_get_valid_name(struct net *net, struct net_device *dev,
const char *name)
{
int ret;
ret = dev_prep_valid_name(net, dev, name, dev->name, EEXIST);
return ret < 0 ? ret : 0;
}
int netif_change_name(struct net_device *dev, const char *newname)
{
struct net *net = dev_net(dev);
unsigned char old_assign_type;
char oldname[IFNAMSIZ];
int err = 0;
int ret;
ASSERT_RTNL_NET(net);
if (!strncmp(newname, dev->name, IFNAMSIZ))
return 0;
memcpy(oldname, dev->name, IFNAMSIZ);
write_seqlock_bh(&netdev_rename_lock);
err = dev_get_valid_name(net, dev, newname);
write_sequnlock_bh(&netdev_rename_lock);
if (err < 0)
return err;
if (oldname[0] && !strchr(oldname, '%'))
netdev_info(dev, "renamed from %s%s\n", oldname,
dev->flags & IFF_UP ? " (while UP)" : "");
old_assign_type = dev->name_assign_type;
WRITE_ONCE(dev->name_assign_type, NET_NAME_RENAMED);
rollback:
ret = device_rename(&dev->dev, dev->name);
if (ret) {
write_seqlock_bh(&netdev_rename_lock);
memcpy(dev->name, oldname, IFNAMSIZ);
write_sequnlock_bh(&netdev_rename_lock);
WRITE_ONCE(dev->name_assign_type, old_assign_type);
return ret;
}
netdev_adjacent_rename_links(dev, oldname);
netdev_name_node_del(dev->name_node);
synchronize_net();
netdev_name_node_add(net, dev->name_node);
ret = call_netdevice_notifiers(NETDEV_CHANGENAME, dev);
ret = notifier_to_errno(ret);
if (ret) {
/* err >= 0 after dev_alloc_name() or stores the first errno */
if (err >= 0) {
err = ret;
write_seqlock_bh(&netdev_rename_lock);
memcpy(dev->name, oldname, IFNAMSIZ);
write_sequnlock_bh(&netdev_rename_lock);
memcpy(oldname, newname, IFNAMSIZ);
WRITE_ONCE(dev->name_assign_type, old_assign_type);
old_assign_type = NET_NAME_RENAMED;
goto rollback;
} else {
netdev_err(dev, "name change rollback failed: %d\n",
ret);
}
}
return err;
}
int netif_set_alias(struct net_device *dev, const char *alias, size_t len)
{
struct dev_ifalias *new_alias = NULL;
if (len >= IFALIASZ)
return -EINVAL;
if (len) {
new_alias = kmalloc(sizeof(*new_alias) + len + 1, GFP_KERNEL);
if (!new_alias)
return -ENOMEM;
memcpy(new_alias->ifalias, alias, len);
new_alias->ifalias[len] = 0;
}
mutex_lock(&ifalias_mutex);
new_alias = rcu_replace_pointer(dev->ifalias, new_alias,
mutex_is_locked(&ifalias_mutex));
mutex_unlock(&ifalias_mutex);
if (new_alias)
kfree_rcu(new_alias, rcuhead);
return len;
}
/**
* dev_get_alias - get ifalias of a device
* @dev: device
* @name: buffer to store name of ifalias
* @len: size of buffer
*
* get ifalias for a device. Caller must make sure dev cannot go
* away, e.g. rcu read lock or own a reference count to device.
*/
int dev_get_alias(const struct net_device *dev, char *name, size_t len)
{
const struct dev_ifalias *alias;
int ret = 0;
rcu_read_lock();
alias = rcu_dereference(dev->ifalias);
if (alias)
ret = snprintf(name, len, "%s", alias->ifalias);
rcu_read_unlock();
return ret;
}
/**
* netdev_features_change - device changes features
* @dev: device to cause notification
*
* Called to indicate a device has changed features.
*/
void netdev_features_change(struct net_device *dev)
{
call_netdevice_notifiers(NETDEV_FEAT_CHANGE, dev);
}
EXPORT_SYMBOL(netdev_features_change);
void netif_state_change(struct net_device *dev)
{
netdev_ops_assert_locked_or_invisible(dev);
if (dev->flags & IFF_UP) {
struct netdev_notifier_change_info change_info = {
.info.dev = dev,
};
call_netdevice_notifiers_info(NETDEV_CHANGE,
&change_info.info);
rtmsg_ifinfo(RTM_NEWLINK, dev, 0, GFP_KERNEL, 0, NULL);
}
}
/**
* __netdev_notify_peers - notify network peers about existence of @dev,
* to be called when rtnl lock is already held.
* @dev: network device
*
* Generate traffic such that interested network peers are aware of
* @dev, such as by generating a gratuitous ARP. This may be used when
* a device wants to inform the rest of the network about some sort of
* reconfiguration such as a failover event or virtual machine
* migration.
*/
void __netdev_notify_peers(struct net_device *dev)
{
ASSERT_RTNL();
call_netdevice_notifiers(NETDEV_NOTIFY_PEERS, dev);
call_netdevice_notifiers(NETDEV_RESEND_IGMP, dev);
}
EXPORT_SYMBOL(__netdev_notify_peers);
/**
* netdev_notify_peers - notify network peers about existence of @dev
* @dev: network device
*
* Generate traffic such that interested network peers are aware of
* @dev, such as by generating a gratuitous ARP. This may be used when
* a device wants to inform the rest of the network about some sort of
* reconfiguration such as a failover event or virtual machine
* migration.
*/
void netdev_notify_peers(struct net_device *dev)
{
rtnl_lock();
__netdev_notify_peers(dev);
rtnl_unlock();
}
EXPORT_SYMBOL(netdev_notify_peers);
static int napi_threaded_poll(void *data);
static int napi_kthread_create(struct napi_struct *n)
{
int err = 0;
/* Create and wake up the kthread once to put it in
* TASK_INTERRUPTIBLE mode to avoid the blocked task
* warning and work with loadavg.
*/
n->thread = kthread_run(napi_threaded_poll, n, "napi/%s-%d",
n->dev->name, n->napi_id);
if (IS_ERR(n->thread)) {
err = PTR_ERR(n->thread);
pr_err("kthread_run failed with err %d\n", err);
n->thread = NULL;
}
return err;
}
static int __dev_open(struct net_device *dev, struct netlink_ext_ack *extack)
{
const struct net_device_ops *ops = dev->netdev_ops;
int ret;
ASSERT_RTNL();
dev_addr_check(dev);
if (!netif_device_present(dev)) {
/* may be detached because parent is runtime-suspended */
if (dev->dev.parent)
pm_runtime_resume(dev->dev.parent);
if (!netif_device_present(dev))
return -ENODEV;
}
/* Block netpoll from trying to do any rx path servicing.
* If we don't do this there is a chance ndo_poll_controller
* or ndo_poll may be running while we open the device
*/
netpoll_poll_disable(dev);
ret = call_netdevice_notifiers_extack(NETDEV_PRE_UP, dev, extack);
ret = notifier_to_errno(ret);
if (ret)
return ret;
set_bit(__LINK_STATE_START, &dev->state);
netdev_ops_assert_locked(dev);
if (ops->ndo_validate_addr)
ret = ops->ndo_validate_addr(dev);
if (!ret && ops->ndo_open)
ret = ops->ndo_open(dev);
netpoll_poll_enable(dev);
if (ret)
clear_bit(__LINK_STATE_START, &dev->state);
else {
netif_set_up(dev, true);
dev_set_rx_mode(dev);
dev_activate(dev);
add_device_randomness(dev->dev_addr, dev->addr_len);
}
return ret;
}
int netif_open(struct net_device *dev, struct netlink_ext_ack *extack)
{
int ret;
if (dev->flags & IFF_UP)
return 0;
ret = __dev_open(dev, extack);
if (ret < 0)
return ret;
rtmsg_ifinfo(RTM_NEWLINK, dev, IFF_UP | IFF_RUNNING, GFP_KERNEL, 0, NULL);
call_netdevice_notifiers(NETDEV_UP, dev);
return ret;
}
static void __dev_close_many(struct list_head *head)
{
struct net_device *dev;
ASSERT_RTNL();
might_sleep();
list_for_each_entry(dev, head, close_list) {
/* Temporarily disable netpoll until the interface is down */
netpoll_poll_disable(dev);
call_netdevice_notifiers(NETDEV_GOING_DOWN, dev);
clear_bit(__LINK_STATE_START, &dev->state);
/* Synchronize to scheduled poll. We cannot touch poll list, it
* can be even on different cpu. So just clear netif_running().
*
* dev->stop() will invoke napi_disable() on all of it's
* napi_struct instances on this device.
*/
smp_mb__after_atomic(); /* Commit netif_running(). */
}
dev_deactivate_many(head);
list_for_each_entry(dev, head, close_list) {
const struct net_device_ops *ops = dev->netdev_ops;
/*
* Call the device specific close. This cannot fail.
* Only if device is UP
*
* We allow it to be called even after a DETACH hot-plug
* event.
*/
netdev_ops_assert_locked(dev);
if (ops->ndo_stop)
ops->ndo_stop(dev);
netif_set_up(dev, false);
netpoll_poll_enable(dev);
}
}
static void __dev_close(struct net_device *dev)
{
LIST_HEAD(single);
list_add(&dev->close_list, &single);
__dev_close_many(&single);
list_del(&single);
}
void netif_close_many(struct list_head *head, bool unlink)
{
struct net_device *dev, *tmp;
/* Remove the devices that don't need to be closed */
list_for_each_entry_safe(dev, tmp, head, close_list)
if (!(dev->flags & IFF_UP))
list_del_init(&dev->close_list);
__dev_close_many(head);
list_for_each_entry_safe(dev, tmp, head, close_list) {
rtmsg_ifinfo(RTM_NEWLINK, dev, IFF_UP | IFF_RUNNING, GFP_KERNEL, 0, NULL);
call_netdevice_notifiers(NETDEV_DOWN, dev);
if (unlink)
list_del_init(&dev->close_list);
}
}
EXPORT_SYMBOL_NS_GPL(netif_close_many, "NETDEV_INTERNAL");
void netif_close(struct net_device *dev)
{
if (dev->flags & IFF_UP) {
LIST_HEAD(single);
list_add(&dev->close_list, &single);
netif_close_many(&single, true);
list_del(&single);
}
}
EXPORT_SYMBOL(netif_close);
void netif_disable_lro(struct net_device *dev)
{
struct net_device *lower_dev;
struct list_head *iter;
dev->wanted_features &= ~NETIF_F_LRO;
netdev_update_features(dev);
if (unlikely(dev->features & NETIF_F_LRO))
netdev_WARN(dev, "failed to disable LRO!\n");
netdev_for_each_lower_dev(dev, lower_dev, iter) {
netdev_lock_ops(lower_dev);
netif_disable_lro(lower_dev);
netdev_unlock_ops(lower_dev);
}
}
EXPORT_IPV6_MOD(netif_disable_lro);
/**
* dev_disable_gro_hw - disable HW Generic Receive Offload on a device
* @dev: device
*
* Disable HW Generic Receive Offload (GRO_HW) on a net device. Must be
* called under RTNL. This is needed if Generic XDP is installed on
* the device.
*/
static void dev_disable_gro_hw(struct net_device *dev)
{
dev->wanted_features &= ~NETIF_F_GRO_HW;
netdev_update_features(dev);
if (unlikely(dev->features & NETIF_F_GRO_HW))
netdev_WARN(dev, "failed to disable GRO_HW!\n");
}
const char *netdev_cmd_to_name(enum netdev_cmd cmd)
{
#define N(val) \
case NETDEV_##val: \
return "NETDEV_" __stringify(val);
switch (cmd) {
N(UP) N(DOWN) N(REBOOT) N(CHANGE) N(REGISTER) N(UNREGISTER)
N(CHANGEMTU) N(CHANGEADDR) N(GOING_DOWN) N(CHANGENAME) N(FEAT_CHANGE)
N(BONDING_FAILOVER) N(PRE_UP) N(PRE_TYPE_CHANGE) N(POST_TYPE_CHANGE)
N(POST_INIT) N(PRE_UNINIT) N(RELEASE) N(NOTIFY_PEERS) N(JOIN)
N(CHANGEUPPER) N(RESEND_IGMP) N(PRECHANGEMTU) N(CHANGEINFODATA)
N(BONDING_INFO) N(PRECHANGEUPPER) N(CHANGELOWERSTATE)
N(UDP_TUNNEL_PUSH_INFO) N(UDP_TUNNEL_DROP_INFO) N(CHANGE_TX_QUEUE_LEN)
N(CVLAN_FILTER_PUSH_INFO) N(CVLAN_FILTER_DROP_INFO)
N(SVLAN_FILTER_PUSH_INFO) N(SVLAN_FILTER_DROP_INFO)
N(PRE_CHANGEADDR) N(OFFLOAD_XSTATS_ENABLE) N(OFFLOAD_XSTATS_DISABLE)
N(OFFLOAD_XSTATS_REPORT_USED) N(OFFLOAD_XSTATS_REPORT_DELTA)
N(XDP_FEAT_CHANGE)
}
#undef N
return "UNKNOWN_NETDEV_EVENT";
}
EXPORT_SYMBOL_GPL(netdev_cmd_to_name);
static int call_netdevice_notifier(struct notifier_block *nb, unsigned long val,
struct net_device *dev)
{
struct netdev_notifier_info info = {
.dev = dev,
};
return nb->notifier_call(nb, val, &info);
}
static int call_netdevice_register_notifiers(struct notifier_block *nb,
struct net_device *dev)
{
int err;
err = call_netdevice_notifier(nb, NETDEV_REGISTER, dev);
err = notifier_to_errno(err);
if (err)
return err;
if (!(dev->flags & IFF_UP))
return 0;
call_netdevice_notifier(nb, NETDEV_UP, dev);
return 0;
}
static void call_netdevice_unregister_notifiers(struct notifier_block *nb,
struct net_device *dev)
{
if (dev->flags & IFF_UP) {
call_netdevice_notifier(nb, NETDEV_GOING_DOWN,
dev);
call_netdevice_notifier(nb, NETDEV_DOWN, dev);
}
call_netdevice_notifier(nb, NETDEV_UNREGISTER, dev);
}
static int call_netdevice_register_net_notifiers(struct notifier_block *nb,
struct net *net)
{
struct net_device *dev;
int err;
for_each_netdev(net, dev) {
netdev_lock_ops(dev);
err = call_netdevice_register_notifiers(nb, dev);
netdev_unlock_ops(dev);
if (err)
goto rollback;
}
return 0;
rollback:
for_each_netdev_continue_reverse(net, dev)
call_netdevice_unregister_notifiers(nb, dev);
return err;
}
static void call_netdevice_unregister_net_notifiers(struct notifier_block *nb,
struct net *net)
{
struct net_device *dev;
for_each_netdev(net, dev)
call_netdevice_unregister_notifiers(nb, dev);
}
static int dev_boot_phase = 1;
/**
* register_netdevice_notifier - register a network notifier block
* @nb: notifier
*
* Register a notifier to be called when network device events occur.
* The notifier passed is linked into the kernel structures and must
* not be reused until it has been unregistered. A negative errno code
* is returned on a failure.
*
* When registered all registration and up events are replayed
* to the new notifier to allow device to have a race free
* view of the network device list.
*/
int register_netdevice_notifier(struct notifier_block *nb)
{
struct net *net;
int err;
/* Close race with setup_net() and cleanup_net() */
down_write(&pernet_ops_rwsem);
/* When RTNL is removed, we need protection for netdev_chain. */
rtnl_lock();
err = raw_notifier_chain_register(&netdev_chain, nb);
if (err)
goto unlock;
if (dev_boot_phase)
goto unlock;
for_each_net(net) {
__rtnl_net_lock(net);
err = call_netdevice_register_net_notifiers(nb, net);
__rtnl_net_unlock(net);
if (err)
goto rollback;
}
unlock:
rtnl_unlock();
up_write(&pernet_ops_rwsem);
return err;
rollback:
for_each_net_continue_reverse(net) {
__rtnl_net_lock(net);
call_netdevice_unregister_net_notifiers(nb, net);
__rtnl_net_unlock(net);
}
raw_notifier_chain_unregister(&netdev_chain, nb);
goto unlock;
}
EXPORT_SYMBOL(register_netdevice_notifier);
/**
* unregister_netdevice_notifier - unregister a network notifier block
* @nb: notifier
*
* Unregister a notifier previously registered by
* register_netdevice_notifier(). The notifier is unlinked into the
* kernel structures and may then be reused. A negative errno code
* is returned on a failure.
*
* After unregistering unregister and down device events are synthesized
* for all devices on the device list to the removed notifier to remove
* the need for special case cleanup code.
*/
int unregister_netdevice_notifier(struct notifier_block *nb)
{
struct net *net;
int err;
/* Close race with setup_net() and cleanup_net() */
down_write(&pernet_ops_rwsem);
rtnl_lock();
err = raw_notifier_chain_unregister(&netdev_chain, nb);
if (err)
goto unlock;
for_each_net(net) {
__rtnl_net_lock(net);
call_netdevice_unregister_net_notifiers(nb, net);
__rtnl_net_unlock(net);
}
unlock:
rtnl_unlock();
up_write(&pernet_ops_rwsem);
return err;
}
EXPORT_SYMBOL(unregister_netdevice_notifier);
static int __register_netdevice_notifier_net(struct net *net,
struct notifier_block *nb,
bool ignore_call_fail)
{
int err;
err = raw_notifier_chain_register(&net->netdev_chain, nb);
if (err)
return err;
if (dev_boot_phase)
return 0;
err = call_netdevice_register_net_notifiers(nb, net);
if (err && !ignore_call_fail)
goto chain_unregister;
return 0;
chain_unregister:
raw_notifier_chain_unregister(&net->netdev_chain, nb);
return err;
}
static int __unregister_netdevice_notifier_net(struct net *net,
struct notifier_block *nb)
{
int err;
err = raw_notifier_chain_unregister(&net->netdev_chain, nb);
if (err)
return err;
call_netdevice_unregister_net_notifiers(nb, net);
return 0;
}
/**
* register_netdevice_notifier_net - register a per-netns network notifier block
* @net: network namespace
* @nb: notifier
*
* Register a notifier to be called when network device events occur.
* The notifier passed is linked into the kernel structures and must
* not be reused until it has been unregistered. A negative errno code
* is returned on a failure.
*
* When registered all registration and up events are replayed
* to the new notifier to allow device to have a race free
* view of the network device list.
*/
int register_netdevice_notifier_net(struct net *net, struct notifier_block *nb)
{
int err;
rtnl_net_lock(net);
err = __register_netdevice_notifier_net(net, nb, false);
rtnl_net_unlock(net);
return err;
}
EXPORT_SYMBOL(register_netdevice_notifier_net);
/**
* unregister_netdevice_notifier_net - unregister a per-netns
* network notifier block
* @net: network namespace
* @nb: notifier
*
* Unregister a notifier previously registered by
* register_netdevice_notifier_net(). The notifier is unlinked from the
* kernel structures and may then be reused. A negative errno code
* is returned on a failure.
*
* After unregistering unregister and down device events are synthesized
* for all devices on the device list to the removed notifier to remove
* the need for special case cleanup code.
*/
int unregister_netdevice_notifier_net(struct net *net,
struct notifier_block *nb)
{
int err;
rtnl_net_lock(net);
err = __unregister_netdevice_notifier_net(net, nb);
rtnl_net_unlock(net);
return err;
}
EXPORT_SYMBOL(unregister_netdevice_notifier_net);
static void __move_netdevice_notifier_net(struct net *src_net,
struct net *dst_net,
struct notifier_block *nb)
{
__unregister_netdevice_notifier_net(src_net, nb);
__register_netdevice_notifier_net(dst_net, nb, true);
}
static void rtnl_net_dev_lock(struct net_device *dev)
{
bool again;
do {
struct net *net;
again = false;
/* netns might be being dismantled. */
rcu_read_lock();
net = dev_net_rcu(dev);
net_passive_inc(net);
rcu_read_unlock();
rtnl_net_lock(net);
#ifdef CONFIG_NET_NS
/* dev might have been moved to another netns. */
if (!net_eq(net, rcu_access_pointer(dev->nd_net.net))) {
rtnl_net_unlock(net);
net_passive_dec(net);
again = true;
}
#endif
} while (again);
}
static void rtnl_net_dev_unlock(struct net_device *dev)
{
struct net *net = dev_net(dev);
rtnl_net_unlock(net);
net_passive_dec(net);
}
int register_netdevice_notifier_dev_net(struct net_device *dev,
struct notifier_block *nb,
struct netdev_net_notifier *nn)
{
int err;
rtnl_net_dev_lock(dev);
err = __register_netdevice_notifier_net(dev_net(dev), nb, false);
if (!err) {
nn->nb = nb;
list_add(&nn->list, &dev->net_notifier_list);
}
rtnl_net_dev_unlock(dev);
return err;
}
EXPORT_SYMBOL(register_netdevice_notifier_dev_net);
int unregister_netdevice_notifier_dev_net(struct net_device *dev,
struct notifier_block *nb,
struct netdev_net_notifier *nn)
{
int err;
rtnl_net_dev_lock(dev);
list_del(&nn->list);
err = __unregister_netdevice_notifier_net(dev_net(dev), nb);
rtnl_net_dev_unlock(dev);
return err;
}
EXPORT_SYMBOL(unregister_netdevice_notifier_dev_net);
static void move_netdevice_notifiers_dev_net(struct net_device *dev,
struct net *net)
{
struct netdev_net_notifier *nn;
list_for_each_entry(nn, &dev->net_notifier_list, list)
__move_netdevice_notifier_net(dev_net(dev), net, nn->nb);
}
/**
* call_netdevice_notifiers_info - call all network notifier blocks
* @val: value passed unmodified to notifier function
* @info: notifier information data
*
* Call all network notifier blocks. Parameters and return value
* are as for raw_notifier_call_chain().
*/
int call_netdevice_notifiers_info(unsigned long val,
struct netdev_notifier_info *info)
{
struct net *net = dev_net(info->dev);
int ret;
ASSERT_RTNL();
/* Run per-netns notifier block chain first, then run the global one.
* Hopefully, one day, the global one is going to be removed after
* all notifier block registrators get converted to be per-netns.
*/
ret = raw_notifier_call_chain(&net->netdev_chain, val, info);
if (ret & NOTIFY_STOP_MASK)
return ret;
return raw_notifier_call_chain(&netdev_chain, val, info);
}
/**
* call_netdevice_notifiers_info_robust - call per-netns notifier blocks
* for and rollback on error
* @val_up: value passed unmodified to notifier function
* @val_down: value passed unmodified to the notifier function when
* recovering from an error on @val_up
* @info: notifier information data
*
* Call all per-netns network notifier blocks, but not notifier blocks on
* the global notifier chain. Parameters and return value are as for
* raw_notifier_call_chain_robust().
*/
static int
call_netdevice_notifiers_info_robust(unsigned long val_up,
unsigned long val_down,
struct netdev_notifier_info *info)
{
struct net *net = dev_net(info->dev);
ASSERT_RTNL();
return raw_notifier_call_chain_robust(&net->netdev_chain,
val_up, val_down, info);
}
static int call_netdevice_notifiers_extack(unsigned long val,
struct net_device *dev,
struct netlink_ext_ack *extack)
{
struct netdev_notifier_info info = {
.dev = dev,
.extack = extack,
};
return call_netdevice_notifiers_info(val, &info);
}
/**
* call_netdevice_notifiers - call all network notifier blocks
* @val: value passed unmodified to notifier function
* @dev: net_device pointer passed unmodified to notifier function
*
* Call all network notifier blocks. Parameters and return value
* are as for raw_notifier_call_chain().
*/
int call_netdevice_notifiers(unsigned long val, struct net_device *dev)
{
return call_netdevice_notifiers_extack(val, dev, NULL);
}
EXPORT_SYMBOL(call_netdevice_notifiers);
/**
* call_netdevice_notifiers_mtu - call all network notifier blocks
* @val: value passed unmodified to notifier function
* @dev: net_device pointer passed unmodified to notifier function
* @arg: additional u32 argument passed to the notifier function
*
* Call all network notifier blocks. Parameters and return value
* are as for raw_notifier_call_chain().
*/
static int call_netdevice_notifiers_mtu(unsigned long val,
struct net_device *dev, u32 arg)
{
struct netdev_notifier_info_ext info = {
.info.dev = dev,
.ext.mtu = arg,
};
BUILD_BUG_ON(offsetof(struct netdev_notifier_info_ext, info) != 0);
return call_netdevice_notifiers_info(val, &info.info);
}
#ifdef CONFIG_NET_INGRESS
static DEFINE_STATIC_KEY_FALSE(ingress_needed_key);
void net_inc_ingress_queue(void)
{
static_branch_inc(&ingress_needed_key);
}
EXPORT_SYMBOL_GPL(net_inc_ingress_queue);
void net_dec_ingress_queue(void)
{
static_branch_dec(&ingress_needed_key);
}
EXPORT_SYMBOL_GPL(net_dec_ingress_queue);
#endif
#ifdef CONFIG_NET_EGRESS
static DEFINE_STATIC_KEY_FALSE(egress_needed_key);
void net_inc_egress_queue(void)
{
static_branch_inc(&egress_needed_key);
}
EXPORT_SYMBOL_GPL(net_inc_egress_queue);
void net_dec_egress_queue(void)
{
static_branch_dec(&egress_needed_key);
}
EXPORT_SYMBOL_GPL(net_dec_egress_queue);
#endif
#ifdef CONFIG_NET_CLS_ACT
DEFINE_STATIC_KEY_FALSE(tcf_sw_enabled_key);
EXPORT_SYMBOL(tcf_sw_enabled_key);
#endif
DEFINE_STATIC_KEY_FALSE(netstamp_needed_key);
EXPORT_SYMBOL(netstamp_needed_key);
#ifdef CONFIG_JUMP_LABEL
static atomic_t netstamp_needed_deferred;
static atomic_t netstamp_wanted;
static void netstamp_clear(struct work_struct *work)
{
int deferred = atomic_xchg(&netstamp_needed_deferred, 0);
int wanted;
wanted = atomic_add_return(deferred, &netstamp_wanted);
if (wanted > 0)
static_branch_enable(&netstamp_needed_key);
else
static_branch_disable(&netstamp_needed_key);
}
static DECLARE_WORK(netstamp_work, netstamp_clear);
#endif
void net_enable_timestamp(void)
{
#ifdef CONFIG_JUMP_LABEL
int wanted = atomic_read(&netstamp_wanted);
while (wanted > 0) {
if (atomic_try_cmpxchg(&netstamp_wanted, &wanted, wanted + 1))
return;
}
atomic_inc(&netstamp_needed_deferred);
schedule_work(&netstamp_work);
#else
static_branch_inc(&netstamp_needed_key);
#endif
}
EXPORT_SYMBOL(net_enable_timestamp);
void net_disable_timestamp(void)
{
#ifdef CONFIG_JUMP_LABEL
int wanted = atomic_read(&netstamp_wanted);
while (wanted > 1) {
if (atomic_try_cmpxchg(&netstamp_wanted, &wanted, wanted - 1))
return;
}
atomic_dec(&netstamp_needed_deferred);
schedule_work(&netstamp_work);
#else
static_branch_dec(&netstamp_needed_key);
#endif
}
EXPORT_SYMBOL(net_disable_timestamp);
static inline void net_timestamp_set(struct sk_buff *skb)
{
skb->tstamp = 0;
skb->tstamp_type = SKB_CLOCK_REALTIME;
if (static_branch_unlikely(&netstamp_needed_key))
skb->tstamp = ktime_get_real();
}
#define net_timestamp_check(COND, SKB) \
if (static_branch_unlikely(&netstamp_needed_key)) { \
if ((COND) && !(SKB)->tstamp) \
(SKB)->tstamp = ktime_get_real(); \
} \
bool is_skb_forwardable(const struct net_device *dev, const struct sk_buff *skb)
{
return __is_skb_forwardable(dev, skb, true);
}
EXPORT_SYMBOL_GPL(is_skb_forwardable);
static int __dev_forward_skb2(struct net_device *dev, struct sk_buff *skb,
bool check_mtu)
{
int ret = ____dev_forward_skb(dev, skb, check_mtu);
if (likely(!ret)) {
skb->protocol = eth_type_trans(skb, dev);
skb_postpull_rcsum(skb, eth_hdr(skb), ETH_HLEN);
}
return ret;
}
int __dev_forward_skb(struct net_device *dev, struct sk_buff *skb)
{
return __dev_forward_skb2(dev, skb, true);
}
EXPORT_SYMBOL_GPL(__dev_forward_skb);
/**
* dev_forward_skb - loopback an skb to another netif
*
* @dev: destination network device
* @skb: buffer to forward
*
* return values:
* NET_RX_SUCCESS (no congestion)
* NET_RX_DROP (packet was dropped, but freed)
*
* dev_forward_skb can be used for injecting an skb from the
* start_xmit function of one device into the receive queue
* of another device.
*
* The receiving device may be in another namespace, so
* we have to clear all information in the skb that could
* impact namespace isolation.
*/
int dev_forward_skb(struct net_device *dev, struct sk_buff *skb)
{
return __dev_forward_skb(dev, skb) ?: netif_rx_internal(skb);
}
EXPORT_SYMBOL_GPL(dev_forward_skb);
int dev_forward_skb_nomtu(struct net_device *dev, struct sk_buff *skb)
{
return __dev_forward_skb2(dev, skb, false) ?: netif_rx_internal(skb);
}
static int deliver_skb(struct sk_buff *skb,
struct packet_type *pt_prev,
struct net_device *orig_dev)
{
if (unlikely(skb_orphan_frags_rx(skb, GFP_ATOMIC)))
return -ENOMEM;
refcount_inc(&skb->users);
return pt_prev->func(skb, skb->dev, pt_prev, orig_dev);
}
static inline void deliver_ptype_list_skb(struct sk_buff *skb,
struct packet_type **pt,
struct net_device *orig_dev,
__be16 type,
struct list_head *ptype_list)
{
struct packet_type *ptype, *pt_prev = *pt;
list_for_each_entry_rcu(ptype, ptype_list, list) {
if (ptype->type != type)
continue;
if (unlikely(pt_prev))
deliver_skb(skb, pt_prev, orig_dev);
pt_prev = ptype;
}
*pt = pt_prev;
}
static inline bool skb_loop_sk(struct packet_type *ptype, struct sk_buff *skb)
{
if (!ptype->af_packet_priv || !skb->sk)
return false;
if (ptype->id_match)
return ptype->id_match(ptype, skb->sk);
else if ((struct sock *)ptype->af_packet_priv == skb->sk)
return true;
return false;
}
/**
* dev_nit_active_rcu - return true if any network interface taps are in use
*
* The caller must hold the RCU lock
*
* @dev: network device to check for the presence of taps
*/
bool dev_nit_active_rcu(const struct net_device *dev)
{
/* Callers may hold either RCU or RCU BH lock */
WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held());
return !list_empty(&dev_net(dev)->ptype_all) ||
!list_empty(&dev->ptype_all);
}
EXPORT_SYMBOL_GPL(dev_nit_active_rcu);
/*
* Support routine. Sends outgoing frames to any network
* taps currently in use.
*/
void dev_queue_xmit_nit(struct sk_buff *skb, struct net_device *dev)
{
struct packet_type *ptype, *pt_prev = NULL;
struct list_head *ptype_list;
struct sk_buff *skb2 = NULL;
rcu_read_lock();
ptype_list = &dev_net_rcu(dev)->ptype_all;
again:
list_for_each_entry_rcu(ptype, ptype_list, list) {
if (READ_ONCE(ptype->ignore_outgoing))
continue;
/* Never send packets back to the socket
* they originated from - MvS (miquels@drinkel.ow.org)
*/
if (skb_loop_sk(ptype, skb))
continue;
if (unlikely(pt_prev)) {
deliver_skb(skb2, pt_prev, skb->dev);
pt_prev = ptype;
continue;
}
/* need to clone skb, done only once */
skb2 = skb_clone(skb, GFP_ATOMIC);
if (!skb2)
goto out_unlock;
net_timestamp_set(skb2);
/* skb->nh should be correctly
* set by sender, so that the second statement is
* just protection against buggy protocols.
*/
skb_reset_mac_header(skb2);
if (skb_network_header(skb2) < skb2->data ||
skb_network_header(skb2) > skb_tail_pointer(skb2)) {
net_crit_ratelimited("protocol %04x is buggy, dev %s\n",
ntohs(skb2->protocol),
dev->name);
skb_reset_network_header(skb2);
}
skb2->transport_header = skb2->network_header;
skb2->pkt_type = PACKET_OUTGOING;
pt_prev = ptype;
}
if (ptype_list != &dev->ptype_all) {
ptype_list = &dev->ptype_all;
goto again;
}
out_unlock:
if (pt_prev) {
if (!skb_orphan_frags_rx(skb2, GFP_ATOMIC))
pt_prev->func(skb2, skb->dev, pt_prev, skb->dev);
else
kfree_skb(skb2);
}
rcu_read_unlock();
}
EXPORT_SYMBOL_GPL(dev_queue_xmit_nit);
/**
* netif_setup_tc - Handle tc mappings on real_num_tx_queues change
* @dev: Network device
* @txq: number of queues available
*
* If real_num_tx_queues is changed the tc mappings may no longer be
* valid. To resolve this verify the tc mapping remains valid and if
* not NULL the mapping. With no priorities mapping to this
* offset/count pair it will no longer be used. In the worst case TC0
* is invalid nothing can be done so disable priority mappings. If is
* expected that drivers will fix this mapping if they can before
* calling netif_set_real_num_tx_queues.
*/
static void netif_setup_tc(struct net_device *dev, unsigned int txq)
{
int i;
struct netdev_tc_txq *tc = &dev->tc_to_txq[0];
/* If TC0 is invalidated disable TC mapping */
if (tc->offset + tc->count > txq) {
netdev_warn(dev, "Number of in use tx queues changed invalidating tc mappings. Priority traffic classification disabled!\n");
dev->num_tc = 0;
return;
}
/* Invalidated prio to tc mappings set to TC0 */
for (i = 1; i < TC_BITMASK + 1; i++) {
int q = netdev_get_prio_tc_map(dev, i);
tc = &dev->tc_to_txq[q];
if (tc->offset + tc->count > txq) {
netdev_warn(dev, "Number of in use tx queues changed. Priority %i to tc mapping %i is no longer valid. Setting map to 0\n",
i, q);
netdev_set_prio_tc_map(dev, i, 0);
}
}
}
int netdev_txq_to_tc(struct net_device *dev, unsigned int txq)
{
if (dev->num_tc) {
struct netdev_tc_txq *tc = &dev->tc_to_txq[0];
int i;
/* walk through the TCs and see if it falls into any of them */
for (i = 0; i < TC_MAX_QUEUE; i++, tc++) {
if ((txq - tc->offset) < tc->count)
return i;
}
/* didn't find it, just return -1 to indicate no match */
return -1;
}
return 0;
}
EXPORT_SYMBOL(netdev_txq_to_tc);
#ifdef CONFIG_XPS
static struct static_key xps_needed __read_mostly;
static struct static_key xps_rxqs_needed __read_mostly;
static DEFINE_MUTEX(xps_map_mutex);
#define xmap_dereference(P) \
rcu_dereference_protected((P), lockdep_is_held(&xps_map_mutex))
static bool remove_xps_queue(struct xps_dev_maps *dev_maps,
struct xps_dev_maps *old_maps, int tci, u16 index)
{
struct xps_map *map = NULL;
int pos;
map = xmap_dereference(dev_maps->attr_map[tci]);
if (!map)
return false;
for (pos = map->len; pos--;) {
if (map->queues[pos] != index)
continue;
if (map->len > 1) {
map->queues[pos] = map->queues[--map->len];
break;
}
if (old_maps)
RCU_INIT_POINTER(old_maps->attr_map[tci], NULL);
RCU_INIT_POINTER(dev_maps->attr_map[tci], NULL);
kfree_rcu(map, rcu);
return false;
}
return true;
}
static bool remove_xps_queue_cpu(struct net_device *dev,
struct xps_dev_maps *dev_maps,
int cpu, u16 offset, u16 count)
{
int num_tc = dev_maps->num_tc;
bool active = false;
int tci;
for (tci = cpu * num_tc; num_tc--; tci++) {
int i, j;
for (i = count, j = offset; i--; j++) {
if (!remove_xps_queue(dev_maps, NULL, tci, j))
break;
}
active |= i < 0;
}
return active;
}
static void reset_xps_maps(struct net_device *dev,
struct xps_dev_maps *dev_maps,
enum xps_map_type type)
{
static_key_slow_dec_cpuslocked(&xps_needed);
if (type == XPS_RXQS)
static_key_slow_dec_cpuslocked(&xps_rxqs_needed);
RCU_INIT_POINTER(dev->xps_maps[type], NULL);
kfree_rcu(dev_maps, rcu);
}
static void clean_xps_maps(struct net_device *dev, enum xps_map_type type,
u16 offset, u16 count)
{
struct xps_dev_maps *dev_maps;
bool active = false;
int i, j;
dev_maps = xmap_dereference(dev->xps_maps[type]);
if (!dev_maps)
return;
for (j = 0; j < dev_maps->nr_ids; j++)
active |= remove_xps_queue_cpu(dev, dev_maps, j, offset, count);
if (!active)
reset_xps_maps(dev, dev_maps, type);
if (type == XPS_CPUS) {
for (i = offset + (count - 1); count--; i--)
netdev_queue_numa_node_write(
netdev_get_tx_queue(dev, i), NUMA_NO_NODE);
}
}
static void netif_reset_xps_queues(struct net_device *dev, u16 offset,
u16 count)
{
if (!static_key_false(&xps_needed))
return;
cpus_read_lock();
mutex_lock(&xps_map_mutex);
if (static_key_false(&xps_rxqs_needed))
clean_xps_maps(dev, XPS_RXQS, offset, count);
clean_xps_maps(dev, XPS_CPUS, offset, count);
mutex_unlock(&xps_map_mutex);
cpus_read_unlock();
}
static void netif_reset_xps_queues_gt(struct net_device *dev, u16 index)
{
netif_reset_xps_queues(dev, index, dev->num_tx_queues - index);
}
static struct xps_map *expand_xps_map(struct xps_map *map, int attr_index,
u16 index, bool is_rxqs_map)
{
struct xps_map *new_map;
int alloc_len = XPS_MIN_MAP_ALLOC;
int i, pos;
for (pos = 0; map && pos < map->len; pos++) {
if (map->queues[pos] != index)
continue;
return map;
}
/* Need to add tx-queue to this CPU's/rx-queue's existing map */
if (map) {
if (pos < map->alloc_len)
return map;
alloc_len = map->alloc_len * 2;
}
/* Need to allocate new map to store tx-queue on this CPU's/rx-queue's
* map
*/
if (is_rxqs_map)
new_map = kzalloc(XPS_MAP_SIZE(alloc_len), GFP_KERNEL);
else
new_map = kzalloc_node(XPS_MAP_SIZE(alloc_len), GFP_KERNEL,
cpu_to_node(attr_index));
if (!new_map)
return NULL;
for (i = 0; i < pos; i++)
new_map->queues[i] = map->queues[i];
new_map->alloc_len = alloc_len;
new_map->len = pos;
return new_map;
}
/* Copy xps maps at a given index */
static void xps_copy_dev_maps(struct xps_dev_maps *dev_maps,
struct xps_dev_maps *new_dev_maps, int index,
int tc, bool skip_tc)
{
int i, tci = index * dev_maps->num_tc;
struct xps_map *map;
/* copy maps belonging to foreign traffic classes */
for (i = 0; i < dev_maps->num_tc; i++, tci++) {
if (i == tc && skip_tc)
continue;
/* fill in the new device map from the old device map */
map = xmap_dereference(dev_maps->attr_map[tci]);
RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map);
}
}
/* Must be called under cpus_read_lock */
int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask,
u16 index, enum xps_map_type type)
{
struct xps_dev_maps *dev_maps, *new_dev_maps = NULL, *old_dev_maps = NULL;
const unsigned long *online_mask = NULL;
bool active = false, copy = false;
int i, j, tci, numa_node_id = -2;
int maps_sz, num_tc = 1, tc = 0;
struct xps_map *map, *new_map;
unsigned int nr_ids;
WARN_ON_ONCE(index >= dev->num_tx_queues);
if (dev->num_tc) {
/* Do not allow XPS on subordinate device directly */
num_tc = dev->num_tc;
if (num_tc < 0)
return -EINVAL;
/* If queue belongs to subordinate dev use its map */
dev = netdev_get_tx_queue(dev, index)->sb_dev ? : dev;
tc = netdev_txq_to_tc(dev, index);
if (tc < 0)
return -EINVAL;
}
mutex_lock(&xps_map_mutex);
dev_maps = xmap_dereference(dev->xps_maps[type]);
if (type == XPS_RXQS) {
maps_sz = XPS_RXQ_DEV_MAPS_SIZE(num_tc, dev->num_rx_queues);
nr_ids = dev->num_rx_queues;
} else {
maps_sz = XPS_CPU_DEV_MAPS_SIZE(num_tc);
if (num_possible_cpus() > 1)
online_mask = cpumask_bits(cpu_online_mask);
nr_ids = nr_cpu_ids;
}
if (maps_sz < L1_CACHE_BYTES)
maps_sz = L1_CACHE_BYTES;
/* The old dev_maps could be larger or smaller than the one we're
* setting up now, as dev->num_tc or nr_ids could have been updated in
* between. We could try to be smart, but let's be safe instead and only
* copy foreign traffic classes if the two map sizes match.
*/
if (dev_maps &&
dev_maps->num_tc == num_tc && dev_maps->nr_ids == nr_ids)
copy = true;
/* allocate memory for queue storage */
for (j = -1; j = netif_attrmask_next_and(j, online_mask, mask, nr_ids),
j < nr_ids;) {
if (!new_dev_maps) {
new_dev_maps = kzalloc(maps_sz, GFP_KERNEL);
if (!new_dev_maps) {
mutex_unlock(&xps_map_mutex);
return -ENOMEM;
}
new_dev_maps->nr_ids = nr_ids;
new_dev_maps->num_tc = num_tc;
}
tci = j * num_tc + tc;
map = copy ? xmap_dereference(dev_maps->attr_map[tci]) : NULL;
map = expand_xps_map(map, j, index, type == XPS_RXQS);
if (!map)
goto error;
RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map);
}
if (!new_dev_maps)
goto out_no_new_maps;
if (!dev_maps) {
/* Increment static keys at most once per type */
static_key_slow_inc_cpuslocked(&xps_needed);
if (type == XPS_RXQS)
static_key_slow_inc_cpuslocked(&xps_rxqs_needed);
}
for (j = 0; j < nr_ids; j++) {
bool skip_tc = false;
tci = j * num_tc + tc;
if (netif_attr_test_mask(j, mask, nr_ids) &&
netif_attr_test_online(j, online_mask, nr_ids)) {
/* add tx-queue to CPU/rx-queue maps */
int pos = 0;
skip_tc = true;
map = xmap_dereference(new_dev_maps->attr_map[tci]);
while ((pos < map->len) && (map->queues[pos] != index))
pos++;
if (pos == map->len)
map->queues[map->len++] = index;
#ifdef CONFIG_NUMA
if (type == XPS_CPUS) {
if (numa_node_id == -2)
numa_node_id = cpu_to_node(j);
else if (numa_node_id != cpu_to_node(j))
numa_node_id = -1;
}
#endif
}
if (copy)
xps_copy_dev_maps(dev_maps, new_dev_maps, j, tc,
skip_tc);
}
rcu_assign_pointer(dev->xps_maps[type], new_dev_maps);
/* Cleanup old maps */
if (!dev_maps)
goto out_no_old_maps;
for (j = 0; j < dev_maps->nr_ids; j++) {
for (i = num_tc, tci = j * dev_maps->num_tc; i--; tci++) {
map = xmap_dereference(dev_maps->attr_map[tci]);
if (!map)
continue;
if (copy) {
new_map = xmap_dereference(new_dev_maps->attr_map[tci]);
if (map == new_map)
continue;
}
RCU_INIT_POINTER(dev_maps->attr_map[tci], NULL);
kfree_rcu(map, rcu);
}
}
old_dev_maps = dev_maps;
out_no_old_maps:
dev_maps = new_dev_maps;
active = true;
out_no_new_maps:
if (type == XPS_CPUS)
/* update Tx queue numa node */
netdev_queue_numa_node_write(netdev_get_tx_queue(dev, index),
(numa_node_id >= 0) ?
numa_node_id : NUMA_NO_NODE);
if (!dev_maps)
goto out_no_maps;
/* removes tx-queue from unused CPUs/rx-queues */
for (j = 0; j < dev_maps->nr_ids; j++) {
tci = j * dev_maps->num_tc;
for (i = 0; i < dev_maps->num_tc; i++, tci++) {
if (i == tc &&
netif_attr_test_mask(j, mask, dev_maps->nr_ids) &&
netif_attr_test_online(j, online_mask, dev_maps->nr_ids))
continue;
active |= remove_xps_queue(dev_maps,
copy ? old_dev_maps : NULL,
tci, index);
}
}
if (old_dev_maps)
kfree_rcu(old_dev_maps, rcu);
/* free map if not active */
if (!active)
reset_xps_maps(dev, dev_maps, type);
out_no_maps:
mutex_unlock(&xps_map_mutex);
return 0;
error:
/* remove any maps that we added */
for (j = 0; j < nr_ids; j++) {
for (i = num_tc, tci = j * num_tc; i--; tci++) {
new_map = xmap_dereference(new_dev_maps->attr_map[tci]);
map = copy ?
xmap_dereference(dev_maps->attr_map[tci]) :
NULL;
if (new_map && new_map != map)
kfree(new_map);
}
}
mutex_unlock(&xps_map_mutex);
kfree(new_dev_maps);
return -ENOMEM;
}
EXPORT_SYMBOL_GPL(__netif_set_xps_queue);
int netif_set_xps_queue(struct net_device *dev, const struct cpumask *mask,
u16 index)
{
int ret;
cpus_read_lock();
ret = __netif_set_xps_queue(dev, cpumask_bits(mask), index, XPS_CPUS);
cpus_read_unlock();
return ret;
}
EXPORT_SYMBOL(netif_set_xps_queue);
#endif
static void netdev_unbind_all_sb_channels(struct net_device *dev)
{
struct netdev_queue *txq = &dev->_tx[dev->num_tx_queues];
/* Unbind any subordinate channels */
while (txq-- != &dev->_tx[0]) {
if (txq->sb_dev)
netdev_unbind_sb_channel(dev, txq->sb_dev);
}
}
void netdev_reset_tc(struct net_device *dev)
{
#ifdef CONFIG_XPS
netif_reset_xps_queues_gt(dev, 0);
#endif
netdev_unbind_all_sb_channels(dev);
/* Reset TC configuration of device */
dev->num_tc = 0;
memset(dev->tc_to_txq, 0, sizeof(dev->tc_to_txq));
memset(dev->prio_tc_map, 0, sizeof(dev->prio_tc_map));
}
EXPORT_SYMBOL(netdev_reset_tc);
int netdev_set_tc_queue(struct net_device *dev, u8 tc, u16 count, u16 offset)
{
if (tc >= dev->num_tc)
return -EINVAL;
#ifdef CONFIG_XPS
netif_reset_xps_queues(dev, offset, count);
#endif
dev->tc_to_txq[tc].count = count;
dev->tc_to_txq[tc].offset = offset;
return 0;
}
EXPORT_SYMBOL(netdev_set_tc_queue);
int netdev_set_num_tc(struct net_device *dev, u8 num_tc)
{
if (num_tc > TC_MAX_QUEUE)
return -EINVAL;
#ifdef CONFIG_XPS
netif_reset_xps_queues_gt(dev, 0);
#endif
netdev_unbind_all_sb_channels(dev);
dev->num_tc = num_tc;
return 0;
}
EXPORT_SYMBOL(netdev_set_num_tc);
void netdev_unbind_sb_channel(struct net_device *dev,
struct net_device *sb_dev)
{
struct netdev_queue *txq = &dev->_tx[dev->num_tx_queues];
#ifdef CONFIG_XPS
netif_reset_xps_queues_gt(sb_dev, 0);
#endif
memset(sb_dev->tc_to_txq, 0, sizeof(sb_dev->tc_to_txq));
memset(sb_dev->prio_tc_map, 0, sizeof(sb_dev->prio_tc_map));
while (txq-- != &dev->_tx[0]) {
if (txq->sb_dev == sb_dev)
txq->sb_dev = NULL;
}
}
EXPORT_SYMBOL(netdev_unbind_sb_channel);
int netdev_bind_sb_channel_queue(struct net_device *dev,
struct net_device *sb_dev,
u8 tc, u16 count, u16 offset)
{
/* Make certain the sb_dev and dev are already configured */
if (sb_dev->num_tc >= 0 || tc >= dev->num_tc)
return -EINVAL;
/* We cannot hand out queues we don't have */
if ((offset + count) > dev->real_num_tx_queues)
return -EINVAL;
/* Record the mapping */
sb_dev->tc_to_txq[tc].count = count;
sb_dev->tc_to_txq[tc].offset = offset;
/* Provide a way for Tx queue to find the tc_to_txq map or
* XPS map for itself.
*/
while (count--)
netdev_get_tx_queue(dev, count + offset)->sb_dev = sb_dev;
return 0;
}
EXPORT_SYMBOL(netdev_bind_sb_channel_queue);
int netdev_set_sb_channel(struct net_device *dev, u16 channel)
{
/* Do not use a multiqueue device to represent a subordinate channel */
if (netif_is_multiqueue(dev))
return -ENODEV;
/* We allow channels 1 - 32767 to be used for subordinate channels.
* Channel 0 is meant to be "native" mode and used only to represent
* the main root device. We allow writing 0 to reset the device back
* to normal mode after being used as a subordinate channel.
*/
if (channel > S16_MAX)
return -EINVAL;
dev->num_tc = -channel;
return 0;
}
EXPORT_SYMBOL(netdev_set_sb_channel);
/*
* Routine to help set real_num_tx_queues. To avoid skbs mapped to queues
* greater than real_num_tx_queues stale skbs on the qdisc must be flushed.
*/
int netif_set_real_num_tx_queues(struct net_device *dev, unsigned int txq)
{
bool disabling;
int rc;
disabling = txq < dev->real_num_tx_queues;
if (txq < 1 || txq > dev->num_tx_queues)
return -EINVAL;
if (dev->reg_state == NETREG_REGISTERED ||
dev->reg_state == NETREG_UNREGISTERING) {
netdev_ops_assert_locked(dev);
rc = netdev_queue_update_kobjects(dev, dev->real_num_tx_queues,
txq);
if (rc)
return rc;
if (dev->num_tc)
netif_setup_tc(dev, txq);
net_shaper_set_real_num_tx_queues(dev, txq);
dev_qdisc_change_real_num_tx(dev, txq);
dev->real_num_tx_queues = txq;
if (disabling) {
synchronize_net();
qdisc_reset_all_tx_gt(dev, txq);
#ifdef CONFIG_XPS
netif_reset_xps_queues_gt(dev, txq);
#endif
}
} else {
dev->real_num_tx_queues = txq;
}
return 0;
}
EXPORT_SYMBOL(netif_set_real_num_tx_queues);
/**
* netif_set_real_num_rx_queues - set actual number of RX queues used
* @dev: Network device
* @rxq: Actual number of RX queues
*
* This must be called either with the rtnl_lock held or before
* registration of the net device. Returns 0 on success, or a
* negative error code. If called before registration, it always
* succeeds.
*/
int netif_set_real_num_rx_queues(struct net_device *dev, unsigned int rxq)
{
int rc;
if (rxq < 1 || rxq > dev->num_rx_queues)
return -EINVAL;
if (dev->reg_state == NETREG_REGISTERED) {
netdev_ops_assert_locked(dev);
rc = net_rx_queue_update_kobjects(dev, dev->real_num_rx_queues,
rxq);
if (rc)
return rc;
}
dev->real_num_rx_queues = rxq;
return 0;
}
EXPORT_SYMBOL(netif_set_real_num_rx_queues);
/**
* netif_set_real_num_queues - set actual number of RX and TX queues used
* @dev: Network device
* @txq: Actual number of TX queues
* @rxq: Actual number of RX queues
*
* Set the real number of both TX and RX queues.
* Does nothing if the number of queues is already correct.
*/
int netif_set_real_num_queues(struct net_device *dev,
unsigned int txq, unsigned int rxq)
{
unsigned int old_rxq = dev->real_num_rx_queues;
int err;
if (txq < 1 || txq > dev->num_tx_queues ||
rxq < 1 || rxq > dev->num_rx_queues)
return -EINVAL;
/* Start from increases, so the error path only does decreases -
* decreases can't fail.
*/
if (rxq > dev->real_num_rx_queues) {
err = netif_set_real_num_rx_queues(dev, rxq);
if (err)
return err;
}
if (txq > dev->real_num_tx_queues) {
err = netif_set_real_num_tx_queues(dev, txq);
if (err)
goto undo_rx;
}
if (rxq < dev->real_num_rx_queues)
WARN_ON(netif_set_real_num_rx_queues(dev, rxq));
if (txq < dev->real_num_tx_queues)
WARN_ON(netif_set_real_num_tx_queues(dev, txq));
return 0;
undo_rx:
WARN_ON(netif_set_real_num_rx_queues(dev, old_rxq));
return err;
}
EXPORT_SYMBOL(netif_set_real_num_queues);
/**
* netif_set_tso_max_size() - set the max size of TSO frames supported
* @dev: netdev to update
* @size: max skb->len of a TSO frame
*
* Set the limit on the size of TSO super-frames the device can handle.
* Unless explicitly set the stack will assume the value of
* %GSO_LEGACY_MAX_SIZE.
*/
void netif_set_tso_max_size(struct net_device *dev, unsigned int size)
{
dev->tso_max_size = min(GSO_MAX_SIZE, size);
if (size < READ_ONCE(dev->gso_max_size))
netif_set_gso_max_size(dev, size);
if (size < READ_ONCE(dev->gso_ipv4_max_size))
netif_set_gso_ipv4_max_size(dev, size);
}
EXPORT_SYMBOL(netif_set_tso_max_size);
/**
* netif_set_tso_max_segs() - set the max number of segs supported for TSO
* @dev: netdev to update
* @segs: max number of TCP segments
*
* Set the limit on the number of TCP segments the device can generate from
* a single TSO super-frame.
* Unless explicitly set the stack will assume the value of %GSO_MAX_SEGS.
*/
void netif_set_tso_max_segs(struct net_device *dev, unsigned int segs)
{
dev->tso_max_segs = segs;
if (segs < READ_ONCE(dev->gso_max_segs))
netif_set_gso_max_segs(dev, segs);
}
EXPORT_SYMBOL(netif_set_tso_max_segs);
/**
* netif_inherit_tso_max() - copy all TSO limits from a lower device to an upper
* @to: netdev to update
* @from: netdev from which to copy the limits
*/
void netif_inherit_tso_max(struct net_device *to, const struct net_device *from)
{
netif_set_tso_max_size(to, from->tso_max_size);
netif_set_tso_max_segs(to, from->tso_max_segs);
}
EXPORT_SYMBOL(netif_inherit_tso_max);
/**
* netif_get_num_default_rss_queues - default number of RSS queues
*
* Default value is the number of physical cores if there are only 1 or 2, or
* divided by 2 if there are more.
*/
int netif_get_num_default_rss_queues(void)
{
cpumask_var_t cpus;
int cpu, count = 0;
if (unlikely(is_kdump_kernel() || !zalloc_cpumask_var(&cpus, GFP_KERNEL)))
return 1;
cpumask_copy(cpus, cpu_online_mask);
for_each_cpu(cpu, cpus) {
++count;
cpumask_andnot(cpus, cpus, topology_sibling_cpumask(cpu));
}
free_cpumask_var(cpus);
return count > 2 ? DIV_ROUND_UP(count, 2) : count;
}
EXPORT_SYMBOL(netif_get_num_default_rss_queues);
static void __netif_reschedule(struct Qdisc *q)
{
struct softnet_data *sd;
unsigned long flags;
local_irq_save(flags);
sd = this_cpu_ptr(&softnet_data);
q->next_sched = NULL;
*sd->output_queue_tailp = q;
sd->output_queue_tailp = &q->next_sched;
raise_softirq_irqoff(NET_TX_SOFTIRQ);
local_irq_restore(flags);
}
void __netif_schedule(struct Qdisc *q)
{
/* If q->defer_list is not empty, at least one thread is
* in __dev_xmit_skb() before llist_del_all(&q->defer_list).
* This thread will attempt to run the queue.
*/
if (!llist_empty(&q->defer_list))
return;
if (!test_and_set_bit(__QDISC_STATE_SCHED, &q->state))
__netif_reschedule(q);
}
EXPORT_SYMBOL(__netif_schedule);
struct dev_kfree_skb_cb {
enum skb_drop_reason reason;
};
static struct dev_kfree_skb_cb *get_kfree_skb_cb(const struct sk_buff *skb)
{
return (struct dev_kfree_skb_cb *)skb->cb;
}
void netif_schedule_queue(struct netdev_queue *txq)
{
rcu_read_lock();
if (!netif_xmit_stopped(txq)) {
struct Qdisc *q = rcu_dereference(txq->qdisc);
__netif_schedule(q);
}
rcu_read_unlock();
}
EXPORT_SYMBOL(netif_schedule_queue);
void netif_tx_wake_queue(struct netdev_queue *dev_queue)
{
if (test_and_clear_bit(__QUEUE_STATE_DRV_XOFF, &dev_queue->state)) {
struct Qdisc *q;
rcu_read_lock();
q = rcu_dereference(dev_queue->qdisc);
__netif_schedule(q);
rcu_read_unlock();
}
}
EXPORT_SYMBOL(netif_tx_wake_queue);
void dev_kfree_skb_irq_reason(struct sk_buff *skb, enum skb_drop_reason reason)
{
unsigned long flags;
if (unlikely(!skb))
return;
if (likely(refcount_read(&skb->users) == 1)) {
smp_rmb();
refcount_set(&skb->users, 0);
} else if (likely(!refcount_dec_and_test(&skb->users))) {
return;
}
get_kfree_skb_cb(skb)->reason = reason;
local_irq_save(flags);
skb->next = __this_cpu_read(softnet_data.completion_queue);
__this_cpu_write(softnet_data.completion_queue, skb);
raise_softirq_irqoff(NET_TX_SOFTIRQ);
local_irq_restore(flags);
}
EXPORT_SYMBOL(dev_kfree_skb_irq_reason);
void dev_kfree_skb_any_reason(struct sk_buff *skb, enum skb_drop_reason reason)
{
if (in_hardirq() || irqs_disabled())
dev_kfree_skb_irq_reason(skb, reason);
else
kfree_skb_reason(skb, reason);
}
EXPORT_SYMBOL(dev_kfree_skb_any_reason);
/**
* netif_device_detach - mark device as removed
* @dev: network device
*
* Mark device as removed from system and therefore no longer available.
*/
void netif_device_detach(struct net_device *dev)
{
if (test_and_clear_bit(__LINK_STATE_PRESENT, &dev->state) &&
netif_running(dev)) {
netif_tx_stop_all_queues(dev);
}
}
EXPORT_SYMBOL(netif_device_detach);
/**
* netif_device_attach - mark device as attached
* @dev: network device
*
* Mark device as attached from system and restart if needed.
*/
void netif_device_attach(struct net_device *dev)
{
if (!test_and_set_bit(__LINK_STATE_PRESENT, &dev->state) &&
netif_running(dev)) {
netif_tx_wake_all_queues(dev);
netdev_watchdog_up(dev);
}
}
EXPORT_SYMBOL(netif_device_attach);
/*
* Returns a Tx hash based on the given packet descriptor a Tx queues' number
* to be used as a distribution range.
*/
static u16 skb_tx_hash(const struct net_device *dev,
const struct net_device *sb_dev,
struct sk_buff *skb)
{
u32 hash;
u16 qoffset = 0;
u16 qcount = dev->real_num_tx_queues;
if (dev->num_tc) {
u8 tc = netdev_get_prio_tc_map(dev, skb->priority);
qoffset = sb_dev->tc_to_txq[tc].offset;
qcount = sb_dev->tc_to_txq[tc].count;
if (unlikely(!qcount)) {
net_warn_ratelimited("%s: invalid qcount, qoffset %u for tc %u\n",
sb_dev->name, qoffset, tc);
qoffset = 0;
qcount = dev->real_num_tx_queues;
}
}
if (skb_rx_queue_recorded(skb)) {
DEBUG_NET_WARN_ON_ONCE(qcount == 0);
hash = skb_get_rx_queue(skb);
if (hash >= qoffset)
hash -= qoffset;
while (unlikely(hash >= qcount))
hash -= qcount;
return hash + qoffset;
}
return (u16) reciprocal_scale(skb_get_hash(skb), qcount) + qoffset;
}
void skb_warn_bad_offload(const struct sk_buff *skb)
{
static const netdev_features_t null_features;
struct net_device *dev = skb->dev;
const char *name = "";
if (!net_ratelimit())
return;
if (dev) {
if (dev->dev.parent)
name = dev_driver_string(dev->dev.parent);
else
name = netdev_name(dev);
}
skb_dump(KERN_WARNING, skb, false);
WARN(1, "%s: caps=(%pNF, %pNF)\n",
name, dev ? &dev->features : &null_features,
skb->sk ? &skb->sk->sk_route_caps : &null_features);
}
/*
* Invalidate hardware checksum when packet is to be mangled, and
* complete checksum manually on outgoing path.
*/
int skb_checksum_help(struct sk_buff *skb)
{
__wsum csum;
int ret = 0, offset;
if (skb->ip_summed == CHECKSUM_COMPLETE)
goto out_set_summed;
if (unlikely(skb_is_gso(skb))) {
skb_warn_bad_offload(skb);
return -EINVAL;
}
if (!skb_frags_readable(skb)) {
return -EFAULT;
}
/* Before computing a checksum, we should make sure no frag could
* be modified by an external entity : checksum could be wrong.
*/
if (skb_has_shared_frag(skb)) {
ret = __skb_linearize(skb);
if (ret)
goto out;
}
offset = skb_checksum_start_offset(skb);
ret = -EINVAL;
if (unlikely(offset >= skb_headlen(skb))) {
DO_ONCE_LITE(skb_dump, KERN_ERR, skb, false);
WARN_ONCE(true, "offset (%d) >= skb_headlen() (%u)\n",
offset, skb_headlen(skb));
goto out;
}
csum = skb_checksum(skb, offset, skb->len - offset, 0);
offset += skb->csum_offset;
if (unlikely(offset + sizeof(__sum16) > skb_headlen(skb))) {
DO_ONCE_LITE(skb_dump, KERN_ERR, skb, false);
WARN_ONCE(true, "offset+2 (%zu) > skb_headlen() (%u)\n",
offset + sizeof(__sum16), skb_headlen(skb));
goto out;
}
ret = skb_ensure_writable(skb, offset + sizeof(__sum16));
if (ret)
goto out;
*(__sum16 *)(skb->data + offset) = csum_fold(csum) ?: CSUM_MANGLED_0;
out_set_summed:
skb->ip_summed = CHECKSUM_NONE;
out:
return ret;
}
EXPORT_SYMBOL(skb_checksum_help);
#ifdef CONFIG_NET_CRC32C
int skb_crc32c_csum_help(struct sk_buff *skb)
{
u32 crc;
int ret = 0, offset, start;
if (skb->ip_summed != CHECKSUM_PARTIAL)
goto out;
if (unlikely(skb_is_gso(skb)))
goto out;
/* Before computing a checksum, we should make sure no frag could
* be modified by an external entity : checksum could be wrong.
*/
if (unlikely(skb_has_shared_frag(skb))) {
ret = __skb_linearize(skb);
if (ret)
goto out;
}
start = skb_checksum_start_offset(skb);
offset = start + offsetof(struct sctphdr, checksum);
if (WARN_ON_ONCE(offset >= skb_headlen(skb))) {
ret = -EINVAL;
goto out;
}
ret = skb_ensure_writable(skb, offset + sizeof(__le32));
if (ret)
goto out;
crc = ~skb_crc32c(skb, start, skb->len - start, ~0);
*(__le32 *)(skb->data + offset) = cpu_to_le32(crc);
skb_reset_csum_not_inet(skb);
out:
return ret;
}
EXPORT_SYMBOL(skb_crc32c_csum_help);
#endif /* CONFIG_NET_CRC32C */
__be16 skb_network_protocol(struct sk_buff *skb, int *depth)
{
__be16 type = skb->protocol;
/* Tunnel gso handlers can set protocol to ethernet. */
if (type == htons(ETH_P_TEB)) {
struct ethhdr *eth;
if (unlikely(!pskb_may_pull(skb, sizeof(struct ethhdr))))
return 0;
eth = (struct ethhdr *)skb->data;
type = eth->h_proto;
}
return vlan_get_protocol_and_depth(skb, type, depth);
}
/* Take action when hardware reception checksum errors are detected. */
#ifdef CONFIG_BUG
static void do_netdev_rx_csum_fault(struct net_device *dev, struct sk_buff *skb)
{
netdev_err(dev, "hw csum failure\n");
skb_dump(KERN_ERR, skb, true);
dump_stack();
}
void netdev_rx_csum_fault(struct net_device *dev, struct sk_buff *skb)
{
DO_ONCE_LITE(do_netdev_rx_csum_fault, dev, skb);
}
EXPORT_SYMBOL(netdev_rx_csum_fault);
#endif
/* XXX: check that highmem exists at all on the given machine. */
static int illegal_highdma(struct net_device *dev, struct sk_buff *skb)
{
#ifdef CONFIG_HIGHMEM
int i;
if (!(dev->features & NETIF_F_HIGHDMA)) {
for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) {
skb_frag_t *frag = &skb_shinfo(skb)->frags[i];
struct page *page = skb_frag_page(frag);
if (page && PageHighMem(page))
return 1;
}
}
#endif
return 0;
}
/* If MPLS offload request, verify we are testing hardware MPLS features
* instead of standard features for the netdev.
*/
#if IS_ENABLED(CONFIG_NET_MPLS_GSO)
static netdev_features_t net_mpls_features(struct sk_buff *skb,
netdev_features_t features,
__be16 type)
{
if (eth_p_mpls(type))
features &= skb->dev->mpls_features;
return features;
}
#else
static netdev_features_t net_mpls_features(struct sk_buff *skb,
netdev_features_t features,
__be16 type)
{
return features;
}
#endif
static netdev_features_t harmonize_features(struct sk_buff *skb,
netdev_features_t features)
{
__be16 type;
type = skb_network_protocol(skb, NULL);
features = net_mpls_features(skb, features, type);
if (skb->ip_summed != CHECKSUM_NONE &&
!can_checksum_protocol(features, type)) {
features &= ~(NETIF_F_CSUM_MASK | NETIF_F_GSO_MASK);
}
if (illegal_highdma(skb->dev, skb))
features &= ~NETIF_F_SG;
return features;
}
netdev_features_t passthru_features_check(struct sk_buff *skb,
struct net_device *dev,
netdev_features_t features)
{
return features;
}
EXPORT_SYMBOL(passthru_features_check);
static netdev_features_t dflt_features_check(struct sk_buff *skb,
struct net_device *dev,
netdev_features_t features)
{
return vlan_features_check(skb, features);
}
static netdev_features_t gso_features_check(const struct sk_buff *skb,
struct net_device *dev,
netdev_features_t features)
{
u16 gso_segs = skb_shinfo(skb)->gso_segs;
if (gso_segs > READ_ONCE(dev->gso_max_segs))
return features & ~NETIF_F_GSO_MASK;
if (unlikely(skb->len >= netif_get_gso_max_size(dev, skb)))
return features & ~NETIF_F_GSO_MASK;
if (!skb_shinfo(skb)->gso_type) {
skb_warn_bad_offload(skb);
return features & ~NETIF_F_GSO_MASK;
}
/* Support for GSO partial features requires software
* intervention before we can actually process the packets
* so we need to strip support for any partial features now
* and we can pull them back in after we have partially
* segmented the frame.
*/
if (!(skb_shinfo(skb)->gso_type & SKB_GSO_PARTIAL))
features &= ~dev->gso_partial_features;
/* Make sure to clear the IPv4 ID mangling feature if the IPv4 header
* has the potential to be fragmented so that TSO does not generate
* segments with the same ID. For encapsulated packets, the ID mangling
* feature is guaranteed not to use the same ID for the outer IPv4
* headers of the generated segments if the headers have the potential
* to be fragmented, so there is no need to clear the IPv4 ID mangling
* feature (see the section about NETIF_F_TSO_MANGLEID in
* segmentation-offloads.rst).
*/
if (skb_shinfo(skb)->gso_type & SKB_GSO_TCPV4) {
struct iphdr *iph = skb->encapsulation ?
inner_ip_hdr(skb) : ip_hdr(skb);
if (!(iph->frag_off & htons(IP_DF)))
features &= ~NETIF_F_TSO_MANGLEID;
}
/* NETIF_F_IPV6_CSUM does not support IPv6 extension headers,
* so neither does TSO that depends on it.
*/
if (features & NETIF_F_IPV6_CSUM &&
(skb_shinfo(skb)->gso_type & SKB_GSO_TCPV6 ||
(skb_shinfo(skb)->gso_type & SKB_GSO_UDP_L4 &&
vlan_get_protocol(skb) == htons(ETH_P_IPV6))) &&
skb_transport_header_was_set(skb) &&
skb_network_header_len(skb) != sizeof(struct ipv6hdr) &&
!ipv6_has_hopopt_jumbo(skb))
features &= ~(NETIF_F_IPV6_CSUM | NETIF_F_TSO6 | NETIF_F_GSO_UDP_L4);
return features;
}
netdev_features_t netif_skb_features(struct sk_buff *skb)
{
struct net_device *dev = skb->dev;
netdev_features_t features = dev->features;
if (skb_is_gso(skb))
features = gso_features_check(skb, dev, features);
/* If encapsulation offload request, verify we are testing
* hardware encapsulation features instead of standard
* features for the netdev
*/
if (skb->encapsulation)
features &= dev->hw_enc_features;
if (skb_vlan_tagged(skb))
features = netdev_intersect_features(features,
dev->vlan_features |
NETIF_F_HW_VLAN_CTAG_TX |
NETIF_F_HW_VLAN_STAG_TX);
if (dev->netdev_ops->ndo_features_check)
features &= dev->netdev_ops->ndo_features_check(skb, dev,
features);
else
features &= dflt_features_check(skb, dev, features);
return harmonize_features(skb, features);
}
EXPORT_SYMBOL(netif_skb_features);
static int xmit_one(struct sk_buff *skb, struct net_device *dev,
struct netdev_queue *txq, bool more)
{
unsigned int len;
int rc;
if (dev_nit_active_rcu(dev))
dev_queue_xmit_nit(skb, dev);
len = skb->len;
trace_net_dev_start_xmit(skb, dev);
rc = netdev_start_xmit(skb, dev, txq, more);
trace_net_dev_xmit(skb, rc, dev, len);
return rc;
}
struct sk_buff *dev_hard_start_xmit(struct sk_buff *first, struct net_device *dev,
struct netdev_queue *txq, int *ret)
{
struct sk_buff *skb = first;
int rc = NETDEV_TX_OK;
while (skb) {
struct sk_buff *next = skb->next;
skb_mark_not_on_list(skb);
rc = xmit_one(skb, dev, txq, next != NULL);
if (unlikely(!dev_xmit_complete(rc))) {
skb->next = next;
goto out;
}
skb = next;
if (netif_tx_queue_stopped(txq) && skb) {
rc = NETDEV_TX_BUSY;
break;
}
}
out:
*ret = rc;
return skb;
}
static struct sk_buff *validate_xmit_vlan(struct sk_buff *skb,
netdev_features_t features)
{
if (skb_vlan_tag_present(skb) &&
!vlan_hw_offload_capable(features, skb->vlan_proto))
skb = __vlan_hwaccel_push_inside(skb);
return skb;
}
int skb_csum_hwoffload_help(struct sk_buff *skb,
const netdev_features_t features)
{
if (unlikely(skb_csum_is_sctp(skb)))
return !!(features & NETIF_F_SCTP_CRC) ? 0 :
skb_crc32c_csum_help(skb);
if (features & NETIF_F_HW_CSUM)
return 0;
if (features & (NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM)) {
if (vlan_get_protocol(skb) == htons(ETH_P_IPV6) &&
skb_network_header_len(skb) != sizeof(struct ipv6hdr) &&
!ipv6_has_hopopt_jumbo(skb))
goto sw_checksum;
switch (skb->csum_offset) {
case offsetof(struct tcphdr, check):
case offsetof(struct udphdr, check):
return 0;
}
}
sw_checksum:
return skb_checksum_help(skb);
}
EXPORT_SYMBOL(skb_csum_hwoffload_help);
/* Checks if this SKB belongs to an HW offloaded socket
* and whether any SW fallbacks are required based on dev.
* Check decrypted mark in case skb_orphan() cleared socket.
*/
static struct sk_buff *sk_validate_xmit_skb(struct sk_buff *skb,
struct net_device *dev)
{
#ifdef CONFIG_SOCK_VALIDATE_XMIT
struct sk_buff *(*sk_validate)(struct sock *sk, struct net_device *dev,
struct sk_buff *skb);
struct sock *sk = skb->sk;
sk_validate = NULL;
if (sk) {
if (sk_fullsock(sk))
sk_validate = sk->sk_validate_xmit_skb;
else if (sk_is_inet(sk) && sk->sk_state == TCP_TIME_WAIT)
sk_validate = inet_twsk(sk)->tw_validate_xmit_skb;
}
if (sk_validate) {
skb = sk_validate(sk, dev, skb);
} else if (unlikely(skb_is_decrypted(skb))) {
pr_warn_ratelimited("unencrypted skb with no associated socket - dropping\n");
kfree_skb(skb);
skb = NULL;
}
#endif
return skb;
}
static struct sk_buff *validate_xmit_unreadable_skb(struct sk_buff *skb,
struct net_device *dev)
{
struct skb_shared_info *shinfo;
struct net_iov *niov;
if (likely(skb_frags_readable(skb)))
goto out;
if (!dev->netmem_tx)
goto out_free;
shinfo = skb_shinfo(skb);
if (shinfo->nr_frags > 0) {
niov = netmem_to_net_iov(skb_frag_netmem(&shinfo->frags[0]));
if (net_is_devmem_iov(niov) &&
net_devmem_iov_binding(niov)->dev != dev)
goto out_free;
}
out:
return skb;
out_free:
kfree_skb(skb);
return NULL;
}
static struct sk_buff *validate_xmit_skb(struct sk_buff *skb, struct net_device *dev, bool *again)
{
netdev_features_t features;
skb = validate_xmit_unreadable_skb(skb, dev);
if (unlikely(!skb))
goto out_null;
features = netif_skb_features(skb);
skb = validate_xmit_vlan(skb, features);
if (unlikely(!skb))
goto out_null;
skb = sk_validate_xmit_skb(skb, dev);
if (unlikely(!skb))
goto out_null;
if (netif_needs_gso(skb, features)) {
struct sk_buff *segs;
segs = skb_gso_segment(skb, features);
if (IS_ERR(segs)) {
goto out_kfree_skb;
} else if (segs) {
consume_skb(skb);
skb = segs;
}
} else {
if (skb_needs_linearize(skb, features) &&
__skb_linearize(skb))
goto out_kfree_skb;
/* If packet is not checksummed and device does not
* support checksumming for this protocol, complete
* checksumming here.
*/
if (skb->ip_summed == CHECKSUM_PARTIAL) {
if (skb->encapsulation)
skb_set_inner_transport_header(skb,
skb_checksum_start_offset(skb));
else
skb_set_transport_header(skb,
skb_checksum_start_offset(skb));
if (skb_csum_hwoffload_help(skb, features))
goto out_kfree_skb;
}
}
skb = validate_xmit_xfrm(skb, features, again);
return skb;
out_kfree_skb:
kfree_skb(skb);
out_null:
dev_core_stats_tx_dropped_inc(dev);
return NULL;
}
struct sk_buff *validate_xmit_skb_list(struct sk_buff *skb, struct net_device *dev, bool *again)
{
struct sk_buff *next, *head = NULL, *tail;
for (; skb != NULL; skb = next) {
next = skb->next;
skb_mark_not_on_list(skb);
/* in case skb won't be segmented, point to itself */
skb->prev = skb;
skb = validate_xmit_skb(skb, dev, again);
if (!skb)
continue;
if (!head)
head = skb;
else
tail->next = skb;
/* If skb was segmented, skb->prev points to
* the last segment. If not, it still contains skb.
*/
tail = skb->prev;
}
return head;
}
EXPORT_SYMBOL_GPL(validate_xmit_skb_list);
static void qdisc_pkt_len_segs_init(struct sk_buff *skb)
{
struct skb_shared_info *shinfo = skb_shinfo(skb);
u16 gso_segs;
qdisc_skb_cb(skb)->pkt_len = skb->len;
if (!shinfo->gso_size) {
qdisc_skb_cb(skb)->pkt_segs = 1;
return;
}
qdisc_skb_cb(skb)->pkt_segs = gso_segs = shinfo->gso_segs;
/* To get more precise estimation of bytes sent on wire,
* we add to pkt_len the headers size of all segments
*/
if (skb_transport_header_was_set(skb)) {
unsigned int hdr_len;
/* mac layer + network layer */
if (!skb->encapsulation)
hdr_len = skb_transport_offset(skb);
else
hdr_len = skb_inner_transport_offset(skb);
/* + transport layer */
if (likely(shinfo->gso_type & (SKB_GSO_TCPV4 | SKB_GSO_TCPV6))) {
const struct tcphdr *th;
struct tcphdr _tcphdr;
th = skb_header_pointer(skb, hdr_len,
sizeof(_tcphdr), &_tcphdr);
if (likely(th))
hdr_len += __tcp_hdrlen(th);
} else if (shinfo->gso_type & SKB_GSO_UDP_L4) {
struct udphdr _udphdr;
if (skb_header_pointer(skb, hdr_len,
sizeof(_udphdr), &_udphdr))
hdr_len += sizeof(struct udphdr);
}
if (unlikely(shinfo->gso_type & SKB_GSO_DODGY)) {
int payload = skb->len - hdr_len;
/* Malicious packet. */
if (payload <= 0)
return;
gso_segs = DIV_ROUND_UP(payload, shinfo->gso_size);
shinfo->gso_segs = gso_segs;
qdisc_skb_cb(skb)->pkt_segs = gso_segs;
}
qdisc_skb_cb(skb)->pkt_len += (gso_segs - 1) * hdr_len;
}
}
static int dev_qdisc_enqueue(struct sk_buff *skb, struct Qdisc *q,
struct sk_buff **to_free,
struct netdev_queue *txq)
{
int rc;
rc = q->enqueue(skb, q, to_free) & NET_XMIT_MASK;
if (rc == NET_XMIT_SUCCESS)
trace_qdisc_enqueue(q, txq, skb);
return rc;
}
static inline int __dev_xmit_skb(struct sk_buff *skb, struct Qdisc *q,
struct net_device *dev,
struct netdev_queue *txq)
{
struct sk_buff *next, *to_free = NULL, *to_free2 = NULL;
spinlock_t *root_lock = qdisc_lock(q);
struct llist_node *ll_list, *first_n;
unsigned long defer_count = 0;
int rc;
qdisc_calculate_pkt_len(skb, q);
tcf_set_drop_reason(skb, SKB_DROP_REASON_QDISC_DROP);
if (q->flags & TCQ_F_NOLOCK) {
if (q->flags & TCQ_F_CAN_BYPASS && nolock_qdisc_is_empty(q) &&
qdisc_run_begin(q)) {
/* Retest nolock_qdisc_is_empty() within the protection
* of q->seqlock to protect from racing with requeuing.
*/
if (unlikely(!nolock_qdisc_is_empty(q))) {
rc = dev_qdisc_enqueue(skb, q, &to_free, txq);
__qdisc_run(q);
to_free2 = qdisc_run_end(q);
goto free_skbs;
}
qdisc_bstats_cpu_update(q, skb);
if (sch_direct_xmit(skb, q, dev, txq, NULL, true) &&
!nolock_qdisc_is_empty(q))
__qdisc_run(q);
to_free2 = qdisc_run_end(q);
rc = NET_XMIT_SUCCESS;
goto free_skbs;
}
rc = dev_qdisc_enqueue(skb, q, &to_free, txq);
to_free2 = qdisc_run(q);
goto free_skbs;
}
/* Open code llist_add(&skb->ll_node, &q->defer_list) + queue limit.
* In the try_cmpxchg() loop, we want to increment q->defer_count
* at most once to limit the number of skbs in defer_list.
* We perform the defer_count increment only if the list is not empty,
* because some arches have slow atomic_long_inc_return().
*/
first_n = READ_ONCE(q->defer_list.first);
do {
if (first_n && !defer_count) {
defer_count = atomic_long_inc_return(&q->defer_count);
if (unlikely(defer_count > READ_ONCE(q->limit))) {
kfree_skb_reason(skb, SKB_DROP_REASON_QDISC_DROP);
return NET_XMIT_DROP;
}
}
skb->ll_node.next = first_n;
} while (!try_cmpxchg(&q->defer_list.first, &first_n, &skb->ll_node));
/* If defer_list was not empty, we know the cpu which queued
* the first skb will process the whole list for us.
*/
if (first_n)
return NET_XMIT_SUCCESS;
spin_lock(root_lock);
ll_list = llist_del_all(&q->defer_list);
/* There is a small race because we clear defer_count not atomically
* with the prior llist_del_all(). This means defer_list could grow
* over q->limit.
*/
atomic_long_set(&q->defer_count, 0);
ll_list = llist_reverse_order(ll_list);
if (unlikely(test_bit(__QDISC_STATE_DEACTIVATED, &q->state))) {
llist_for_each_entry_safe(skb, next, ll_list, ll_node)
__qdisc_drop(skb, &to_free);
rc = NET_XMIT_DROP;
goto unlock;
}
if ((q->flags & TCQ_F_CAN_BYPASS) && !qdisc_qlen(q) &&
!llist_next(ll_list) && qdisc_run_begin(q)) {
/*
* This is a work-conserving queue; there are no old skbs
* waiting to be sent out; and the qdisc is not running -
* xmit the skb directly.
*/
DEBUG_NET_WARN_ON_ONCE(skb != llist_entry(ll_list,
struct sk_buff,
ll_node));
qdisc_bstats_update(q, skb);
if (sch_direct_xmit(skb, q, dev, txq, root_lock, true))
__qdisc_run(q);
to_free2 = qdisc_run_end(q);
rc = NET_XMIT_SUCCESS;
} else {
int count = 0;
llist_for_each_entry_safe(skb, next, ll_list, ll_node) {
prefetch(next);
prefetch(&next->priority);
skb_mark_not_on_list(skb);
rc = dev_qdisc_enqueue(skb, q, &to_free, txq);
count++;
}
to_free2 = qdisc_run(q);
if (count != 1)
rc = NET_XMIT_SUCCESS;
}
unlock:
spin_unlock(root_lock);
free_skbs:
tcf_kfree_skb_list(to_free);
tcf_kfree_skb_list(to_free2);
return rc;
}
#if IS_ENABLED(CONFIG_CGROUP_NET_PRIO)
static void skb_update_prio(struct sk_buff *skb)
{
const struct netprio_map *map;
const struct sock *sk;
unsigned int prioidx;
if (skb->priority)
return;
map = rcu_dereference_bh(skb->dev->priomap);
if (!map)
return;
sk = skb_to_full_sk(skb);
if (!sk)
return;
prioidx = sock_cgroup_prioidx(&sk->sk_cgrp_data);
if (prioidx < map->priomap_len)
skb->priority = map->priomap[prioidx];
}
#else
#define skb_update_prio(skb)
#endif
/**
* dev_loopback_xmit - loop back @skb
* @net: network namespace this loopback is happening in
* @sk: sk needed to be a netfilter okfn
* @skb: buffer to transmit
*/
int dev_loopback_xmit(struct net *net, struct sock *sk, struct sk_buff *skb)
{
skb_reset_mac_header(skb);
__skb_pull(skb, skb_network_offset(skb));
skb->pkt_type = PACKET_LOOPBACK;
if (skb->ip_summed == CHECKSUM_NONE)
skb->ip_summed = CHECKSUM_UNNECESSARY;
DEBUG_NET_WARN_ON_ONCE(!skb_dst(skb));
skb_dst_force(skb);
netif_rx(skb);
return 0;
}
EXPORT_SYMBOL(dev_loopback_xmit);
#ifdef CONFIG_NET_EGRESS
static struct netdev_queue *
netdev_tx_queue_mapping(struct net_device *dev, struct sk_buff *skb)
{
int qm = skb_get_queue_mapping(skb);
return netdev_get_tx_queue(dev, netdev_cap_txqueue(dev, qm));
}
#ifndef CONFIG_PREEMPT_RT
static bool netdev_xmit_txqueue_skipped(void)
{
return __this_cpu_read(softnet_data.xmit.skip_txqueue);
}
void netdev_xmit_skip_txqueue(bool skip)
{
__this_cpu_write(softnet_data.xmit.skip_txqueue, skip);
}
EXPORT_SYMBOL_GPL(netdev_xmit_skip_txqueue);
#else
static bool netdev_xmit_txqueue_skipped(void)
{
return current->net_xmit.skip_txqueue;
}
void netdev_xmit_skip_txqueue(bool skip)
{
current->net_xmit.skip_txqueue = skip;
}
EXPORT_SYMBOL_GPL(netdev_xmit_skip_txqueue);
#endif
#endif /* CONFIG_NET_EGRESS */
#ifdef CONFIG_NET_XGRESS
static int tc_run(struct tcx_entry *entry, struct sk_buff *skb,
enum skb_drop_reason *drop_reason)
{
int ret = TC_ACT_UNSPEC;
#ifdef CONFIG_NET_CLS_ACT
struct mini_Qdisc *miniq = rcu_dereference_bh(entry->miniq);
struct tcf_result res;
if (!miniq)
return ret;
/* Global bypass */
if (!static_branch_likely(&tcf_sw_enabled_key))
return ret;
/* Block-wise bypass */
if (tcf_block_bypass_sw(miniq->block))
return ret;
tc_skb_cb(skb)->mru = 0;
qdisc_skb_cb(skb)->post_ct = false;
tcf_set_drop_reason(skb, *drop_reason);
mini_qdisc_bstats_cpu_update(miniq, skb);
ret = tcf_classify(skb, miniq->block, miniq->filter_list, &res, false);
/* Only tcf related quirks below. */
switch (ret) {
case TC_ACT_SHOT:
*drop_reason = tcf_get_drop_reason(skb);
mini_qdisc_qstats_cpu_drop(miniq);
break;
case TC_ACT_OK:
case TC_ACT_RECLASSIFY:
skb->tc_index = TC_H_MIN(res.classid);
break;
}
#endif /* CONFIG_NET_CLS_ACT */
return ret;
}
static DEFINE_STATIC_KEY_FALSE(tcx_needed_key);
void tcx_inc(void)
{
static_branch_inc(&tcx_needed_key);
}
void tcx_dec(void)
{
static_branch_dec(&tcx_needed_key);
}
static __always_inline enum tcx_action_base
tcx_run(const struct bpf_mprog_entry *entry, struct sk_buff *skb,
const bool needs_mac)
{
const struct bpf_mprog_fp *fp;
const struct bpf_prog *prog;
int ret = TCX_NEXT;
if (needs_mac)
__skb_push(skb, skb->mac_len);
bpf_mprog_foreach_prog(entry, fp, prog) {
bpf_compute_data_pointers(skb);
ret = bpf_prog_run(prog, skb);
if (ret != TCX_NEXT)
break;
}
if (needs_mac)
__skb_pull(skb, skb->mac_len);
return tcx_action_code(skb, ret);
}
static __always_inline struct sk_buff *
sch_handle_ingress(struct sk_buff *skb, struct packet_type **pt_prev, int *ret,
struct net_device *orig_dev, bool *another)
{
struct bpf_mprog_entry *entry = rcu_dereference_bh(skb->dev->tcx_ingress);
enum skb_drop_reason drop_reason = SKB_DROP_REASON_TC_INGRESS;
struct bpf_net_context __bpf_net_ctx, *bpf_net_ctx;
int sch_ret;
if (!entry)
return skb;
bpf_net_ctx = bpf_net_ctx_set(&__bpf_net_ctx);
if (unlikely(*pt_prev)) {
*ret = deliver_skb(skb, *pt_prev, orig_dev);
*pt_prev = NULL;
}
qdisc_pkt_len_segs_init(skb);
tcx_set_ingress(skb, true);
if (static_branch_unlikely(&tcx_needed_key)) {
sch_ret = tcx_run(entry, skb, true);
if (sch_ret != TC_ACT_UNSPEC)
goto ingress_verdict;
}
sch_ret = tc_run(tcx_entry(entry), skb, &drop_reason);
ingress_verdict:
switch (sch_ret) {
case TC_ACT_REDIRECT:
/* skb_mac_header check was done by BPF, so we can safely
* push the L2 header back before redirecting to another
* netdev.
*/
__skb_push(skb, skb->mac_len);
if (skb_do_redirect(skb) == -EAGAIN) {
__skb_pull(skb, skb->mac_len);
*another = true;
break;
}
*ret = NET_RX_SUCCESS;
bpf_net_ctx_clear(bpf_net_ctx);
return NULL;
case TC_ACT_SHOT:
kfree_skb_reason(skb, drop_reason);
*ret = NET_RX_DROP;
bpf_net_ctx_clear(bpf_net_ctx);
return NULL;
/* used by tc_run */
case TC_ACT_STOLEN:
case TC_ACT_QUEUED:
case TC_ACT_TRAP:
consume_skb(skb);
fallthrough;
case TC_ACT_CONSUMED:
*ret = NET_RX_SUCCESS;
bpf_net_ctx_clear(bpf_net_ctx);
return NULL;
}
bpf_net_ctx_clear(bpf_net_ctx);
return skb;
}
static __always_inline struct sk_buff *
sch_handle_egress(struct sk_buff *skb, int *ret, struct net_device *dev)
{
struct bpf_mprog_entry *entry = rcu_dereference_bh(dev->tcx_egress);
enum skb_drop_reason drop_reason = SKB_DROP_REASON_TC_EGRESS;
struct bpf_net_context __bpf_net_ctx, *bpf_net_ctx;
int sch_ret;
if (!entry)
return skb;
bpf_net_ctx = bpf_net_ctx_set(&__bpf_net_ctx);
/* qdisc_skb_cb(skb)->pkt_len & tcx_set_ingress() was
* already set by the caller.
*/
if (static_branch_unlikely(&tcx_needed_key)) {
sch_ret = tcx_run(entry, skb, false);
if (sch_ret != TC_ACT_UNSPEC)
goto egress_verdict;
}
sch_ret = tc_run(tcx_entry(entry), skb, &drop_reason);
egress_verdict:
switch (sch_ret) {
case TC_ACT_REDIRECT:
/* No need to push/pop skb's mac_header here on egress! */
skb_do_redirect(skb);
*ret = NET_XMIT_SUCCESS;
bpf_net_ctx_clear(bpf_net_ctx);
return NULL;
case TC_ACT_SHOT:
kfree_skb_reason(skb, drop_reason);
*ret = NET_XMIT_DROP;
bpf_net_ctx_clear(bpf_net_ctx);
return NULL;
/* used by tc_run */
case TC_ACT_STOLEN:
case TC_ACT_QUEUED:
case TC_ACT_TRAP:
consume_skb(skb);
fallthrough;
case TC_ACT_CONSUMED:
*ret = NET_XMIT_SUCCESS;
bpf_net_ctx_clear(bpf_net_ctx);
return NULL;
}
bpf_net_ctx_clear(bpf_net_ctx);
return skb;
}
#else
static __always_inline struct sk_buff *
sch_handle_ingress(struct sk_buff *skb, struct packet_type **pt_prev, int *ret,
struct net_device *orig_dev, bool *another)
{
return skb;
}
static __always_inline struct sk_buff *
sch_handle_egress(struct sk_buff *skb, int *ret, struct net_device *dev)
{
return skb;
}
#endif /* CONFIG_NET_XGRESS */
#ifdef CONFIG_XPS
static int __get_xps_queue_idx(struct net_device *dev, struct sk_buff *skb,
struct xps_dev_maps *dev_maps, unsigned int tci)
{
int tc = netdev_get_prio_tc_map(dev, skb->priority);
struct xps_map *map;
int queue_index = -1;
if (tc >= dev_maps->num_tc || tci >= dev_maps->nr_ids)
return queue_index;
tci *= dev_maps->num_tc;
tci += tc;
map = rcu_dereference(dev_maps->attr_map[tci]);
if (map) {
if (map->len == 1)
queue_index = map->queues[0];
else
queue_index = map->queues[reciprocal_scale(
skb_get_hash(skb), map->len)];
if (unlikely(queue_index >= dev->real_num_tx_queues))
queue_index = -1;
}
return queue_index;
}
#endif
static int get_xps_queue(struct net_device *dev, struct net_device *sb_dev,
struct sk_buff *skb)
{
#ifdef CONFIG_XPS
struct xps_dev_maps *dev_maps;
struct sock *sk = skb->sk;
int queue_index = -1;
if (!static_key_false(&xps_needed))
return -1;
rcu_read_lock();
if (!static_key_false(&xps_rxqs_needed))
goto get_cpus_map;
dev_maps = rcu_dereference(sb_dev->xps_maps[XPS_RXQS]);
if (dev_maps) {
int tci = sk_rx_queue_get(sk);
if (tci >= 0)
queue_index = __get_xps_queue_idx(dev, skb, dev_maps,
tci);
}
get_cpus_map:
if (queue_index < 0) {
dev_maps = rcu_dereference(sb_dev->xps_maps[XPS_CPUS]);
if (dev_maps) {
unsigned int tci = skb->sender_cpu - 1;
queue_index = __get_xps_queue_idx(dev, skb, dev_maps,
tci);
}
}
rcu_read_unlock();
return queue_index;
#else
return -1;
#endif
}
u16 dev_pick_tx_zero(struct net_device *dev, struct sk_buff *skb,
struct net_device *sb_dev)
{
return 0;
}
EXPORT_SYMBOL(dev_pick_tx_zero);
int sk_tx_queue_get(const struct sock *sk)
{
int resel, val;
if (!sk)
return -1;
/* Paired with WRITE_ONCE() in sk_tx_queue_clear()
* and sk_tx_queue_set().
*/
val = READ_ONCE(sk->sk_tx_queue_mapping);
if (val == NO_QUEUE_MAPPING)
return -1;
if (!sk_fullsock(sk))
return val;
resel = READ_ONCE(sock_net(sk)->core.sysctl_txq_reselection);
if (resel && time_is_before_jiffies(
READ_ONCE(sk->sk_tx_queue_mapping_jiffies) + resel))
return -1;
return val;
}
EXPORT_SYMBOL(sk_tx_queue_get);
u16 netdev_pick_tx(struct net_device *dev, struct sk_buff *skb,
struct net_device *sb_dev)
{
struct sock *sk = skb->sk;
int queue_index = sk_tx_queue_get(sk);
sb_dev = sb_dev ? : dev;
if (queue_index < 0 || skb->ooo_okay ||
queue_index >= dev->real_num_tx_queues) {
int new_index = get_xps_queue(dev, sb_dev, skb);
if (new_index < 0)
new_index = skb_tx_hash(dev, sb_dev, skb);
if (sk && sk_fullsock(sk) &&
rcu_access_pointer(sk->sk_dst_cache))
sk_tx_queue_set(sk, new_index);
queue_index = new_index;
}
return queue_index;
}
EXPORT_SYMBOL(netdev_pick_tx);
struct netdev_queue *netdev_core_pick_tx(struct net_device *dev,
struct sk_buff *skb,
struct net_device *sb_dev)
{
int queue_index = 0;
#ifdef CONFIG_XPS
u32 sender_cpu = skb->sender_cpu - 1;
if (sender_cpu >= (u32)NR_CPUS)
skb->sender_cpu = raw_smp_processor_id() + 1;
#endif
if (dev->real_num_tx_queues != 1) {
const struct net_device_ops *ops = dev->netdev_ops;
if (ops->ndo_select_queue)
queue_index = ops->ndo_select_queue(dev, skb, sb_dev);
else
queue_index = netdev_pick_tx(dev, skb, sb_dev);
queue_index = netdev_cap_txqueue(dev, queue_index);
}
skb_set_queue_mapping(skb, queue_index);
return netdev_get_tx_queue(dev, queue_index);
}
/**
* __dev_queue_xmit() - transmit a buffer
* @skb: buffer to transmit
* @sb_dev: suboordinate device used for L2 forwarding offload
*
* Queue a buffer for transmission to a network device. The caller must
* have set the device and priority and built the buffer before calling
* this function. The function can be called from an interrupt.
*
* When calling this method, interrupts MUST be enabled. This is because
* the BH enable code must have IRQs enabled so that it will not deadlock.
*
* Regardless of the return value, the skb is consumed, so it is currently
* difficult to retry a send to this method. (You can bump the ref count
* before sending to hold a reference for retry if you are careful.)
*
* Return:
* * 0 - buffer successfully transmitted
* * positive qdisc return code - NET_XMIT_DROP etc.
* * negative errno - other errors
*/
int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev)
{
struct net_device *dev = skb->dev;
struct netdev_queue *txq = NULL;
struct Qdisc *q;
int rc = -ENOMEM;
bool again = false;
skb_reset_mac_header(skb);
skb_assert_len(skb);
if (unlikely(skb_shinfo(skb)->tx_flags &
(SKBTX_SCHED_TSTAMP | SKBTX_BPF)))
__skb_tstamp_tx(skb, NULL, NULL, skb->sk, SCM_TSTAMP_SCHED);
/* Disable soft irqs for various locks below. Also
* stops preemption for RCU.
*/
rcu_read_lock_bh();
skb_update_prio(skb);
qdisc_pkt_len_segs_init(skb);
tcx_set_ingress(skb, false);
#ifdef CONFIG_NET_EGRESS
if (static_branch_unlikely(&egress_needed_key)) {
if (nf_hook_egress_active()) {
skb = nf_hook_egress(skb, &rc, dev);
if (!skb)
goto out;
}
netdev_xmit_skip_txqueue(false);
nf_skip_egress(skb, true);
skb = sch_handle_egress(skb, &rc, dev);
if (!skb)
goto out;
nf_skip_egress(skb, false);
if (netdev_xmit_txqueue_skipped())
txq = netdev_tx_queue_mapping(dev, skb);
}
#endif
/* If device/qdisc don't need skb->dst, release it right now while
* its hot in this cpu cache.
*/
if (dev->priv_flags & IFF_XMIT_DST_RELEASE)
skb_dst_drop(skb);
else
skb_dst_force(skb);
if (!txq)
txq = netdev_core_pick_tx(dev, skb, sb_dev);
q = rcu_dereference_bh(txq->qdisc);
trace_net_dev_queue(skb);
if (q->enqueue) {
rc = __dev_xmit_skb(skb, q, dev, txq);
goto out;
}
/* The device has no queue. Common case for software devices:
* loopback, all the sorts of tunnels...
* Really, it is unlikely that netif_tx_lock protection is necessary
* here. (f.e. loopback and IP tunnels are clean ignoring statistics
* counters.)
* However, it is possible, that they rely on protection
* made by us here.
* Check this and shot the lock. It is not prone from deadlocks.
*Either shot noqueue qdisc, it is even simpler 8)
*/
if (dev->flags & IFF_UP) {
int cpu = smp_processor_id(); /* ok because BHs are off */
/* Other cpus might concurrently change txq->xmit_lock_owner
* to -1 or to their cpu id, but not to our id.
*/
if (READ_ONCE(txq->xmit_lock_owner) != cpu) {
if (dev_xmit_recursion())
goto recursion_alert;
skb = validate_xmit_skb(skb, dev, &again);
if (!skb)
goto out;
HARD_TX_LOCK(dev, txq, cpu);
if (!netif_xmit_stopped(txq)) {
dev_xmit_recursion_inc();
skb = dev_hard_start_xmit(skb, dev, txq, &rc);
dev_xmit_recursion_dec();
if (dev_xmit_complete(rc)) {
HARD_TX_UNLOCK(dev, txq);
goto out;
}
}
HARD_TX_UNLOCK(dev, txq);
net_crit_ratelimited("Virtual device %s asks to queue packet!\n",
dev->name);
} else {
/* Recursion is detected! It is possible,
* unfortunately
*/
recursion_alert:
net_crit_ratelimited("Dead loop on virtual device %s, fix it urgently!\n",
dev->name);
}
}
rc = -ENETDOWN;
rcu_read_unlock_bh();
dev_core_stats_tx_dropped_inc(dev);
kfree_skb_list(skb);
return rc;
out:
rcu_read_unlock_bh();
return rc;
}
EXPORT_SYMBOL(__dev_queue_xmit);
int __dev_direct_xmit(struct sk_buff *skb, u16 queue_id)
{
struct net_device *dev = skb->dev;
struct sk_buff *orig_skb = skb;
struct netdev_queue *txq;
int ret = NETDEV_TX_BUSY;
bool again = false;
if (unlikely(!netif_running(dev) ||
!netif_carrier_ok(dev)))
goto drop;
skb = validate_xmit_skb_list(skb, dev, &again);
if (skb != orig_skb)
goto drop;
skb_set_queue_mapping(skb, queue_id);
txq = skb_get_tx_queue(dev, skb);
local_bh_disable();
dev_xmit_recursion_inc();
HARD_TX_LOCK(dev, txq, smp_processor_id());
if (!netif_xmit_frozen_or_drv_stopped(txq))
ret = netdev_start_xmit(skb, dev, txq, false);
HARD_TX_UNLOCK(dev, txq);
dev_xmit_recursion_dec();
local_bh_enable();
return ret;
drop:
dev_core_stats_tx_dropped_inc(dev);
kfree_skb_list(skb);
return NET_XMIT_DROP;
}
EXPORT_SYMBOL(__dev_direct_xmit);
/*************************************************************************
* Receiver routines
*************************************************************************/
static DEFINE_PER_CPU(struct task_struct *, backlog_napi);
int weight_p __read_mostly = 64; /* old backlog weight */
int dev_weight_rx_bias __read_mostly = 1; /* bias for backlog weight */
int dev_weight_tx_bias __read_mostly = 1; /* bias for output_queue quota */
/* Called with irq disabled */
static inline void ____napi_schedule(struct softnet_data *sd,
struct napi_struct *napi)
{
struct task_struct *thread;
lockdep_assert_irqs_disabled();
if (test_bit(NAPI_STATE_THREADED, &napi->state)) {
/* Paired with smp_mb__before_atomic() in
* napi_enable()/netif_set_threaded().
* Use READ_ONCE() to guarantee a complete
* read on napi->thread. Only call
* wake_up_process() when it's not NULL.
*/
thread = READ_ONCE(napi->thread);
if (thread) {
if (use_backlog_threads() && thread == raw_cpu_read(backlog_napi))
goto use_local_napi;
set_bit(NAPI_STATE_SCHED_THREADED, &napi->state);
wake_up_process(thread);
return;
}
}
use_local_napi:
DEBUG_NET_WARN_ON_ONCE(!list_empty(&napi->poll_list));
list_add_tail(&napi->poll_list, &sd->poll_list);
WRITE_ONCE(napi->list_owner, smp_processor_id());
/* If not called from net_rx_action()
* we have to raise NET_RX_SOFTIRQ.
*/
if (!sd->in_net_rx_action)
raise_softirq_irqoff(NET_RX_SOFTIRQ);
}
#ifdef CONFIG_RPS
struct static_key_false rps_needed __read_mostly;
EXPORT_SYMBOL(rps_needed);
struct static_key_false rfs_needed __read_mostly;
EXPORT_SYMBOL(rfs_needed);
static u32 rfs_slot(u32 hash, const struct rps_dev_flow_table *flow_table)
{
return hash_32(hash, flow_table->log);
}
#ifdef CONFIG_RFS_ACCEL
/**
* rps_flow_is_active - check whether the flow is recently active.
* @rflow: Specific flow to check activity.
* @flow_table: per-queue flowtable that @rflow belongs to.
* @cpu: CPU saved in @rflow.
*
* If the CPU has processed many packets since the flow's last activity
* (beyond 10 times the table size), the flow is considered stale.
*
* Return: true if flow was recently active.
*/
static bool rps_flow_is_active(struct rps_dev_flow *rflow,
struct rps_dev_flow_table *flow_table,
unsigned int cpu)
{
unsigned int flow_last_active;
unsigned int sd_input_head;
if (cpu >= nr_cpu_ids)
return false;
sd_input_head = READ_ONCE(per_cpu(softnet_data, cpu).input_queue_head);
flow_last_active = READ_ONCE(rflow->last_qtail);
return (int)(sd_input_head - flow_last_active) <
(int)(10 << flow_table->log);
}
#endif
static struct rps_dev_flow *
set_rps_cpu(struct net_device *dev, struct sk_buff *skb,
struct rps_dev_flow *rflow, u16 next_cpu, u32 hash,
u32 flow_id)
{
if (next_cpu < nr_cpu_ids) {
u32 head;
#ifdef CONFIG_RFS_ACCEL
struct netdev_rx_queue *rxqueue;
struct rps_dev_flow_table *flow_table;
struct rps_dev_flow *old_rflow;
struct rps_dev_flow *tmp_rflow;
unsigned int tmp_cpu;
u16 rxq_index;
int rc;
/* Should we steer this flow to a different hardware queue? */
if (!skb_rx_queue_recorded(skb) || !dev->rx_cpu_rmap ||
!(dev->features & NETIF_F_NTUPLE))
goto out;
rxq_index = cpu_rmap_lookup_index(dev->rx_cpu_rmap, next_cpu);
if (rxq_index == skb_get_rx_queue(skb))
goto out;
rxqueue = dev->_rx + rxq_index;
flow_table = rcu_dereference(rxqueue->rps_flow_table);
if (!flow_table)
goto out;
tmp_rflow = &flow_table->flows[flow_id];
tmp_cpu = READ_ONCE(tmp_rflow->cpu);
if (READ_ONCE(tmp_rflow->filter) != RPS_NO_FILTER) {
if (rps_flow_is_active(tmp_rflow, flow_table,
tmp_cpu)) {
if (hash != READ_ONCE(tmp_rflow->hash) ||
next_cpu == tmp_cpu)
goto out;
}
}
rc = dev->netdev_ops->ndo_rx_flow_steer(dev, skb,
rxq_index, flow_id);
if (rc < 0)
goto out;
old_rflow = rflow;
rflow = tmp_rflow;
WRITE_ONCE(rflow->filter, rc);
WRITE_ONCE(rflow->hash, hash);
if (old_rflow->filter == rc)
WRITE_ONCE(old_rflow->filter, RPS_NO_FILTER);
out:
#endif
head = READ_ONCE(per_cpu(softnet_data, next_cpu).input_queue_head);
rps_input_queue_tail_save(&rflow->last_qtail, head);
}
WRITE_ONCE(rflow->cpu, next_cpu);
return rflow;
}
/*
* get_rps_cpu is called from netif_receive_skb and returns the target
* CPU from the RPS map of the receiving queue for a given skb.
* rcu_read_lock must be held on entry.
*/
static int get_rps_cpu(struct net_device *dev, struct sk_buff *skb,
struct rps_dev_flow **rflowp)
{
const struct rps_sock_flow_table *sock_flow_table;
struct netdev_rx_queue *rxqueue = dev->_rx;
struct rps_dev_flow_table *flow_table;
struct rps_map *map;
int cpu = -1;
u32 flow_id;
u32 tcpu;
u32 hash;
if (skb_rx_queue_recorded(skb)) {
u16 index = skb_get_rx_queue(skb);
if (unlikely(index >= dev->real_num_rx_queues)) {
WARN_ONCE(dev->real_num_rx_queues > 1,
"%s received packet on queue %u, but number "
"of RX queues is %u\n",
dev->name, index, dev->real_num_rx_queues);
goto done;
}
rxqueue += index;
}
/* Avoid computing hash if RFS/RPS is not active for this rxqueue */
flow_table = rcu_dereference(rxqueue->rps_flow_table);
map = rcu_dereference(rxqueue->rps_map);
if (!flow_table && !map)
goto done;
skb_reset_network_header(skb);
hash = skb_get_hash(skb);
if (!hash)
goto done;
sock_flow_table = rcu_dereference(net_hotdata.rps_sock_flow_table);
if (flow_table && sock_flow_table) {
struct rps_dev_flow *rflow;
u32 next_cpu;
u32 ident;
/* First check into global flow table if there is a match.
* This READ_ONCE() pairs with WRITE_ONCE() from rps_record_sock_flow().
*/
ident = READ_ONCE(sock_flow_table->ents[hash & sock_flow_table->mask]);
if ((ident ^ hash) & ~net_hotdata.rps_cpu_mask)
goto try_rps;
next_cpu = ident & net_hotdata.rps_cpu_mask;
/* OK, now we know there is a match,
* we can look at the local (per receive queue) flow table
*/
flow_id = rfs_slot(hash, flow_table);
rflow = &flow_table->flows[flow_id];
tcpu = rflow->cpu;
/*
* If the desired CPU (where last recvmsg was done) is
* different from current CPU (one in the rx-queue flow
* table entry), switch if one of the following holds:
* - Current CPU is unset (>= nr_cpu_ids).
* - Current CPU is offline.
* - The current CPU's queue tail has advanced beyond the
* last packet that was enqueued using this table entry.
* This guarantees that all previous packets for the flow
* have been dequeued, thus preserving in order delivery.
*/
if (unlikely(tcpu != next_cpu) &&
(tcpu >= nr_cpu_ids || !cpu_online(tcpu) ||
((int)(READ_ONCE(per_cpu(softnet_data, tcpu).input_queue_head) -
rflow->last_qtail)) >= 0)) {
tcpu = next_cpu;
rflow = set_rps_cpu(dev, skb, rflow, next_cpu, hash,
flow_id);
}
if (tcpu < nr_cpu_ids && cpu_online(tcpu)) {
*rflowp = rflow;
cpu = tcpu;
goto done;
}
}
try_rps:
if (map) {
tcpu = map->cpus[reciprocal_scale(hash, map->len)];
if (cpu_online(tcpu)) {
cpu = tcpu;
goto done;
}
}
done:
return cpu;
}
#ifdef CONFIG_RFS_ACCEL
/**
* rps_may_expire_flow - check whether an RFS hardware filter may be removed
* @dev: Device on which the filter was set
* @rxq_index: RX queue index
* @flow_id: Flow ID passed to ndo_rx_flow_steer()
* @filter_id: Filter ID returned by ndo_rx_flow_steer()
*
* Drivers that implement ndo_rx_flow_steer() should periodically call
* this function for each installed filter and remove the filters for
* which it returns %true.
*/
bool rps_may_expire_flow(struct net_device *dev, u16 rxq_index,
u32 flow_id, u16 filter_id)
{
struct netdev_rx_queue *rxqueue = dev->_rx + rxq_index;
struct rps_dev_flow_table *flow_table;
struct rps_dev_flow *rflow;
bool expire = true;
rcu_read_lock();
flow_table = rcu_dereference(rxqueue->rps_flow_table);
if (flow_table && flow_id < (1UL << flow_table->log)) {
unsigned int cpu;
rflow = &flow_table->flows[flow_id];
cpu = READ_ONCE(rflow->cpu);
if (READ_ONCE(rflow->filter) == filter_id &&
rps_flow_is_active(rflow, flow_table, cpu))
expire = false;
}
rcu_read_unlock();
return expire;
}
EXPORT_SYMBOL(rps_may_expire_flow);
#endif /* CONFIG_RFS_ACCEL */
/* Called from hardirq (IPI) context */
static void rps_trigger_softirq(void *data)
{
struct softnet_data *sd = data;
____napi_schedule(sd, &sd->backlog);
/* Pairs with READ_ONCE() in softnet_seq_show() */
WRITE_ONCE(sd->received_rps, sd->received_rps + 1);
}
#endif /* CONFIG_RPS */
/* Called from hardirq (IPI) context */
static void trigger_rx_softirq(void *data)
{
struct softnet_data *sd = data;
__raise_softirq_irqoff(NET_RX_SOFTIRQ);
smp_store_release(&sd->defer_ipi_scheduled, 0);
}
/*
* After we queued a packet into sd->input_pkt_queue,
* we need to make sure this queue is serviced soon.
*
* - If this is another cpu queue, link it to our rps_ipi_list,
* and make sure we will process rps_ipi_list from net_rx_action().
*
* - If this is our own queue, NAPI schedule our backlog.
* Note that this also raises NET_RX_SOFTIRQ.
*/
static void napi_schedule_rps(struct softnet_data *sd)
{
struct softnet_data *mysd = this_cpu_ptr(&softnet_data);
#ifdef CONFIG_RPS
if (sd != mysd) {
if (use_backlog_threads()) {
__napi_schedule_irqoff(&sd->backlog);
return;
}
sd->rps_ipi_next = mysd->rps_ipi_list;
mysd->rps_ipi_list = sd;
/* If not called from net_rx_action() or napi_threaded_poll()
* we have to raise NET_RX_SOFTIRQ.
*/
if (!mysd->in_net_rx_action && !mysd->in_napi_threaded_poll)
__raise_softirq_irqoff(NET_RX_SOFTIRQ);
return;
}
#endif /* CONFIG_RPS */
__napi_schedule_irqoff(&mysd->backlog);
}
void kick_defer_list_purge(unsigned int cpu)
{
struct softnet_data *sd = &per_cpu(softnet_data, cpu);
unsigned long flags;
if (use_backlog_threads()) {
backlog_lock_irq_save(sd, &flags);
if (!__test_and_set_bit(NAPI_STATE_SCHED, &sd->backlog.state))
__napi_schedule_irqoff(&sd->backlog);
backlog_unlock_irq_restore(sd, &flags);
} else if (!cmpxchg(&sd->defer_ipi_scheduled, 0, 1)) {
smp_call_function_single_async(cpu, &sd->defer_csd);
}
}
#ifdef CONFIG_NET_FLOW_LIMIT
int netdev_flow_limit_table_len __read_mostly = (1 << 12);
#endif
static bool skb_flow_limit(struct sk_buff *skb, unsigned int qlen,
int max_backlog)
{
#ifdef CONFIG_NET_FLOW_LIMIT
unsigned int old_flow, new_flow;
const struct softnet_data *sd;
struct sd_flow_limit *fl;
if (likely(qlen < (max_backlog >> 1)))
return false;
sd = this_cpu_ptr(&softnet_data);
rcu_read_lock();
fl = rcu_dereference(sd->flow_limit);
if (fl) {
new_flow = hash_32(skb_get_hash(skb), fl->log_buckets);
old_flow = fl->history[fl->history_head];
fl->history[fl->history_head] = new_flow;
fl->history_head++;
fl->history_head &= FLOW_LIMIT_HISTORY - 1;
if (likely(fl->buckets[old_flow]))
fl->buckets[old_flow]--;
if (++fl->buckets[new_flow] > (FLOW_LIMIT_HISTORY >> 1)) {
/* Pairs with READ_ONCE() in softnet_seq_show() */
WRITE_ONCE(fl->count, fl->count + 1);
rcu_read_unlock();
return true;
}
}
rcu_read_unlock();
#endif
return false;
}
/*
* enqueue_to_backlog is called to queue an skb to a per CPU backlog
* queue (may be a remote CPU queue).
*/
static int enqueue_to_backlog(struct sk_buff *skb, int cpu,
unsigned int *qtail)
{
enum skb_drop_reason reason;
struct softnet_data *sd;
unsigned long flags;
unsigned int qlen;
int max_backlog;
u32 tail;
reason = SKB_DROP_REASON_DEV_READY;
if (unlikely(!netif_running(skb->dev)))
goto bad_dev;
sd = &per_cpu(softnet_data, cpu);
qlen = skb_queue_len_lockless(&sd->input_pkt_queue);
max_backlog = READ_ONCE(net_hotdata.max_backlog);
if (unlikely(qlen > max_backlog) ||
skb_flow_limit(skb, qlen, max_backlog))
goto cpu_backlog_drop;
backlog_lock_irq_save(sd, &flags);
qlen = skb_queue_len(&sd->input_pkt_queue);
if (likely(qlen <= max_backlog)) {
if (!qlen) {
/* Schedule NAPI for backlog device. We can use
* non atomic operation as we own the queue lock.
*/
if (!__test_and_set_bit(NAPI_STATE_SCHED,
&sd->backlog.state))
napi_schedule_rps(sd);
}
__skb_queue_tail(&sd->input_pkt_queue, skb);
tail = rps_input_queue_tail_incr(sd);
backlog_unlock_irq_restore(sd, &flags);
/* save the tail outside of the critical section */
rps_input_queue_tail_save(qtail, tail);
return NET_RX_SUCCESS;
}
backlog_unlock_irq_restore(sd, &flags);
cpu_backlog_drop:
reason = SKB_DROP_REASON_CPU_BACKLOG;
numa_drop_add(&sd->drop_counters, 1);
bad_dev:
dev_core_stats_rx_dropped_inc(skb->dev);
kfree_skb_reason(skb, reason);
return NET_RX_DROP;
}
static struct netdev_rx_queue *netif_get_rxqueue(struct sk_buff *skb)
{
struct net_device *dev = skb->dev;
struct netdev_rx_queue *rxqueue;
rxqueue = dev->_rx;
if (skb_rx_queue_recorded(skb)) {
u16 index = skb_get_rx_queue(skb);
if (unlikely(index >= dev->real_num_rx_queues)) {
WARN_ONCE(dev->real_num_rx_queues > 1,
"%s received packet on queue %u, but number "
"of RX queues is %u\n",
dev->name, index, dev->real_num_rx_queues);
return rxqueue; /* Return first rxqueue */
}
rxqueue += index;
}
return rxqueue;
}
u32 bpf_prog_run_generic_xdp(struct sk_buff *skb, struct xdp_buff *xdp,
const struct bpf_prog *xdp_prog)
{
void *orig_data, *orig_data_end, *hard_start;
struct netdev_rx_queue *rxqueue;
bool orig_bcast, orig_host;
u32 mac_len, frame_sz;
__be16 orig_eth_type;
struct ethhdr *eth;
u32 metalen, act;
int off;
/* The XDP program wants to see the packet starting at the MAC
* header.
*/
mac_len = skb->data - skb_mac_header(skb);
hard_start = skb->data - skb_headroom(skb);
/* SKB "head" area always have tailroom for skb_shared_info */
frame_sz = (void *)skb_end_pointer(skb) - hard_start;
frame_sz += SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
rxqueue = netif_get_rxqueue(skb);
xdp_init_buff(xdp, frame_sz, &rxqueue->xdp_rxq);
xdp_prepare_buff(xdp, hard_start, skb_headroom(skb) - mac_len,
skb_headlen(skb) + mac_len, true);
if (skb_is_nonlinear(skb)) {
skb_shinfo(skb)->xdp_frags_size = skb->data_len;
xdp_buff_set_frags_flag(xdp);
} else {
xdp_buff_clear_frags_flag(xdp);
}
orig_data_end = xdp->data_end;
orig_data = xdp->data;
eth = (struct ethhdr *)xdp->data;
orig_host = ether_addr_equal_64bits(eth->h_dest, skb->dev->dev_addr);
orig_bcast = is_multicast_ether_addr_64bits(eth->h_dest);
orig_eth_type = eth->h_proto;
act = bpf_prog_run_xdp(xdp_prog, xdp);
/* check if bpf_xdp_adjust_head was used */
off = xdp->data - orig_data;
if (off) {
if (off > 0)
__skb_pull(skb, off);
else if (off < 0)
__skb_push(skb, -off);
skb->mac_header += off;
skb_reset_network_header(skb);
}
/* check if bpf_xdp_adjust_tail was used */
off = xdp->data_end - orig_data_end;
if (off != 0) {
skb_set_tail_pointer(skb, xdp->data_end - xdp->data);
skb->len += off; /* positive on grow, negative on shrink */
}
/* XDP frag metadata (e.g. nr_frags) are updated in eBPF helpers
* (e.g. bpf_xdp_adjust_tail), we need to update data_len here.
*/
if (xdp_buff_has_frags(xdp))
skb->data_len = skb_shinfo(skb)->xdp_frags_size;
else
skb->data_len = 0;
/* check if XDP changed eth hdr such SKB needs update */
eth = (struct ethhdr *)xdp->data;
if ((orig_eth_type != eth->h_proto) ||
(orig_host != ether_addr_equal_64bits(eth->h_dest,
skb->dev->dev_addr)) ||
(orig_bcast != is_multicast_ether_addr_64bits(eth->h_dest))) {
__skb_push(skb, ETH_HLEN);
skb->pkt_type = PACKET_HOST;
skb->protocol = eth_type_trans(skb, skb->dev);
}
/* Redirect/Tx gives L2 packet, code that will reuse skb must __skb_pull
* before calling us again on redirect path. We do not call do_redirect
* as we leave that up to the caller.
*
* Caller is responsible for managing lifetime of skb (i.e. calling
* kfree_skb in response to actions it cannot handle/XDP_DROP).
*/
switch (act) {
case XDP_REDIRECT:
case XDP_TX:
__skb_push(skb, mac_len);
break;
case XDP_PASS:
metalen = xdp->data - xdp->data_meta;
if (metalen)
skb_metadata_set(skb, metalen);
break;
}
return act;
}
static int
netif_skb_check_for_xdp(struct sk_buff **pskb, const struct bpf_prog *prog)
{
struct sk_buff *skb = *pskb;
int err, hroom, troom;
local_lock_nested_bh(&system_page_pool.bh_lock);
err = skb_cow_data_for_xdp(this_cpu_read(system_page_pool.pool), pskb, prog);
local_unlock_nested_bh(&system_page_pool.bh_lock);
if (!err)
return 0;
/* In case we have to go down the path and also linearize,
* then lets do the pskb_expand_head() work just once here.
*/
hroom = XDP_PACKET_HEADROOM - skb_headroom(skb);
troom = skb->tail + skb->data_len - skb->end;
err = pskb_expand_head(skb,
hroom > 0 ? ALIGN(hroom, NET_SKB_PAD) : 0,
troom > 0 ? troom + 128 : 0, GFP_ATOMIC);
if (err)
return err;
return skb_linearize(skb);
}
static u32 netif_receive_generic_xdp(struct sk_buff **pskb,
struct xdp_buff *xdp,
const struct bpf_prog *xdp_prog)
{
struct sk_buff *skb = *pskb;
u32 mac_len, act = XDP_DROP;
/* Reinjected packets coming from act_mirred or similar should
* not get XDP generic processing.
*/
if (skb_is_redirected(skb))
return XDP_PASS;
/* XDP packets must have sufficient headroom of XDP_PACKET_HEADROOM
* bytes. This is the guarantee that also native XDP provides,
* thus we need to do it here as well.
*/
mac_len = skb->data - skb_mac_header(skb);
__skb_push(skb, mac_len);
if (skb_cloned(skb) || skb_is_nonlinear(skb) ||
skb_headroom(skb) < XDP_PACKET_HEADROOM) {
if (netif_skb_check_for_xdp(pskb, xdp_prog))
goto do_drop;
}
__skb_pull(*pskb, mac_len);
act = bpf_prog_run_generic_xdp(*pskb, xdp, xdp_prog);
switch (act) {
case XDP_REDIRECT:
case XDP_TX:
case XDP_PASS:
break;
default:
bpf_warn_invalid_xdp_action((*pskb)->dev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception((*pskb)->dev, xdp_prog, act);
fallthrough;
case XDP_DROP:
do_drop:
kfree_skb(*pskb);
break;
}
return act;
}
/* When doing generic XDP we have to bypass the qdisc layer and the
* network taps in order to match in-driver-XDP behavior. This also means
* that XDP packets are able to starve other packets going through a qdisc,
* and DDOS attacks will be more effective. In-driver-XDP use dedicated TX
* queues, so they do not have this starvation issue.
*/
void generic_xdp_tx(struct sk_buff *skb, const struct bpf_prog *xdp_prog)
{
struct net_device *dev = skb->dev;
struct netdev_queue *txq;
bool free_skb = true;
int cpu, rc;
txq = netdev_core_pick_tx(dev, skb, NULL);
cpu = smp_processor_id();
HARD_TX_LOCK(dev, txq, cpu);
if (!netif_xmit_frozen_or_drv_stopped(txq)) {
rc = netdev_start_xmit(skb, dev, txq, 0);
if (dev_xmit_complete(rc))
free_skb = false;
}
HARD_TX_UNLOCK(dev, txq);
if (free_skb) {
trace_xdp_exception(dev, xdp_prog, XDP_TX);
dev_core_stats_tx_dropped_inc(dev);
kfree_skb(skb);
}
}
static DEFINE_STATIC_KEY_FALSE(generic_xdp_needed_key);
int do_xdp_generic(const struct bpf_prog *xdp_prog, struct sk_buff **pskb)
{
struct bpf_net_context __bpf_net_ctx, *bpf_net_ctx;
if (xdp_prog) {
struct xdp_buff xdp;
u32 act;
int err;
bpf_net_ctx = bpf_net_ctx_set(&__bpf_net_ctx);
act = netif_receive_generic_xdp(pskb, &xdp, xdp_prog);
if (act != XDP_PASS) {
switch (act) {
case XDP_REDIRECT:
err = xdp_do_generic_redirect((*pskb)->dev, *pskb,
&xdp, xdp_prog);
if (err)
goto out_redir;
break;
case XDP_TX:
generic_xdp_tx(*pskb, xdp_prog);
break;
}
bpf_net_ctx_clear(bpf_net_ctx);
return XDP_DROP;
}
bpf_net_ctx_clear(bpf_net_ctx);
}
return XDP_PASS;
out_redir:
bpf_net_ctx_clear(bpf_net_ctx);
kfree_skb_reason(*pskb, SKB_DROP_REASON_XDP);
return XDP_DROP;
}
EXPORT_SYMBOL_GPL(do_xdp_generic);
static int netif_rx_internal(struct sk_buff *skb)
{
int ret;
net_timestamp_check(READ_ONCE(net_hotdata.tstamp_prequeue), skb);
trace_netif_rx(skb);
#ifdef CONFIG_RPS
if (static_branch_unlikely(&rps_needed)) {
struct rps_dev_flow voidflow, *rflow = &voidflow;
int cpu;
rcu_read_lock();
cpu = get_rps_cpu(skb->dev, skb, &rflow);
if (cpu < 0)
cpu = smp_processor_id();
ret = enqueue_to_backlog(skb, cpu, &rflow->last_qtail);
rcu_read_unlock();
} else
#endif
{
unsigned int qtail;
ret = enqueue_to_backlog(skb, smp_processor_id(), &qtail);
}
return ret;
}
/**
* __netif_rx - Slightly optimized version of netif_rx
* @skb: buffer to post
*
* This behaves as netif_rx except that it does not disable bottom halves.
* As a result this function may only be invoked from the interrupt context
* (either hard or soft interrupt).
*/
int __netif_rx(struct sk_buff *skb)
{
int ret;
lockdep_assert_once(hardirq_count() | softirq_count());
trace_netif_rx_entry(skb);
ret = netif_rx_internal(skb);
trace_netif_rx_exit(ret);
return ret;
}
EXPORT_SYMBOL(__netif_rx);
/**
* netif_rx - post buffer to the network code
* @skb: buffer to post
*
* This function receives a packet from a device driver and queues it for
* the upper (protocol) levels to process via the backlog NAPI device. It
* always succeeds. The buffer may be dropped during processing for
* congestion control or by the protocol layers.
* The network buffer is passed via the backlog NAPI device. Modern NIC
* driver should use NAPI and GRO.
* This function can used from interrupt and from process context. The
* caller from process context must not disable interrupts before invoking
* this function.
*
* return values:
* NET_RX_SUCCESS (no congestion)
* NET_RX_DROP (packet was dropped)
*
*/
int netif_rx(struct sk_buff *skb)
{
bool need_bh_off = !(hardirq_count() | softirq_count());
int ret;
if (need_bh_off)
local_bh_disable();
trace_netif_rx_entry(skb);
ret = netif_rx_internal(skb);
trace_netif_rx_exit(ret);
if (need_bh_off)
local_bh_enable();
return ret;
}
EXPORT_SYMBOL(netif_rx);
static __latent_entropy void net_tx_action(void)
{
struct softnet_data *sd = this_cpu_ptr(&softnet_data);
if (sd->completion_queue) {
struct sk_buff *clist;
local_irq_disable();
clist = sd->completion_queue;
sd->completion_queue = NULL;
local_irq_enable();
while (clist) {
struct sk_buff *skb = clist;
clist = clist->next;
WARN_ON(refcount_read(&skb->users));
if (likely(get_kfree_skb_cb(skb)->reason == SKB_CONSUMED))
trace_consume_skb(skb, net_tx_action);
else
trace_kfree_skb(skb, net_tx_action,
get_kfree_skb_cb(skb)->reason, NULL);
if (skb->fclone != SKB_FCLONE_UNAVAILABLE)
__kfree_skb(skb);
else
__napi_kfree_skb(skb,
get_kfree_skb_cb(skb)->reason);
}
}
if (sd->output_queue) {
struct Qdisc *head;
local_irq_disable();
head = sd->output_queue;
sd->output_queue = NULL;
sd->output_queue_tailp = &sd->output_queue;
local_irq_enable();
rcu_read_lock();
while (head) {
spinlock_t *root_lock = NULL;
struct sk_buff *to_free;
struct Qdisc *q = head;
head = head->next_sched;
/* We need to make sure head->next_sched is read
* before clearing __QDISC_STATE_SCHED
*/
smp_mb__before_atomic();
if (!(q->flags & TCQ_F_NOLOCK)) {
root_lock = qdisc_lock(q);
spin_lock(root_lock);
} else if (unlikely(test_bit(__QDISC_STATE_DEACTIVATED,
&q->state))) {
/* There is a synchronize_net() between
* STATE_DEACTIVATED flag being set and
* qdisc_reset()/some_qdisc_is_busy() in
* dev_deactivate(), so we can safely bail out
* early here to avoid data race between
* qdisc_deactivate() and some_qdisc_is_busy()
* for lockless qdisc.
*/
clear_bit(__QDISC_STATE_SCHED, &q->state);
continue;
}
clear_bit(__QDISC_STATE_SCHED, &q->state);
to_free = qdisc_run(q);
if (root_lock)
spin_unlock(root_lock);
tcf_kfree_skb_list(to_free);
}
rcu_read_unlock();
}
xfrm_dev_backlog(sd);
}
#if IS_ENABLED(CONFIG_BRIDGE) && IS_ENABLED(CONFIG_ATM_LANE)
/* This hook is defined here for ATM LANE */
int (*br_fdb_test_addr_hook)(struct net_device *dev,
unsigned char *addr) __read_mostly;
EXPORT_SYMBOL_GPL(br_fdb_test_addr_hook);
#endif
/**
* netdev_is_rx_handler_busy - check if receive handler is registered
* @dev: device to check
*
* Check if a receive handler is already registered for a given device.
* Return true if there one.
*
* The caller must hold the rtnl_mutex.
*/
bool netdev_is_rx_handler_busy(struct net_device *dev)
{
ASSERT_RTNL();
return dev && rtnl_dereference(dev->rx_handler);
}
EXPORT_SYMBOL_GPL(netdev_is_rx_handler_busy);
/**
* netdev_rx_handler_register - register receive handler
* @dev: device to register a handler for
* @rx_handler: receive handler to register
* @rx_handler_data: data pointer that is used by rx handler
*
* Register a receive handler for a device. This handler will then be
* called from __netif_receive_skb. A negative errno code is returned
* on a failure.
*
* The caller must hold the rtnl_mutex.
*
* For a general description of rx_handler, see enum rx_handler_result.
*/
int netdev_rx_handler_register(struct net_device *dev,
rx_handler_func_t *rx_handler,
void *rx_handler_data)
{
if (netdev_is_rx_handler_busy(dev))
return -EBUSY;
if (dev->priv_flags & IFF_NO_RX_HANDLER)
return -EINVAL;
/* Note: rx_handler_data must be set before rx_handler */
rcu_assign_pointer(dev->rx_handler_data, rx_handler_data);
rcu_assign_pointer(dev->rx_handler, rx_handler);
return 0;
}
EXPORT_SYMBOL_GPL(netdev_rx_handler_register);
/**
* netdev_rx_handler_unregister - unregister receive handler
* @dev: device to unregister a handler from
*
* Unregister a receive handler from a device.
*
* The caller must hold the rtnl_mutex.
*/
void netdev_rx_handler_unregister(struct net_device *dev)
{
ASSERT_RTNL();
RCU_INIT_POINTER(dev->rx_handler, NULL);
/* a reader seeing a non NULL rx_handler in a rcu_read_lock()
* section has a guarantee to see a non NULL rx_handler_data
* as well.
*/
synchronize_net();
RCU_INIT_POINTER(dev->rx_handler_data, NULL);
}
EXPORT_SYMBOL_GPL(netdev_rx_handler_unregister);
/*
* Limit the use of PFMEMALLOC reserves to those protocols that implement
* the special handling of PFMEMALLOC skbs.
*/
static bool skb_pfmemalloc_protocol(struct sk_buff *skb)
{
switch (skb->protocol) {
case htons(ETH_P_ARP):
case htons(ETH_P_IP):
case htons(ETH_P_IPV6):
case htons(ETH_P_8021Q):
case htons(ETH_P_8021AD):
return true;
default:
return false;
}
}
static inline int nf_ingress(struct sk_buff *skb, struct packet_type **pt_prev,
int *ret, struct net_device *orig_dev)
{
if (nf_hook_ingress_active(skb)) {
int ingress_retval;
if (unlikely(*pt_prev)) {
*ret = deliver_skb(skb, *pt_prev, orig_dev);
*pt_prev = NULL;
}
rcu_read_lock();
ingress_retval = nf_hook_ingress(skb);
rcu_read_unlock();
return ingress_retval;
}
return 0;
}
static int __netif_receive_skb_core(struct sk_buff **pskb, bool pfmemalloc,
struct packet_type **ppt_prev)
{
enum skb_drop_reason drop_reason = SKB_DROP_REASON_UNHANDLED_PROTO;
struct packet_type *ptype, *pt_prev;
rx_handler_func_t *rx_handler;
struct sk_buff *skb = *pskb;
struct net_device *orig_dev;
bool deliver_exact = false;
int ret = NET_RX_DROP;
__be16 type;
net_timestamp_check(!READ_ONCE(net_hotdata.tstamp_prequeue), skb);
trace_netif_receive_skb(skb);
orig_dev = skb->dev;
skb_reset_network_header(skb);
#if !defined(CONFIG_DEBUG_NET)
/* We plan to no longer reset the transport header here.
* Give some time to fuzzers and dev build to catch bugs
* in network stacks.
*/
if (!skb_transport_header_was_set(skb))
skb_reset_transport_header(skb);
#endif
skb_reset_mac_len(skb);
pt_prev = NULL;
another_round:
skb->skb_iif = skb->dev->ifindex;
__this_cpu_inc(softnet_data.processed);
if (static_branch_unlikely(&generic_xdp_needed_key)) {
int ret2;
migrate_disable();
ret2 = do_xdp_generic(rcu_dereference(skb->dev->xdp_prog),
&skb);
migrate_enable();
if (ret2 != XDP_PASS) {
ret = NET_RX_DROP;
goto out;
}
}
if (eth_type_vlan(skb->protocol)) {
skb = skb_vlan_untag(skb);
if (unlikely(!skb))
goto out;
}
if (skb_skip_tc_classify(skb))
goto skip_classify;
if (pfmemalloc)
goto skip_taps;
list_for_each_entry_rcu(ptype, &dev_net_rcu(skb->dev)->ptype_all,
list) {
if (unlikely(pt_prev))
ret = deliver_skb(skb, pt_prev, orig_dev);
pt_prev = ptype;
}
list_for_each_entry_rcu(ptype, &skb->dev->ptype_all, list) {
if (unlikely(pt_prev))
ret = deliver_skb(skb, pt_prev, orig_dev);
pt_prev = ptype;
}
skip_taps:
#ifdef CONFIG_NET_INGRESS
if (static_branch_unlikely(&ingress_needed_key)) {
bool another = false;
nf_skip_egress(skb, true);
skb = sch_handle_ingress(skb, &pt_prev, &ret, orig_dev,
&another);
if (another)
goto another_round;
if (!skb)
goto out;
nf_skip_egress(skb, false);
if (nf_ingress(skb, &pt_prev, &ret, orig_dev) < 0)
goto out;
}
#endif
skb_reset_redirect(skb);
skip_classify:
if (pfmemalloc && !skb_pfmemalloc_protocol(skb)) {
drop_reason = SKB_DROP_REASON_PFMEMALLOC;
goto drop;
}
if (skb_vlan_tag_present(skb)) {
if (unlikely(pt_prev)) {
ret = deliver_skb(skb, pt_prev, orig_dev);
pt_prev = NULL;
}
if (vlan_do_receive(&skb))
goto another_round;
else if (unlikely(!skb))
goto out;
}
rx_handler = rcu_dereference(skb->dev->rx_handler);
if (rx_handler) {
if (unlikely(pt_prev)) {
ret = deliver_skb(skb, pt_prev, orig_dev);
pt_prev = NULL;
}
switch (rx_handler(&skb)) {
case RX_HANDLER_CONSUMED:
ret = NET_RX_SUCCESS;
goto out;
case RX_HANDLER_ANOTHER:
goto another_round;
case RX_HANDLER_EXACT:
deliver_exact = true;
break;
case RX_HANDLER_PASS:
break;
default:
BUG();
}
}
if (unlikely(skb_vlan_tag_present(skb)) && !netdev_uses_dsa(skb->dev)) {
check_vlan_id:
if (skb_vlan_tag_get_id(skb)) {
/* Vlan id is non 0 and vlan_do_receive() above couldn't
* find vlan device.
*/
skb->pkt_type = PACKET_OTHERHOST;
} else if (eth_type_vlan(skb->protocol)) {
/* Outer header is 802.1P with vlan 0, inner header is
* 802.1Q or 802.1AD and vlan_do_receive() above could
* not find vlan dev for vlan id 0.
*/
__vlan_hwaccel_clear_tag(skb);
skb = skb_vlan_untag(skb);
if (unlikely(!skb))
goto out;
if (vlan_do_receive(&skb))
/* After stripping off 802.1P header with vlan 0
* vlan dev is found for inner header.
*/
goto another_round;
else if (unlikely(!skb))
goto out;
else
/* We have stripped outer 802.1P vlan 0 header.
* But could not find vlan dev.
* check again for vlan id to set OTHERHOST.
*/
goto check_vlan_id;
}
/* Note: we might in the future use prio bits
* and set skb->priority like in vlan_do_receive()
* For the time being, just ignore Priority Code Point
*/
__vlan_hwaccel_clear_tag(skb);
}
type = skb->protocol;
/* deliver only exact match when indicated */
if (likely(!deliver_exact)) {
deliver_ptype_list_skb(skb, &pt_prev, orig_dev, type,
&ptype_base[ntohs(type) &
PTYPE_HASH_MASK]);
/* orig_dev and skb->dev could belong to different netns;
* Even in such case we need to traverse only the list
* coming from skb->dev, as the ptype owner (packet socket)
* will use dev_net(skb->dev) to do namespace filtering.
*/
deliver_ptype_list_skb(skb, &pt_prev, orig_dev, type,
&dev_net_rcu(skb->dev)->ptype_specific);
}
deliver_ptype_list_skb(skb, &pt_prev, orig_dev, type,
&orig_dev->ptype_specific);
if (unlikely(skb->dev != orig_dev)) {
deliver_ptype_list_skb(skb, &pt_prev, orig_dev, type,
&skb->dev->ptype_specific);
}
if (pt_prev) {
*ppt_prev = pt_prev;
} else {
drop:
if (!deliver_exact)
dev_core_stats_rx_dropped_inc(skb->dev);
else
dev_core_stats_rx_nohandler_inc(skb->dev);
kfree_skb_reason(skb, drop_reason);
/* Jamal, now you will not able to escape explaining
* me how you were going to use this. :-)
*/
ret = NET_RX_DROP;
}
out:
/* The invariant here is that if *ppt_prev is not NULL
* then skb should also be non-NULL.
*
* Apparently *ppt_prev assignment above holds this invariant due to
* skb dereferencing near it.
*/
*pskb = skb;
return ret;
}
static int __netif_receive_skb_one_core(struct sk_buff *skb, bool pfmemalloc)
{
struct net_device *orig_dev = skb->dev;
struct packet_type *pt_prev = NULL;
int ret;
ret = __netif_receive_skb_core(&skb, pfmemalloc, &pt_prev);
if (pt_prev)
ret = INDIRECT_CALL_INET(pt_prev->func, ipv6_rcv, ip_rcv, skb,
skb->dev, pt_prev, orig_dev);
return ret;
}
/**
* netif_receive_skb_core - special purpose version of netif_receive_skb
* @skb: buffer to process
*
* More direct receive version of netif_receive_skb(). It should
* only be used by callers that have a need to skip RPS and Generic XDP.
* Caller must also take care of handling if ``(page_is_)pfmemalloc``.
*
* This function may only be called from softirq context and interrupts
* should be enabled.
*
* Return values (usually ignored):
* NET_RX_SUCCESS: no congestion
* NET_RX_DROP: packet was dropped
*/
int netif_receive_skb_core(struct sk_buff *skb)
{
int ret;
rcu_read_lock();
ret = __netif_receive_skb_one_core(skb, false);
rcu_read_unlock();
return ret;
}
EXPORT_SYMBOL(netif_receive_skb_core);
static inline void __netif_receive_skb_list_ptype(struct list_head *head,
struct packet_type *pt_prev,
struct net_device *orig_dev)
{
struct sk_buff *skb, *next;
if (!pt_prev)
return;
if (list_empty(head))
return;
if (pt_prev->list_func != NULL)
INDIRECT_CALL_INET(pt_prev->list_func, ipv6_list_rcv,
ip_list_rcv, head, pt_prev, orig_dev);
else
list_for_each_entry_safe(skb, next, head, list) {
skb_list_del_init(skb);
pt_prev->func(skb, skb->dev, pt_prev, orig_dev);
}
}
static void __netif_receive_skb_list_core(struct list_head *head, bool pfmemalloc)
{
/* Fast-path assumptions:
* - There is no RX handler.
* - Only one packet_type matches.
* If either of these fails, we will end up doing some per-packet
* processing in-line, then handling the 'last ptype' for the whole
* sublist. This can't cause out-of-order delivery to any single ptype,
* because the 'last ptype' must be constant across the sublist, and all
* other ptypes are handled per-packet.
*/
/* Current (common) ptype of sublist */
struct packet_type *pt_curr = NULL;
/* Current (common) orig_dev of sublist */
struct net_device *od_curr = NULL;
struct sk_buff *skb, *next;
LIST_HEAD(sublist);
list_for_each_entry_safe(skb, next, head, list) {
struct net_device *orig_dev = skb->dev;
struct packet_type *pt_prev = NULL;
skb_list_del_init(skb);
__netif_receive_skb_core(&skb, pfmemalloc, &pt_prev);
if (!pt_prev)
continue;
if (pt_curr != pt_prev || od_curr != orig_dev) {
/* dispatch old sublist */
__netif_receive_skb_list_ptype(&sublist, pt_curr, od_curr);
/* start new sublist */
INIT_LIST_HEAD(&sublist);
pt_curr = pt_prev;
od_curr = orig_dev;
}
list_add_tail(&skb->list, &sublist);
}
/* dispatch final sublist */
__netif_receive_skb_list_ptype(&sublist, pt_curr, od_curr);
}
static int __netif_receive_skb(struct sk_buff *skb)
{
int ret;
if (sk_memalloc_socks() && skb_pfmemalloc(skb)) {
unsigned int noreclaim_flag;
/*
* PFMEMALLOC skbs are special, they should
* - be delivered to SOCK_MEMALLOC sockets only
* - stay away from userspace
* - have bounded memory usage
*
* Use PF_MEMALLOC as this saves us from propagating the allocation
* context down to all allocation sites.
*/
noreclaim_flag = memalloc_noreclaim_save();
ret = __netif_receive_skb_one_core(skb, true);
memalloc_noreclaim_restore(noreclaim_flag);
} else
ret = __netif_receive_skb_one_core(skb, false);
return ret;
}
static void __netif_receive_skb_list(struct list_head *head)
{
unsigned long noreclaim_flag = 0;
struct sk_buff *skb, *next;
bool pfmemalloc = false; /* Is current sublist PF_MEMALLOC? */
list_for_each_entry_safe(skb, next, head, list) {
if ((sk_memalloc_socks() && skb_pfmemalloc(skb)) != pfmemalloc) {
struct list_head sublist;
/* Handle the previous sublist */
list_cut_before(&sublist, head, &skb->list);
if (!list_empty(&sublist))
__netif_receive_skb_list_core(&sublist, pfmemalloc);
pfmemalloc = !pfmemalloc;
/* See comments in __netif_receive_skb */
if (pfmemalloc)
noreclaim_flag = memalloc_noreclaim_save();
else
memalloc_noreclaim_restore(noreclaim_flag);
}
}
/* Handle the remaining sublist */
if (!list_empty(head))
__netif_receive_skb_list_core(head, pfmemalloc);
/* Restore pflags */
if (pfmemalloc)
memalloc_noreclaim_restore(noreclaim_flag);
}
static int generic_xdp_install(struct net_device *dev, struct netdev_bpf *xdp)
{
struct bpf_prog *old = rtnl_dereference(dev->xdp_prog);
struct bpf_prog *new = xdp->prog;
int ret = 0;
switch (xdp->command) {
case XDP_SETUP_PROG:
rcu_assign_pointer(dev->xdp_prog, new);
if (old)
bpf_prog_put(old);
if (old && !new) {
static_branch_dec(&generic_xdp_needed_key);
} else if (new && !old) {
static_branch_inc(&generic_xdp_needed_key);
netif_disable_lro(dev);
dev_disable_gro_hw(dev);
}
break;
default:
ret = -EINVAL;
break;
}
return ret;
}
static int netif_receive_skb_internal(struct sk_buff *skb)
{
int ret;
net_timestamp_check(READ_ONCE(net_hotdata.tstamp_prequeue), skb);
if (skb_defer_rx_timestamp(skb))
return NET_RX_SUCCESS;
rcu_read_lock();
#ifdef CONFIG_RPS
if (static_branch_unlikely(&rps_needed)) {
struct rps_dev_flow voidflow, *rflow = &voidflow;
int cpu = get_rps_cpu(skb->dev, skb, &rflow);
if (cpu >= 0) {
ret = enqueue_to_backlog(skb, cpu, &rflow->last_qtail);
rcu_read_unlock();
return ret;
}
}
#endif
ret = __netif_receive_skb(skb);
rcu_read_unlock();
return ret;
}
void netif_receive_skb_list_internal(struct list_head *head)
{
struct sk_buff *skb, *next;
LIST_HEAD(sublist);
list_for_each_entry_safe(skb, next, head, list) {
net_timestamp_check(READ_ONCE(net_hotdata.tstamp_prequeue),
skb);
skb_list_del_init(skb);
if (!skb_defer_rx_timestamp(skb))
list_add_tail(&skb->list, &sublist);
}
list_splice_init(&sublist, head);
rcu_read_lock();
#ifdef CONFIG_RPS
if (static_branch_unlikely(&rps_needed)) {
list_for_each_entry_safe(skb, next, head, list) {
struct rps_dev_flow voidflow, *rflow = &voidflow;
int cpu = get_rps_cpu(skb->dev, skb, &rflow);
if (cpu >= 0) {
/* Will be handled, remove from list */
skb_list_del_init(skb);
enqueue_to_backlog(skb, cpu, &rflow->last_qtail);
}
}
}
#endif
__netif_receive_skb_list(head);
rcu_read_unlock();
}
/**
* netif_receive_skb - process receive buffer from network
* @skb: buffer to process
*
* netif_receive_skb() is the main receive data processing function.
* It always succeeds. The buffer may be dropped during processing
* for congestion control or by the protocol layers.
*
* This function may only be called from softirq context and interrupts
* should be enabled.
*
* Return values (usually ignored):
* NET_RX_SUCCESS: no congestion
* NET_RX_DROP: packet was dropped
*/
int netif_receive_skb(struct sk_buff *skb)
{
int ret;
trace_netif_receive_skb_entry(skb);
ret = netif_receive_skb_internal(skb);
trace_netif_receive_skb_exit(ret);
return ret;
}
EXPORT_SYMBOL(netif_receive_skb);
/**
* netif_receive_skb_list - process many receive buffers from network
* @head: list of skbs to process.
*
* Since return value of netif_receive_skb() is normally ignored, and
* wouldn't be meaningful for a list, this function returns void.
*
* This function may only be called from softirq context and interrupts
* should be enabled.
*/
void netif_receive_skb_list(struct list_head *head)
{
struct sk_buff *skb;
if (list_empty(head))
return;
if (trace_netif_receive_skb_list_entry_enabled()) {
list_for_each_entry(skb, head, list)
trace_netif_receive_skb_list_entry(skb);
}
netif_receive_skb_list_internal(head);
trace_netif_receive_skb_list_exit(0);
}
EXPORT_SYMBOL(netif_receive_skb_list);
/* Network device is going away, flush any packets still pending */
static void flush_backlog(struct work_struct *work)
{
struct sk_buff *skb, *tmp;
struct sk_buff_head list;
struct softnet_data *sd;
__skb_queue_head_init(&list);
local_bh_disable();
sd = this_cpu_ptr(&softnet_data);
backlog_lock_irq_disable(sd);
skb_queue_walk_safe(&sd->input_pkt_queue, skb, tmp) {
if (READ_ONCE(skb->dev->reg_state) == NETREG_UNREGISTERING) {
__skb_unlink(skb, &sd->input_pkt_queue);
__skb_queue_tail(&list, skb);
rps_input_queue_head_incr(sd);
}
}
backlog_unlock_irq_enable(sd);
local_lock_nested_bh(&softnet_data.process_queue_bh_lock);
skb_queue_walk_safe(&sd->process_queue, skb, tmp) {
if (READ_ONCE(skb->dev->reg_state) == NETREG_UNREGISTERING) {
__skb_unlink(skb, &sd->process_queue);
__skb_queue_tail(&list, skb);
rps_input_queue_head_incr(sd);
}
}
local_unlock_nested_bh(&softnet_data.process_queue_bh_lock);
local_bh_enable();
__skb_queue_purge_reason(&list, SKB_DROP_REASON_DEV_READY);
}
static bool flush_required(int cpu)
{
#if IS_ENABLED(CONFIG_RPS)
struct softnet_data *sd = &per_cpu(softnet_data, cpu);
bool do_flush;
backlog_lock_irq_disable(sd);
/* as insertion into process_queue happens with the rps lock held,
* process_queue access may race only with dequeue
*/
do_flush = !skb_queue_empty(&sd->input_pkt_queue) ||
!skb_queue_empty_lockless(&sd->process_queue);
backlog_unlock_irq_enable(sd);
return do_flush;
#endif
/* without RPS we can't safely check input_pkt_queue: during a
* concurrent remote skb_queue_splice() we can detect as empty both
* input_pkt_queue and process_queue even if the latter could end-up
* containing a lot of packets.
*/
return true;
}
struct flush_backlogs {
cpumask_t flush_cpus;
struct work_struct w[];
};
static struct flush_backlogs *flush_backlogs_alloc(void)
{
return kmalloc(struct_size_t(struct flush_backlogs, w, nr_cpu_ids),
GFP_KERNEL);
}
static struct flush_backlogs *flush_backlogs_fallback;
static DEFINE_MUTEX(flush_backlogs_mutex);
static void flush_all_backlogs(void)
{
struct flush_backlogs *ptr = flush_backlogs_alloc();
unsigned int cpu;
if (!ptr) {
mutex_lock(&flush_backlogs_mutex);
ptr = flush_backlogs_fallback;
}
cpumask_clear(&ptr->flush_cpus);
cpus_read_lock();
for_each_online_cpu(cpu) {
if (flush_required(cpu)) {
INIT_WORK(&ptr->w[cpu], flush_backlog);
queue_work_on(cpu, system_highpri_wq, &ptr->w[cpu]);
__cpumask_set_cpu(cpu, &ptr->flush_cpus);
}
}
/* we can have in flight packet[s] on the cpus we are not flushing,
* synchronize_net() in unregister_netdevice_many() will take care of
* them.
*/
for_each_cpu(cpu, &ptr->flush_cpus)
flush_work(&ptr->w[cpu]);
cpus_read_unlock();
if (ptr != flush_backlogs_fallback)
kfree(ptr);
else
mutex_unlock(&flush_backlogs_mutex);
}
static void net_rps_send_ipi(struct softnet_data *remsd)
{
#ifdef CONFIG_RPS
while (remsd) {
struct softnet_data *next = remsd->rps_ipi_next;
if (cpu_online(remsd->cpu))
smp_call_function_single_async(remsd->cpu, &remsd->csd);
remsd = next;
}
#endif
}
/*
* net_rps_action_and_irq_enable sends any pending IPI's for rps.
* Note: called with local irq disabled, but exits with local irq enabled.
*/
static void net_rps_action_and_irq_enable(struct softnet_data *sd)
{
#ifdef CONFIG_RPS
struct softnet_data *remsd = sd->rps_ipi_list;
if (!use_backlog_threads() && remsd) {
sd->rps_ipi_list = NULL;
local_irq_enable();
/* Send pending IPI's to kick RPS processing on remote cpus. */
net_rps_send_ipi(remsd);
} else
#endif
local_irq_enable();
}
static bool sd_has_rps_ipi_waiting(struct softnet_data *sd)
{
#ifdef CONFIG_RPS
return !use_backlog_threads() && sd->rps_ipi_list;
#else
return false;
#endif
}
static int process_backlog(struct napi_struct *napi, int quota)
{
struct softnet_data *sd = container_of(napi, struct softnet_data, backlog);
bool again = true;
int work = 0;
/* Check if we have pending ipi, its better to send them now,
* not waiting net_rx_action() end.
*/
if (sd_has_rps_ipi_waiting(sd)) {
local_irq_disable();
net_rps_action_and_irq_enable(sd);
}
napi->weight = READ_ONCE(net_hotdata.dev_rx_weight);
while (again) {
struct sk_buff *skb;
local_lock_nested_bh(&softnet_data.process_queue_bh_lock);
while ((skb = __skb_dequeue(&sd->process_queue))) {
local_unlock_nested_bh(&softnet_data.process_queue_bh_lock);
rcu_read_lock();
__netif_receive_skb(skb);
rcu_read_unlock();
if (++work >= quota) {
rps_input_queue_head_add(sd, work);
return work;
}
local_lock_nested_bh(&softnet_data.process_queue_bh_lock);
}
local_unlock_nested_bh(&softnet_data.process_queue_bh_lock);
backlog_lock_irq_disable(sd);
if (skb_queue_empty(&sd->input_pkt_queue)) {
/*
* Inline a custom version of __napi_complete().
* only current cpu owns and manipulates this napi,
* and NAPI_STATE_SCHED is the only possible flag set
* on backlog.
* We can use a plain write instead of clear_bit(),
* and we dont need an smp_mb() memory barrier.
*/
napi->state &= NAPIF_STATE_THREADED;
again = false;
} else {
local_lock_nested_bh(&softnet_data.process_queue_bh_lock);
skb_queue_splice_tail_init(&sd->input_pkt_queue,
&sd->process_queue);
local_unlock_nested_bh(&softnet_data.process_queue_bh_lock);
}
backlog_unlock_irq_enable(sd);
}
if (work)
rps_input_queue_head_add(sd, work);
return work;
}
/**
* __napi_schedule - schedule for receive
* @n: entry to schedule
*
* The entry's receive function will be scheduled to run.
* Consider using __napi_schedule_irqoff() if hard irqs are masked.
*/
void __napi_schedule(struct napi_struct *n)
{
unsigned long flags;
local_irq_save(flags);
____napi_schedule(this_cpu_ptr(&softnet_data), n);
local_irq_restore(flags);
}
EXPORT_SYMBOL(__napi_schedule);
/**
* napi_schedule_prep - check if napi can be scheduled
* @n: napi context
*
* Test if NAPI routine is already running, and if not mark
* it as running. This is used as a condition variable to
* insure only one NAPI poll instance runs. We also make
* sure there is no pending NAPI disable.
*/
bool napi_schedule_prep(struct napi_struct *n)
{
unsigned long new, val = READ_ONCE(n->state);
do {
if (unlikely(val & NAPIF_STATE_DISABLE))
return false;
new = val | NAPIF_STATE_SCHED;
/* Sets STATE_MISSED bit if STATE_SCHED was already set
* This was suggested by Alexander Duyck, as compiler
* emits better code than :
* if (val & NAPIF_STATE_SCHED)
* new |= NAPIF_STATE_MISSED;
*/
new |= (val & NAPIF_STATE_SCHED) / NAPIF_STATE_SCHED *
NAPIF_STATE_MISSED;
} while (!try_cmpxchg(&n->state, &val, new));
return !(val & NAPIF_STATE_SCHED);
}
EXPORT_SYMBOL(napi_schedule_prep);
/**
* __napi_schedule_irqoff - schedule for receive
* @n: entry to schedule
*
* Variant of __napi_schedule() assuming hard irqs are masked.
*
* On PREEMPT_RT enabled kernels this maps to __napi_schedule()
* because the interrupt disabled assumption might not be true
* due to force-threaded interrupts and spinlock substitution.
*/
void __napi_schedule_irqoff(struct napi_struct *n)
{
if (!IS_ENABLED(CONFIG_PREEMPT_RT))
____napi_schedule(this_cpu_ptr(&softnet_data), n);
else
__napi_schedule(n);
}
EXPORT_SYMBOL(__napi_schedule_irqoff);
bool napi_complete_done(struct napi_struct *n, int work_done)
{
unsigned long flags, val, new, timeout = 0;
bool ret = true;
/*
* 1) Don't let napi dequeue from the cpu poll list
* just in case its running on a different cpu.
* 2) If we are busy polling, do nothing here, we have
* the guarantee we will be called later.
*/
if (unlikely(n->state & (NAPIF_STATE_NPSVC |
NAPIF_STATE_IN_BUSY_POLL)))
return false;
if (work_done) {
if (n->gro.bitmask)
timeout = napi_get_gro_flush_timeout(n);
n->defer_hard_irqs_count = napi_get_defer_hard_irqs(n);
}
if (n->defer_hard_irqs_count > 0) {
n->defer_hard_irqs_count--;
timeout = napi_get_gro_flush_timeout(n);
if (timeout)
ret = false;
}
/*
* When the NAPI instance uses a timeout and keeps postponing
* it, we need to bound somehow the time packets are kept in
* the GRO layer.
*/
gro_flush_normal(&n->gro, !!timeout);
if (unlikely(!list_empty(&n->poll_list))) {
/* If n->poll_list is not empty, we need to mask irqs */
local_irq_save(flags);
list_del_init(&n->poll_list);
local_irq_restore(flags);
}
WRITE_ONCE(n->list_owner, -1);
val = READ_ONCE(n->state);
do {
WARN_ON_ONCE(!(val & NAPIF_STATE_SCHED));
new = val & ~(NAPIF_STATE_MISSED | NAPIF_STATE_SCHED |
NAPIF_STATE_SCHED_THREADED |
NAPIF_STATE_PREFER_BUSY_POLL);
/* If STATE_MISSED was set, leave STATE_SCHED set,
* because we will call napi->poll() one more time.
* This C code was suggested by Alexander Duyck to help gcc.
*/
new |= (val & NAPIF_STATE_MISSED) / NAPIF_STATE_MISSED *
NAPIF_STATE_SCHED;
} while (!try_cmpxchg(&n->state, &val, new));
if (unlikely(val & NAPIF_STATE_MISSED)) {
__napi_schedule(n);
return false;
}
if (timeout)
hrtimer_start(&n->timer, ns_to_ktime(timeout),
HRTIMER_MODE_REL_PINNED);
return ret;
}
EXPORT_SYMBOL(napi_complete_done);
static void skb_defer_free_flush(void)
{
struct llist_node *free_list;
struct sk_buff *skb, *next;
struct skb_defer_node *sdn;
int node;
for_each_node(node) {
sdn = this_cpu_ptr(net_hotdata.skb_defer_nodes) + node;
if (llist_empty(&sdn->defer_list))
continue;
atomic_long_set(&sdn->defer_count, 0);
free_list = llist_del_all(&sdn->defer_list);
llist_for_each_entry_safe(skb, next, free_list, ll_node) {
prefetch(next);
napi_consume_skb(skb, 1);
}
}
}
#if defined(CONFIG_NET_RX_BUSY_POLL)
static void __busy_poll_stop(struct napi_struct *napi, bool skip_schedule)
{
if (!skip_schedule) {
gro_normal_list(&napi->gro);
__napi_schedule(napi);
return;
}
/* Flush too old packets. If HZ < 1000, flush all packets */
gro_flush_normal(&napi->gro, HZ >= 1000);
clear_bit(NAPI_STATE_SCHED, &napi->state);
}
enum {
NAPI_F_PREFER_BUSY_POLL = 1,
NAPI_F_END_ON_RESCHED = 2,
};
static void busy_poll_stop(struct napi_struct *napi, void *have_poll_lock,
unsigned flags, u16 budget)
{
struct bpf_net_context __bpf_net_ctx, *bpf_net_ctx;
bool skip_schedule = false;
unsigned long timeout;
int rc;
/* Busy polling means there is a high chance device driver hard irq
* could not grab NAPI_STATE_SCHED, and that NAPI_STATE_MISSED was
* set in napi_schedule_prep().
* Since we are about to call napi->poll() once more, we can safely
* clear NAPI_STATE_MISSED.
*
* Note: x86 could use a single "lock and ..." instruction
* to perform these two clear_bit()
*/
clear_bit(NAPI_STATE_MISSED, &napi->state);
clear_bit(NAPI_STATE_IN_BUSY_POLL, &napi->state);
local_bh_disable();
bpf_net_ctx = bpf_net_ctx_set(&__bpf_net_ctx);
if (flags & NAPI_F_PREFER_BUSY_POLL) {
napi->defer_hard_irqs_count = napi_get_defer_hard_irqs(napi);
timeout = napi_get_gro_flush_timeout(napi);
if (napi->defer_hard_irqs_count && timeout) {
hrtimer_start(&napi->timer, ns_to_ktime(timeout), HRTIMER_MODE_REL_PINNED);
skip_schedule = true;
}
}
/* All we really want here is to re-enable device interrupts.
* Ideally, a new ndo_busy_poll_stop() could avoid another round.
*/
rc = napi->poll(napi, budget);
/* We can't gro_normal_list() here, because napi->poll() might have
* rearmed the napi (napi_complete_done()) in which case it could
* already be running on another CPU.
*/
trace_napi_poll(napi, rc, budget);
netpoll_poll_unlock(have_poll_lock);
if (rc == budget)
__busy_poll_stop(napi, skip_schedule);
bpf_net_ctx_clear(bpf_net_ctx);
local_bh_enable();
}
static void __napi_busy_loop(unsigned int napi_id,
bool (*loop_end)(void *, unsigned long),
void *loop_end_arg, unsigned flags, u16 budget)
{
unsigned long start_time = loop_end ? busy_loop_current_time() : 0;
int (*napi_poll)(struct napi_struct *napi, int budget);
struct bpf_net_context __bpf_net_ctx, *bpf_net_ctx;
void *have_poll_lock = NULL;
struct napi_struct *napi;
WARN_ON_ONCE(!rcu_read_lock_held());
restart:
napi_poll = NULL;
napi = napi_by_id(napi_id);
if (!napi)
return;
if (!IS_ENABLED(CONFIG_PREEMPT_RT))
preempt_disable();
for (;;) {
int work = 0;
local_bh_disable();
bpf_net_ctx = bpf_net_ctx_set(&__bpf_net_ctx);
if (!napi_poll) {
unsigned long val = READ_ONCE(napi->state);
/* If multiple threads are competing for this napi,
* we avoid dirtying napi->state as much as we can.
*/
if (val & (NAPIF_STATE_DISABLE | NAPIF_STATE_SCHED |
NAPIF_STATE_IN_BUSY_POLL)) {
if (flags & NAPI_F_PREFER_BUSY_POLL)
set_bit(NAPI_STATE_PREFER_BUSY_POLL, &napi->state);
goto count;
}
if (cmpxchg(&napi->state, val,
val | NAPIF_STATE_IN_BUSY_POLL |
NAPIF_STATE_SCHED) != val) {
if (flags & NAPI_F_PREFER_BUSY_POLL)
set_bit(NAPI_STATE_PREFER_BUSY_POLL, &napi->state);
goto count;
}
have_poll_lock = netpoll_poll_lock(napi);
napi_poll = napi->poll;
}
work = napi_poll(napi, budget);
trace_napi_poll(napi, work, budget);
gro_normal_list(&napi->gro);
count:
if (work > 0)
__NET_ADD_STATS(dev_net(napi->dev),
LINUX_MIB_BUSYPOLLRXPACKETS, work);
skb_defer_free_flush();
bpf_net_ctx_clear(bpf_net_ctx);
local_bh_enable();
if (!loop_end || loop_end(loop_end_arg, start_time))
break;
if (unlikely(need_resched())) {
if (flags & NAPI_F_END_ON_RESCHED)
break;
if (napi_poll)
busy_poll_stop(napi, have_poll_lock, flags, budget);
if (!IS_ENABLED(CONFIG_PREEMPT_RT))
preempt_enable();
rcu_read_unlock();
cond_resched();
rcu_read_lock();
if (loop_end(loop_end_arg, start_time))
return;
goto restart;
}
cpu_relax();
}
if (napi_poll)
busy_poll_stop(napi, have_poll_lock, flags, budget);
if (!IS_ENABLED(CONFIG_PREEMPT_RT))
preempt_enable();
}
void napi_busy_loop_rcu(unsigned int napi_id,
bool (*loop_end)(void *, unsigned long),
void *loop_end_arg, bool prefer_busy_poll, u16 budget)
{
unsigned flags = NAPI_F_END_ON_RESCHED;
if (prefer_busy_poll)
flags |= NAPI_F_PREFER_BUSY_POLL;
__napi_busy_loop(napi_id, loop_end, loop_end_arg, flags, budget);
}
void napi_busy_loop(unsigned int napi_id,
bool (*loop_end)(void *, unsigned long),
void *loop_end_arg, bool prefer_busy_poll, u16 budget)
{
unsigned flags = prefer_busy_poll ? NAPI_F_PREFER_BUSY_POLL : 0;
rcu_read_lock();
__napi_busy_loop(napi_id, loop_end, loop_end_arg, flags, budget);
rcu_read_unlock();
}
EXPORT_SYMBOL(napi_busy_loop);
void napi_suspend_irqs(unsigned int napi_id)
{
struct napi_struct *napi;
rcu_read_lock();
napi = napi_by_id(napi_id);
if (napi) {
unsigned long timeout = napi_get_irq_suspend_timeout(napi);
if (timeout)
hrtimer_start(&napi->timer, ns_to_ktime(timeout),
HRTIMER_MODE_REL_PINNED);
}
rcu_read_unlock();
}
void napi_resume_irqs(unsigned int napi_id)
{
struct napi_struct *napi;
rcu_read_lock();
napi = napi_by_id(napi_id);
if (napi) {
/* If irq_suspend_timeout is set to 0 between the call to
* napi_suspend_irqs and now, the original value still
* determines the safety timeout as intended and napi_watchdog
* will resume irq processing.
*/
if (napi_get_irq_suspend_timeout(napi)) {
local_bh_disable();
napi_schedule(napi);
local_bh_enable();
}
}
rcu_read_unlock();
}
#endif /* CONFIG_NET_RX_BUSY_POLL */
static void __napi_hash_add_with_id(struct napi_struct *napi,
unsigned int napi_id)
{
napi->gro.cached_napi_id = napi_id;
WRITE_ONCE(napi->napi_id, napi_id);
hlist_add_head_rcu(&napi->napi_hash_node,
&napi_hash[napi->napi_id % HASH_SIZE(napi_hash)]);
}
static void napi_hash_add_with_id(struct napi_struct *napi,
unsigned int napi_id)
{
unsigned long flags;
spin_lock_irqsave(&napi_hash_lock, flags);
WARN_ON_ONCE(napi_by_id(napi_id));
__napi_hash_add_with_id(napi, napi_id);
spin_unlock_irqrestore(&napi_hash_lock, flags);
}
static void napi_hash_add(struct napi_struct *napi)
{
unsigned long flags;
if (test_bit(NAPI_STATE_NO_BUSY_POLL, &napi->state))
return;
spin_lock_irqsave(&napi_hash_lock, flags);
/* 0..NR_CPUS range is reserved for sender_cpu use */
do {
if (unlikely(!napi_id_valid(++napi_gen_id)))
napi_gen_id = MIN_NAPI_ID;
} while (napi_by_id(napi_gen_id));
__napi_hash_add_with_id(napi, napi_gen_id);
spin_unlock_irqrestore(&napi_hash_lock, flags);
}
/* Warning : caller is responsible to make sure rcu grace period
* is respected before freeing memory containing @napi
*/
static void napi_hash_del(struct napi_struct *napi)
{
unsigned long flags;
spin_lock_irqsave(&napi_hash_lock, flags);
hlist_del_init_rcu(&napi->napi_hash_node);
spin_unlock_irqrestore(&napi_hash_lock, flags);
}
static enum hrtimer_restart napi_watchdog(struct hrtimer *timer)
{
struct napi_struct *napi;
napi = container_of(timer, struct napi_struct, timer);
/* Note : we use a relaxed variant of napi_schedule_prep() not setting
* NAPI_STATE_MISSED, since we do not react to a device IRQ.
*/
if (!napi_disable_pending(napi) &&
!test_and_set_bit(NAPI_STATE_SCHED, &napi->state)) {
clear_bit(NAPI_STATE_PREFER_BUSY_POLL, &napi->state);
__napi_schedule_irqoff(napi);
}
return HRTIMER_NORESTART;
}
static void napi_stop_kthread(struct napi_struct *napi)
{
unsigned long val, new;
/* Wait until the napi STATE_THREADED is unset. */
while (true) {
val = READ_ONCE(napi->state);
/* If napi kthread own this napi or the napi is idle,
* STATE_THREADED can be unset here.
*/
if ((val & NAPIF_STATE_SCHED_THREADED) ||
!(val & NAPIF_STATE_SCHED)) {
new = val & (~(NAPIF_STATE_THREADED |
NAPIF_STATE_THREADED_BUSY_POLL));
} else {
msleep(20);
continue;
}
if (try_cmpxchg(&napi->state, &val, new))
break;
}
/* Once STATE_THREADED is unset, wait for SCHED_THREADED to be unset by
* the kthread.
*/
while (true) {
if (!test_bit(NAPI_STATE_SCHED_THREADED, &napi->state))
break;
msleep(20);
}
kthread_stop(napi->thread);
napi->thread = NULL;
}
static void napi_set_threaded_state(struct napi_struct *napi,
enum netdev_napi_threaded threaded_mode)
{
bool threaded = threaded_mode != NETDEV_NAPI_THREADED_DISABLED;
bool busy_poll = threaded_mode == NETDEV_NAPI_THREADED_BUSY_POLL;
assign_bit(NAPI_STATE_THREADED, &napi->state, threaded);
assign_bit(NAPI_STATE_THREADED_BUSY_POLL, &napi->state, busy_poll);
}
int napi_set_threaded(struct napi_struct *napi,
enum netdev_napi_threaded threaded)
{
if (threaded) {
if (!napi->thread) {
int err = napi_kthread_create(napi);
if (err)
return err;
}
}
if (napi->config)
napi->config->threaded = threaded;
/* Setting/unsetting threaded mode on a napi might not immediately
* take effect, if the current napi instance is actively being
* polled. In this case, the switch between threaded mode and
* softirq mode will happen in the next round of napi_schedule().
* This should not cause hiccups/stalls to the live traffic.
*/
if (!threaded && napi->thread) {
napi_stop_kthread(napi);
} else {
/* Make sure kthread is created before THREADED bit is set. */
smp_mb__before_atomic();
napi_set_threaded_state(napi, threaded);
}
return 0;
}
int netif_set_threaded(struct net_device *dev,
enum netdev_napi_threaded threaded)
{
struct napi_struct *napi;
int i, err = 0;
netdev_assert_locked_or_invisible(dev);
if (threaded) {
list_for_each_entry(napi, &dev->napi_list, dev_list) {
if (!napi->thread) {
err = napi_kthread_create(napi);
if (err) {
threaded = NETDEV_NAPI_THREADED_DISABLED;
break;
}
}
}
}
WRITE_ONCE(dev->threaded, threaded);
/* The error should not occur as the kthreads are already created. */
list_for_each_entry(napi, &dev->napi_list, dev_list)
WARN_ON_ONCE(napi_set_threaded(napi, threaded));
/* Override the config for all NAPIs even if currently not listed */
for (i = 0; i < dev->num_napi_configs; i++)
dev->napi_config[i].threaded = threaded;
return err;
}
/**
* netif_threaded_enable() - enable threaded NAPIs
* @dev: net_device instance
*
* Enable threaded mode for the NAPI instances of the device. This may be useful
* for devices where multiple NAPI instances get scheduled by a single
* interrupt. Threaded NAPI allows moving the NAPI processing to cores other
* than the core where IRQ is mapped.
*
* This function should be called before @dev is registered.
*/
void netif_threaded_enable(struct net_device *dev)
{
WARN_ON_ONCE(netif_set_threaded(dev, NETDEV_NAPI_THREADED_ENABLED));
}
EXPORT_SYMBOL(netif_threaded_enable);
/**
* netif_queue_set_napi - Associate queue with the napi
* @dev: device to which NAPI and queue belong
* @queue_index: Index of queue
* @type: queue type as RX or TX
* @napi: NAPI context, pass NULL to clear previously set NAPI
*
* Set queue with its corresponding napi context. This should be done after
* registering the NAPI handler for the queue-vector and the queues have been
* mapped to the corresponding interrupt vector.
*/
void netif_queue_set_napi(struct net_device *dev, unsigned int queue_index,
enum netdev_queue_type type, struct napi_struct *napi)
{
struct netdev_rx_queue *rxq;
struct netdev_queue *txq;
if (WARN_ON_ONCE(napi && !napi->dev))
return;
netdev_ops_assert_locked_or_invisible(dev);
switch (type) {
case NETDEV_QUEUE_TYPE_RX:
rxq = __netif_get_rx_queue(dev, queue_index);
rxq->napi = napi;
return;
case NETDEV_QUEUE_TYPE_TX:
txq = netdev_get_tx_queue(dev, queue_index);
txq->napi = napi;
return;
default:
return;
}
}
EXPORT_SYMBOL(netif_queue_set_napi);
static void
netif_napi_irq_notify(struct irq_affinity_notify *notify,
const cpumask_t *mask)
{
struct napi_struct *napi =
container_of(notify, struct napi_struct, notify);
#ifdef CONFIG_RFS_ACCEL
struct cpu_rmap *rmap = napi->dev->rx_cpu_rmap;
int err;
#endif
if (napi->config && napi->dev->irq_affinity_auto)
cpumask_copy(&napi->config->affinity_mask, mask);
#ifdef CONFIG_RFS_ACCEL
if (napi->dev->rx_cpu_rmap_auto) {
err = cpu_rmap_update(rmap, napi->napi_rmap_idx, mask);
if (err)
netdev_warn(napi->dev, "RMAP update failed (%d)\n",
err);
}
#endif
}
#ifdef CONFIG_RFS_ACCEL
static void netif_napi_affinity_release(struct kref *ref)
{
struct napi_struct *napi =
container_of(ref, struct napi_struct, notify.kref);
struct cpu_rmap *rmap = napi->dev->rx_cpu_rmap;
netdev_assert_locked(napi->dev);
WARN_ON(test_and_clear_bit(NAPI_STATE_HAS_NOTIFIER,
&napi->state));
if (!napi->dev->rx_cpu_rmap_auto)
return;
rmap->obj[napi->napi_rmap_idx] = NULL;
napi->napi_rmap_idx = -1;
cpu_rmap_put(rmap);
}
int netif_enable_cpu_rmap(struct net_device *dev, unsigned int num_irqs)
{
if (dev->rx_cpu_rmap_auto)
return 0;
dev->rx_cpu_rmap = alloc_irq_cpu_rmap(num_irqs);
if (!dev->rx_cpu_rmap)
return -ENOMEM;
dev->rx_cpu_rmap_auto = true;
return 0;
}
EXPORT_SYMBOL(netif_enable_cpu_rmap);
static void netif_del_cpu_rmap(struct net_device *dev)
{
struct cpu_rmap *rmap = dev->rx_cpu_rmap;
if (!dev->rx_cpu_rmap_auto)
return;
/* Free the rmap */
cpu_rmap_put(rmap);
dev->rx_cpu_rmap = NULL;
dev->rx_cpu_rmap_auto = false;
}
#else
static void netif_napi_affinity_release(struct kref *ref)
{
}
int netif_enable_cpu_rmap(struct net_device *dev, unsigned int num_irqs)
{
return 0;
}
EXPORT_SYMBOL(netif_enable_cpu_rmap);
static void netif_del_cpu_rmap(struct net_device *dev)
{
}
#endif
void netif_set_affinity_auto(struct net_device *dev)
{
unsigned int i, maxqs, numa;
maxqs = max(dev->num_tx_queues, dev->num_rx_queues);
numa = dev_to_node(&dev->dev);
for (i = 0; i < maxqs; i++)
cpumask_set_cpu(cpumask_local_spread(i, numa),
&dev->napi_config[i].affinity_mask);
dev->irq_affinity_auto = true;
}
EXPORT_SYMBOL(netif_set_affinity_auto);
void netif_napi_set_irq_locked(struct napi_struct *napi, int irq)
{
int rc;
netdev_assert_locked_or_invisible(napi->dev);
if (napi->irq == irq)
return;
/* Remove existing resources */
if (test_and_clear_bit(NAPI_STATE_HAS_NOTIFIER, &napi->state))
irq_set_affinity_notifier(napi->irq, NULL);
napi->irq = irq;
if (irq < 0 ||
(!napi->dev->rx_cpu_rmap_auto && !napi->dev->irq_affinity_auto))
return;
/* Abort for buggy drivers */
if (napi->dev->irq_affinity_auto && WARN_ON_ONCE(!napi->config))
return;
#ifdef CONFIG_RFS_ACCEL
if (napi->dev->rx_cpu_rmap_auto) {
rc = cpu_rmap_add(napi->dev->rx_cpu_rmap, napi);
if (rc < 0)
return;
cpu_rmap_get(napi->dev->rx_cpu_rmap);
napi->napi_rmap_idx = rc;
}
#endif
/* Use core IRQ notifier */
napi->notify.notify = netif_napi_irq_notify;
napi->notify.release = netif_napi_affinity_release;
rc = irq_set_affinity_notifier(irq, &napi->notify);
if (rc) {
netdev_warn(napi->dev, "Unable to set IRQ notifier (%d)\n",
rc);
goto put_rmap;
}
set_bit(NAPI_STATE_HAS_NOTIFIER, &napi->state);
return;
put_rmap:
#ifdef CONFIG_RFS_ACCEL
if (napi->dev->rx_cpu_rmap_auto) {
napi->dev->rx_cpu_rmap->obj[napi->napi_rmap_idx] = NULL;
cpu_rmap_put(napi->dev->rx_cpu_rmap);
napi->napi_rmap_idx = -1;
}
#endif
napi->notify.notify = NULL;
napi->notify.release = NULL;
}
EXPORT_SYMBOL(netif_napi_set_irq_locked);
static void napi_restore_config(struct napi_struct *n)
{
n->defer_hard_irqs = n->config->defer_hard_irqs;
n->gro_flush_timeout = n->config->gro_flush_timeout;
n->irq_suspend_timeout = n->config->irq_suspend_timeout;
if (n->dev->irq_affinity_auto &&
test_bit(NAPI_STATE_HAS_NOTIFIER, &n->state))
irq_set_affinity(n->irq, &n->config->affinity_mask);
/* a NAPI ID might be stored in the config, if so use it. if not, use
* napi_hash_add to generate one for us.
*/
if (n->config->napi_id) {
napi_hash_add_with_id(n, n->config->napi_id);
} else {
napi_hash_add(n);
n->config->napi_id = n->napi_id;
}
WARN_ON_ONCE(napi_set_threaded(n, n->config->threaded));
}
static void napi_save_config(struct napi_struct *n)
{
n->config->defer_hard_irqs = n->defer_hard_irqs;
n->config->gro_flush_timeout = n->gro_flush_timeout;
n->config->irq_suspend_timeout = n->irq_suspend_timeout;
napi_hash_del(n);
}
/* Netlink wants the NAPI list to be sorted by ID, if adding a NAPI which will
* inherit an existing ID try to insert it at the right position.
*/
static void
netif_napi_dev_list_add(struct net_device *dev, struct napi_struct *napi)
{
unsigned int new_id, pos_id;
struct list_head *higher;
struct napi_struct *pos;
new_id = UINT_MAX;
if (napi->config && napi->config->napi_id)
new_id = napi->config->napi_id;
higher = &dev->napi_list;
list_for_each_entry(pos, &dev->napi_list, dev_list) {
if (napi_id_valid(pos->napi_id))
pos_id = pos->napi_id;
else if (pos->config)
pos_id = pos->config->napi_id;
else
pos_id = UINT_MAX;
if (pos_id <= new_id)
break;
higher = &pos->dev_list;
}
list_add_rcu(&napi->dev_list, higher); /* adds after higher */
}
/* Double check that napi_get_frags() allocates skbs with
* skb->head being backed by slab, not a page fragment.
* This is to make sure bug fixed in 3226b158e67c
* ("net: avoid 32 x truesize under-estimation for tiny skbs")
* does not accidentally come back.
*/
static void napi_get_frags_check(struct napi_struct *napi)
{
struct sk_buff *skb;
local_bh_disable();
skb = napi_get_frags(napi);
WARN_ON_ONCE(skb && skb->head_frag);
napi_free_frags(napi);
local_bh_enable();
}
void netif_napi_add_weight_locked(struct net_device *dev,
struct napi_struct *napi,
int (*poll)(struct napi_struct *, int),
int weight)
{
netdev_assert_locked(dev);
if (WARN_ON(test_and_set_bit(NAPI_STATE_LISTED, &napi->state)))
return;
INIT_LIST_HEAD(&napi->poll_list);
INIT_HLIST_NODE(&napi->napi_hash_node);
hrtimer_setup(&napi->timer, napi_watchdog, CLOCK_MONOTONIC, HRTIMER_MODE_REL_PINNED);
gro_init(&napi->gro);
napi->skb = NULL;
napi->poll = poll;
if (weight > NAPI_POLL_WEIGHT)
netdev_err_once(dev, "%s() called with weight %d\n", __func__,
weight);
napi->weight = weight;
napi->dev = dev;
#ifdef CONFIG_NETPOLL
napi->poll_owner = -1;
#endif
napi->list_owner = -1;
set_bit(NAPI_STATE_SCHED, &napi->state);
set_bit(NAPI_STATE_NPSVC, &napi->state);
netif_napi_dev_list_add(dev, napi);
/* default settings from sysfs are applied to all NAPIs. any per-NAPI
* configuration will be loaded in napi_enable
*/
napi_set_defer_hard_irqs(napi, READ_ONCE(dev->napi_defer_hard_irqs));
napi_set_gro_flush_timeout(napi, READ_ONCE(dev->gro_flush_timeout));
napi_get_frags_check(napi);
/* Create kthread for this napi if dev->threaded is set.
* Clear dev->threaded if kthread creation failed so that
* threaded mode will not be enabled in napi_enable().
*/
if (napi_get_threaded_config(dev, napi))
if (napi_kthread_create(napi))
dev->threaded = NETDEV_NAPI_THREADED_DISABLED;
netif_napi_set_irq_locked(napi, -1);
}
EXPORT_SYMBOL(netif_napi_add_weight_locked);
void napi_disable_locked(struct napi_struct *n)
{
unsigned long val, new;
might_sleep();
netdev_assert_locked(n->dev);
set_bit(NAPI_STATE_DISABLE, &n->state);
val = READ_ONCE(n->state);
do {
while (val & (NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC)) {
usleep_range(20, 200);
val = READ_ONCE(n->state);
}
new = val | NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC;
new &= ~(NAPIF_STATE_THREADED |
NAPIF_STATE_THREADED_BUSY_POLL |
NAPIF_STATE_PREFER_BUSY_POLL);
} while (!try_cmpxchg(&n->state, &val, new));
hrtimer_cancel(&n->timer);
if (n->config)
napi_save_config(n);
else
napi_hash_del(n);
clear_bit(NAPI_STATE_DISABLE, &n->state);
}
EXPORT_SYMBOL(napi_disable_locked);
/**
* napi_disable() - prevent NAPI from scheduling
* @n: NAPI context
*
* Stop NAPI from being scheduled on this context.
* Waits till any outstanding processing completes.
* Takes netdev_lock() for associated net_device.
*/
void napi_disable(struct napi_struct *n)
{
netdev_lock(n->dev);
napi_disable_locked(n);
netdev_unlock(n->dev);
}
EXPORT_SYMBOL(napi_disable);
void napi_enable_locked(struct napi_struct *n)
{
unsigned long new, val = READ_ONCE(n->state);
if (n->config)
napi_restore_config(n);
else
napi_hash_add(n);
do {
BUG_ON(!test_bit(NAPI_STATE_SCHED, &val));
new = val & ~(NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC);
if (n->dev->threaded && n->thread)
new |= NAPIF_STATE_THREADED;
} while (!try_cmpxchg(&n->state, &val, new));
}
EXPORT_SYMBOL(napi_enable_locked);
/**
* napi_enable() - enable NAPI scheduling
* @n: NAPI context
*
* Enable scheduling of a NAPI instance.
* Must be paired with napi_disable().
* Takes netdev_lock() for associated net_device.
*/
void napi_enable(struct napi_struct *n)
{
netdev_lock(n->dev);
napi_enable_locked(n);
netdev_unlock(n->dev);
}
EXPORT_SYMBOL(napi_enable);
/* Must be called in process context */
void __netif_napi_del_locked(struct napi_struct *napi)
{
netdev_assert_locked(napi->dev);
if (!test_and_clear_bit(NAPI_STATE_LISTED, &napi->state))
return;
/* Make sure NAPI is disabled (or was never enabled). */
WARN_ON(!test_bit(NAPI_STATE_SCHED, &napi->state));
if (test_and_clear_bit(NAPI_STATE_HAS_NOTIFIER, &napi->state))
irq_set_affinity_notifier(napi->irq, NULL);
if (napi->config) {
napi->index = -1;
napi->config = NULL;
}
list_del_rcu(&napi->dev_list);
napi_free_frags(napi);
gro_cleanup(&napi->gro);
if (napi->thread) {
kthread_stop(napi->thread);
napi->thread = NULL;
}
}
EXPORT_SYMBOL(__netif_napi_del_locked);
static int __napi_poll(struct napi_struct *n, bool *repoll)
{
int work, weight;
weight = n->weight;
/* This NAPI_STATE_SCHED test is for avoiding a race
* with netpoll's poll_napi(). Only the entity which
* obtains the lock and sees NAPI_STATE_SCHED set will
* actually make the ->poll() call. Therefore we avoid
* accidentally calling ->poll() when NAPI is not scheduled.
*/
work = 0;
if (napi_is_scheduled(n)) {
work = n->poll(n, weight);
trace_napi_poll(n, work, weight);
xdp_do_check_flushed(n);
}
if (unlikely(work > weight))
netdev_err_once(n->dev, "NAPI poll function %pS returned %d, exceeding its budget of %d.\n",
n->poll, work, weight);
if (likely(work < weight))
return work;
/* Drivers must not modify the NAPI state if they
* consume the entire weight. In such cases this code
* still "owns" the NAPI instance and therefore can
* move the instance around on the list at-will.
*/
if (unlikely(napi_disable_pending(n))) {
napi_complete(n);
return work;
}
/* The NAPI context has more processing work, but busy-polling
* is preferred. Exit early.
*/
if (napi_prefer_busy_poll(n)) {
if (napi_complete_done(n, work)) {
/* If timeout is not set, we need to make sure
* that the NAPI is re-scheduled.
*/
napi_schedule(n);
}
return work;
}
/* Flush too old packets. If HZ < 1000, flush all packets */
gro_flush_normal(&n->gro, HZ >= 1000);
/* Some drivers may have called napi_schedule
* prior to exhausting their budget.
*/
if (unlikely(!list_empty(&n->poll_list))) {
pr_warn_once("%s: Budget exhausted after napi rescheduled\n",
n->dev ? n->dev->name : "backlog");
return work;
}
*repoll = true;
return work;
}
static int napi_poll(struct napi_struct *n, struct list_head *repoll)
{
bool do_repoll = false;
void *have;
int work;
list_del_init(&n->poll_list);
have = netpoll_poll_lock(n);
work = __napi_poll(n, &do_repoll);
if (do_repoll) {
#if defined(CONFIG_DEBUG_NET)
if (unlikely(!napi_is_scheduled(n)))
pr_crit("repoll requested for device %s %ps but napi is not scheduled.\n",
n->dev->name, n->poll);
#endif
list_add_tail(&n->poll_list, repoll);
}
netpoll_poll_unlock(have);
return work;
}
static int napi_thread_wait(struct napi_struct *napi)
{
set_current_state(TASK_INTERRUPTIBLE);
while (!kthread_should_stop()) {
/* Testing SCHED_THREADED bit here to make sure the current
* kthread owns this napi and could poll on this napi.
* Testing SCHED bit is not enough because SCHED bit might be
* set by some other busy poll thread or by napi_disable().
*/
if (test_bit(NAPI_STATE_SCHED_THREADED, &napi->state)) {
WARN_ON(!list_empty(&napi->poll_list));
__set_current_state(TASK_RUNNING);
return 0;
}
schedule();
set_current_state(TASK_INTERRUPTIBLE);
}
__set_current_state(TASK_RUNNING);
return -1;
}
static void napi_threaded_poll_loop(struct napi_struct *napi, bool busy_poll)
{
struct bpf_net_context __bpf_net_ctx, *bpf_net_ctx;
struct softnet_data *sd;
unsigned long last_qs = jiffies;
for (;;) {
bool repoll = false;
void *have;
local_bh_disable();
bpf_net_ctx = bpf_net_ctx_set(&__bpf_net_ctx);
sd = this_cpu_ptr(&softnet_data);
sd->in_napi_threaded_poll = true;
have = netpoll_poll_lock(napi);
__napi_poll(napi, &repoll);
netpoll_poll_unlock(have);
sd->in_napi_threaded_poll = false;
barrier();
if (sd_has_rps_ipi_waiting(sd)) {
local_irq_disable();
net_rps_action_and_irq_enable(sd);
}
skb_defer_free_flush();
bpf_net_ctx_clear(bpf_net_ctx);
/* When busy poll is enabled, the old packets are not flushed in
* napi_complete_done. So flush them here.
*/
if (busy_poll)
gro_flush_normal(&napi->gro, HZ >= 1000);
local_bh_enable();
/* Call cond_resched here to avoid watchdog warnings. */
if (repoll || busy_poll) {
rcu_softirq_qs_periodic(last_qs);
cond_resched();
}
if (!repoll)
break;
}
}
static int napi_threaded_poll(void *data)
{
struct napi_struct *napi = data;
bool want_busy_poll;
bool in_busy_poll;
unsigned long val;
while (!napi_thread_wait(napi)) {
val = READ_ONCE(napi->state);
want_busy_poll = val & NAPIF_STATE_THREADED_BUSY_POLL;
in_busy_poll = val & NAPIF_STATE_IN_BUSY_POLL;
if (unlikely(val & NAPIF_STATE_DISABLE))
want_busy_poll = false;
if (want_busy_poll != in_busy_poll)
assign_bit(NAPI_STATE_IN_BUSY_POLL, &napi->state,
want_busy_poll);
napi_threaded_poll_loop(napi, want_busy_poll);
}
return 0;
}
static __latent_entropy void net_rx_action(void)
{
struct softnet_data *sd = this_cpu_ptr(&softnet_data);
unsigned long time_limit = jiffies +
usecs_to_jiffies(READ_ONCE(net_hotdata.netdev_budget_usecs));
struct bpf_net_context __bpf_net_ctx, *bpf_net_ctx;
int budget = READ_ONCE(net_hotdata.netdev_budget);
LIST_HEAD(list);
LIST_HEAD(repoll);
bpf_net_ctx = bpf_net_ctx_set(&__bpf_net_ctx);
start:
sd->in_net_rx_action = true;
local_irq_disable();
list_splice_init(&sd->poll_list, &list);
local_irq_enable();
for (;;) {
struct napi_struct *n;
skb_defer_free_flush();
if (list_empty(&list)) {
if (list_empty(&repoll)) {
sd->in_net_rx_action = false;
barrier();
/* We need to check if ____napi_schedule()
* had refilled poll_list while
* sd->in_net_rx_action was true.
*/
if (!list_empty(&sd->poll_list))
goto start;
if (!sd_has_rps_ipi_waiting(sd))
goto end;
}
break;
}
n = list_first_entry(&list, struct napi_struct, poll_list);
budget -= napi_poll(n, &repoll);
/* If softirq window is exhausted then punt.
* Allow this to run for 2 jiffies since which will allow
* an average latency of 1.5/HZ.
*/
if (unlikely(budget <= 0 ||
time_after_eq(jiffies, time_limit))) {
/* Pairs with READ_ONCE() in softnet_seq_show() */
WRITE_ONCE(sd->time_squeeze, sd->time_squeeze + 1);
break;
}
}
local_irq_disable();
list_splice_tail_init(&sd->poll_list, &list);
list_splice_tail(&repoll, &list);
list_splice(&list, &sd->poll_list);
if (!list_empty(&sd->poll_list))
__raise_softirq_irqoff(NET_RX_SOFTIRQ);
else
sd->in_net_rx_action = false;
net_rps_action_and_irq_enable(sd);
end:
bpf_net_ctx_clear(bpf_net_ctx);
}
struct netdev_adjacent {
struct net_device *dev;
netdevice_tracker dev_tracker;
/* upper master flag, there can only be one master device per list */
bool master;
/* lookup ignore flag */
bool ignore;
/* counter for the number of times this device was added to us */
u16 ref_nr;
/* private field for the users */
void *private;
struct list_head list;
struct rcu_head rcu;
};
static struct netdev_adjacent *__netdev_find_adj(struct net_device *adj_dev,
struct list_head *adj_list)
{
struct netdev_adjacent *adj;
list_for_each_entry(adj, adj_list, list) {
if (adj->dev == adj_dev)
return adj;
}
return NULL;
}
static int ____netdev_has_upper_dev(struct net_device *upper_dev,
struct netdev_nested_priv *priv)
{
struct net_device *dev = (struct net_device *)priv->data;
return upper_dev == dev;
}
/**
* netdev_has_upper_dev - Check if device is linked to an upper device
* @dev: device
* @upper_dev: upper device to check
*
* Find out if a device is linked to specified upper device and return true
* in case it is. Note that this checks only immediate upper device,
* not through a complete stack of devices. The caller must hold the RTNL lock.
*/
bool netdev_has_upper_dev(struct net_device *dev,
struct net_device *upper_dev)
{
struct netdev_nested_priv priv = {
.data = (void *)upper_dev,
};
ASSERT_RTNL();
return netdev_walk_all_upper_dev_rcu(dev, ____netdev_has_upper_dev,
&priv);
}
EXPORT_SYMBOL(netdev_has_upper_dev);
/**
* netdev_has_upper_dev_all_rcu - Check if device is linked to an upper device
* @dev: device
* @upper_dev: upper device to check
*
* Find out if a device is linked to specified upper device and return true
* in case it is. Note that this checks the entire upper device chain.
* The caller must hold rcu lock.
*/
bool netdev_has_upper_dev_all_rcu(struct net_device *dev,
struct net_device *upper_dev)
{
struct netdev_nested_priv priv = {
.data = (void *)upper_dev,
};
return !!netdev_walk_all_upper_dev_rcu(dev, ____netdev_has_upper_dev,
&priv);
}
EXPORT_SYMBOL(netdev_has_upper_dev_all_rcu);
/**
* netdev_has_any_upper_dev - Check if device is linked to some device
* @dev: device
*
* Find out if a device is linked to an upper device and return true in case
* it is. The caller must hold the RTNL lock.
*/
bool netdev_has_any_upper_dev(struct net_device *dev)
{
ASSERT_RTNL();
return !list_empty(&dev->adj_list.upper);
}
EXPORT_SYMBOL(netdev_has_any_upper_dev);
/**
* netdev_master_upper_dev_get - Get master upper device
* @dev: device
*
* Find a master upper device and return pointer to it or NULL in case
* it's not there. The caller must hold the RTNL lock.
*/
struct net_device *netdev_master_upper_dev_get(struct net_device *dev)
{
struct netdev_adjacent *upper;
ASSERT_RTNL();
if (list_empty(&dev->adj_list.upper))
return NULL;
upper = list_first_entry(&dev->adj_list.upper,
struct netdev_adjacent, list);
if (likely(upper->master))
return upper->dev;
return NULL;
}
EXPORT_SYMBOL(netdev_master_upper_dev_get);
static struct net_device *__netdev_master_upper_dev_get(struct net_device *dev)
{
struct netdev_adjacent *upper;
ASSERT_RTNL();
if (list_empty(&dev->adj_list.upper))
return NULL;
upper = list_first_entry(&dev->adj_list.upper,
struct netdev_adjacent, list);
if (likely(upper->master) && !upper->ignore)
return upper->dev;
return NULL;
}
/**
* netdev_has_any_lower_dev - Check if device is linked to some device
* @dev: device
*
* Find out if a device is linked to a lower device and return true in case
* it is. The caller must hold the RTNL lock.
*/
static bool netdev_has_any_lower_dev(struct net_device *dev)
{
ASSERT_RTNL();
return !list_empty(&dev->adj_list.lower);
}
void *netdev_adjacent_get_private(struct list_head *adj_list)
{
struct netdev_adjacent *adj;
adj = list_entry(adj_list, struct netdev_adjacent, list);
return adj->private;
}
EXPORT_SYMBOL(netdev_adjacent_get_private);
/**
* netdev_upper_get_next_dev_rcu - Get the next dev from upper list
* @dev: device
* @iter: list_head ** of the current position
*
* Gets the next device from the dev's upper list, starting from iter
* position. The caller must hold RCU read lock.
*/
struct net_device *netdev_upper_get_next_dev_rcu(struct net_device *dev,
struct list_head **iter)
{
struct netdev_adjacent *upper;
WARN_ON_ONCE(!rcu_read_lock_held() && !lockdep_rtnl_is_held());
upper = list_entry_rcu((*iter)->next, struct netdev_adjacent, list);
if (&upper->list == &dev->adj_list.upper)
return NULL;
*iter = &upper->list;
return upper->dev;
}
EXPORT_SYMBOL(netdev_upper_get_next_dev_rcu);
static struct net_device *__netdev_next_upper_dev(struct net_device *dev,
struct list_head **iter,
bool *ignore)
{
struct netdev_adjacent *upper;
upper = list_entry((*iter)->next, struct netdev_adjacent, list);
if (&upper->list == &dev->adj_list.upper)
return NULL;
*iter = &upper->list;
*ignore = upper->ignore;
return upper->dev;
}
static struct net_device *netdev_next_upper_dev_rcu(struct net_device *dev,
struct list_head **iter)
{
struct netdev_adjacent *upper;
WARN_ON_ONCE(!rcu_read_lock_held() && !lockdep_rtnl_is_held());
upper = list_entry_rcu((*iter)->next, struct netdev_adjacent, list);
if (&upper->list == &dev->adj_list.upper)
return NULL;
*iter = &upper->list;
return upper->dev;
}
static int __netdev_walk_all_upper_dev(struct net_device *dev,
int (*fn)(struct net_device *dev,
struct netdev_nested_priv *priv),
struct netdev_nested_priv *priv)
{
struct net_device *udev, *next, *now, *dev_stack[MAX_NEST_DEV + 1];
struct list_head *niter, *iter, *iter_stack[MAX_NEST_DEV + 1];
int ret, cur = 0;
bool ignore;
now = dev;
iter = &dev->adj_list.upper;
while (1) {
if (now != dev) {
ret = fn(now, priv);
if (ret)
return ret;
}
next = NULL;
while (1) {
udev = __netdev_next_upper_dev(now, &iter, &ignore);
if (!udev)
break;
if (ignore)
continue;
next = udev;
niter = &udev->adj_list.upper;
dev_stack[cur] = now;
iter_stack[cur++] = iter;
break;
}
if (!next) {
if (!cur)
return 0;
next = dev_stack[--cur];
niter = iter_stack[cur];
}
now = next;
iter = niter;
}
return 0;
}
int netdev_walk_all_upper_dev_rcu(struct net_device *dev,
int (*fn)(struct net_device *dev,
struct netdev_nested_priv *priv),
struct netdev_nested_priv *priv)
{
struct net_device *udev, *next, *now, *dev_stack[MAX_NEST_DEV + 1];
struct list_head *niter, *iter, *iter_stack[MAX_NEST_DEV + 1];
int ret, cur = 0;
now = dev;
iter = &dev->adj_list.upper;
while (1) {
if (now != dev) {
ret = fn(now, priv);
if (ret)
return ret;
}
next = NULL;
while (1) {
udev = netdev_next_upper_dev_rcu(now, &iter);
if (!udev)
break;
next = udev;
niter = &udev->adj_list.upper;
dev_stack[cur] = now;
iter_stack[cur++] = iter;
break;
}
if (!next) {
if (!cur)
return 0;
next = dev_stack[--cur];
niter = iter_stack[cur];
}
now = next;
iter = niter;
}
return 0;
}
EXPORT_SYMBOL_GPL(netdev_walk_all_upper_dev_rcu);
static bool __netdev_has_upper_dev(struct net_device *dev,
struct net_device *upper_dev)
{
struct netdev_nested_priv priv = {
.flags = 0,
.data = (void *)upper_dev,
};
ASSERT_RTNL();
return __netdev_walk_all_upper_dev(dev, ____netdev_has_upper_dev,
&priv);
}
/**
* netdev_lower_get_next_private - Get the next ->private from the
* lower neighbour list
* @dev: device
* @iter: list_head ** of the current position
*
* Gets the next netdev_adjacent->private from the dev's lower neighbour
* list, starting from iter position. The caller must hold either hold the
* RTNL lock or its own locking that guarantees that the neighbour lower
* list will remain unchanged.
*/
void *netdev_lower_get_next_private(struct net_device *dev,
struct list_head **iter)
{
struct netdev_adjacent *lower;
lower = list_entry(*iter, struct netdev_adjacent, list);
if (&lower->list == &dev->adj_list.lower)
return NULL;
*iter = lower->list.next;
return lower->private;
}
EXPORT_SYMBOL(netdev_lower_get_next_private);
/**
* netdev_lower_get_next_private_rcu - Get the next ->private from the
* lower neighbour list, RCU
* variant
* @dev: device
* @iter: list_head ** of the current position
*
* Gets the next netdev_adjacent->private from the dev's lower neighbour
* list, starting from iter position. The caller must hold RCU read lock.
*/
void *netdev_lower_get_next_private_rcu(struct net_device *dev,
struct list_head **iter)
{
struct netdev_adjacent *lower;
WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held());
lower = list_entry_rcu((*iter)->next, struct netdev_adjacent, list);
if (&lower->list == &dev->adj_list.lower)
return NULL;
*iter = &lower->list;
return lower->private;
}
EXPORT_SYMBOL(netdev_lower_get_next_private_rcu);
/**
* netdev_lower_get_next - Get the next device from the lower neighbour
* list
* @dev: device
* @iter: list_head ** of the current position
*
* Gets the next netdev_adjacent from the dev's lower neighbour
* list, starting from iter position. The caller must hold RTNL lock or
* its own locking that guarantees that the neighbour lower
* list will remain unchanged.
*/
void *netdev_lower_get_next(struct net_device *dev, struct list_head **iter)
{
struct netdev_adjacent *lower;
lower = list_entry(*iter, struct netdev_adjacent, list);
if (&lower->list == &dev->adj_list.lower)
return NULL;
*iter = lower->list.next;
return lower->dev;
}
EXPORT_SYMBOL(netdev_lower_get_next);
static struct net_device *netdev_next_lower_dev(struct net_device *dev,
struct list_head **iter)
{
struct netdev_adjacent *lower;
lower = list_entry((*iter)->next, struct netdev_adjacent, list);
if (&lower->list == &dev->adj_list.lower)
return NULL;
*iter = &lower->list;
return lower->dev;
}
static struct net_device *__netdev_next_lower_dev(struct net_device *dev,
struct list_head **iter,
bool *ignore)
{
struct netdev_adjacent *lower;
lower = list_entry((*iter)->next, struct netdev_adjacent, list);
if (&lower->list == &dev->adj_list.lower)
return NULL;
*iter = &lower->list;
*ignore = lower->ignore;
return lower->dev;
}
int netdev_walk_all_lower_dev(struct net_device *dev,
int (*fn)(struct net_device *dev,
struct netdev_nested_priv *priv),
struct netdev_nested_priv *priv)
{
struct net_device *ldev, *next, *now, *dev_stack[MAX_NEST_DEV + 1];
struct list_head *niter, *iter, *iter_stack[MAX_NEST_DEV + 1];
int ret, cur = 0;
now = dev;
iter = &dev->adj_list.lower;
while (1) {
if (now != dev) {
ret = fn(now, priv);
if (ret)
return ret;
}
next = NULL;
while (1) {
ldev = netdev_next_lower_dev(now, &iter);
if (!ldev)
break;
next = ldev;
niter = &ldev->adj_list.lower;
dev_stack[cur] = now;
iter_stack[cur++] = iter;
break;
}
if (!next) {
if (!cur)
return 0;
next = dev_stack[--cur];
niter = iter_stack[cur];
}
now = next;
iter = niter;
}
return 0;
}
EXPORT_SYMBOL_GPL(netdev_walk_all_lower_dev);
static int __netdev_walk_all_lower_dev(struct net_device *dev,
int (*fn)(struct net_device *dev,
struct netdev_nested_priv *priv),
struct netdev_nested_priv *priv)
{
struct net_device *ldev, *next, *now, *dev_stack[MAX_NEST_DEV + 1];
struct list_head *niter, *iter, *iter_stack[MAX_NEST_DEV + 1];
int ret, cur = 0;
bool ignore;
now = dev;
iter = &dev->adj_list.lower;
while (1) {
if (now != dev) {
ret = fn(now, priv);
if (ret)
return ret;
}
next = NULL;
while (1) {
ldev = __netdev_next_lower_dev(now, &iter, &ignore);
if (!ldev)
break;
if (ignore)
continue;
next = ldev;
niter = &ldev->adj_list.lower;
dev_stack[cur] = now;
iter_stack[cur++] = iter;
break;
}
if (!next) {
if (!cur)
return 0;
next = dev_stack[--cur];
niter = iter_stack[cur];
}
now = next;
iter = niter;
}
return 0;
}
struct net_device *netdev_next_lower_dev_rcu(struct net_device *dev,
struct list_head **iter)
{
struct netdev_adjacent *lower;
lower = list_entry_rcu((*iter)->next, struct netdev_adjacent, list);
if (&lower->list == &dev->adj_list.lower)
return NULL;
*iter = &lower->list;
return lower->dev;
}
EXPORT_SYMBOL(netdev_next_lower_dev_rcu);
static u8 __netdev_upper_depth(struct net_device *dev)
{
struct net_device *udev;
struct list_head *iter;
u8 max_depth = 0;
bool ignore;
for (iter = &dev->adj_list.upper,
udev = __netdev_next_upper_dev(dev, &iter, &ignore);
udev;
udev = __netdev_next_upper_dev(dev, &iter, &ignore)) {
if (ignore)
continue;
if (max_depth < udev->upper_level)
max_depth = udev->upper_level;
}
return max_depth;
}
static u8 __netdev_lower_depth(struct net_device *dev)
{
struct net_device *ldev;
struct list_head *iter;
u8 max_depth = 0;
bool ignore;
for (iter = &dev->adj_list.lower,
ldev = __netdev_next_lower_dev(dev, &iter, &ignore);
ldev;
ldev = __netdev_next_lower_dev(dev, &iter, &ignore)) {
if (ignore)
continue;
if (max_depth < ldev->lower_level)
max_depth = ldev->lower_level;
}
return max_depth;
}
static int __netdev_update_upper_level(struct net_device *dev,
struct netdev_nested_priv *__unused)
{
dev->upper_level = __netdev_upper_depth(dev) + 1;
return 0;
}
#ifdef CONFIG_LOCKDEP
static LIST_HEAD(net_unlink_list);
static void net_unlink_todo(struct net_device *dev)
{
if (list_empty(&dev->unlink_list))
list_add_tail(&dev->unlink_list, &net_unlink_list);
}
#endif
static int __netdev_update_lower_level(struct net_device *dev,
struct netdev_nested_priv *priv)
{
dev->lower_level = __netdev_lower_depth(dev) + 1;
#ifdef CONFIG_LOCKDEP
if (!priv)
return 0;
if (priv->flags & NESTED_SYNC_IMM)
dev->nested_level = dev->lower_level - 1;
if (priv->flags & NESTED_SYNC_TODO)
net_unlink_todo(dev);
#endif
return 0;
}
int netdev_walk_all_lower_dev_rcu(struct net_device *dev,
int (*fn)(struct net_device *dev,
struct netdev_nested_priv *priv),
struct netdev_nested_priv *priv)
{
struct net_device *ldev, *next, *now, *dev_stack[MAX_NEST_DEV + 1];
struct list_head *niter, *iter, *iter_stack[MAX_NEST_DEV + 1];
int ret, cur = 0;
now = dev;
iter = &dev->adj_list.lower;
while (1) {
if (now != dev) {
ret = fn(now, priv);
if (ret)
return ret;
}
next = NULL;
while (1) {
ldev = netdev_next_lower_dev_rcu(now, &iter);
if (!ldev)
break;
next = ldev;
niter = &ldev->adj_list.lower;
dev_stack[cur] = now;
iter_stack[cur++] = iter;
break;
}
if (!next) {
if (!cur)
return 0;
next = dev_stack[--cur];
niter = iter_stack[cur];
}
now = next;
iter = niter;
}
return 0;
}
EXPORT_SYMBOL_GPL(netdev_walk_all_lower_dev_rcu);
/**
* netdev_lower_get_first_private_rcu - Get the first ->private from the
* lower neighbour list, RCU
* variant
* @dev: device
*
* Gets the first netdev_adjacent->private from the dev's lower neighbour
* list. The caller must hold RCU read lock.
*/
void *netdev_lower_get_first_private_rcu(struct net_device *dev)
{
struct netdev_adjacent *lower;
lower = list_first_or_null_rcu(&dev->adj_list.lower,
struct netdev_adjacent, list);
if (lower)
return lower->private;
return NULL;
}
EXPORT_SYMBOL(netdev_lower_get_first_private_rcu);
/**
* netdev_master_upper_dev_get_rcu - Get master upper device
* @dev: device
*
* Find a master upper device and return pointer to it or NULL in case
* it's not there. The caller must hold the RCU read lock.
*/
struct net_device *netdev_master_upper_dev_get_rcu(struct net_device *dev)
{
struct netdev_adjacent *upper;
upper = list_first_or_null_rcu(&dev->adj_list.upper,
struct netdev_adjacent, list);
if (upper && likely(upper->master))
return upper->dev;
return NULL;
}
EXPORT_SYMBOL(netdev_master_upper_dev_get_rcu);
static int netdev_adjacent_sysfs_add(struct net_device *dev,
struct net_device *adj_dev,
struct list_head *dev_list)
{
char linkname[IFNAMSIZ+7];
sprintf(linkname, dev_list == &dev->adj_list.upper ?
"upper_%s" : "lower_%s", adj_dev->name);
return sysfs_create_link(&(dev->dev.kobj), &(adj_dev->dev.kobj),
linkname);
}
static void netdev_adjacent_sysfs_del(struct net_device *dev,
char *name,
struct list_head *dev_list)
{
char linkname[IFNAMSIZ+7];
sprintf(linkname, dev_list == &dev->adj_list.upper ?
"upper_%s" : "lower_%s", name);
sysfs_remove_link(&(dev->dev.kobj), linkname);
}
static inline bool netdev_adjacent_is_neigh_list(struct net_device *dev,
struct net_device *adj_dev,
struct list_head *dev_list)
{
return (dev_list == &dev->adj_list.upper ||
dev_list == &dev->adj_list.lower) &&
net_eq(dev_net(dev), dev_net(adj_dev));
}
static int __netdev_adjacent_dev_insert(struct net_device *dev,
struct net_device *adj_dev,
struct list_head *dev_list,
void *private, bool master)
{
struct netdev_adjacent *adj;
int ret;
adj = __netdev_find_adj(adj_dev, dev_list);
if (adj) {
adj->ref_nr += 1;
pr_debug("Insert adjacency: dev %s adj_dev %s adj->ref_nr %d\n",
dev->name, adj_dev->name, adj->ref_nr);
return 0;
}
adj = kmalloc(sizeof(*adj), GFP_KERNEL);
if (!adj)
return -ENOMEM;
adj->dev = adj_dev;
adj->master = master;
adj->ref_nr = 1;
adj->private = private;
adj->ignore = false;
netdev_hold(adj_dev, &adj->dev_tracker, GFP_KERNEL);
pr_debug("Insert adjacency: dev %s adj_dev %s adj->ref_nr %d; dev_hold on %s\n",
dev->name, adj_dev->name, adj->ref_nr, adj_dev->name);
if (netdev_adjacent_is_neigh_list(dev, adj_dev, dev_list)) {
ret = netdev_adjacent_sysfs_add(dev, adj_dev, dev_list);
if (ret)
goto free_adj;
}
/* Ensure that master link is always the first item in list. */
if (master) {
ret = sysfs_create_link(&(dev->dev.kobj),
&(adj_dev->dev.kobj), "master");
if (ret)
goto remove_symlinks;
list_add_rcu(&adj->list, dev_list);
} else {
list_add_tail_rcu(&adj->list, dev_list);
}
return 0;
remove_symlinks:
if (netdev_adjacent_is_neigh_list(dev, adj_dev, dev_list))
netdev_adjacent_sysfs_del(dev, adj_dev->name, dev_list);
free_adj:
netdev_put(adj_dev, &adj->dev_tracker);
kfree(adj);
return ret;
}
static void __netdev_adjacent_dev_remove(struct net_device *dev,
struct net_device *adj_dev,
u16 ref_nr,
struct list_head *dev_list)
{
struct netdev_adjacent *adj;
pr_debug("Remove adjacency: dev %s adj_dev %s ref_nr %d\n",
dev->name, adj_dev->name, ref_nr);
adj = __netdev_find_adj(adj_dev, dev_list);
if (!adj) {
pr_err("Adjacency does not exist for device %s from %s\n",
dev->name, adj_dev->name);
WARN_ON(1);
return;
}
if (adj->ref_nr > ref_nr) {
pr_debug("adjacency: %s to %s ref_nr - %d = %d\n",
dev->name, adj_dev->name, ref_nr,
adj->ref_nr - ref_nr);
adj->ref_nr -= ref_nr;
return;
}
if (adj->master)
sysfs_remove_link(&(dev->dev.kobj), "master");
if (netdev_adjacent_is_neigh_list(dev, adj_dev, dev_list))
netdev_adjacent_sysfs_del(dev, adj_dev->name, dev_list);
list_del_rcu(&adj->list);
pr_debug("adjacency: dev_put for %s, because link removed from %s to %s\n",
adj_dev->name, dev->name, adj_dev->name);
netdev_put(adj_dev, &adj->dev_tracker);
kfree_rcu(adj, rcu);
}
static int __netdev_adjacent_dev_link_lists(struct net_device *dev,
struct net_device *upper_dev,
struct list_head *up_list,
struct list_head *down_list,
void *private, bool master)
{
int ret;
ret = __netdev_adjacent_dev_insert(dev, upper_dev, up_list,
private, master);
if (ret)
return ret;
ret = __netdev_adjacent_dev_insert(upper_dev, dev, down_list,
private, false);
if (ret) {
__netdev_adjacent_dev_remove(dev, upper_dev, 1, up_list);
return ret;
}
return 0;
}
static void __netdev_adjacent_dev_unlink_lists(struct net_device *dev,
struct net_device *upper_dev,
u16 ref_nr,
struct list_head *up_list,
struct list_head *down_list)
{
__netdev_adjacent_dev_remove(dev, upper_dev, ref_nr, up_list);
__netdev_adjacent_dev_remove(upper_dev, dev, ref_nr, down_list);
}
static int __netdev_adjacent_dev_link_neighbour(struct net_device *dev,
struct net_device *upper_dev,
void *private, bool master)
{
return __netdev_adjacent_dev_link_lists(dev, upper_dev,
&dev->adj_list.upper,
&upper_dev->adj_list.lower,
private, master);
}
static void __netdev_adjacent_dev_unlink_neighbour(struct net_device *dev,
struct net_device *upper_dev)
{
__netdev_adjacent_dev_unlink_lists(dev, upper_dev, 1,
&dev->adj_list.upper,
&upper_dev->adj_list.lower);
}
static int __netdev_upper_dev_link(struct net_device *dev,
struct net_device *upper_dev, bool master,
void *upper_priv, void *upper_info,
struct netdev_nested_priv *priv,
struct netlink_ext_ack *extack)
{
struct netdev_notifier_changeupper_info changeupper_info = {
.info = {
.dev = dev,
.extack = extack,
},
.upper_dev = upper_dev,
.master = master,
.linking = true,
.upper_info = upper_info,
};
struct net_device *master_dev;
int ret = 0;
ASSERT_RTNL();
if (dev == upper_dev)
return -EBUSY;
/* To prevent loops, check if dev is not upper device to upper_dev. */
if (__netdev_has_upper_dev(upper_dev, dev))
return -EBUSY;
if ((dev->lower_level + upper_dev->upper_level) > MAX_NEST_DEV)
return -EMLINK;
if (!master) {
if (__netdev_has_upper_dev(dev, upper_dev))
return -EEXIST;
} else {
master_dev = __netdev_master_upper_dev_get(dev);
if (master_dev)
return master_dev == upper_dev ? -EEXIST : -EBUSY;
}
ret = call_netdevice_notifiers_info(NETDEV_PRECHANGEUPPER,
&changeupper_info.info);
ret = notifier_to_errno(ret);
if (ret)
return ret;
ret = __netdev_adjacent_dev_link_neighbour(dev, upper_dev, upper_priv,
master);
if (ret)
return ret;
ret = call_netdevice_notifiers_info(NETDEV_CHANGEUPPER,
&changeupper_info.info);
ret = notifier_to_errno(ret);
if (ret)
goto rollback;
__netdev_update_upper_level(dev, NULL);
__netdev_walk_all_lower_dev(dev, __netdev_update_upper_level, NULL);
__netdev_update_lower_level(upper_dev, priv);
__netdev_walk_all_upper_dev(upper_dev, __netdev_update_lower_level,
priv);
return 0;
rollback:
__netdev_adjacent_dev_unlink_neighbour(dev, upper_dev);
return ret;
}
/**
* netdev_upper_dev_link - Add a link to the upper device
* @dev: device
* @upper_dev: new upper device
* @extack: netlink extended ack
*
* Adds a link to device which is upper to this one. The caller must hold
* the RTNL lock. On a failure a negative errno code is returned.
* On success the reference counts are adjusted and the function
* returns zero.
*/
int netdev_upper_dev_link(struct net_device *dev,
struct net_device *upper_dev,
struct netlink_ext_ack *extack)
{
struct netdev_nested_priv priv = {
.flags = NESTED_SYNC_IMM | NESTED_SYNC_TODO,
.data = NULL,
};
return __netdev_upper_dev_link(dev, upper_dev, false,
NULL, NULL, &priv, extack);
}
EXPORT_SYMBOL(netdev_upper_dev_link);
/**
* netdev_master_upper_dev_link - Add a master link to the upper device
* @dev: device
* @upper_dev: new upper device
* @upper_priv: upper device private
* @upper_info: upper info to be passed down via notifier
* @extack: netlink extended ack
*
* Adds a link to device which is upper to this one. In this case, only
* one master upper device can be linked, although other non-master devices
* might be linked as well. The caller must hold the RTNL lock.
* On a failure a negative errno code is returned. On success the reference
* counts are adjusted and the function returns zero.
*/
int netdev_master_upper_dev_link(struct net_device *dev,
struct net_device *upper_dev,
void *upper_priv, void *upper_info,
struct netlink_ext_ack *extack)
{
struct netdev_nested_priv priv = {
.flags = NESTED_SYNC_IMM | NESTED_SYNC_TODO,
.data = NULL,
};
return __netdev_upper_dev_link(dev, upper_dev, true,
upper_priv, upper_info, &priv, extack);
}
EXPORT_SYMBOL(netdev_master_upper_dev_link);
static void __netdev_upper_dev_unlink(struct net_device *dev,
struct net_device *upper_dev,
struct netdev_nested_priv *priv)
{
struct netdev_notifier_changeupper_info changeupper_info = {
.info = {
.dev = dev,
},
.upper_dev = upper_dev,
.linking = false,
};
ASSERT_RTNL();
changeupper_info.master = netdev_master_upper_dev_get(dev) == upper_dev;
call_netdevice_notifiers_info(NETDEV_PRECHANGEUPPER,
&changeupper_info.info);
__netdev_adjacent_dev_unlink_neighbour(dev, upper_dev);
call_netdevice_notifiers_info(NETDEV_CHANGEUPPER,
&changeupper_info.info);
__netdev_update_upper_level(dev, NULL);
__netdev_walk_all_lower_dev(dev, __netdev_update_upper_level, NULL);
__netdev_update_lower_level(upper_dev, priv);
__netdev_walk_all_upper_dev(upper_dev, __netdev_update_lower_level,
priv);
}
/**
* netdev_upper_dev_unlink - Removes a link to upper device
* @dev: device
* @upper_dev: new upper device
*
* Removes a link to device which is upper to this one. The caller must hold
* the RTNL lock.
*/
void netdev_upper_dev_unlink(struct net_device *dev,
struct net_device *upper_dev)
{
struct netdev_nested_priv priv = {
.flags = NESTED_SYNC_TODO,
.data = NULL,
};
__netdev_upper_dev_unlink(dev, upper_dev, &priv);
}
EXPORT_SYMBOL(netdev_upper_dev_unlink);
static void __netdev_adjacent_dev_set(struct net_device *upper_dev,
struct net_device *lower_dev,
bool val)
{
struct netdev_adjacent *adj;
adj = __netdev_find_adj(lower_dev, &upper_dev->adj_list.lower);
if (adj)
adj->ignore = val;
adj = __netdev_find_adj(upper_dev, &lower_dev->adj_list.upper);
if (adj)
adj->ignore = val;
}
static void netdev_adjacent_dev_disable(struct net_device *upper_dev,
struct net_device *lower_dev)
{
__netdev_adjacent_dev_set(upper_dev, lower_dev, true);
}
static void netdev_adjacent_dev_enable(struct net_device *upper_dev,
struct net_device *lower_dev)
{
__netdev_adjacent_dev_set(upper_dev, lower_dev, false);
}
int netdev_adjacent_change_prepare(struct net_device *old_dev,
struct net_device *new_dev,
struct net_device *dev,
struct netlink_ext_ack *extack)
{
struct netdev_nested_priv priv = {
.flags = 0,
.data = NULL,
};
int err;
if (!new_dev)
return 0;
if (old_dev && new_dev != old_dev)
netdev_adjacent_dev_disable(dev, old_dev);
err = __netdev_upper_dev_link(new_dev, dev, false, NULL, NULL, &priv,
extack);
if (err) {
if (old_dev && new_dev != old_dev)
netdev_adjacent_dev_enable(dev, old_dev);
return err;
}
return 0;
}
EXPORT_SYMBOL(netdev_adjacent_change_prepare);
void netdev_adjacent_change_commit(struct net_device *old_dev,
struct net_device *new_dev,
struct net_device *dev)
{
struct netdev_nested_priv priv = {
.flags = NESTED_SYNC_IMM | NESTED_SYNC_TODO,
.data = NULL,
};
if (!new_dev || !old_dev)
return;
if (new_dev == old_dev)
return;
netdev_adjacent_dev_enable(dev, old_dev);
__netdev_upper_dev_unlink(old_dev, dev, &priv);
}
EXPORT_SYMBOL(netdev_adjacent_change_commit);
void netdev_adjacent_change_abort(struct net_device *old_dev,
struct net_device *new_dev,
struct net_device *dev)
{
struct netdev_nested_priv priv = {
.flags = 0,
.data = NULL,
};
if (!new_dev)
return;
if (old_dev && new_dev != old_dev)
netdev_adjacent_dev_enable(dev, old_dev);
__netdev_upper_dev_unlink(new_dev, dev, &priv);
}
EXPORT_SYMBOL(netdev_adjacent_change_abort);
/**
* netdev_bonding_info_change - Dispatch event about slave change
* @dev: device
* @bonding_info: info to dispatch
*
* Send NETDEV_BONDING_INFO to netdev notifiers with info.
* The caller must hold the RTNL lock.
*/
void netdev_bonding_info_change(struct net_device *dev,
struct netdev_bonding_info *bonding_info)
{
struct netdev_notifier_bonding_info info = {
.info.dev = dev,
};
memcpy(&info.bonding_info, bonding_info,
sizeof(struct netdev_bonding_info));
call_netdevice_notifiers_info(NETDEV_BONDING_INFO,
&info.info);
}
EXPORT_SYMBOL(netdev_bonding_info_change);
static int netdev_offload_xstats_enable_l3(struct net_device *dev,
struct netlink_ext_ack *extack)
{
struct netdev_notifier_offload_xstats_info info = {
.info.dev = dev,
.info.extack = extack,
.type = NETDEV_OFFLOAD_XSTATS_TYPE_L3,
};
int err;
int rc;
dev->offload_xstats_l3 = kzalloc(sizeof(*dev->offload_xstats_l3),
GFP_KERNEL);
if (!dev->offload_xstats_l3)
return -ENOMEM;
rc = call_netdevice_notifiers_info_robust(NETDEV_OFFLOAD_XSTATS_ENABLE,
NETDEV_OFFLOAD_XSTATS_DISABLE,
&info.info);
err = notifier_to_errno(rc);
if (err)
goto free_stats;
return 0;
free_stats:
kfree(dev->offload_xstats_l3);
dev->offload_xstats_l3 = NULL;
return err;
}
int netdev_offload_xstats_enable(struct net_device *dev,
enum netdev_offload_xstats_type type,
struct netlink_ext_ack *extack)
{
ASSERT_RTNL();
if (netdev_offload_xstats_enabled(dev, type))
return -EALREADY;
switch (type) {
case NETDEV_OFFLOAD_XSTATS_TYPE_L3:
return netdev_offload_xstats_enable_l3(dev, extack);
}
WARN_ON(1);
return -EINVAL;
}
EXPORT_SYMBOL(netdev_offload_xstats_enable);
static void netdev_offload_xstats_disable_l3(struct net_device *dev)
{
struct netdev_notifier_offload_xstats_info info = {
.info.dev = dev,
.type = NETDEV_OFFLOAD_XSTATS_TYPE_L3,
};
call_netdevice_notifiers_info(NETDEV_OFFLOAD_XSTATS_DISABLE,
&info.info);
kfree(dev->offload_xstats_l3);
dev->offload_xstats_l3 = NULL;
}
int netdev_offload_xstats_disable(struct net_device *dev,
enum netdev_offload_xstats_type type)
{
ASSERT_RTNL();
if (!netdev_offload_xstats_enabled(dev, type))
return -EALREADY;
switch (type) {
case NETDEV_OFFLOAD_XSTATS_TYPE_L3:
netdev_offload_xstats_disable_l3(dev);
return 0;
}
WARN_ON(1);
return -EINVAL;
}
EXPORT_SYMBOL(netdev_offload_xstats_disable);
static void netdev_offload_xstats_disable_all(struct net_device *dev)
{
netdev_offload_xstats_disable(dev, NETDEV_OFFLOAD_XSTATS_TYPE_L3);
}
static struct rtnl_hw_stats64 *
netdev_offload_xstats_get_ptr(const struct net_device *dev,
enum netdev_offload_xstats_type type)
{
switch (type) {
case NETDEV_OFFLOAD_XSTATS_TYPE_L3:
return dev->offload_xstats_l3;
}
WARN_ON(1);
return NULL;
}
bool netdev_offload_xstats_enabled(const struct net_device *dev,
enum netdev_offload_xstats_type type)
{
ASSERT_RTNL();
return netdev_offload_xstats_get_ptr(dev, type);
}
EXPORT_SYMBOL(netdev_offload_xstats_enabled);
struct netdev_notifier_offload_xstats_ru {
bool used;
};
struct netdev_notifier_offload_xstats_rd {
struct rtnl_hw_stats64 stats;
bool used;
};
static void netdev_hw_stats64_add(struct rtnl_hw_stats64 *dest,
const struct rtnl_hw_stats64 *src)
{
dest->rx_packets += src->rx_packets;
dest->tx_packets += src->tx_packets;
dest->rx_bytes += src->rx_bytes;
dest->tx_bytes += src->tx_bytes;
dest->rx_errors += src->rx_errors;
dest->tx_errors += src->tx_errors;
dest->rx_dropped += src->rx_dropped;
dest->tx_dropped += src->tx_dropped;
dest->multicast += src->multicast;
}
static int netdev_offload_xstats_get_used(struct net_device *dev,
enum netdev_offload_xstats_type type,
bool *p_used,
struct netlink_ext_ack *extack)
{
struct netdev_notifier_offload_xstats_ru report_used = {};
struct netdev_notifier_offload_xstats_info info = {
.info.dev = dev,
.info.extack = extack,
.type = type,
.report_used = &report_used,
};
int rc;
WARN_ON(!netdev_offload_xstats_enabled(dev, type));
rc = call_netdevice_notifiers_info(NETDEV_OFFLOAD_XSTATS_REPORT_USED,
&info.info);
*p_used = report_used.used;
return notifier_to_errno(rc);
}
static int netdev_offload_xstats_get_stats(struct net_device *dev,
enum netdev_offload_xstats_type type,
struct rtnl_hw_stats64 *p_stats,
bool *p_used,
struct netlink_ext_ack *extack)
{
struct netdev_notifier_offload_xstats_rd report_delta = {};
struct netdev_notifier_offload_xstats_info info = {
.info.dev = dev,
.info.extack = extack,
.type = type,
.report_delta = &report_delta,
};
struct rtnl_hw_stats64 *stats;
int rc;
stats = netdev_offload_xstats_get_ptr(dev, type);
if (WARN_ON(!stats))
return -EINVAL;
rc = call_netdevice_notifiers_info(NETDEV_OFFLOAD_XSTATS_REPORT_DELTA,
&info.info);
/* Cache whatever we got, even if there was an error, otherwise the
* successful stats retrievals would get lost.
*/
netdev_hw_stats64_add(stats, &report_delta.stats);
if (p_stats)
*p_stats = *stats;
*p_used = report_delta.used;
return notifier_to_errno(rc);
}
int netdev_offload_xstats_get(struct net_device *dev,
enum netdev_offload_xstats_type type,
struct rtnl_hw_stats64 *p_stats, bool *p_used,
struct netlink_ext_ack *extack)
{
ASSERT_RTNL();
if (p_stats)
return netdev_offload_xstats_get_stats(dev, type, p_stats,
p_used, extack);
else
return netdev_offload_xstats_get_used(dev, type, p_used,
extack);
}
EXPORT_SYMBOL(netdev_offload_xstats_get);
void
netdev_offload_xstats_report_delta(struct netdev_notifier_offload_xstats_rd *report_delta,
const struct rtnl_hw_stats64 *stats)
{
report_delta->used = true;
netdev_hw_stats64_add(&report_delta->stats, stats);
}
EXPORT_SYMBOL(netdev_offload_xstats_report_delta);
void
netdev_offload_xstats_report_used(struct netdev_notifier_offload_xstats_ru *report_used)
{
report_used->used = true;
}
EXPORT_SYMBOL(netdev_offload_xstats_report_used);
void netdev_offload_xstats_push_delta(struct net_device *dev,
enum netdev_offload_xstats_type type,
const struct rtnl_hw_stats64 *p_stats)
{
struct rtnl_hw_stats64 *stats;
ASSERT_RTNL();
stats = netdev_offload_xstats_get_ptr(dev, type);
if (WARN_ON(!stats))
return;
netdev_hw_stats64_add(stats, p_stats);
}
EXPORT_SYMBOL(netdev_offload_xstats_push_delta);
/**
* netdev_get_xmit_slave - Get the xmit slave of master device
* @dev: device
* @skb: The packet
* @all_slaves: assume all the slaves are active
*
* The reference counters are not incremented so the caller must be
* careful with locks. The caller must hold RCU lock.
* %NULL is returned if no slave is found.
*/
struct net_device *netdev_get_xmit_slave(struct net_device *dev,
struct sk_buff *skb,
bool all_slaves)
{
const struct net_device_ops *ops = dev->netdev_ops;
if (!ops->ndo_get_xmit_slave)
return NULL;
return ops->ndo_get_xmit_slave(dev, skb, all_slaves);
}
EXPORT_SYMBOL(netdev_get_xmit_slave);
static struct net_device *netdev_sk_get_lower_dev(struct net_device *dev,
struct sock *sk)
{
const struct net_device_ops *ops = dev->netdev_ops;
if (!ops->ndo_sk_get_lower_dev)
return NULL;
return ops->ndo_sk_get_lower_dev(dev, sk);
}
/**
* netdev_sk_get_lowest_dev - Get the lowest device in chain given device and socket
* @dev: device
* @sk: the socket
*
* %NULL is returned if no lower device is found.
*/
struct net_device *netdev_sk_get_lowest_dev(struct net_device *dev,
struct sock *sk)
{
struct net_device *lower;
lower = netdev_sk_get_lower_dev(dev, sk);
while (lower) {
dev = lower;
lower = netdev_sk_get_lower_dev(dev, sk);
}
return dev;
}
EXPORT_SYMBOL(netdev_sk_get_lowest_dev);
static void netdev_adjacent_add_links(struct net_device *dev)
{
struct netdev_adjacent *iter;
struct net *net = dev_net(dev);
list_for_each_entry(iter, &dev->adj_list.upper, list) {
if (!net_eq(net, dev_net(iter->dev)))
continue;
netdev_adjacent_sysfs_add(iter->dev, dev,
&iter->dev->adj_list.lower);
netdev_adjacent_sysfs_add(dev, iter->dev,
&dev->adj_list.upper);
}
list_for_each_entry(iter, &dev->adj_list.lower, list) {
if (!net_eq(net, dev_net(iter->dev)))
continue;
netdev_adjacent_sysfs_add(iter->dev, dev,
&iter->dev->adj_list.upper);
netdev_adjacent_sysfs_add(dev, iter->dev,
&dev->adj_list.lower);
}
}
static void netdev_adjacent_del_links(struct net_device *dev)
{
struct netdev_adjacent *iter;
struct net *net = dev_net(dev);
list_for_each_entry(iter, &dev->adj_list.upper, list) {
if (!net_eq(net, dev_net(iter->dev)))
continue;
netdev_adjacent_sysfs_del(iter->dev, dev->name,
&iter->dev->adj_list.lower);
netdev_adjacent_sysfs_del(dev, iter->dev->name,
&dev->adj_list.upper);
}
list_for_each_entry(iter, &dev->adj_list.lower, list) {
if (!net_eq(net, dev_net(iter->dev)))
continue;
netdev_adjacent_sysfs_del(iter->dev, dev->name,
&iter->dev->adj_list.upper);
netdev_adjacent_sysfs_del(dev, iter->dev->name,
&dev->adj_list.lower);
}
}
void netdev_adjacent_rename_links(struct net_device *dev, char *oldname)
{
struct netdev_adjacent *iter;
struct net *net = dev_net(dev);
list_for_each_entry(iter, &dev->adj_list.upper, list) {
if (!net_eq(net, dev_net(iter->dev)))
continue;
netdev_adjacent_sysfs_del(iter->dev, oldname,
&iter->dev->adj_list.lower);
netdev_adjacent_sysfs_add(iter->dev, dev,
&iter->dev->adj_list.lower);
}
list_for_each_entry(iter, &dev->adj_list.lower, list) {
if (!net_eq(net, dev_net(iter->dev)))
continue;
netdev_adjacent_sysfs_del(iter->dev, oldname,
&iter->dev->adj_list.upper);
netdev_adjacent_sysfs_add(iter->dev, dev,
&iter->dev->adj_list.upper);
}
}
void *netdev_lower_dev_get_private(struct net_device *dev,
struct net_device *lower_dev)
{
struct netdev_adjacent *lower;
if (!lower_dev)
return NULL;
lower = __netdev_find_adj(lower_dev, &dev->adj_list.lower);
if (!lower)
return NULL;
return lower->private;
}
EXPORT_SYMBOL(netdev_lower_dev_get_private);
/**
* netdev_lower_state_changed - Dispatch event about lower device state change
* @lower_dev: device
* @lower_state_info: state to dispatch
*
* Send NETDEV_CHANGELOWERSTATE to netdev notifiers with info.
* The caller must hold the RTNL lock.
*/
void netdev_lower_state_changed(struct net_device *lower_dev,
void *lower_state_info)
{
struct netdev_notifier_changelowerstate_info changelowerstate_info = {
.info.dev = lower_dev,
};
ASSERT_RTNL();
changelowerstate_info.lower_state_info = lower_state_info;
call_netdevice_notifiers_info(NETDEV_CHANGELOWERSTATE,
&changelowerstate_info.info);
}
EXPORT_SYMBOL(netdev_lower_state_changed);
static void dev_change_rx_flags(struct net_device *dev, int flags)
{
const struct net_device_ops *ops = dev->netdev_ops;
if (ops->ndo_change_rx_flags)
ops->ndo_change_rx_flags(dev, flags);
}
static int __dev_set_promiscuity(struct net_device *dev, int inc, bool notify)
{
unsigned int old_flags = dev->flags;
unsigned int promiscuity, flags;
kuid_t uid;
kgid_t gid;
ASSERT_RTNL();
promiscuity = dev->promiscuity + inc;
if (promiscuity == 0) {
/*
* Avoid overflow.
* If inc causes overflow, untouch promisc and return error.
*/
if (unlikely(inc > 0)) {
netdev_warn(dev, "promiscuity touches roof, set promiscuity failed. promiscuity feature of device might be broken.\n");
return -EOVERFLOW;
}
flags = old_flags & ~IFF_PROMISC;
} else {
flags = old_flags | IFF_PROMISC;
}
WRITE_ONCE(dev->promiscuity, promiscuity);
if (flags != old_flags) {
WRITE_ONCE(dev->flags, flags);
netdev_info(dev, "%s promiscuous mode\n",
dev->flags & IFF_PROMISC ? "entered" : "left");
if (audit_enabled) {
current_uid_gid(&uid, &gid);
audit_log(audit_context(), GFP_ATOMIC,
AUDIT_ANOM_PROMISCUOUS,
"dev=%s prom=%d old_prom=%d auid=%u uid=%u gid=%u ses=%u",
dev->name, (dev->flags & IFF_PROMISC),
(old_flags & IFF_PROMISC),
from_kuid(&init_user_ns, audit_get_loginuid(current)),
from_kuid(&init_user_ns, uid),
from_kgid(&init_user_ns, gid),
audit_get_sessionid(current));
}
dev_change_rx_flags(dev, IFF_PROMISC);
}
if (notify) {
/* The ops lock is only required to ensure consistent locking
* for `NETDEV_CHANGE` notifiers. This function is sometimes
* called without the lock, even for devices that are ops
* locked, such as in `dev_uc_sync_multiple` when using
* bonding or teaming.
*/
netdev_ops_assert_locked(dev);
__dev_notify_flags(dev, old_flags, IFF_PROMISC, 0, NULL);
}
return 0;
}
int netif_set_promiscuity(struct net_device *dev, int inc)
{
unsigned int old_flags = dev->flags;
int err;
err = __dev_set_promiscuity(dev, inc, true);
if (err < 0)
return err;
if (dev->flags != old_flags)
dev_set_rx_mode(dev);
return err;
}
int netif_set_allmulti(struct net_device *dev, int inc, bool notify)
{
unsigned int old_flags = dev->flags, old_gflags = dev->gflags;
unsigned int allmulti, flags;
ASSERT_RTNL();
allmulti = dev->allmulti + inc;
if (allmulti == 0) {
/*
* Avoid overflow.
* If inc causes overflow, untouch allmulti and return error.
*/
if (unlikely(inc > 0)) {
netdev_warn(dev, "allmulti touches roof, set allmulti failed. allmulti feature of device might be broken.\n");
return -EOVERFLOW;
}
flags = old_flags & ~IFF_ALLMULTI;
} else {
flags = old_flags | IFF_ALLMULTI;
}
WRITE_ONCE(dev->allmulti, allmulti);
if (flags != old_flags) {
WRITE_ONCE(dev->flags, flags);
netdev_info(dev, "%s allmulticast mode\n",
dev->flags & IFF_ALLMULTI ? "entered" : "left");
dev_change_rx_flags(dev, IFF_ALLMULTI);
dev_set_rx_mode(dev);
if (notify)
__dev_notify_flags(dev, old_flags,
dev->gflags ^ old_gflags, 0, NULL);
}
return 0;
}
/*
* Upload unicast and multicast address lists to device and
* configure RX filtering. When the device doesn't support unicast
* filtering it is put in promiscuous mode while unicast addresses
* are present.
*/
void __dev_set_rx_mode(struct net_device *dev)
{
const struct net_device_ops *ops = dev->netdev_ops;
/* dev_open will call this function so the list will stay sane. */
if (!(dev->flags&IFF_UP))
return;
if (!netif_device_present(dev))
return;
if (!(dev->priv_flags & IFF_UNICAST_FLT)) {
/* Unicast addresses changes may only happen under the rtnl,
* therefore calling __dev_set_promiscuity here is safe.
*/
if (!netdev_uc_empty(dev) && !dev->uc_promisc) {
__dev_set_promiscuity(dev, 1, false);
dev->uc_promisc = true;
} else if (netdev_uc_empty(dev) && dev->uc_promisc) {
__dev_set_promiscuity(dev, -1, false);
dev->uc_promisc = false;
}
}
if (ops->ndo_set_rx_mode)
ops->ndo_set_rx_mode(dev);
}
void dev_set_rx_mode(struct net_device *dev)
{
netif_addr_lock_bh(dev);
__dev_set_rx_mode(dev);
netif_addr_unlock_bh(dev);
}
/**
* netif_get_flags() - get flags reported to userspace
* @dev: device
*
* Get the combination of flag bits exported through APIs to userspace.
*/
unsigned int netif_get_flags(const struct net_device *dev)
{
unsigned int flags;
flags = (READ_ONCE(dev->flags) & ~(IFF_PROMISC |
IFF_ALLMULTI |
IFF_RUNNING |
IFF_LOWER_UP |
IFF_DORMANT)) |
(READ_ONCE(dev->gflags) & (IFF_PROMISC |
IFF_ALLMULTI));
if (netif_running(dev)) {
if (netif_oper_up(dev))
flags |= IFF_RUNNING;
if (netif_carrier_ok(dev))
flags |= IFF_LOWER_UP;
if (netif_dormant(dev))
flags |= IFF_DORMANT;
}
return flags;
}
EXPORT_SYMBOL(netif_get_flags);
int __dev_change_flags(struct net_device *dev, unsigned int flags,
struct netlink_ext_ack *extack)
{
unsigned int old_flags = dev->flags;
int ret;
ASSERT_RTNL();
/*
* Set the flags on our device.
*/
dev->flags = (flags & (IFF_DEBUG | IFF_NOTRAILERS | IFF_NOARP |
IFF_DYNAMIC | IFF_MULTICAST | IFF_PORTSEL |
IFF_AUTOMEDIA)) |
(dev->flags & (IFF_UP | IFF_VOLATILE | IFF_PROMISC |
IFF_ALLMULTI));
/*
* Load in the correct multicast list now the flags have changed.
*/
if ((old_flags ^ flags) & IFF_MULTICAST)
dev_change_rx_flags(dev, IFF_MULTICAST);
dev_set_rx_mode(dev);
/*
* Have we downed the interface. We handle IFF_UP ourselves
* according to user attempts to set it, rather than blindly
* setting it.
*/
ret = 0;
if ((old_flags ^ flags) & IFF_UP) {
if (old_flags & IFF_UP)
__dev_close(dev);
else
ret = __dev_open(dev, extack);
}
if ((flags ^ dev->gflags) & IFF_PROMISC) {
int inc = (flags & IFF_PROMISC) ? 1 : -1;
old_flags = dev->flags;
dev->gflags ^= IFF_PROMISC;
if (__dev_set_promiscuity(dev, inc, false) >= 0)
if (dev->flags != old_flags)
dev_set_rx_mode(dev);
}
/* NOTE: order of synchronization of IFF_PROMISC and IFF_ALLMULTI
* is important. Some (broken) drivers set IFF_PROMISC, when
* IFF_ALLMULTI is requested not asking us and not reporting.
*/
if ((flags ^ dev->gflags) & IFF_ALLMULTI) {
int inc = (flags & IFF_ALLMULTI) ? 1 : -1;
dev->gflags ^= IFF_ALLMULTI;
netif_set_allmulti(dev, inc, false);
}
return ret;
}
void __dev_notify_flags(struct net_device *dev, unsigned int old_flags,
unsigned int gchanges, u32 portid,
const struct nlmsghdr *nlh)
{
unsigned int changes = dev->flags ^ old_flags;
if (gchanges)
rtmsg_ifinfo(RTM_NEWLINK, dev, gchanges, GFP_ATOMIC, portid, nlh);
if (changes & IFF_UP) {
if (dev->flags & IFF_UP)
call_netdevice_notifiers(NETDEV_UP, dev);
else
call_netdevice_notifiers(NETDEV_DOWN, dev);
}
if (dev->flags & IFF_UP &&
(changes & ~(IFF_UP | IFF_PROMISC | IFF_ALLMULTI | IFF_VOLATILE))) {
struct netdev_notifier_change_info change_info = {
.info = {
.dev = dev,
},
.flags_changed = changes,
};
call_netdevice_notifiers_info(NETDEV_CHANGE, &change_info.info);
}
}
int netif_change_flags(struct net_device *dev, unsigned int flags,
struct netlink_ext_ack *extack)
{
int ret;
unsigned int changes, old_flags = dev->flags, old_gflags = dev->gflags;
ret = __dev_change_flags(dev, flags, extack);
if (ret < 0)
return ret;
changes = (old_flags ^ dev->flags) | (old_gflags ^ dev->gflags);
__dev_notify_flags(dev, old_flags, changes, 0, NULL);
return ret;
}
int __netif_set_mtu(struct net_device *dev, int new_mtu)
{
const struct net_device_ops *ops = dev->netdev_ops;
if (ops->ndo_change_mtu)
return ops->ndo_change_mtu(dev, new_mtu);
/* Pairs with all the lockless reads of dev->mtu in the stack */
WRITE_ONCE(dev->mtu, new_mtu);
return 0;
}
EXPORT_SYMBOL_NS_GPL(__netif_set_mtu, "NETDEV_INTERNAL");
int dev_validate_mtu(struct net_device *dev, int new_mtu,
struct netlink_ext_ack *extack)
{
/* MTU must be positive, and in range */
if (new_mtu < 0 || new_mtu < dev->min_mtu) {
NL_SET_ERR_MSG(extack, "mtu less than device minimum");
return -EINVAL;
}
if (dev->max_mtu > 0 && new_mtu > dev->max_mtu) {
NL_SET_ERR_MSG(extack, "mtu greater than device maximum");
return -EINVAL;
}
return 0;
}
/**
* netif_set_mtu_ext() - Change maximum transfer unit
* @dev: device
* @new_mtu: new transfer unit
* @extack: netlink extended ack
*
* Change the maximum transfer size of the network device.
*
* Return: 0 on success, -errno on failure.
*/
int netif_set_mtu_ext(struct net_device *dev, int new_mtu,
struct netlink_ext_ack *extack)
{
int err, orig_mtu;
netdev_ops_assert_locked(dev);
if (new_mtu == dev->mtu)
return 0;
err = dev_validate_mtu(dev, new_mtu, extack);
if (err)
return err;
if (!netif_device_present(dev))
return -ENODEV;
err = call_netdevice_notifiers(NETDEV_PRECHANGEMTU, dev);
err = notifier_to_errno(err);
if (err)
return err;
orig_mtu = dev->mtu;
err = __netif_set_mtu(dev, new_mtu);
if (!err) {
err = call_netdevice_notifiers_mtu(NETDEV_CHANGEMTU, dev,
orig_mtu);
err = notifier_to_errno(err);
if (err) {
/* setting mtu back and notifying everyone again,
* so that they have a chance to revert changes.
*/
__netif_set_mtu(dev, orig_mtu);
call_netdevice_notifiers_mtu(NETDEV_CHANGEMTU, dev,
new_mtu);
}
}
return err;
}
int netif_set_mtu(struct net_device *dev, int new_mtu)
{
struct netlink_ext_ack extack;
int err;
memset(&extack, 0, sizeof(extack));
err = netif_set_mtu_ext(dev, new_mtu, &extack);
if (err && extack._msg)
net_err_ratelimited("%s: %s\n", dev->name, extack._msg);
return err;
}
EXPORT_SYMBOL(netif_set_mtu);
int netif_change_tx_queue_len(struct net_device *dev, unsigned long new_len)
{
unsigned int orig_len = dev->tx_queue_len;
int res;
if (new_len != (unsigned int)new_len)
return -ERANGE;
if (new_len != orig_len) {
WRITE_ONCE(dev->tx_queue_len, new_len);
res = call_netdevice_notifiers(NETDEV_CHANGE_TX_QUEUE_LEN, dev);
res = notifier_to_errno(res);
if (res)
goto err_rollback;
res = dev_qdisc_change_tx_queue_len(dev);
if (res)
goto err_rollback;
}
return 0;
err_rollback:
netdev_err(dev, "refused to change device tx_queue_len\n");
WRITE_ONCE(dev->tx_queue_len, orig_len);
return res;
}
void netif_set_group(struct net_device *dev, int new_group)
{
dev->group = new_group;
}
/**
* netif_pre_changeaddr_notify() - Call NETDEV_PRE_CHANGEADDR.
* @dev: device
* @addr: new address
* @extack: netlink extended ack
*
* Return: 0 on success, -errno on failure.
*/
int netif_pre_changeaddr_notify(struct net_device *dev, const char *addr,
struct netlink_ext_ack *extack)
{
struct netdev_notifier_pre_changeaddr_info info = {
.info.dev = dev,
.info.extack = extack,
.dev_addr = addr,
};
int rc;
rc = call_netdevice_notifiers_info(NETDEV_PRE_CHANGEADDR, &info.info);
return notifier_to_errno(rc);
}
EXPORT_SYMBOL_NS_GPL(netif_pre_changeaddr_notify, "NETDEV_INTERNAL");
int netif_set_mac_address(struct net_device *dev, struct sockaddr_storage *ss,
struct netlink_ext_ack *extack)
{
const struct net_device_ops *ops = dev->netdev_ops;
int err;
if (!ops->ndo_set_mac_address)
return -EOPNOTSUPP;
if (ss->ss_family != dev->type)
return -EINVAL;
if (!netif_device_present(dev))
return -ENODEV;
err = netif_pre_changeaddr_notify(dev, ss->__data, extack);
if (err)
return err;
if (memcmp(dev->dev_addr, ss->__data, dev->addr_len)) {
err = ops->ndo_set_mac_address(dev, ss);
if (err)
return err;
}
dev->addr_assign_type = NET_ADDR_SET;
call_netdevice_notifiers(NETDEV_CHANGEADDR, dev);
add_device_randomness(dev->dev_addr, dev->addr_len);
return 0;
}
DECLARE_RWSEM(dev_addr_sem);
/* "sa" is a true struct sockaddr with limited "sa_data" member. */
int netif_get_mac_address(struct sockaddr *sa, struct net *net, char *dev_name)
{
size_t size = sizeof(sa->sa_data);
struct net_device *dev;
int ret = 0;
down_read(&dev_addr_sem);
rcu_read_lock();
dev = dev_get_by_name_rcu(net, dev_name);
if (!dev) {
ret = -ENODEV;
goto unlock;
}
if (!dev->addr_len)
memset(sa->sa_data, 0, size);
else
memcpy(sa->sa_data, dev->dev_addr,
min_t(size_t, size, dev->addr_len));
sa->sa_family = dev->type;
unlock:
rcu_read_unlock();
up_read(&dev_addr_sem);
return ret;
}
EXPORT_SYMBOL_NS_GPL(netif_get_mac_address, "NETDEV_INTERNAL");
int netif_change_carrier(struct net_device *dev, bool new_carrier)
{
const struct net_device_ops *ops = dev->netdev_ops;
if (!ops->ndo_change_carrier)
return -EOPNOTSUPP;
if (!netif_device_present(dev))
return -ENODEV;
return ops->ndo_change_carrier(dev, new_carrier);
}
/**
* dev_get_phys_port_id - Get device physical port ID
* @dev: device
* @ppid: port ID
*
* Get device physical port ID
*/
int dev_get_phys_port_id(struct net_device *dev,
struct netdev_phys_item_id *ppid)
{
const struct net_device_ops *ops = dev->netdev_ops;
if (!ops->ndo_get_phys_port_id)
return -EOPNOTSUPP;
return ops->ndo_get_phys_port_id(dev, ppid);
}
/**
* dev_get_phys_port_name - Get device physical port name
* @dev: device
* @name: port name
* @len: limit of bytes to copy to name
*
* Get device physical port name
*/
int dev_get_phys_port_name(struct net_device *dev,
char *name, size_t len)
{
const struct net_device_ops *ops = dev->netdev_ops;
int err;
if (ops->ndo_get_phys_port_name) {
err = ops->ndo_get_phys_port_name(dev, name, len);
if (err != -EOPNOTSUPP)
return err;
}
return devlink_compat_phys_port_name_get(dev, name, len);
}
/**
* netif_get_port_parent_id() - Get the device's port parent identifier
* @dev: network device
* @ppid: pointer to a storage for the port's parent identifier
* @recurse: allow/disallow recursion to lower devices
*
* Get the devices's port parent identifier.
*
* Return: 0 on success, -errno on failure.
*/
int netif_get_port_parent_id(struct net_device *dev,
struct netdev_phys_item_id *ppid, bool recurse)
{
const struct net_device_ops *ops = dev->netdev_ops;
struct netdev_phys_item_id first = { };
struct net_device *lower_dev;
struct list_head *iter;
int err;
if (ops->ndo_get_port_parent_id) {
err = ops->ndo_get_port_parent_id(dev, ppid);
if (err != -EOPNOTSUPP)
return err;
}
err = devlink_compat_switch_id_get(dev, ppid);
if (!recurse || err != -EOPNOTSUPP)
return err;
netdev_for_each_lower_dev(dev, lower_dev, iter) {
err = netif_get_port_parent_id(lower_dev, ppid, true);
if (err)
break;
if (!first.id_len)
first = *ppid;
else if (memcmp(&first, ppid, sizeof(*ppid)))
return -EOPNOTSUPP;
}
return err;
}
EXPORT_SYMBOL(netif_get_port_parent_id);
/**
* netdev_port_same_parent_id - Indicate if two network devices have
* the same port parent identifier
* @a: first network device
* @b: second network device
*/
bool netdev_port_same_parent_id(struct net_device *a, struct net_device *b)
{
struct netdev_phys_item_id a_id = { };
struct netdev_phys_item_id b_id = { };
if (netif_get_port_parent_id(a, &a_id, true) ||
netif_get_port_parent_id(b, &b_id, true))
return false;
return netdev_phys_item_id_same(&a_id, &b_id);
}
EXPORT_SYMBOL(netdev_port_same_parent_id);
int netif_change_proto_down(struct net_device *dev, bool proto_down)
{
if (!dev->change_proto_down)
return -EOPNOTSUPP;
if (!netif_device_present(dev))
return -ENODEV;
if (proto_down)
netif_carrier_off(dev);
else
netif_carrier_on(dev);
WRITE_ONCE(dev->proto_down, proto_down);
return 0;
}
/**
* netdev_change_proto_down_reason_locked - proto down reason
*
* @dev: device
* @mask: proto down mask
* @value: proto down value
*/
void netdev_change_proto_down_reason_locked(struct net_device *dev,
unsigned long mask, u32 value)
{
u32 proto_down_reason;
int b;
if (!mask) {
proto_down_reason = value;
} else {
proto_down_reason = dev->proto_down_reason;
for_each_set_bit(b, &mask, 32) {
if (value & (1 << b))
proto_down_reason |= BIT(b);
else
proto_down_reason &= ~BIT(b);
}
}
WRITE_ONCE(dev->proto_down_reason, proto_down_reason);
}
struct bpf_xdp_link {
struct bpf_link link;
struct net_device *dev; /* protected by rtnl_lock, no refcnt held */
int flags;
};
static enum bpf_xdp_mode dev_xdp_mode(struct net_device *dev, u32 flags)
{
if (flags & XDP_FLAGS_HW_MODE)
return XDP_MODE_HW;
if (flags & XDP_FLAGS_DRV_MODE)
return XDP_MODE_DRV;
if (flags & XDP_FLAGS_SKB_MODE)
return XDP_MODE_SKB;
return dev->netdev_ops->ndo_bpf ? XDP_MODE_DRV : XDP_MODE_SKB;
}
static bpf_op_t dev_xdp_bpf_op(struct net_device *dev, enum bpf_xdp_mode mode)
{
switch (mode) {
case XDP_MODE_SKB:
return generic_xdp_install;
case XDP_MODE_DRV:
case XDP_MODE_HW:
return dev->netdev_ops->ndo_bpf;
default:
return NULL;
}
}
static struct bpf_xdp_link *dev_xdp_link(struct net_device *dev,
enum bpf_xdp_mode mode)
{
return dev->xdp_state[mode].link;
}
static struct bpf_prog *dev_xdp_prog(struct net_device *dev,
enum bpf_xdp_mode mode)
{
struct bpf_xdp_link *link = dev_xdp_link(dev, mode);
if (link)
return link->link.prog;
return dev->xdp_state[mode].prog;
}
u8 dev_xdp_prog_count(struct net_device *dev)
{
u8 count = 0;
int i;
for (i = 0; i < __MAX_XDP_MODE; i++)
if (dev->xdp_state[i].prog || dev->xdp_state[i].link)
count++;
return count;
}
EXPORT_SYMBOL_GPL(dev_xdp_prog_count);
u8 dev_xdp_sb_prog_count(struct net_device *dev)
{
u8 count = 0;
int i;
for (i = 0; i < __MAX_XDP_MODE; i++)
if (dev->xdp_state[i].prog &&
!dev->xdp_state[i].prog->aux->xdp_has_frags)
count++;
return count;
}
int netif_xdp_propagate(struct net_device *dev, struct netdev_bpf *bpf)
{
if (!dev->netdev_ops->ndo_bpf)
return -EOPNOTSUPP;
if (dev->cfg->hds_config == ETHTOOL_TCP_DATA_SPLIT_ENABLED &&
bpf->command == XDP_SETUP_PROG &&
bpf->prog && !bpf->prog->aux->xdp_has_frags) {
NL_SET_ERR_MSG(bpf->extack,
"unable to propagate XDP to device using tcp-data-split");
return -EBUSY;
}
if (dev_get_min_mp_channel_count(dev)) {
NL_SET_ERR_MSG(bpf->extack, "unable to propagate XDP to device using memory provider");
return -EBUSY;
}
return dev->netdev_ops->ndo_bpf(dev, bpf);
}
EXPORT_SYMBOL_GPL(netif_xdp_propagate);
u32 dev_xdp_prog_id(struct net_device *dev, enum bpf_xdp_mode mode)
{
struct bpf_prog *prog = dev_xdp_prog(dev, mode);
return prog ? prog->aux->id : 0;
}
static void dev_xdp_set_link(struct net_device *dev, enum bpf_xdp_mode mode,
struct bpf_xdp_link *link)
{
dev->xdp_state[mode].link = link;
dev->xdp_state[mode].prog = NULL;
}
static void dev_xdp_set_prog(struct net_device *dev, enum bpf_xdp_mode mode,
struct bpf_prog *prog)
{
dev->xdp_state[mode].link = NULL;
dev->xdp_state[mode].prog = prog;
}
static int dev_xdp_install(struct net_device *dev, enum bpf_xdp_mode mode,
bpf_op_t bpf_op, struct netlink_ext_ack *extack,
u32 flags, struct bpf_prog *prog)
{
struct netdev_bpf xdp;
int err;
netdev_ops_assert_locked(dev);
if (dev->cfg->hds_config == ETHTOOL_TCP_DATA_SPLIT_ENABLED &&
prog && !prog->aux->xdp_has_frags) {
NL_SET_ERR_MSG(extack, "unable to install XDP to device using tcp-data-split");
return -EBUSY;
}
if (dev_get_min_mp_channel_count(dev)) {
NL_SET_ERR_MSG(extack, "unable to install XDP to device using memory provider");
return -EBUSY;
}
memset(&xdp, 0, sizeof(xdp));
xdp.command = mode == XDP_MODE_HW ? XDP_SETUP_PROG_HW : XDP_SETUP_PROG;
xdp.extack = extack;
xdp.flags = flags;
xdp.prog = prog;
/* Drivers assume refcnt is already incremented (i.e, prog pointer is
* "moved" into driver), so they don't increment it on their own, but
* they do decrement refcnt when program is detached or replaced.
* Given net_device also owns link/prog, we need to bump refcnt here
* to prevent drivers from underflowing it.
*/
if (prog)
bpf_prog_inc(prog);
err = bpf_op(dev, &xdp);
if (err) {
if (prog)
bpf_prog_put(prog);
return err;
}
if (mode != XDP_MODE_HW)
bpf_prog_change_xdp(dev_xdp_prog(dev, mode), prog);
return 0;
}
static void dev_xdp_uninstall(struct net_device *dev)
{
struct bpf_xdp_link *link;
struct bpf_prog *prog;
enum bpf_xdp_mode mode;
bpf_op_t bpf_op;
ASSERT_RTNL();
for (mode = XDP_MODE_SKB; mode < __MAX_XDP_MODE; mode++) {
prog = dev_xdp_prog(dev, mode);
if (!prog)
continue;
bpf_op = dev_xdp_bpf_op(dev, mode);
if (!bpf_op)
continue;
WARN_ON(dev_xdp_install(dev, mode, bpf_op, NULL, 0, NULL));
/* auto-detach link from net device */
link = dev_xdp_link(dev, mode);
if (link)
link->dev = NULL;
else
bpf_prog_put(prog);
dev_xdp_set_link(dev, mode, NULL);
}
}
static int dev_xdp_attach(struct net_device *dev, struct netlink_ext_ack *extack,
struct bpf_xdp_link *link, struct bpf_prog *new_prog,
struct bpf_prog *old_prog, u32 flags)
{
unsigned int num_modes = hweight32(flags & XDP_FLAGS_MODES);
struct bpf_prog *cur_prog;
struct net_device *upper;
struct list_head *iter;
enum bpf_xdp_mode mode;
bpf_op_t bpf_op;
int err;
ASSERT_RTNL();
/* either link or prog attachment, never both */
if (link && (new_prog || old_prog))
return -EINVAL;
/* link supports only XDP mode flags */
if (link && (flags & ~XDP_FLAGS_MODES)) {
NL_SET_ERR_MSG(extack, "Invalid XDP flags for BPF link attachment");
return -EINVAL;
}
/* just one XDP mode bit should be set, zero defaults to drv/skb mode */
if (num_modes > 1) {
NL_SET_ERR_MSG(extack, "Only one XDP mode flag can be set");
return -EINVAL;
}
/* avoid ambiguity if offload + drv/skb mode progs are both loaded */
if (!num_modes && dev_xdp_prog_count(dev) > 1) {
NL_SET_ERR_MSG(extack,
"More than one program loaded, unset mode is ambiguous");
return -EINVAL;
}
/* old_prog != NULL implies XDP_FLAGS_REPLACE is set */
if (old_prog && !(flags & XDP_FLAGS_REPLACE)) {
NL_SET_ERR_MSG(extack, "XDP_FLAGS_REPLACE is not specified");
return -EINVAL;
}
mode = dev_xdp_mode(dev, flags);
/* can't replace attached link */
if (dev_xdp_link(dev, mode)) {
NL_SET_ERR_MSG(extack, "Can't replace active BPF XDP link");
return -EBUSY;
}
/* don't allow if an upper device already has a program */
netdev_for_each_upper_dev_rcu(dev, upper, iter) {
if (dev_xdp_prog_count(upper) > 0) {
NL_SET_ERR_MSG(extack, "Cannot attach when an upper device already has a program");
return -EEXIST;
}
}
cur_prog = dev_xdp_prog(dev, mode);
/* can't replace attached prog with link */
if (link && cur_prog) {
NL_SET_ERR_MSG(extack, "Can't replace active XDP program with BPF link");
return -EBUSY;
}
if ((flags & XDP_FLAGS_REPLACE) && cur_prog != old_prog) {
NL_SET_ERR_MSG(extack, "Active program does not match expected");
return -EEXIST;
}
/* put effective new program into new_prog */
if (link)
new_prog = link->link.prog;
if (new_prog) {
bool offload = mode == XDP_MODE_HW;
enum bpf_xdp_mode other_mode = mode == XDP_MODE_SKB
? XDP_MODE_DRV : XDP_MODE_SKB;
if ((flags & XDP_FLAGS_UPDATE_IF_NOEXIST) && cur_prog) {
NL_SET_ERR_MSG(extack, "XDP program already attached");
return -EBUSY;
}
if (!offload && dev_xdp_prog(dev, other_mode)) {
NL_SET_ERR_MSG(extack, "Native and generic XDP can't be active at the same time");
return -EEXIST;
}
if (!offload && bpf_prog_is_offloaded(new_prog->aux)) {
NL_SET_ERR_MSG(extack, "Using offloaded program without HW_MODE flag is not supported");
return -EINVAL;
}
if (bpf_prog_is_dev_bound(new_prog->aux) && !bpf_offload_dev_match(new_prog, dev)) {
NL_SET_ERR_MSG(extack, "Program bound to different device");
return -EINVAL;
}
if (bpf_prog_is_dev_bound(new_prog->aux) && mode == XDP_MODE_SKB) {
NL_SET_ERR_MSG(extack, "Can't attach device-bound programs in generic mode");
return -EINVAL;
}
if (new_prog->expected_attach_type == BPF_XDP_DEVMAP) {
NL_SET_ERR_MSG(extack, "BPF_XDP_DEVMAP programs can not be attached to a device");
return -EINVAL;
}
if (new_prog->expected_attach_type == BPF_XDP_CPUMAP) {
NL_SET_ERR_MSG(extack, "BPF_XDP_CPUMAP programs can not be attached to a device");
return -EINVAL;
}
}
/* don't call drivers if the effective program didn't change */
if (new_prog != cur_prog) {
bpf_op = dev_xdp_bpf_op(dev, mode);
if (!bpf_op) {
NL_SET_ERR_MSG(extack, "Underlying driver does not support XDP in native mode");
return -EOPNOTSUPP;
}
err = dev_xdp_install(dev, mode, bpf_op, extack, flags, new_prog);
if (err)
return err;
}
if (link)
dev_xdp_set_link(dev, mode, link);
else
dev_xdp_set_prog(dev, mode, new_prog);
if (cur_prog)
bpf_prog_put(cur_prog);
return 0;
}
static int dev_xdp_attach_link(struct net_device *dev,
struct netlink_ext_ack *extack,
struct bpf_xdp_link *link)
{
return dev_xdp_attach(dev, extack, link, NULL, NULL, link->flags);
}
static int dev_xdp_detach_link(struct net_device *dev,
struct netlink_ext_ack *extack,
struct bpf_xdp_link *link)
{
enum bpf_xdp_mode mode;
bpf_op_t bpf_op;
ASSERT_RTNL();
mode = dev_xdp_mode(dev, link->flags);
if (dev_xdp_link(dev, mode) != link)
return -EINVAL;
bpf_op = dev_xdp_bpf_op(dev, mode);
WARN_ON(dev_xdp_install(dev, mode, bpf_op, NULL, 0, NULL));
dev_xdp_set_link(dev, mode, NULL);
return 0;
}
static void bpf_xdp_link_release(struct bpf_link *link)
{
struct bpf_xdp_link *xdp_link = container_of(link, struct bpf_xdp_link, link);
rtnl_lock();
/* if racing with net_device's tear down, xdp_link->dev might be
* already NULL, in which case link was already auto-detached
*/
if (xdp_link->dev) {
netdev_lock_ops(xdp_link->dev);
WARN_ON(dev_xdp_detach_link(xdp_link->dev, NULL, xdp_link));
netdev_unlock_ops(xdp_link->dev);
xdp_link->dev = NULL;
}
rtnl_unlock();
}
static int bpf_xdp_link_detach(struct bpf_link *link)
{
bpf_xdp_link_release(link);
return 0;
}
static void bpf_xdp_link_dealloc(struct bpf_link *link)
{
struct bpf_xdp_link *xdp_link = container_of(link, struct bpf_xdp_link, link);
kfree(xdp_link);
}
static void bpf_xdp_link_show_fdinfo(const struct bpf_link *link,
struct seq_file *seq)
{
struct bpf_xdp_link *xdp_link = container_of(link, struct bpf_xdp_link, link);
u32 ifindex = 0;
rtnl_lock();
if (xdp_link->dev)
ifindex = xdp_link->dev->ifindex;
rtnl_unlock();
seq_printf(seq, "ifindex:\t%u\n", ifindex);
}
static int bpf_xdp_link_fill_link_info(const struct bpf_link *link,
struct bpf_link_info *info)
{
struct bpf_xdp_link *xdp_link = container_of(link, struct bpf_xdp_link, link);
u32 ifindex = 0;
rtnl_lock();
if (xdp_link->dev)
ifindex = xdp_link->dev->ifindex;
rtnl_unlock();
info->xdp.ifindex = ifindex;
return 0;
}
static int bpf_xdp_link_update(struct bpf_link *link, struct bpf_prog *new_prog,
struct bpf_prog *old_prog)
{
struct bpf_xdp_link *xdp_link = container_of(link, struct bpf_xdp_link, link);
enum bpf_xdp_mode mode;
bpf_op_t bpf_op;
int err = 0;
rtnl_lock();
/* link might have been auto-released already, so fail */
if (!xdp_link->dev) {
err = -ENOLINK;
goto out_unlock;
}
if (old_prog && link->prog != old_prog) {
err = -EPERM;
goto out_unlock;
}
old_prog = link->prog;
if (old_prog->type != new_prog->type ||
old_prog->expected_attach_type != new_prog->expected_attach_type) {
err = -EINVAL;
goto out_unlock;
}
if (old_prog == new_prog) {
/* no-op, don't disturb drivers */
bpf_prog_put(new_prog);
goto out_unlock;
}
netdev_lock_ops(xdp_link->dev);
mode = dev_xdp_mode(xdp_link->dev, xdp_link->flags);
bpf_op = dev_xdp_bpf_op(xdp_link->dev, mode);
err = dev_xdp_install(xdp_link->dev, mode, bpf_op, NULL,
xdp_link->flags, new_prog);
netdev_unlock_ops(xdp_link->dev);
if (err)
goto out_unlock;
old_prog = xchg(&link->prog, new_prog);
bpf_prog_put(old_prog);
out_unlock:
rtnl_unlock();
return err;
}
static const struct bpf_link_ops bpf_xdp_link_lops = {
.release = bpf_xdp_link_release,
.dealloc = bpf_xdp_link_dealloc,
.detach = bpf_xdp_link_detach,
.show_fdinfo = bpf_xdp_link_show_fdinfo,
.fill_link_info = bpf_xdp_link_fill_link_info,
.update_prog = bpf_xdp_link_update,
};
int bpf_xdp_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
{
struct net *net = current->nsproxy->net_ns;
struct bpf_link_primer link_primer;
struct netlink_ext_ack extack = {};
struct bpf_xdp_link *link;
struct net_device *dev;
int err, fd;
rtnl_lock();
dev = dev_get_by_index(net, attr->link_create.target_ifindex);
if (!dev) {
rtnl_unlock();
return -EINVAL;
}
link = kzalloc(sizeof(*link), GFP_USER);
if (!link) {
err = -ENOMEM;
goto unlock;
}
bpf_link_init(&link->link, BPF_LINK_TYPE_XDP, &bpf_xdp_link_lops, prog,
attr->link_create.attach_type);
link->dev = dev;
link->flags = attr->link_create.flags;
err = bpf_link_prime(&link->link, &link_primer);
if (err) {
kfree(link);
goto unlock;
}
netdev_lock_ops(dev);
err = dev_xdp_attach_link(dev, &extack, link);
netdev_unlock_ops(dev);
rtnl_unlock();
if (err) {
link->dev = NULL;
bpf_link_cleanup(&link_primer);
trace_bpf_xdp_link_attach_failed(extack._msg);
goto out_put_dev;
}
fd = bpf_link_settle(&link_primer);
/* link itself doesn't hold dev's refcnt to not complicate shutdown */
dev_put(dev);
return fd;
unlock:
rtnl_unlock();
out_put_dev:
dev_put(dev);
return err;
}
/**
* dev_change_xdp_fd - set or clear a bpf program for a device rx path
* @dev: device
* @extack: netlink extended ack
* @fd: new program fd or negative value to clear
* @expected_fd: old program fd that userspace expects to replace or clear
* @flags: xdp-related flags
*
* Set or clear a bpf program for a device
*/
int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack,
int fd, int expected_fd, u32 flags)
{
enum bpf_xdp_mode mode = dev_xdp_mode(dev, flags);
struct bpf_prog *new_prog = NULL, *old_prog = NULL;
int err;
ASSERT_RTNL();
if (fd >= 0) {
new_prog = bpf_prog_get_type_dev(fd, BPF_PROG_TYPE_XDP,
mode != XDP_MODE_SKB);
if (IS_ERR(new_prog))
return PTR_ERR(new_prog);
}
if (expected_fd >= 0) {
old_prog = bpf_prog_get_type_dev(expected_fd, BPF_PROG_TYPE_XDP,
mode != XDP_MODE_SKB);
if (IS_ERR(old_prog)) {
err = PTR_ERR(old_prog);
old_prog = NULL;
goto err_out;
}
}
err = dev_xdp_attach(dev, extack, NULL, new_prog, old_prog, flags);
err_out:
if (err && new_prog)
bpf_prog_put(new_prog);
if (old_prog)
bpf_prog_put(old_prog);
return err;
}
u32 dev_get_min_mp_channel_count(const struct net_device *dev)
{
int i;
netdev_ops_assert_locked(dev);
for (i = dev->real_num_rx_queues - 1; i >= 0; i--)
if (dev->_rx[i].mp_params.mp_priv)
/* The channel count is the idx plus 1. */
return i + 1;
return 0;
}
/**
* dev_index_reserve() - allocate an ifindex in a namespace
* @net: the applicable net namespace
* @ifindex: requested ifindex, pass %0 to get one allocated
*
* Allocate a ifindex for a new device. Caller must either use the ifindex
* to store the device (via list_netdevice()) or call dev_index_release()
* to give the index up.
*
* Return: a suitable unique value for a new device interface number or -errno.
*/
static int dev_index_reserve(struct net *net, u32 ifindex)
{
int err;
if (ifindex > INT_MAX) {
DEBUG_NET_WARN_ON_ONCE(1);
return -EINVAL;
}
if (!ifindex)
err = xa_alloc_cyclic(&net->dev_by_index, &ifindex, NULL,
xa_limit_31b, &net->ifindex, GFP_KERNEL);
else
err = xa_insert(&net->dev_by_index, ifindex, NULL, GFP_KERNEL);
if (err < 0)
return err;
return ifindex;
}
static void dev_index_release(struct net *net, int ifindex)
{
/* Expect only unused indexes, unlist_netdevice() removes the used */
WARN_ON(xa_erase(&net->dev_by_index, ifindex));
}
static bool from_cleanup_net(void)
{
#ifdef CONFIG_NET_NS
return current == READ_ONCE(cleanup_net_task);
#else
return false;
#endif
}
/* Delayed registration/unregisteration */
LIST_HEAD(net_todo_list);
DECLARE_WAIT_QUEUE_HEAD(netdev_unregistering_wq);
atomic_t dev_unreg_count = ATOMIC_INIT(0);
static void net_set_todo(struct net_device *dev)
{
list_add_tail(&dev->todo_list, &net_todo_list);
}
static netdev_features_t netdev_sync_upper_features(struct net_device *lower,
struct net_device *upper, netdev_features_t features)
{
netdev_features_t upper_disables = NETIF_F_UPPER_DISABLES;
netdev_features_t feature;
int feature_bit;
for_each_netdev_feature(upper_disables, feature_bit) {
feature = __NETIF_F_BIT(feature_bit);
if (!(upper->wanted_features & feature)
&& (features & feature)) {
netdev_dbg(lower, "Dropping feature %pNF, upper dev %s has it off.\n",
&feature, upper->name);
features &= ~feature;
}
}
return features;
}
static void netdev_sync_lower_features(struct net_device *upper,
struct net_device *lower, netdev_features_t features)
{
netdev_features_t upper_disables = NETIF_F_UPPER_DISABLES;
netdev_features_t feature;
int feature_bit;
for_each_netdev_feature(upper_disables, feature_bit) {
feature = __NETIF_F_BIT(feature_bit);
if (!(features & feature) && (lower->features & feature)) {
netdev_dbg(upper, "Disabling feature %pNF on lower dev %s.\n",
&feature, lower->name);
netdev_lock_ops(lower);
lower->wanted_features &= ~feature;
__netdev_update_features(lower);
if (unlikely(lower->features & feature))
netdev_WARN(upper, "failed to disable %pNF on %s!\n",
&feature, lower->name);
else
netdev_features_change(lower);
netdev_unlock_ops(lower);
}
}
}
static bool netdev_has_ip_or_hw_csum(netdev_features_t features)
{
netdev_features_t ip_csum_mask = NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM;
bool ip_csum = (features & ip_csum_mask) == ip_csum_mask;
bool hw_csum = features & NETIF_F_HW_CSUM;
return ip_csum || hw_csum;
}
static netdev_features_t netdev_fix_features(struct net_device *dev,
netdev_features_t features)
{
/* Fix illegal checksum combinations */
if ((features & NETIF_F_HW_CSUM) &&
(features & (NETIF_F_IP_CSUM|NETIF_F_IPV6_CSUM))) {
netdev_warn(dev, "mixed HW and IP checksum settings.\n");
features &= ~(NETIF_F_IP_CSUM|NETIF_F_IPV6_CSUM);
}
/* TSO requires that SG is present as well. */
if ((features & NETIF_F_ALL_TSO) && !(features & NETIF_F_SG)) {
netdev_dbg(dev, "Dropping TSO features since no SG feature.\n");
features &= ~NETIF_F_ALL_TSO;
}
if ((features & NETIF_F_TSO) && !(features & NETIF_F_HW_CSUM) &&
!(features & NETIF_F_IP_CSUM)) {
netdev_dbg(dev, "Dropping TSO features since no CSUM feature.\n");
features &= ~NETIF_F_TSO;
features &= ~NETIF_F_TSO_ECN;
}
if ((features & NETIF_F_TSO6) && !(features & NETIF_F_HW_CSUM) &&
!(features & NETIF_F_IPV6_CSUM)) {
netdev_dbg(dev, "Dropping TSO6 features since no CSUM feature.\n");
features &= ~NETIF_F_TSO6;
}
/* TSO with IPv4 ID mangling requires IPv4 TSO be enabled */
if ((features & NETIF_F_TSO_MANGLEID) && !(features & NETIF_F_TSO))
features &= ~NETIF_F_TSO_MANGLEID;
/* TSO ECN requires that TSO is present as well. */
if ((features & NETIF_F_ALL_TSO) == NETIF_F_TSO_ECN)
features &= ~NETIF_F_TSO_ECN;
/* Software GSO depends on SG. */
if ((features & NETIF_F_GSO) && !(features & NETIF_F_SG)) {
netdev_dbg(dev, "Dropping NETIF_F_GSO since no SG feature.\n");
features &= ~NETIF_F_GSO;
}
/* GSO partial features require GSO partial be set */
if ((features & dev->gso_partial_features) &&
!(features & NETIF_F_GSO_PARTIAL)) {
netdev_dbg(dev,
"Dropping partially supported GSO features since no GSO partial.\n");
features &= ~dev->gso_partial_features;
}
if (!(features & NETIF_F_RXCSUM)) {
/* NETIF_F_GRO_HW implies doing RXCSUM since every packet
* successfully merged by hardware must also have the
* checksum verified by hardware. If the user does not
* want to enable RXCSUM, logically, we should disable GRO_HW.
*/
if (features & NETIF_F_GRO_HW) {
netdev_dbg(dev, "Dropping NETIF_F_GRO_HW since no RXCSUM feature.\n");
features &= ~NETIF_F_GRO_HW;
}
}
/* LRO/HW-GRO features cannot be combined with RX-FCS */
if (features & NETIF_F_RXFCS) {
if (features & NETIF_F_LRO) {
netdev_dbg(dev, "Dropping LRO feature since RX-FCS is requested.\n");
features &= ~NETIF_F_LRO;
}
if (features & NETIF_F_GRO_HW) {
netdev_dbg(dev, "Dropping HW-GRO feature since RX-FCS is requested.\n");
features &= ~NETIF_F_GRO_HW;
}
}
if ((features & NETIF_F_GRO_HW) && (features & NETIF_F_LRO)) {
netdev_dbg(dev, "Dropping LRO feature since HW-GRO is requested.\n");
features &= ~NETIF_F_LRO;
}
if ((features & NETIF_F_HW_TLS_TX) && !netdev_has_ip_or_hw_csum(features)) {
netdev_dbg(dev, "Dropping TLS TX HW offload feature since no CSUM feature.\n");
features &= ~NETIF_F_HW_TLS_TX;
}
if ((features & NETIF_F_HW_TLS_RX) && !(features & NETIF_F_RXCSUM)) {
netdev_dbg(dev, "Dropping TLS RX HW offload feature since no RXCSUM feature.\n");
features &= ~NETIF_F_HW_TLS_RX;
}
if ((features & NETIF_F_GSO_UDP_L4) && !netdev_has_ip_or_hw_csum(features)) {
netdev_dbg(dev, "Dropping USO feature since no CSUM feature.\n");
features &= ~NETIF_F_GSO_UDP_L4;
}
return features;
}
int __netdev_update_features(struct net_device *dev)
{
struct net_device *upper, *lower;
netdev_features_t features;
struct list_head *iter;
int err = -1;
ASSERT_RTNL();
netdev_ops_assert_locked(dev);
features = netdev_get_wanted_features(dev);
if (dev->netdev_ops->ndo_fix_features)
features = dev->netdev_ops->ndo_fix_features(dev, features);
/* driver might be less strict about feature dependencies */
features = netdev_fix_features(dev, features);
/* some features can't be enabled if they're off on an upper device */
netdev_for_each_upper_dev_rcu(dev, upper, iter)
features = netdev_sync_upper_features(dev, upper, features);
if (dev->features == features)
goto sync_lower;
netdev_dbg(dev, "Features changed: %pNF -> %pNF\n",
&dev->features, &features);
if (dev->netdev_ops->ndo_set_features)
err = dev->netdev_ops->ndo_set_features(dev, features);
else
err = 0;
if (unlikely(err < 0)) {
netdev_err(dev,
"set_features() failed (%d); wanted %pNF, left %pNF\n",
err, &features, &dev->features);
/* return non-0 since some features might have changed and
* it's better to fire a spurious notification than miss it
*/
return -1;
}
sync_lower:
/* some features must be disabled on lower devices when disabled
* on an upper device (think: bonding master or bridge)
*/
netdev_for_each_lower_dev(dev, lower, iter)
netdev_sync_lower_features(dev, lower, features);
if (!err) {
netdev_features_t diff = features ^ dev->features;
if (diff & NETIF_F_RX_UDP_TUNNEL_PORT) {
/* udp_tunnel_{get,drop}_rx_info both need
* NETIF_F_RX_UDP_TUNNEL_PORT enabled on the
* device, or they won't do anything.
* Thus we need to update dev->features
* *before* calling udp_tunnel_get_rx_info,
* but *after* calling udp_tunnel_drop_rx_info.
*/
udp_tunnel_nic_lock(dev);
if (features & NETIF_F_RX_UDP_TUNNEL_PORT) {
dev->features = features;
udp_tunnel_get_rx_info(dev);
} else {
udp_tunnel_drop_rx_info(dev);
}
udp_tunnel_nic_unlock(dev);
}
if (diff & NETIF_F_HW_VLAN_CTAG_FILTER) {
if (features & NETIF_F_HW_VLAN_CTAG_FILTER) {
dev->features = features;
err |= vlan_get_rx_ctag_filter_info(dev);
} else {
vlan_drop_rx_ctag_filter_info(dev);
}
}
if (diff & NETIF_F_HW_VLAN_STAG_FILTER) {
if (features & NETIF_F_HW_VLAN_STAG_FILTER) {
dev->features = features;
err |= vlan_get_rx_stag_filter_info(dev);
} else {
vlan_drop_rx_stag_filter_info(dev);
}
}
dev->features = features;
}
return err < 0 ? 0 : 1;
}
/**
* netdev_update_features - recalculate device features
* @dev: the device to check
*
* Recalculate dev->features set and send notifications if it
* has changed. Should be called after driver or hardware dependent
* conditions might have changed that influence the features.
*/
void netdev_update_features(struct net_device *dev)
{
if (__netdev_update_features(dev))
netdev_features_change(dev);
}
EXPORT_SYMBOL(netdev_update_features);
/**
* netdev_change_features - recalculate device features
* @dev: the device to check
*
* Recalculate dev->features set and send notifications even
* if they have not changed. Should be called instead of
* netdev_update_features() if also dev->vlan_features might
* have changed to allow the changes to be propagated to stacked
* VLAN devices.
*/
void netdev_change_features(struct net_device *dev)
{
__netdev_update_features(dev);
netdev_features_change(dev);
}
EXPORT_SYMBOL(netdev_change_features);
/**
* netif_stacked_transfer_operstate - transfer operstate
* @rootdev: the root or lower level device to transfer state from
* @dev: the device to transfer operstate to
*
* Transfer operational state from root to device. This is normally
* called when a stacking relationship exists between the root
* device and the device(a leaf device).
*/
void netif_stacked_transfer_operstate(const struct net_device *rootdev,
struct net_device *dev)
{
if (rootdev->operstate == IF_OPER_DORMANT)
netif_dormant_on(dev);
else
netif_dormant_off(dev);
if (rootdev->operstate == IF_OPER_TESTING)
netif_testing_on(dev);
else
netif_testing_off(dev);
if (netif_carrier_ok(rootdev))
netif_carrier_on(dev);
else
netif_carrier_off(dev);
}
EXPORT_SYMBOL(netif_stacked_transfer_operstate);
static int netif_alloc_rx_queues(struct net_device *dev)
{
unsigned int i, count = dev->num_rx_queues;
struct netdev_rx_queue *rx;
size_t sz = count * sizeof(*rx);
int err = 0;
BUG_ON(count < 1);
rx = kvzalloc(sz, GFP_KERNEL_ACCOUNT | __GFP_RETRY_MAYFAIL);
if (!rx)
return -ENOMEM;
dev->_rx = rx;
for (i = 0; i < count; i++) {
rx[i].dev = dev;
/* XDP RX-queue setup */
err = xdp_rxq_info_reg(&rx[i].xdp_rxq, dev, i, 0);
if (err < 0)
goto err_rxq_info;
}
return 0;
err_rxq_info:
/* Rollback successful reg's and free other resources */
while (i--)
xdp_rxq_info_unreg(&rx[i].xdp_rxq);
kvfree(dev->_rx);
dev->_rx = NULL;
return err;
}
static void netif_free_rx_queues(struct net_device *dev)
{
unsigned int i, count = dev->num_rx_queues;
/* netif_alloc_rx_queues alloc failed, resources have been unreg'ed */
if (!dev->_rx)
return;
for (i = 0; i < count; i++)
xdp_rxq_info_unreg(&dev->_rx[i].xdp_rxq);
kvfree(dev->_rx);
}
static void netdev_init_one_queue(struct net_device *dev,
struct netdev_queue *queue, void *_unused)
{
/* Initialize queue lock */
spin_lock_init(&queue->_xmit_lock);
netdev_set_xmit_lockdep_class(&queue->_xmit_lock, dev->type);
queue->xmit_lock_owner = -1;
netdev_queue_numa_node_write(queue, NUMA_NO_NODE);
queue->dev = dev;
#ifdef CONFIG_BQL
dql_init(&queue->dql, HZ);
#endif
}
static void netif_free_tx_queues(struct net_device *dev)
{
kvfree(dev->_tx);
}
static int netif_alloc_netdev_queues(struct net_device *dev)
{
unsigned int count = dev->num_tx_queues;
struct netdev_queue *tx;
size_t sz = count * sizeof(*tx);
if (count < 1 || count > 0xffff)
return -EINVAL;
tx = kvzalloc(sz, GFP_KERNEL_ACCOUNT | __GFP_RETRY_MAYFAIL);
if (!tx)
return -ENOMEM;
dev->_tx = tx;
netdev_for_each_tx_queue(dev, netdev_init_one_queue, NULL);
spin_lock_init(&dev->tx_global_lock);
return 0;
}
void netif_tx_stop_all_queues(struct net_device *dev)
{
unsigned int i;
for (i = 0; i < dev->num_tx_queues; i++) {
struct netdev_queue *txq = netdev_get_tx_queue(dev, i);
netif_tx_stop_queue(txq);
}
}
EXPORT_SYMBOL(netif_tx_stop_all_queues);
static int netdev_do_alloc_pcpu_stats(struct net_device *dev)
{
void __percpu *v;
/* Drivers implementing ndo_get_peer_dev must support tstat
* accounting, so that skb_do_redirect() can bump the dev's
* RX stats upon network namespace switch.
*/
if (dev->netdev_ops->ndo_get_peer_dev &&
dev->pcpu_stat_type != NETDEV_PCPU_STAT_TSTATS)
return -EOPNOTSUPP;
switch (dev->pcpu_stat_type) {
case NETDEV_PCPU_STAT_NONE:
return 0;
case NETDEV_PCPU_STAT_LSTATS:
v = dev->lstats = netdev_alloc_pcpu_stats(struct pcpu_lstats);
break;
case NETDEV_PCPU_STAT_TSTATS:
v = dev->tstats = netdev_alloc_pcpu_stats(struct pcpu_sw_netstats);
break;
case NETDEV_PCPU_STAT_DSTATS:
v = dev->dstats = netdev_alloc_pcpu_stats(struct pcpu_dstats);
break;
default:
return -EINVAL;
}
return v ? 0 : -ENOMEM;
}
static void netdev_do_free_pcpu_stats(struct net_device *dev)
{
switch (dev->pcpu_stat_type) {
case NETDEV_PCPU_STAT_NONE:
return;
case NETDEV_PCPU_STAT_LSTATS:
free_percpu(dev->lstats);
break;
case NETDEV_PCPU_STAT_TSTATS:
free_percpu(dev->tstats);
break;
case NETDEV_PCPU_STAT_DSTATS:
free_percpu(dev->dstats);
break;
}
}
static void netdev_free_phy_link_topology(struct net_device *dev)
{
struct phy_link_topology *topo = dev->link_topo;
if (IS_ENABLED(CONFIG_PHYLIB) && topo) {
xa_destroy(&topo->phys);
kfree(topo);
dev->link_topo = NULL;
}
}
/**
* register_netdevice() - register a network device
* @dev: device to register
*
* Take a prepared network device structure and make it externally accessible.
* A %NETDEV_REGISTER message is sent to the netdev notifier chain.
* Callers must hold the rtnl lock - you may want register_netdev()
* instead of this.
*/
int register_netdevice(struct net_device *dev)
{
int ret;
struct net *net = dev_net(dev);
BUILD_BUG_ON(sizeof(netdev_features_t) * BITS_PER_BYTE <
NETDEV_FEATURE_COUNT);
BUG_ON(dev_boot_phase);
ASSERT_RTNL();
might_sleep();
/* When net_device's are persistent, this will be fatal. */
BUG_ON(dev->reg_state != NETREG_UNINITIALIZED);
BUG_ON(!net);
ret = ethtool_check_ops(dev->ethtool_ops);
if (ret)
return ret;
/* rss ctx ID 0 is reserved for the default context, start from 1 */
xa_init_flags(&dev->ethtool->rss_ctx, XA_FLAGS_ALLOC1);
mutex_init(&dev->ethtool->rss_lock);
spin_lock_init(&dev->addr_list_lock);
netdev_set_addr_lockdep_class(dev);
ret = dev_get_valid_name(net, dev, dev->name);
if (ret < 0)
goto out;
ret = -ENOMEM;
dev->name_node = netdev_name_node_head_alloc(dev);
if (!dev->name_node)
goto out;
/* Init, if this function is available */
if (dev->netdev_ops->ndo_init) {
ret = dev->netdev_ops->ndo_init(dev);
if (ret) {
if (ret > 0)
ret = -EIO;
goto err_free_name;
}
}
if (((dev->hw_features | dev->features) &
NETIF_F_HW_VLAN_CTAG_FILTER) &&
(!dev->netdev_ops->ndo_vlan_rx_add_vid ||
!dev->netdev_ops->ndo_vlan_rx_kill_vid)) {
netdev_WARN(dev, "Buggy VLAN acceleration in driver!\n");
ret = -EINVAL;
goto err_uninit;
}
ret = netdev_do_alloc_pcpu_stats(dev);
if (ret)
goto err_uninit;
ret = dev_index_reserve(net, dev->ifindex);
if (ret < 0)
goto err_free_pcpu;
dev->ifindex = ret;
/* Transfer changeable features to wanted_features and enable
* software offloads (GSO and GRO).
*/
dev->hw_features |= (NETIF_F_SOFT_FEATURES | NETIF_F_SOFT_FEATURES_OFF);
dev->features |= NETIF_F_SOFT_FEATURES;
if (dev->udp_tunnel_nic_info) {
dev->features |= NETIF_F_RX_UDP_TUNNEL_PORT;
dev->hw_features |= NETIF_F_RX_UDP_TUNNEL_PORT;
}
dev->wanted_features = dev->features & dev->hw_features;
if (!(dev->flags & IFF_LOOPBACK))
dev->hw_features |= NETIF_F_NOCACHE_COPY;
/* If IPv4 TCP segmentation offload is supported we should also
* allow the device to enable segmenting the frame with the option
* of ignoring a static IP ID value. This doesn't enable the
* feature itself but allows the user to enable it later.
*/
if (dev->hw_features & NETIF_F_TSO)
dev->hw_features |= NETIF_F_TSO_MANGLEID;
if (dev->vlan_features & NETIF_F_TSO)
dev->vlan_features |= NETIF_F_TSO_MANGLEID;
if (dev->mpls_features & NETIF_F_TSO)
dev->mpls_features |= NETIF_F_TSO_MANGLEID;
if (dev->hw_enc_features & NETIF_F_TSO)
dev->hw_enc_features |= NETIF_F_TSO_MANGLEID;
/* Make NETIF_F_HIGHDMA inheritable to VLAN devices.
*/
dev->vlan_features |= NETIF_F_HIGHDMA;
/* Make NETIF_F_SG inheritable to tunnel devices.
*/
dev->hw_enc_features |= NETIF_F_SG | NETIF_F_GSO_PARTIAL;
/* Make NETIF_F_SG inheritable to MPLS.
*/
dev->mpls_features |= NETIF_F_SG;
ret = call_netdevice_notifiers(NETDEV_POST_INIT, dev);
ret = notifier_to_errno(ret);
if (ret)
goto err_ifindex_release;
ret = netdev_register_kobject(dev);
netdev_lock(dev);
WRITE_ONCE(dev->reg_state, ret ? NETREG_UNREGISTERED : NETREG_REGISTERED);
netdev_unlock(dev);
if (ret)
goto err_uninit_notify;
netdev_lock_ops(dev);
__netdev_update_features(dev);
netdev_unlock_ops(dev);
/*
* Default initial state at registry is that the
* device is present.
*/
set_bit(__LINK_STATE_PRESENT, &dev->state);
linkwatch_init_dev(dev);
dev_init_scheduler(dev);
netdev_hold(dev, &dev->dev_registered_tracker, GFP_KERNEL);
list_netdevice(dev);
add_device_randomness(dev->dev_addr, dev->addr_len);
/* If the device has permanent device address, driver should
* set dev_addr and also addr_assign_type should be set to
* NET_ADDR_PERM (default value).
*/
if (dev->addr_assign_type == NET_ADDR_PERM)
memcpy(dev->perm_addr, dev->dev_addr, dev->addr_len);
/* Notify protocols, that a new device appeared. */
netdev_lock_ops(dev);
ret = call_netdevice_notifiers(NETDEV_REGISTER, dev);
netdev_unlock_ops(dev);
ret = notifier_to_errno(ret);
if (ret) {
/* Expect explicit free_netdev() on failure */
dev->needs_free_netdev = false;
unregister_netdevice_queue(dev, NULL);
goto out;
}
/*
* Prevent userspace races by waiting until the network
* device is fully setup before sending notifications.
*/
if (!(dev->rtnl_link_ops && dev->rtnl_link_initializing))
rtmsg_ifinfo(RTM_NEWLINK, dev, ~0U, GFP_KERNEL, 0, NULL);
out:
return ret;
err_uninit_notify:
call_netdevice_notifiers(NETDEV_PRE_UNINIT, dev);
err_ifindex_release:
dev_index_release(net, dev->ifindex);
err_free_pcpu:
netdev_do_free_pcpu_stats(dev);
err_uninit:
if (dev->netdev_ops->ndo_uninit)
dev->netdev_ops->ndo_uninit(dev);
if (dev->priv_destructor)
dev->priv_destructor(dev);
err_free_name:
netdev_name_node_free(dev->name_node);
goto out;
}
EXPORT_SYMBOL(register_netdevice);
/* Initialize the core of a dummy net device.
* The setup steps dummy netdevs need which normal netdevs get by going
* through register_netdevice().
*/
static void init_dummy_netdev(struct net_device *dev)
{
/* make sure we BUG if trying to hit standard
* register/unregister code path
*/
dev->reg_state = NETREG_DUMMY;
/* a dummy interface is started by default */
set_bit(__LINK_STATE_PRESENT, &dev->state);
set_bit(__LINK_STATE_START, &dev->state);
/* Note : We dont allocate pcpu_refcnt for dummy devices,
* because users of this 'device' dont need to change
* its refcount.
*/
}
/**
* register_netdev - register a network device
* @dev: device to register
*
* Take a completed network device structure and add it to the kernel
* interfaces. A %NETDEV_REGISTER message is sent to the netdev notifier
* chain. 0 is returned on success. A negative errno code is returned
* on a failure to set up the device, or if the name is a duplicate.
*
* This is a wrapper around register_netdevice that takes the rtnl semaphore
* and expands the device name if you passed a format string to
* alloc_netdev.
*/
int register_netdev(struct net_device *dev)
{
struct net *net = dev_net(dev);
int err;
if (rtnl_net_lock_killable(net))
return -EINTR;
err = register_netdevice(dev);
rtnl_net_unlock(net);
return err;
}
EXPORT_SYMBOL(register_netdev);
int netdev_refcnt_read(const struct net_device *dev)
{
#ifdef CONFIG_PCPU_DEV_REFCNT
int i, refcnt = 0;
for_each_possible_cpu(i)
refcnt += *per_cpu_ptr(dev->pcpu_refcnt, i);
return refcnt;
#else
return refcount_read(&dev->dev_refcnt);
#endif
}
EXPORT_SYMBOL(netdev_refcnt_read);
int netdev_unregister_timeout_secs __read_mostly = 10;
#define WAIT_REFS_MIN_MSECS 1
#define WAIT_REFS_MAX_MSECS 250
/**
* netdev_wait_allrefs_any - wait until all references are gone.
* @list: list of net_devices to wait on
*
* This is called when unregistering network devices.
*
* Any protocol or device that holds a reference should register
* for netdevice notification, and cleanup and put back the
* reference if they receive an UNREGISTER event.
* We can get stuck here if buggy protocols don't correctly
* call dev_put.
*/
static struct net_device *netdev_wait_allrefs_any(struct list_head *list)
{
unsigned long rebroadcast_time, warning_time;
struct net_device *dev;
int wait = 0;
rebroadcast_time = warning_time = jiffies;
list_for_each_entry(dev, list, todo_list)
if (netdev_refcnt_read(dev) == 1)
return dev;
while (true) {
if (time_after(jiffies, rebroadcast_time + 1 * HZ)) {
rtnl_lock();
/* Rebroadcast unregister notification */
list_for_each_entry(dev, list, todo_list)
call_netdevice_notifiers(NETDEV_UNREGISTER, dev);
__rtnl_unlock();
rcu_barrier();
rtnl_lock();
list_for_each_entry(dev, list, todo_list)
if (test_bit(__LINK_STATE_LINKWATCH_PENDING,
&dev->state)) {
/* We must not have linkwatch events
* pending on unregister. If this
* happens, we simply run the queue
* unscheduled, resulting in a noop
* for this device.
*/
linkwatch_run_queue();
break;
}
__rtnl_unlock();
rebroadcast_time = jiffies;
}
rcu_barrier();
if (!wait) {
wait = WAIT_REFS_MIN_MSECS;
} else {
msleep(wait);
wait = min(wait << 1, WAIT_REFS_MAX_MSECS);
}
list_for_each_entry(dev, list, todo_list)
if (netdev_refcnt_read(dev) == 1)
return dev;
if (time_after(jiffies, warning_time +
READ_ONCE(netdev_unregister_timeout_secs) * HZ)) {
list_for_each_entry(dev, list, todo_list) {
pr_emerg("unregister_netdevice: waiting for %s to become free. Usage count = %d\n",
dev->name, netdev_refcnt_read(dev));
ref_tracker_dir_print(&dev->refcnt_tracker, 10);
}
warning_time = jiffies;
}
}
}
/* The sequence is:
*
* rtnl_lock();
* ...
* register_netdevice(x1);
* register_netdevice(x2);
* ...
* unregister_netdevice(y1);
* unregister_netdevice(y2);
* ...
* rtnl_unlock();
* free_netdev(y1);
* free_netdev(y2);
*
* We are invoked by rtnl_unlock().
* This allows us to deal with problems:
* 1) We can delete sysfs objects which invoke hotplug
* without deadlocking with linkwatch via keventd.
* 2) Since we run with the RTNL semaphore not held, we can sleep
* safely in order to wait for the netdev refcnt to drop to zero.
*
* We must not return until all unregister events added during
* the interval the lock was held have been completed.
*/
void netdev_run_todo(void)
{
struct net_device *dev, *tmp;
struct list_head list;
int cnt;
#ifdef CONFIG_LOCKDEP
struct list_head unlink_list;
list_replace_init(&net_unlink_list, &unlink_list);
while (!list_empty(&unlink_list)) {
dev = list_first_entry(&unlink_list, struct net_device,
unlink_list);
list_del_init(&dev->unlink_list);
dev->nested_level = dev->lower_level - 1;
}
#endif
/* Snapshot list, allow later requests */
list_replace_init(&net_todo_list, &list);
__rtnl_unlock();
/* Wait for rcu callbacks to finish before next phase */
if (!list_empty(&list))
rcu_barrier();
list_for_each_entry_safe(dev, tmp, &list, todo_list) {
if (unlikely(dev->reg_state != NETREG_UNREGISTERING)) {
netdev_WARN(dev, "run_todo but not unregistering\n");
list_del(&dev->todo_list);
continue;
}
netdev_lock(dev);
WRITE_ONCE(dev->reg_state, NETREG_UNREGISTERED);
netdev_unlock(dev);
linkwatch_sync_dev(dev);
}
cnt = 0;
while (!list_empty(&list)) {
dev = netdev_wait_allrefs_any(&list);
list_del(&dev->todo_list);
/* paranoia */
BUG_ON(netdev_refcnt_read(dev) != 1);
BUG_ON(!list_empty(&dev->ptype_all));
BUG_ON(!list_empty(&dev->ptype_specific));
WARN_ON(rcu_access_pointer(dev->ip_ptr));
WARN_ON(rcu_access_pointer(dev->ip6_ptr));
netdev_do_free_pcpu_stats(dev);
if (dev->priv_destructor)
dev->priv_destructor(dev);
if (dev->needs_free_netdev)
free_netdev(dev);
cnt++;
/* Free network device */
kobject_put(&dev->dev.kobj);
}
if (cnt && atomic_sub_and_test(cnt, &dev_unreg_count))
wake_up(&netdev_unregistering_wq);
}
/* Collate per-cpu network dstats statistics
*
* Read per-cpu network statistics from dev->dstats and populate the related
* fields in @s.
*/
static void dev_fetch_dstats(struct rtnl_link_stats64 *s,
const struct pcpu_dstats __percpu *dstats)
{
int cpu;
for_each_possible_cpu(cpu) {
u64 rx_packets, rx_bytes, rx_drops;
u64 tx_packets, tx_bytes, tx_drops;
const struct pcpu_dstats *stats;
unsigned int start;
stats = per_cpu_ptr(dstats, cpu);
do {
start = u64_stats_fetch_begin(&stats->syncp);
rx_packets = u64_stats_read(&stats->rx_packets);
rx_bytes = u64_stats_read(&stats->rx_bytes);
rx_drops = u64_stats_read(&stats->rx_drops);
tx_packets = u64_stats_read(&stats->tx_packets);
tx_bytes = u64_stats_read(&stats->tx_bytes);
tx_drops = u64_stats_read(&stats->tx_drops);
} while (u64_stats_fetch_retry(&stats->syncp, start));
s->rx_packets += rx_packets;
s->rx_bytes += rx_bytes;
s->rx_dropped += rx_drops;
s->tx_packets += tx_packets;
s->tx_bytes += tx_bytes;
s->tx_dropped += tx_drops;
}
}
/* ndo_get_stats64 implementation for dtstats-based accounting.
*
* Populate @s from dev->stats and dev->dstats. This is used internally by the
* core for NETDEV_PCPU_STAT_DSTAT-type stats collection.
*/
static void dev_get_dstats64(const struct net_device *dev,
struct rtnl_link_stats64 *s)
{
netdev_stats_to_stats64(s, &dev->stats);
dev_fetch_dstats(s, dev->dstats);
}
/* Convert net_device_stats to rtnl_link_stats64. rtnl_link_stats64 has
* all the same fields in the same order as net_device_stats, with only
* the type differing, but rtnl_link_stats64 may have additional fields
* at the end for newer counters.
*/
void netdev_stats_to_stats64(struct rtnl_link_stats64 *stats64,
const struct net_device_stats *netdev_stats)
{
size_t i, n = sizeof(*netdev_stats) / sizeof(atomic_long_t);
const atomic_long_t *src = (atomic_long_t *)netdev_stats;
u64 *dst = (u64 *)stats64;
BUILD_BUG_ON(n > sizeof(*stats64) / sizeof(u64));
for (i = 0; i < n; i++)
dst[i] = (unsigned long)atomic_long_read(&src[i]);
/* zero out counters that only exist in rtnl_link_stats64 */
memset((char *)stats64 + n * sizeof(u64), 0,
sizeof(*stats64) - n * sizeof(u64));
}
EXPORT_SYMBOL(netdev_stats_to_stats64);
static __cold struct net_device_core_stats __percpu *netdev_core_stats_alloc(
struct net_device *dev)
{
struct net_device_core_stats __percpu *p;
p = alloc_percpu_gfp(struct net_device_core_stats,
GFP_ATOMIC | __GFP_NOWARN);
if (p && cmpxchg(&dev->core_stats, NULL, p))
free_percpu(p);
/* This READ_ONCE() pairs with the cmpxchg() above */
return READ_ONCE(dev->core_stats);
}
noinline void netdev_core_stats_inc(struct net_device *dev, u32 offset)
{
/* This READ_ONCE() pairs with the write in netdev_core_stats_alloc() */
struct net_device_core_stats __percpu *p = READ_ONCE(dev->core_stats);
unsigned long __percpu *field;
if (unlikely(!p)) {
p = netdev_core_stats_alloc(dev);
if (!p)
return;
}
field = (unsigned long __percpu *)((void __percpu *)p + offset);
this_cpu_inc(*field);
}
EXPORT_SYMBOL_GPL(netdev_core_stats_inc);
/**
* dev_get_stats - get network device statistics
* @dev: device to get statistics from
* @storage: place to store stats
*
* Get network statistics from device. Return @storage.
* The device driver may provide its own method by setting
* dev->netdev_ops->get_stats64 or dev->netdev_ops->get_stats;
* otherwise the internal statistics structure is used.
*/
struct rtnl_link_stats64 *dev_get_stats(struct net_device *dev,
struct rtnl_link_stats64 *storage)
{
const struct net_device_ops *ops = dev->netdev_ops;
const struct net_device_core_stats __percpu *p;
/*
* IPv{4,6} and udp tunnels share common stat helpers and use
* different stat type (NETDEV_PCPU_STAT_TSTATS vs
* NETDEV_PCPU_STAT_DSTATS). Ensure the accounting is consistent.
*/
BUILD_BUG_ON(offsetof(struct pcpu_sw_netstats, rx_bytes) !=
offsetof(struct pcpu_dstats, rx_bytes));
BUILD_BUG_ON(offsetof(struct pcpu_sw_netstats, rx_packets) !=
offsetof(struct pcpu_dstats, rx_packets));
BUILD_BUG_ON(offsetof(struct pcpu_sw_netstats, tx_bytes) !=
offsetof(struct pcpu_dstats, tx_bytes));
BUILD_BUG_ON(offsetof(struct pcpu_sw_netstats, tx_packets) !=
offsetof(struct pcpu_dstats, tx_packets));
if (ops->ndo_get_stats64) {
memset(storage, 0, sizeof(*storage));
ops->ndo_get_stats64(dev, storage);
} else if (ops->ndo_get_stats) {
netdev_stats_to_stats64(storage, ops->ndo_get_stats(dev));
} else if (dev->pcpu_stat_type == NETDEV_PCPU_STAT_TSTATS) {
dev_get_tstats64(dev, storage);
} else if (dev->pcpu_stat_type == NETDEV_PCPU_STAT_DSTATS) {
dev_get_dstats64(dev, storage);
} else {
netdev_stats_to_stats64(storage, &dev->stats);
}
/* This READ_ONCE() pairs with the write in netdev_core_stats_alloc() */
p = READ_ONCE(dev->core_stats);
if (p) {
const struct net_device_core_stats *core_stats;
int i;
for_each_possible_cpu(i) {
core_stats = per_cpu_ptr(p, i);
storage->rx_dropped += READ_ONCE(core_stats->rx_dropped);
storage->tx_dropped += READ_ONCE(core_stats->tx_dropped);
storage->rx_nohandler += READ_ONCE(core_stats->rx_nohandler);
storage->rx_otherhost_dropped += READ_ONCE(core_stats->rx_otherhost_dropped);
}
}
return storage;
}
EXPORT_SYMBOL(dev_get_stats);
/**
* dev_fetch_sw_netstats - get per-cpu network device statistics
* @s: place to store stats
* @netstats: per-cpu network stats to read from
*
* Read per-cpu network statistics and populate the related fields in @s.
*/
void dev_fetch_sw_netstats(struct rtnl_link_stats64 *s,
const struct pcpu_sw_netstats __percpu *netstats)
{
int cpu;
for_each_possible_cpu(cpu) {
u64 rx_packets, rx_bytes, tx_packets, tx_bytes;
const struct pcpu_sw_netstats *stats;
unsigned int start;
stats = per_cpu_ptr(netstats, cpu);
do {
start = u64_stats_fetch_begin(&stats->syncp);
rx_packets = u64_stats_read(&stats->rx_packets);
rx_bytes = u64_stats_read(&stats->rx_bytes);
tx_packets = u64_stats_read(&stats->tx_packets);
tx_bytes = u64_stats_read(&stats->tx_bytes);
} while (u64_stats_fetch_retry(&stats->syncp, start));
s->rx_packets += rx_packets;
s->rx_bytes += rx_bytes;
s->tx_packets += tx_packets;
s->tx_bytes += tx_bytes;
}
}
EXPORT_SYMBOL_GPL(dev_fetch_sw_netstats);
/**
* dev_get_tstats64 - ndo_get_stats64 implementation
* @dev: device to get statistics from
* @s: place to store stats
*
* Populate @s from dev->stats and dev->tstats. Can be used as
* ndo_get_stats64() callback.
*/
void dev_get_tstats64(struct net_device *dev, struct rtnl_link_stats64 *s)
{
netdev_stats_to_stats64(s, &dev->stats);
dev_fetch_sw_netstats(s, dev->tstats);
}
EXPORT_SYMBOL_GPL(dev_get_tstats64);
struct netdev_queue *dev_ingress_queue_create(struct net_device *dev)
{
struct netdev_queue *queue = dev_ingress_queue(dev);
#ifdef CONFIG_NET_CLS_ACT
if (queue)
return queue;
queue = kzalloc(sizeof(*queue), GFP_KERNEL);
if (!queue)
return NULL;
netdev_init_one_queue(dev, queue, NULL);
RCU_INIT_POINTER(queue->qdisc, &noop_qdisc);
RCU_INIT_POINTER(queue->qdisc_sleeping, &noop_qdisc);
rcu_assign_pointer(dev->ingress_queue, queue);
#endif
return queue;
}
static const struct ethtool_ops default_ethtool_ops;
void netdev_set_default_ethtool_ops(struct net_device *dev,
const struct ethtool_ops *ops)
{
if (dev->ethtool_ops == &default_ethtool_ops)
dev->ethtool_ops = ops;
}
EXPORT_SYMBOL_GPL(netdev_set_default_ethtool_ops);
/**
* netdev_sw_irq_coalesce_default_on() - enable SW IRQ coalescing by default
* @dev: netdev to enable the IRQ coalescing on
*
* Sets a conservative default for SW IRQ coalescing. Users can use
* sysfs attributes to override the default values.
*/
void netdev_sw_irq_coalesce_default_on(struct net_device *dev)
{
WARN_ON(dev->reg_state == NETREG_REGISTERED);
if (!IS_ENABLED(CONFIG_PREEMPT_RT)) {
netdev_set_gro_flush_timeout(dev, 20000);
netdev_set_defer_hard_irqs(dev, 1);
}
}
EXPORT_SYMBOL_GPL(netdev_sw_irq_coalesce_default_on);
/**
* alloc_netdev_mqs - allocate network device
* @sizeof_priv: size of private data to allocate space for
* @name: device name format string
* @name_assign_type: origin of device name
* @setup: callback to initialize device
* @txqs: the number of TX subqueues to allocate
* @rxqs: the number of RX subqueues to allocate
*
* Allocates a struct net_device with private data area for driver use
* and performs basic initialization. Also allocates subqueue structs
* for each queue on the device.
*/
struct net_device *alloc_netdev_mqs(int sizeof_priv, const char *name,
unsigned char name_assign_type,
void (*setup)(struct net_device *),
unsigned int txqs, unsigned int rxqs)
{
struct net_device *dev;
size_t napi_config_sz;
unsigned int maxqs;
BUG_ON(strlen(name) >= sizeof(dev->name));
if (txqs < 1) {
pr_err("alloc_netdev: Unable to allocate device with zero queues\n");
return NULL;
}
if (rxqs < 1) {
pr_err("alloc_netdev: Unable to allocate device with zero RX queues\n");
return NULL;
}
maxqs = max(txqs, rxqs);
dev = kvzalloc(struct_size(dev, priv, sizeof_priv),
GFP_KERNEL_ACCOUNT | __GFP_RETRY_MAYFAIL);
if (!dev)
return NULL;
dev->priv_len = sizeof_priv;
ref_tracker_dir_init(&dev->refcnt_tracker, 128, "netdev");
#ifdef CONFIG_PCPU_DEV_REFCNT
dev->pcpu_refcnt = alloc_percpu(int);
if (!dev->pcpu_refcnt)
goto free_dev;
__dev_hold(dev);
#else
refcount_set(&dev->dev_refcnt, 1);
#endif
if (dev_addr_init(dev))
goto free_pcpu;
dev_mc_init(dev);
dev_uc_init(dev);
dev_net_set(dev, &init_net);
dev->gso_max_size = GSO_LEGACY_MAX_SIZE;
dev->xdp_zc_max_segs = 1;
dev->gso_max_segs = GSO_MAX_SEGS;
dev->gro_max_size = GRO_LEGACY_MAX_SIZE;
dev->gso_ipv4_max_size = GSO_LEGACY_MAX_SIZE;
dev->gro_ipv4_max_size = GRO_LEGACY_MAX_SIZE;
dev->tso_max_size = TSO_LEGACY_MAX_SIZE;
dev->tso_max_segs = TSO_MAX_SEGS;
dev->upper_level = 1;
dev->lower_level = 1;
#ifdef CONFIG_LOCKDEP
dev->nested_level = 0;
INIT_LIST_HEAD(&dev->unlink_list);
#endif
INIT_LIST_HEAD(&dev->napi_list);
INIT_LIST_HEAD(&dev->unreg_list);
INIT_LIST_HEAD(&dev->close_list);
INIT_LIST_HEAD(&dev->link_watch_list);
INIT_LIST_HEAD(&dev->adj_list.upper);
INIT_LIST_HEAD(&dev->adj_list.lower);
INIT_LIST_HEAD(&dev->ptype_all);
INIT_LIST_HEAD(&dev->ptype_specific);
INIT_LIST_HEAD(&dev->net_notifier_list);
#ifdef CONFIG_NET_SCHED
hash_init(dev->qdisc_hash);
#endif
mutex_init(&dev->lock);
dev->priv_flags = IFF_XMIT_DST_RELEASE | IFF_XMIT_DST_RELEASE_PERM;
setup(dev);
if (!dev->tx_queue_len) {
dev->priv_flags |= IFF_NO_QUEUE;
dev->tx_queue_len = DEFAULT_TX_QUEUE_LEN;
}
dev->num_tx_queues = txqs;
dev->real_num_tx_queues = txqs;
if (netif_alloc_netdev_queues(dev))
goto free_all;
dev->num_rx_queues = rxqs;
dev->real_num_rx_queues = rxqs;
if (netif_alloc_rx_queues(dev))
goto free_all;
dev->ethtool = kzalloc(sizeof(*dev->ethtool), GFP_KERNEL_ACCOUNT);
if (!dev->ethtool)
goto free_all;
dev->cfg = kzalloc(sizeof(*dev->cfg), GFP_KERNEL_ACCOUNT);
if (!dev->cfg)
goto free_all;
dev->cfg_pending = dev->cfg;
dev->num_napi_configs = maxqs;
napi_config_sz = array_size(maxqs, sizeof(*dev->napi_config));
dev->napi_config = kvzalloc(napi_config_sz, GFP_KERNEL_ACCOUNT);
if (!dev->napi_config)
goto free_all;
strscpy(dev->name, name);
dev->name_assign_type = name_assign_type;
dev->group = INIT_NETDEV_GROUP;
if (!dev->ethtool_ops)
dev->ethtool_ops = &default_ethtool_ops;
nf_hook_netdev_init(dev);
return dev;
free_all:
free_netdev(dev);
return NULL;
free_pcpu:
#ifdef CONFIG_PCPU_DEV_REFCNT
free_percpu(dev->pcpu_refcnt);
free_dev:
#endif
kvfree(dev);
return NULL;
}
EXPORT_SYMBOL(alloc_netdev_mqs);
static void netdev_napi_exit(struct net_device *dev)
{
if (!list_empty(&dev->napi_list)) {
struct napi_struct *p, *n;
netdev_lock(dev);
list_for_each_entry_safe(p, n, &dev->napi_list, dev_list)
__netif_napi_del_locked(p);
netdev_unlock(dev);
synchronize_net();
}
kvfree(dev->napi_config);
}
/**
* free_netdev - free network device
* @dev: device
*
* This function does the last stage of destroying an allocated device
* interface. The reference to the device object is released. If this
* is the last reference then it will be freed.Must be called in process
* context.
*/
void free_netdev(struct net_device *dev)
{
might_sleep();
/* When called immediately after register_netdevice() failed the unwind
* handling may still be dismantling the device. Handle that case by
* deferring the free.
*/
if (dev->reg_state == NETREG_UNREGISTERING) {
ASSERT_RTNL();
dev->needs_free_netdev = true;
return;
}
WARN_ON(dev->cfg != dev->cfg_pending);
kfree(dev->cfg);
kfree(dev->ethtool);
netif_free_tx_queues(dev);
netif_free_rx_queues(dev);
kfree(rcu_dereference_protected(dev->ingress_queue, 1));
/* Flush device addresses */
dev_addr_flush(dev);
netdev_napi_exit(dev);
netif_del_cpu_rmap(dev);
ref_tracker_dir_exit(&dev->refcnt_tracker);
#ifdef CONFIG_PCPU_DEV_REFCNT
free_percpu(dev->pcpu_refcnt);
dev->pcpu_refcnt = NULL;
#endif
free_percpu(dev->core_stats);
dev->core_stats = NULL;
free_percpu(dev->xdp_bulkq);
dev->xdp_bulkq = NULL;
netdev_free_phy_link_topology(dev);
mutex_destroy(&dev->lock);
/* Compatibility with error handling in drivers */
if (dev->reg_state == NETREG_UNINITIALIZED ||
dev->reg_state == NETREG_DUMMY) {
kvfree(dev);
return;
}
BUG_ON(dev->reg_state != NETREG_UNREGISTERED);
WRITE_ONCE(dev->reg_state, NETREG_RELEASED);
/* will free via device release */
put_device(&dev->dev);
}
EXPORT_SYMBOL(free_netdev);
/**
* alloc_netdev_dummy - Allocate and initialize a dummy net device.
* @sizeof_priv: size of private data to allocate space for
*
* Return: the allocated net_device on success, NULL otherwise
*/
struct net_device *alloc_netdev_dummy(int sizeof_priv)
{
return alloc_netdev(sizeof_priv, "dummy#", NET_NAME_UNKNOWN,
init_dummy_netdev);
}
EXPORT_SYMBOL_GPL(alloc_netdev_dummy);
/**
* synchronize_net - Synchronize with packet receive processing
*
* Wait for packets currently being received to be done.
* Does not block later packets from starting.
*/
void synchronize_net(void)
{
might_sleep();
if (from_cleanup_net() || rtnl_is_locked())
synchronize_rcu_expedited();
else
synchronize_rcu();
}
EXPORT_SYMBOL(synchronize_net);
static void netdev_rss_contexts_free(struct net_device *dev)
{
struct ethtool_rxfh_context *ctx;
unsigned long context;
mutex_lock(&dev->ethtool->rss_lock);
xa_for_each(&dev->ethtool->rss_ctx, context, ctx) {
xa_erase(&dev->ethtool->rss_ctx, context);
dev->ethtool_ops->remove_rxfh_context(dev, ctx, context, NULL);
kfree(ctx);
}
xa_destroy(&dev->ethtool->rss_ctx);
mutex_unlock(&dev->ethtool->rss_lock);
}
/**
* unregister_netdevice_queue - remove device from the kernel
* @dev: device
* @head: list
*
* This function shuts down a device interface and removes it
* from the kernel tables.
* If head not NULL, device is queued to be unregistered later.
*
* Callers must hold the rtnl semaphore. You may want
* unregister_netdev() instead of this.
*/
void unregister_netdevice_queue(struct net_device *dev, struct list_head *head)
{
ASSERT_RTNL();
if (head) {
list_move_tail(&dev->unreg_list, head);
} else {
LIST_HEAD(single);
list_add(&dev->unreg_list, &single);
unregister_netdevice_many(&single);
}
}
EXPORT_SYMBOL(unregister_netdevice_queue);
static void dev_memory_provider_uninstall(struct net_device *dev)
{
unsigned int i;
for (i = 0; i < dev->real_num_rx_queues; i++) {
struct netdev_rx_queue *rxq = &dev->_rx[i];
struct pp_memory_provider_params *p = &rxq->mp_params;
if (p->mp_ops && p->mp_ops->uninstall)
p->mp_ops->uninstall(rxq->mp_params.mp_priv, rxq);
}
}
/* devices must be UP and netdev_lock()'d */
static void netif_close_many_and_unlock(struct list_head *close_head)
{
struct net_device *dev, *tmp;
netif_close_many(close_head, false);
/* ... now unlock them */
list_for_each_entry_safe(dev, tmp, close_head, close_list) {
netdev_unlock(dev);
list_del_init(&dev->close_list);
}
}
static void netif_close_many_and_unlock_cond(struct list_head *close_head)
{
#ifdef CONFIG_LOCKDEP
/* We can only track up to MAX_LOCK_DEPTH locks per task.
*
* Reserve half the available slots for additional locks possibly
* taken by notifiers and (soft)irqs.
*/
unsigned int limit = MAX_LOCK_DEPTH / 2;
if (lockdep_depth(current) > limit)
netif_close_many_and_unlock(close_head);
#endif
}
void unregister_netdevice_many_notify(struct list_head *head,
u32 portid, const struct nlmsghdr *nlh)
{
struct net_device *dev, *tmp;
LIST_HEAD(close_head);
int cnt = 0;
BUG_ON(dev_boot_phase);
ASSERT_RTNL();
if (list_empty(head))
return;
list_for_each_entry_safe(dev, tmp, head, unreg_list) {
/* Some devices call without registering
* for initialization unwind. Remove those
* devices and proceed with the remaining.
*/
if (dev->reg_state == NETREG_UNINITIALIZED) {
pr_debug("unregister_netdevice: device %s/%p never was registered\n",
dev->name, dev);
WARN_ON(1);
list_del(&dev->unreg_list);
continue;
}
dev->dismantle = true;
BUG_ON(dev->reg_state != NETREG_REGISTERED);
}
/* If device is running, close it first. Start with ops locked... */
list_for_each_entry(dev, head, unreg_list) {
if (!(dev->flags & IFF_UP))
continue;
if (netdev_need_ops_lock(dev)) {
list_add_tail(&dev->close_list, &close_head);
netdev_lock(dev);
}
netif_close_many_and_unlock_cond(&close_head);
}
netif_close_many_and_unlock(&close_head);
/* ... now go over the rest. */
list_for_each_entry(dev, head, unreg_list) {
if (!netdev_need_ops_lock(dev))
list_add_tail(&dev->close_list, &close_head);
}
netif_close_many(&close_head, true);
list_for_each_entry(dev, head, unreg_list) {
/* And unlink it from device chain. */
unlist_netdevice(dev);
netdev_lock(dev);
WRITE_ONCE(dev->reg_state, NETREG_UNREGISTERING);
netdev_unlock(dev);
}
flush_all_backlogs();
synchronize_net();
list_for_each_entry(dev, head, unreg_list) {
struct sk_buff *skb = NULL;
/* Shutdown queueing discipline. */
netdev_lock_ops(dev);
dev_shutdown(dev);
dev_tcx_uninstall(dev);
dev_xdp_uninstall(dev);
dev_memory_provider_uninstall(dev);
netdev_unlock_ops(dev);
bpf_dev_bound_netdev_unregister(dev);
netdev_offload_xstats_disable_all(dev);
/* Notify protocols, that we are about to destroy
* this device. They should clean all the things.
*/
call_netdevice_notifiers(NETDEV_UNREGISTER, dev);
if (!(dev->rtnl_link_ops && dev->rtnl_link_initializing))
skb = rtmsg_ifinfo_build_skb(RTM_DELLINK, dev, ~0U, 0,
GFP_KERNEL, NULL, 0,
portid, nlh);
/*
* Flush the unicast and multicast chains
*/
dev_uc_flush(dev);
dev_mc_flush(dev);
netdev_name_node_alt_flush(dev);
netdev_name_node_free(dev->name_node);
netdev_rss_contexts_free(dev);
call_netdevice_notifiers(NETDEV_PRE_UNINIT, dev);
if (dev->netdev_ops->ndo_uninit)
dev->netdev_ops->ndo_uninit(dev);
mutex_destroy(&dev->ethtool->rss_lock);
net_shaper_flush_netdev(dev);
if (skb)
rtmsg_ifinfo_send(skb, dev, GFP_KERNEL, portid, nlh);
/* Notifier chain MUST detach us all upper devices. */
WARN_ON(netdev_has_any_upper_dev(dev));
WARN_ON(netdev_has_any_lower_dev(dev));
/* Remove entries from kobject tree */
netdev_unregister_kobject(dev);
#ifdef CONFIG_XPS
/* Remove XPS queueing entries */
netif_reset_xps_queues_gt(dev, 0);
#endif
}
synchronize_net();
list_for_each_entry(dev, head, unreg_list) {
netdev_put(dev, &dev->dev_registered_tracker);
net_set_todo(dev);
cnt++;
}
atomic_add(cnt, &dev_unreg_count);
list_del(head);
}
/**
* unregister_netdevice_many - unregister many devices
* @head: list of devices
*
* Note: As most callers use a stack allocated list_head,
* we force a list_del() to make sure stack won't be corrupted later.
*/
void unregister_netdevice_many(struct list_head *head)
{
unregister_netdevice_many_notify(head, 0, NULL);
}
EXPORT_SYMBOL(unregister_netdevice_many);
/**
* unregister_netdev - remove device from the kernel
* @dev: device
*
* This function shuts down a device interface and removes it
* from the kernel tables.
*
* This is just a wrapper for unregister_netdevice that takes
* the rtnl semaphore. In general you want to use this and not
* unregister_netdevice.
*/
void unregister_netdev(struct net_device *dev)
{
rtnl_net_dev_lock(dev);
unregister_netdevice(dev);
rtnl_net_dev_unlock(dev);
}
EXPORT_SYMBOL(unregister_netdev);
int __dev_change_net_namespace(struct net_device *dev, struct net *net,
const char *pat, int new_ifindex,
struct netlink_ext_ack *extack)
{
struct netdev_name_node *name_node;
struct net *net_old = dev_net(dev);
char new_name[IFNAMSIZ] = {};
int err, new_nsid;
ASSERT_RTNL();
/* Don't allow namespace local devices to be moved. */
err = -EINVAL;
if (dev->netns_immutable) {
NL_SET_ERR_MSG(extack, "The interface netns is immutable");
goto out;
}
/* Ensure the device has been registered */
if (dev->reg_state != NETREG_REGISTERED) {
NL_SET_ERR_MSG(extack, "The interface isn't registered");
goto out;
}
/* Get out if there is nothing todo */
err = 0;
if (net_eq(net_old, net))
goto out;
/* Pick the destination device name, and ensure
* we can use it in the destination network namespace.
*/
err = -EEXIST;
if (netdev_name_in_use(net, dev->name)) {
/* We get here if we can't use the current device name */
if (!pat) {
NL_SET_ERR_MSG(extack,
"An interface with the same name exists in the target netns");
goto out;
}
err = dev_prep_valid_name(net, dev, pat, new_name, EEXIST);
if (err < 0) {
NL_SET_ERR_MSG_FMT(extack,
"Unable to use '%s' for the new interface name in the target netns",
pat);
goto out;
}
}
/* Check that none of the altnames conflicts. */
err = -EEXIST;
netdev_for_each_altname(dev, name_node) {
if (netdev_name_in_use(net, name_node->name)) {
NL_SET_ERR_MSG_FMT(extack,
"An interface with the altname %s exists in the target netns",
name_node->name);
goto out;
}
}
/* Check that new_ifindex isn't used yet. */
if (new_ifindex) {
err = dev_index_reserve(net, new_ifindex);
if (err < 0) {
NL_SET_ERR_MSG_FMT(extack,
"The ifindex %d is not available in the target netns",
new_ifindex);
goto out;
}
} else {
/* If there is an ifindex conflict assign a new one */
err = dev_index_reserve(net, dev->ifindex);
if (err == -EBUSY)
err = dev_index_reserve(net, 0);
if (err < 0) {
NL_SET_ERR_MSG(extack,
"Unable to allocate a new ifindex in the target netns");
goto out;
}
new_ifindex = err;
}
/*
* And now a mini version of register_netdevice unregister_netdevice.
*/
netdev_lock_ops(dev);
/* If device is running close it first. */
netif_close(dev);
/* And unlink it from device chain */
unlist_netdevice(dev);
if (!netdev_need_ops_lock(dev))
netdev_lock(dev);
dev->moving_ns = true;
netdev_unlock(dev);
synchronize_net();
/* Shutdown queueing discipline. */
netdev_lock_ops(dev);
dev_shutdown(dev);
netdev_unlock_ops(dev);
/* Notify protocols, that we are about to destroy
* this device. They should clean all the things.
*
* Note that dev->reg_state stays at NETREG_REGISTERED.
* This is wanted because this way 8021q and macvlan know
* the device is just moving and can keep their slaves up.
*/
call_netdevice_notifiers(NETDEV_UNREGISTER, dev);
rcu_barrier();
new_nsid = peernet2id_alloc(dev_net(dev), net, GFP_KERNEL);
rtmsg_ifinfo_newnet(RTM_DELLINK, dev, ~0U, GFP_KERNEL, &new_nsid,
new_ifindex);
/*
* Flush the unicast and multicast chains
*/
dev_uc_flush(dev);
dev_mc_flush(dev);
/* Send a netdev-removed uevent to the old namespace */
kobject_uevent(&dev->dev.kobj, KOBJ_REMOVE);
netdev_adjacent_del_links(dev);
/* Move per-net netdevice notifiers that are following the netdevice */
move_netdevice_notifiers_dev_net(dev, net);
/* Actually switch the network namespace */
netdev_lock(dev);
dev_net_set(dev, net);
netdev_unlock(dev);
dev->ifindex = new_ifindex;
if (new_name[0]) {
/* Rename the netdev to prepared name */
write_seqlock_bh(&netdev_rename_lock);
strscpy(dev->name, new_name, IFNAMSIZ);
write_sequnlock_bh(&netdev_rename_lock);
}
/* Fixup kobjects */
dev_set_uevent_suppress(&dev->dev, 1);
err = device_rename(&dev->dev, dev->name);
dev_set_uevent_suppress(&dev->dev, 0);
WARN_ON(err);
/* Send a netdev-add uevent to the new namespace */
kobject_uevent(&dev->dev.kobj, KOBJ_ADD);
netdev_adjacent_add_links(dev);
/* Adapt owner in case owning user namespace of target network
* namespace is different from the original one.
*/
err = netdev_change_owner(dev, net_old, net);
WARN_ON(err);
netdev_lock(dev);
dev->moving_ns = false;
if (!netdev_need_ops_lock(dev))
netdev_unlock(dev);
/* Add the device back in the hashes */
list_netdevice(dev);
/* Notify protocols, that a new device appeared. */
call_netdevice_notifiers(NETDEV_REGISTER, dev);
netdev_unlock_ops(dev);
/*
* Prevent userspace races by waiting until the network
* device is fully setup before sending notifications.
*/
rtmsg_ifinfo(RTM_NEWLINK, dev, ~0U, GFP_KERNEL, 0, NULL);
synchronize_net();
err = 0;
out:
return err;
}
static int dev_cpu_dead(unsigned int oldcpu)
{
struct sk_buff **list_skb;
struct sk_buff *skb;
unsigned int cpu;
struct softnet_data *sd, *oldsd, *remsd = NULL;
local_irq_disable();
cpu = smp_processor_id();
sd = &per_cpu(softnet_data, cpu);
oldsd = &per_cpu(softnet_data, oldcpu);
/* Find end of our completion_queue. */
list_skb = &sd->completion_queue;
while (*list_skb)
list_skb = &(*list_skb)->next;
/* Append completion queue from offline CPU. */
*list_skb = oldsd->completion_queue;
oldsd->completion_queue = NULL;
/* Append output queue from offline CPU. */
if (oldsd->output_queue) {
*sd->output_queue_tailp = oldsd->output_queue;
sd->output_queue_tailp = oldsd->output_queue_tailp;
oldsd->output_queue = NULL;
oldsd->output_queue_tailp = &oldsd->output_queue;
}
/* Append NAPI poll list from offline CPU, with one exception :
* process_backlog() must be called by cpu owning percpu backlog.
* We properly handle process_queue & input_pkt_queue later.
*/
while (!list_empty(&oldsd->poll_list)) {
struct napi_struct *napi = list_first_entry(&oldsd->poll_list,
struct napi_struct,
poll_list);
list_del_init(&napi->poll_list);
if (napi->poll == process_backlog)
napi->state &= NAPIF_STATE_THREADED;
else
____napi_schedule(sd, napi);
}
raise_softirq_irqoff(NET_TX_SOFTIRQ);
local_irq_enable();
if (!use_backlog_threads()) {
#ifdef CONFIG_RPS
remsd = oldsd->rps_ipi_list;
oldsd->rps_ipi_list = NULL;
#endif
/* send out pending IPI's on offline CPU */
net_rps_send_ipi(remsd);
}
/* Process offline CPU's input_pkt_queue */
while ((skb = __skb_dequeue(&oldsd->process_queue))) {
netif_rx(skb);
rps_input_queue_head_incr(oldsd);
}
while ((skb = skb_dequeue(&oldsd->input_pkt_queue))) {
netif_rx(skb);
rps_input_queue_head_incr(oldsd);
}
return 0;
}
/**
* netdev_increment_features - increment feature set by one
* @all: current feature set
* @one: new feature set
* @mask: mask feature set
*
* Computes a new feature set after adding a device with feature set
* @one to the master device with current feature set @all. Will not
* enable anything that is off in @mask. Returns the new feature set.
*/
netdev_features_t netdev_increment_features(netdev_features_t all,
netdev_features_t one, netdev_features_t mask)
{
if (mask & NETIF_F_HW_CSUM)
mask |= NETIF_F_CSUM_MASK;
mask |= NETIF_F_VLAN_CHALLENGED;
all |= one & (NETIF_F_ONE_FOR_ALL | NETIF_F_CSUM_MASK) & mask;
all &= one | ~NETIF_F_ALL_FOR_ALL;
/* If one device supports hw checksumming, set for all. */
if (all & NETIF_F_HW_CSUM)
all &= ~(NETIF_F_CSUM_MASK & ~NETIF_F_HW_CSUM);
return all;
}
EXPORT_SYMBOL(netdev_increment_features);
/**
* netdev_compute_master_upper_features - compute feature from lowers
* @dev: the upper device
* @update_header: whether to update upper device's header_len/headroom/tailroom
*
* Recompute the upper device's feature based on all lower devices.
*/
void netdev_compute_master_upper_features(struct net_device *dev, bool update_header)
{
unsigned int dst_release_flag = IFF_XMIT_DST_RELEASE | IFF_XMIT_DST_RELEASE_PERM;
netdev_features_t gso_partial_features = MASTER_UPPER_DEV_GSO_PARTIAL_FEATURES;
netdev_features_t xfrm_features = MASTER_UPPER_DEV_XFRM_FEATURES;
netdev_features_t mpls_features = MASTER_UPPER_DEV_MPLS_FEATURES;
netdev_features_t vlan_features = MASTER_UPPER_DEV_VLAN_FEATURES;
netdev_features_t enc_features = MASTER_UPPER_DEV_ENC_FEATURES;
unsigned short max_header_len = ETH_HLEN;
unsigned int tso_max_size = TSO_MAX_SIZE;
unsigned short max_headroom = 0;
unsigned short max_tailroom = 0;
u16 tso_max_segs = TSO_MAX_SEGS;
struct net_device *lower_dev;
struct list_head *iter;
mpls_features = netdev_base_features(mpls_features);
vlan_features = netdev_base_features(vlan_features);
enc_features = netdev_base_features(enc_features);
netdev_for_each_lower_dev(dev, lower_dev, iter) {
gso_partial_features = netdev_increment_features(gso_partial_features,
lower_dev->gso_partial_features,
MASTER_UPPER_DEV_GSO_PARTIAL_FEATURES);
vlan_features = netdev_increment_features(vlan_features,
lower_dev->vlan_features,
MASTER_UPPER_DEV_VLAN_FEATURES);
enc_features = netdev_increment_features(enc_features,
lower_dev->hw_enc_features,
MASTER_UPPER_DEV_ENC_FEATURES);
if (IS_ENABLED(CONFIG_XFRM_OFFLOAD))
xfrm_features = netdev_increment_features(xfrm_features,
lower_dev->hw_enc_features,
MASTER_UPPER_DEV_XFRM_FEATURES);
mpls_features = netdev_increment_features(mpls_features,
lower_dev->mpls_features,
MASTER_UPPER_DEV_MPLS_FEATURES);
dst_release_flag &= lower_dev->priv_flags;
if (update_header) {
max_header_len = max(max_header_len, lower_dev->hard_header_len);
max_headroom = max(max_headroom, lower_dev->needed_headroom);
max_tailroom = max(max_tailroom, lower_dev->needed_tailroom);
}
tso_max_size = min(tso_max_size, lower_dev->tso_max_size);
tso_max_segs = min(tso_max_segs, lower_dev->tso_max_segs);
}
dev->gso_partial_features = gso_partial_features;
dev->vlan_features = vlan_features;
dev->hw_enc_features = enc_features | NETIF_F_GSO_ENCAP_ALL |
NETIF_F_HW_VLAN_CTAG_TX |
NETIF_F_HW_VLAN_STAG_TX;
if (IS_ENABLED(CONFIG_XFRM_OFFLOAD))
dev->hw_enc_features |= xfrm_features;
dev->mpls_features = mpls_features;
dev->priv_flags &= ~IFF_XMIT_DST_RELEASE;
if ((dev->priv_flags & IFF_XMIT_DST_RELEASE_PERM) &&
dst_release_flag == (IFF_XMIT_DST_RELEASE | IFF_XMIT_DST_RELEASE_PERM))
dev->priv_flags |= IFF_XMIT_DST_RELEASE;
if (update_header) {
dev->hard_header_len = max_header_len;
dev->needed_headroom = max_headroom;
dev->needed_tailroom = max_tailroom;
}
netif_set_tso_max_segs(dev, tso_max_segs);
netif_set_tso_max_size(dev, tso_max_size);
netdev_change_features(dev);
}
EXPORT_SYMBOL(netdev_compute_master_upper_features);
static struct hlist_head * __net_init netdev_create_hash(void)
{
int i;
struct hlist_head *hash;
hash = kmalloc_array(NETDEV_HASHENTRIES, sizeof(*hash), GFP_KERNEL);
if (hash != NULL)
for (i = 0; i < NETDEV_HASHENTRIES; i++)
INIT_HLIST_HEAD(&hash[i]);
return hash;
}
/* Initialize per network namespace state */
static int __net_init netdev_init(struct net *net)
{
BUILD_BUG_ON(GRO_HASH_BUCKETS >
BITS_PER_BYTE * sizeof_field(struct gro_node, bitmask));
INIT_LIST_HEAD(&net->dev_base_head);
net->dev_name_head = netdev_create_hash();
if (net->dev_name_head == NULL)
goto err_name;
net->dev_index_head = netdev_create_hash();
if (net->dev_index_head == NULL)
goto err_idx;
xa_init_flags(&net->dev_by_index, XA_FLAGS_ALLOC1);
RAW_INIT_NOTIFIER_HEAD(&net->netdev_chain);
return 0;
err_idx:
kfree(net->dev_name_head);
err_name:
return -ENOMEM;
}
/**
* netdev_drivername - network driver for the device
* @dev: network device
*
* Determine network driver for device.
*/
const char *netdev_drivername(const struct net_device *dev)
{
const struct device_driver *driver;
const struct device *parent;
const char *empty = "";
parent = dev->dev.parent;
if (!parent)
return empty;
driver = parent->driver;
if (driver && driver->name)
return driver->name;
return empty;
}
static void __netdev_printk(const char *level, const struct net_device *dev,
struct va_format *vaf)
{
if (dev && dev->dev.parent) {
dev_printk_emit(level[1] - '0',
dev->dev.parent,
"%s %s %s%s: %pV",
dev_driver_string(dev->dev.parent),
dev_name(dev->dev.parent),
netdev_name(dev), netdev_reg_state(dev),
vaf);
} else if (dev) {
printk("%s%s%s: %pV",
level, netdev_name(dev), netdev_reg_state(dev), vaf);
} else {
printk("%s(NULL net_device): %pV", level, vaf);
}
}
void netdev_printk(const char *level, const struct net_device *dev,
const char *format, ...)
{
struct va_format vaf;
va_list args;
va_start(args, format);
vaf.fmt = format;
vaf.va = &args;
__netdev_printk(level, dev, &vaf);
va_end(args);
}
EXPORT_SYMBOL(netdev_printk);
#define define_netdev_printk_level(func, level) \
void func(const struct net_device *dev, const char *fmt, ...) \
{ \
struct va_format vaf; \
va_list args; \
\
va_start(args, fmt); \
\
vaf.fmt = fmt; \
vaf.va = &args; \
\
__netdev_printk(level, dev, &vaf); \
\
va_end(args); \
} \
EXPORT_SYMBOL(func);
define_netdev_printk_level(netdev_emerg, KERN_EMERG);
define_netdev_printk_level(netdev_alert, KERN_ALERT);
define_netdev_printk_level(netdev_crit, KERN_CRIT);
define_netdev_printk_level(netdev_err, KERN_ERR);
define_netdev_printk_level(netdev_warn, KERN_WARNING);
define_netdev_printk_level(netdev_notice, KERN_NOTICE);
define_netdev_printk_level(netdev_info, KERN_INFO);
static void __net_exit netdev_exit(struct net *net)
{
kfree(net->dev_name_head);
kfree(net->dev_index_head);
xa_destroy(&net->dev_by_index);
if (net != &init_net)
WARN_ON_ONCE(!list_empty(&net->dev_base_head));
}
static struct pernet_operations __net_initdata netdev_net_ops = {
.init = netdev_init,
.exit = netdev_exit,
};
static void __net_exit default_device_exit_net(struct net *net)
{
struct netdev_name_node *name_node, *tmp;
struct net_device *dev, *aux;
/*
* Push all migratable network devices back to the
* initial network namespace
*/
ASSERT_RTNL();
for_each_netdev_safe(net, dev, aux) {
int err;
char fb_name[IFNAMSIZ];
/* Ignore unmoveable devices (i.e. loopback) */
if (dev->netns_immutable)
continue;
/* Leave virtual devices for the generic cleanup */
if (dev->rtnl_link_ops && !dev->rtnl_link_ops->netns_refund)
continue;
/* Push remaining network devices to init_net */
snprintf(fb_name, IFNAMSIZ, "dev%d", dev->ifindex);
if (netdev_name_in_use(&init_net, fb_name))
snprintf(fb_name, IFNAMSIZ, "dev%%d");
netdev_for_each_altname_safe(dev, name_node, tmp)
if (netdev_name_in_use(&init_net, name_node->name))
__netdev_name_node_alt_destroy(name_node);
err = dev_change_net_namespace(dev, &init_net, fb_name);
if (err) {
pr_emerg("%s: failed to move %s to init_net: %d\n",
__func__, dev->name, err);
BUG();
}
}
}
static void __net_exit default_device_exit_batch(struct list_head *net_list)
{
/* At exit all network devices most be removed from a network
* namespace. Do this in the reverse order of registration.
* Do this across as many network namespaces as possible to
* improve batching efficiency.
*/
struct net_device *dev;
struct net *net;
LIST_HEAD(dev_kill_list);
rtnl_lock();
list_for_each_entry(net, net_list, exit_list) {
default_device_exit_net(net);
cond_resched();
}
list_for_each_entry(net, net_list, exit_list) {
for_each_netdev_reverse(net, dev) {
if (dev->rtnl_link_ops && dev->rtnl_link_ops->dellink)
dev->rtnl_link_ops->dellink(dev, &dev_kill_list);
else
unregister_netdevice_queue(dev, &dev_kill_list);
}
}
unregister_netdevice_many(&dev_kill_list);
rtnl_unlock();
}
static struct pernet_operations __net_initdata default_device_ops = {
.exit_batch = default_device_exit_batch,
};
static void __init net_dev_struct_check(void)
{
/* TX read-mostly hotpath */
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, priv_flags_fast);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, netdev_ops);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, header_ops);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, _tx);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, real_num_tx_queues);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, gso_max_size);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, gso_ipv4_max_size);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, gso_max_segs);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, gso_partial_features);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, num_tc);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, mtu);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, needed_headroom);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, tc_to_txq);
#ifdef CONFIG_XPS
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, xps_maps);
#endif
#ifdef CONFIG_NETFILTER_EGRESS
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, nf_hooks_egress);
#endif
#ifdef CONFIG_NET_XGRESS
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, tcx_egress);
#endif
CACHELINE_ASSERT_GROUP_SIZE(struct net_device, net_device_read_tx, 160);
/* TXRX read-mostly hotpath */
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_txrx, lstats);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_txrx, state);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_txrx, flags);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_txrx, hard_header_len);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_txrx, features);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_txrx, ip6_ptr);
CACHELINE_ASSERT_GROUP_SIZE(struct net_device, net_device_read_txrx, 46);
/* RX read-mostly hotpath */
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, ptype_specific);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, ifindex);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, real_num_rx_queues);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, _rx);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, gro_max_size);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, gro_ipv4_max_size);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, rx_handler);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, rx_handler_data);
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, nd_net);
#ifdef CONFIG_NETPOLL
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, npinfo);
#endif
#ifdef CONFIG_NET_XGRESS
CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, tcx_ingress);
#endif
CACHELINE_ASSERT_GROUP_SIZE(struct net_device, net_device_read_rx, 92);
}
/*
* Initialize the DEV module. At boot time this walks the device list and
* unhooks any devices that fail to initialise (normally hardware not
* present) and leaves us with a valid list of present and active devices.
*
*/
/* We allocate 256 pages for each CPU if PAGE_SHIFT is 12 */
#define SYSTEM_PERCPU_PAGE_POOL_SIZE ((1 << 20) / PAGE_SIZE)
static int net_page_pool_create(int cpuid)
{
#if IS_ENABLED(CONFIG_PAGE_POOL)
struct page_pool_params page_pool_params = {
.pool_size = SYSTEM_PERCPU_PAGE_POOL_SIZE,
.flags = PP_FLAG_SYSTEM_POOL,
.nid = cpu_to_mem(cpuid),
};
struct page_pool *pp_ptr;
int err;
pp_ptr = page_pool_create_percpu(&page_pool_params, cpuid);
if (IS_ERR(pp_ptr))
return -ENOMEM;
err = xdp_reg_page_pool(pp_ptr);
if (err) {
page_pool_destroy(pp_ptr);
return err;
}
per_cpu(system_page_pool.pool, cpuid) = pp_ptr;
#endif
return 0;
}
static int backlog_napi_should_run(unsigned int cpu)
{
struct softnet_data *sd = per_cpu_ptr(&softnet_data, cpu);
struct napi_struct *napi = &sd->backlog;
return test_bit(NAPI_STATE_SCHED_THREADED, &napi->state);
}
static void run_backlog_napi(unsigned int cpu)
{
struct softnet_data *sd = per_cpu_ptr(&softnet_data, cpu);
napi_threaded_poll_loop(&sd->backlog, false);
}
static void backlog_napi_setup(unsigned int cpu)
{
struct softnet_data *sd = per_cpu_ptr(&softnet_data, cpu);
struct napi_struct *napi = &sd->backlog;
napi->thread = this_cpu_read(backlog_napi);
set_bit(NAPI_STATE_THREADED, &napi->state);
}
static struct smp_hotplug_thread backlog_threads = {
.store = &backlog_napi,
.thread_should_run = backlog_napi_should_run,
.thread_fn = run_backlog_napi,
.thread_comm = "backlog_napi/%u",
.setup = backlog_napi_setup,
};
/*
* This is called single threaded during boot, so no need
* to take the rtnl semaphore.
*/
static int __init net_dev_init(void)
{
int i, rc = -ENOMEM;
BUG_ON(!dev_boot_phase);
net_dev_struct_check();
if (dev_proc_init())
goto out;
if (netdev_kobject_init())
goto out;
for (i = 0; i < PTYPE_HASH_SIZE; i++)
INIT_LIST_HEAD(&ptype_base[i]);
if (register_pernet_subsys(&netdev_net_ops))
goto out;
/*
* Initialise the packet receive queues.
*/
flush_backlogs_fallback = flush_backlogs_alloc();
if (!flush_backlogs_fallback)
goto out;
for_each_possible_cpu(i) {
struct softnet_data *sd = &per_cpu(softnet_data, i);
skb_queue_head_init(&sd->input_pkt_queue);
skb_queue_head_init(&sd->process_queue);
#ifdef CONFIG_XFRM_OFFLOAD
skb_queue_head_init(&sd->xfrm_backlog);
#endif
INIT_LIST_HEAD(&sd->poll_list);
sd->output_queue_tailp = &sd->output_queue;
#ifdef CONFIG_RPS
INIT_CSD(&sd->csd, rps_trigger_softirq, sd);
sd->cpu = i;
#endif
INIT_CSD(&sd->defer_csd, trigger_rx_softirq, sd);
gro_init(&sd->backlog.gro);
sd->backlog.poll = process_backlog;
sd->backlog.weight = weight_p;
INIT_LIST_HEAD(&sd->backlog.poll_list);
if (net_page_pool_create(i))
goto out;
}
net_hotdata.skb_defer_nodes =
__alloc_percpu(sizeof(struct skb_defer_node) * nr_node_ids,
__alignof__(struct skb_defer_node));
if (!net_hotdata.skb_defer_nodes)
goto out;
if (use_backlog_threads())
smpboot_register_percpu_thread(&backlog_threads);
dev_boot_phase = 0;
/* The loopback device is special if any other network devices
* is present in a network namespace the loopback device must
* be present. Since we now dynamically allocate and free the
* loopback device ensure this invariant is maintained by
* keeping the loopback device as the first device on the
* list of network devices. Ensuring the loopback devices
* is the first device that appears and the last network device
* that disappears.
*/
if (register_pernet_device(&loopback_net_ops))
goto out;
if (register_pernet_device(&default_device_ops))
goto out;
open_softirq(NET_TX_SOFTIRQ, net_tx_action);
open_softirq(NET_RX_SOFTIRQ, net_rx_action);
rc = cpuhp_setup_state_nocalls(CPUHP_NET_DEV_DEAD, "net/dev:dead",
NULL, dev_cpu_dead);
WARN_ON(rc < 0);
rc = 0;
/* avoid static key IPIs to isolated CPUs */
if (housekeeping_enabled(HK_TYPE_MISC))
net_enable_timestamp();
out:
if (rc < 0) {
for_each_possible_cpu(i) {
struct page_pool *pp_ptr;
pp_ptr = per_cpu(system_page_pool.pool, i);
if (!pp_ptr)
continue;
xdp_unreg_page_pool(pp_ptr);
page_pool_destroy(pp_ptr);
per_cpu(system_page_pool.pool, i) = NULL;
}
}
return rc;
}
subsys_initcall(net_dev_init);
]
|
| 12/2 |
2026/05/21 11:34 |
llm |
expert |
0m
Model:
Tokens:
input: 172539
output: 21
thoughts: 1261
Thoughts: **Analyzing Network Device Unregistration in `net/core/dev.c`**
Okay, so I'm digging into the network device unregistration process, specifically within `net/core/dev.c`. I'm focusing on the interaction between closing a device and the watchdog timer, trying to understand a specific scenario where the watchdog timer might still be running during unregistration.
First, I see the standard flow: `unregister_netdevice_many_notify` is iterating through a list of devices. Before a device is fully shut down and unregistered, there's a critical sequence. The device has `dev_shutdown` called, but crucial to this analysis is the fact that prior to that call, the system executes `netif_close_many`, which calls `__dev_close_many`.
`__dev_close_many` handles several important steps: notifying the system the device is going down (`NETDEV_GOING_DOWN`), clearing the `__LINK_STATE_START` bit in the `dev->state`, and then synchronizing any scheduled polls. Then, and this is key, it executes the device's specific `ndo_stop` operation, via `ops->ndo_stop(dev)`. This should, in turn, disable any associated NAPI instances. And finally, after calling `ndo_stop`, `netif_set_up` is set to false. After these calls, and before further operations, we have the critical call to `dev_deactivate_many`.
Inside `dev_deactivate_many`, I see that `netdev_watchdog_down(dev)` is called. This is interesting. But what happens if the device wasn't UP to begin with? The original code checks `dev->flags & IFF_UP`, meaning that the system is trying to close devices that were UP, before unregistering the devices.
My question then becomes: can the watchdog timer still be running if a device isn't up? If a device wasn't UP (i.e., `IFF_UP` is false), then `netif_close_many` is skipped entirely for that device! However, `dev_shutdown` checks `WARN_ON(timer_pending(&dev->watchdog_timer));`. So, if the device wasn't UP, does that mean `dev_shutdown` will still warn about a pending timer?
This has to be impossible because, if a device is not UP, the watchdog timer would have to have been started somewhere else. The only other place is `dev_activate`. If the device was not activated (`dev_activate` was not called), or if `dev_deactivate` was called, `netdev_watchdog_down` will be called. So, the only way that `dev->watchdog_timer` is running is if `dev` was UP. In other words, `netif_close_many` must be called to handle `netdev_watchdog_down`.
So, the issue isn't a race with devices that aren't properly closed, but the problem must occur if the unregistration process doesn't see that the device is UP. So, the `IFF_UP` flag is checked, but perhaps it's a timing issue.
Let's double-check the logic in `netdev_watchdog_up` just to be sure...
|
| 13/2 |
2026/05/21 11:34 |
tool |
grepper |
0m
Args: map[Expression:netdev_watchdog_up]
Results: map[Output:drivers/net/ethernet/freescale/ucc_geth.c=1430=static void ugeth_activate(struct ucc_geth_private *ugeth)
--
drivers/net/ethernet/freescale/ucc_geth.c-1436- netif_tx_wake_all_queues(ugeth->ndev);
drivers/net/ethernet/freescale/ucc_geth.c:1437: netdev_watchdog_up(ugeth->ndev);
drivers/net/ethernet/freescale/ucc_geth.c-1438-}
--
include/linux/netdevice.h=4514=unsigned long dev_trans_start(struct net_device *dev);
include/linux/netdevice.h-4515-
include/linux/netdevice.h:4516:void netdev_watchdog_up(struct net_device *dev);
include/linux/netdevice.h-4517-
--
net/core/dev.c=3476=void netif_device_attach(struct net_device *dev)
--
net/core/dev.c-3480- netif_tx_wake_all_queues(dev);
net/core/dev.c:3481: netdev_watchdog_up(dev);
net/core/dev.c-3482- }
--
net/sched/sch_generic.c=500=static void dev_watchdog(struct timer_list *t)
--
net/sched/sch_generic.c-557-
net/sched/sch_generic.c:558:void netdev_watchdog_up(struct net_device *dev)
net/sched/sch_generic.c-559-{
--
net/sched/sch_generic.c-568-}
net/sched/sch_generic.c:569:EXPORT_SYMBOL_GPL(netdev_watchdog_up);
net/sched/sch_generic.c-570-
--
net/sched/sch_generic.c=585=void netif_carrier_on(struct net_device *dev)
--
net/sched/sch_generic.c-592- if (netif_running(dev))
net/sched/sch_generic.c:593: netdev_watchdog_up(dev);
net/sched/sch_generic.c-594- }
--
net/sched/sch_generic.c=1241=void dev_activate(struct net_device *dev)
--
net/sched/sch_generic.c-1263- netif_trans_update(dev);
net/sched/sch_generic.c:1264: netdev_watchdog_up(dev);
net/sched/sch_generic.c-1265- }
]
|
| 14/2 |
2026/05/21 11:34 |
llm |
expert |
0m
Model:
Tokens:
input: 175259
output: 21
thoughts: 2405
Thoughts: **Analysis of `netdev_watchdog_up` and `netdev_watchdog_down` Calls**
Okay, so I'm digging into how the network watchdog timer is managed, specifically focusing on when `netdev_watchdog_up` and `netdev_watchdog_down` are called. Let's break this down systematically.
First, I see `netdev_watchdog_up` is called from `netif_device_attach`, `netif_carrier_on`, and `dev_activate`. That's the starting point.
`netif_device_attach` only calls it if the device is running (IFF_UP). Seems straightforward, so far so good. `netif_carrier_on` also checks `netif_running(dev)` (which is `IFF_UP`). This is good; there's consistency there. `dev_activate` seems a little more complex, but again, from what I can see here it eventually checks if the device can transmit.
The crucial question is, can there be a situation where a device's UP status is out of sync with whether or not its watchdog timer is running? Can we get a false positive?
Let's look at the unregistration path, since that's where things can get interesting. The code uses `unregister_netdevice_many_notify`. In this function, devices on the `unreg_list` are iterated through to close them down. The initial loop includes a check for `IFF_UP`, and if UP, the close path is taken. However, devices are added to the list and removed in the path to calling `netif_close_many`.
Initially, it seemed a device might get added to `close_head` even if not `IFF_UP`, but a further check of `netif_close_many` shows that it removes devices that don't need to be closed (`IFF_UP` is not set) before calling `__dev_close_many`. So, only devices that were UP are passed to `__dev_close_many`. And, importantly, `__dev_close_many` calls `dev_deactivate_many` which eventually calls `netdev_watchdog_down`.
But here's a wrinkle: What if `IFF_UP` gets cleared *before* `dev_deactivate` is called, leaving the watchdog timer running unnecessarily? Where does `IFF_UP` get cleared? It's cleared in `__dev_change_flags`.
The critical point is that `__dev_close` *eventually* calls `__dev_close_many` after `IFF_UP` is cleared. And, `__dev_close_many` handles the call to `dev_deactivate_many` and eventually `netdev_watchdog_down`.
However, the tricky part is to confirm the exact execution path. It appears that `__dev_close` calls `__dev_close_many` directly, *bypassing* the potential filtering in `netif_close_many`. My reading of the code reveals that `__dev_close` *always* calls `__dev_close_many`. So, even if `IFF_UP` is cleared before that call, `__dev_close_many` handles the call.
In the case of calling `netif_close` itself, it does a check to see if the device is `IFF_UP`, so that can't trigger a watchdog event on a down device.
So, in the common case, the watchdog should be consistently brought down when a device is closed, as the check on `IFF_UP` is consistent. The only place `IFF_UP` is cleared, is through a call to `__dev_change_flags` that calls `__dev_close` if it is changing UP/DOWN. This leads to the correct flow of events.
|
| 15/2 |
2026/05/21 11:34 |
tool |
grepper |
0m
Args: map[Expression:netdev_watchdog_up]
Results: map[Output:drivers/net/ethernet/freescale/ucc_geth.c=1430=static void ugeth_activate(struct ucc_geth_private *ugeth)
--
drivers/net/ethernet/freescale/ucc_geth.c-1436- netif_tx_wake_all_queues(ugeth->ndev);
drivers/net/ethernet/freescale/ucc_geth.c:1437: netdev_watchdog_up(ugeth->ndev);
drivers/net/ethernet/freescale/ucc_geth.c-1438-}
--
include/linux/netdevice.h=4514=unsigned long dev_trans_start(struct net_device *dev);
include/linux/netdevice.h-4515-
include/linux/netdevice.h:4516:void netdev_watchdog_up(struct net_device *dev);
include/linux/netdevice.h-4517-
--
net/core/dev.c=3476=void netif_device_attach(struct net_device *dev)
--
net/core/dev.c-3480- netif_tx_wake_all_queues(dev);
net/core/dev.c:3481: netdev_watchdog_up(dev);
net/core/dev.c-3482- }
--
net/sched/sch_generic.c=500=static void dev_watchdog(struct timer_list *t)
--
net/sched/sch_generic.c-557-
net/sched/sch_generic.c:558:void netdev_watchdog_up(struct net_device *dev)
net/sched/sch_generic.c-559-{
--
net/sched/sch_generic.c-568-}
net/sched/sch_generic.c:569:EXPORT_SYMBOL_GPL(netdev_watchdog_up);
net/sched/sch_generic.c-570-
--
net/sched/sch_generic.c=585=void netif_carrier_on(struct net_device *dev)
--
net/sched/sch_generic.c-592- if (netif_running(dev))
net/sched/sch_generic.c:593: netdev_watchdog_up(dev);
net/sched/sch_generic.c-594- }
--
net/sched/sch_generic.c=1241=void dev_activate(struct net_device *dev)
--
net/sched/sch_generic.c-1263- netif_trans_update(dev);
net/sched/sch_generic.c:1264: netdev_watchdog_up(dev);
net/sched/sch_generic.c-1265- }
]
|
| 16/2 |
2026/05/21 11:34 |
llm |
expert |
0m
Model:
Tokens:
input: 179237
output: 21
thoughts: 1881
Thoughts: **Analyzing Potential Race Conditions with Network Device Watchdog Timers During Unregistration**
Okay, so I'm digging into how network devices are unregistered and the interaction with their watchdog timers. My initial concern is this: what if `netdev_watchdog_up` gets called, but `dev_deactivate_many` *isn't* called afterwards? I need to understand when `dev_deactivate_many` is called. It seems to be related to `__dev_close_many` and `dev_deactivate`. I also found that `dev_deactivate` is called by `dev_qdisc_change_tx_queue_len`, which in turn, can be triggered when a device's TX queue length changes. This seems potentially problematic.
Now, let's look at `unregister_netdevice_many_notify`. This is where things get interesting. It appears that if a device is up (`dev->flags & IFF_UP`), it’s added to a close list (`close_head`). Critically, the logic *appears* to handle devices with and without an operational lock (`netdev_need_ops_lock`). The code iterates twice, and in the second loop, it *doesn't* check the `IFF_UP` flag. This means if a device isn't up, but doesn't require an operational lock, it *still* gets added to the close list.
The next call is `netif_close_many`, and this is where things are cleared up slightly. However, inside `netif_close_many`, there's a crucial step: devices that *aren't* up (`!(dev->flags & IFF_UP)`) are *removed* from the close list. Therefore, `__dev_close_many` isn't called for devices that aren't up, meaning that `dev_deactivate_many` isn't called, and crucially, `netdev_watchdog_down` is *also* not called.
The problem arises: if a device isn't `IFF_UP`, but its watchdog timer is *still pending*, and if it gets unregistered, the subsequent `dev_shutdown(dev)` *will* trigger the `WARN_ON(timer_pending(&dev->watchdog_timer))` assertion. So, I have to figure out how a device ends up in that state: not up, but with a pending watchdog timer.
Let’s retrace the calls of `netdev_watchdog_up`. I found that it's called in:
1. `netif_device_attach`: This is called if a device is running which is `IFF_UP`, so no problem here.
2. `netif_carrier_on`: Same check as the previous call, the device has to be running, so `IFF_UP`.
3. `dev_activate`: This is called from `__dev_open` and `dev_qdisc_change_tx_queue_len`. And the second function checks the `IFF_UP` flag.
I then realized I had to consider the watchdog itself. Could the watchdog timer itself re-arm itself, regardless of `IFF_UP`? I had to check that `dev_watchdog` itself. It *does* re-arm the timer, but importantly, it explicitly checks `netif_running(dev)`. If the device isn't running (i.e., `IFF_UP` is clear), the timer is *not* re-armed.
So, where else might `netdev_watchdog_up` be called? I will check other potential callers for this function, such as `drivers/net/usb/usbnet.c`.
|
| 17/2 |
2026/05/21 11:35 |
tool |
grepper |
0m
Args: map[Expression:netdev_watchdog_up]
Results: map[Output:drivers/net/ethernet/freescale/ucc_geth.c=1430=static void ugeth_activate(struct ucc_geth_private *ugeth)
--
drivers/net/ethernet/freescale/ucc_geth.c-1436- netif_tx_wake_all_queues(ugeth->ndev);
drivers/net/ethernet/freescale/ucc_geth.c:1437: netdev_watchdog_up(ugeth->ndev);
drivers/net/ethernet/freescale/ucc_geth.c-1438-}
--
include/linux/netdevice.h=4514=unsigned long dev_trans_start(struct net_device *dev);
include/linux/netdevice.h-4515-
include/linux/netdevice.h:4516:void netdev_watchdog_up(struct net_device *dev);
include/linux/netdevice.h-4517-
--
net/core/dev.c=3476=void netif_device_attach(struct net_device *dev)
--
net/core/dev.c-3480- netif_tx_wake_all_queues(dev);
net/core/dev.c:3481: netdev_watchdog_up(dev);
net/core/dev.c-3482- }
--
net/sched/sch_generic.c=500=static void dev_watchdog(struct timer_list *t)
--
net/sched/sch_generic.c-557-
net/sched/sch_generic.c:558:void netdev_watchdog_up(struct net_device *dev)
net/sched/sch_generic.c-559-{
--
net/sched/sch_generic.c-568-}
net/sched/sch_generic.c:569:EXPORT_SYMBOL_GPL(netdev_watchdog_up);
net/sched/sch_generic.c-570-
--
net/sched/sch_generic.c=585=void netif_carrier_on(struct net_device *dev)
--
net/sched/sch_generic.c-592- if (netif_running(dev))
net/sched/sch_generic.c:593: netdev_watchdog_up(dev);
net/sched/sch_generic.c-594- }
--
net/sched/sch_generic.c=1241=void dev_activate(struct net_device *dev)
--
net/sched/sch_generic.c-1263- netif_trans_update(dev);
net/sched/sch_generic.c:1264: netdev_watchdog_up(dev);
net/sched/sch_generic.c-1265- }
]
|
| 18/2 |
2026/05/21 11:35 |
llm |
expert |
0m
Model:
Tokens:
input: 182654
output: 18
thoughts: 714
Thoughts: **Investigating a Race Condition in Network Device Shutdown**
Alright, so I'm trying to figure out how `timer_pending(&dev->watchdog_timer)` can be true after `dev_shutdown` is called. The assumption is that `netdev_watchdog_up` is only called when the device is up, so how is this possible?
I initially thought maybe `netif_close_many` was missing a call to `netdev_watchdog_down` when the device is still technically UP, but not running. But no, let's double-check `netif_close_many`.
Okay, I see that `netif_close_many` calls `__dev_close_many`, which then calls `dev_deactivate_many`. This in turn does call `netdev_watchdog_down`, as expected. Perfect, so `netdev_watchdog_down` is definitely being called during the close-down.
Let me refresh my memory on what `netdev_watchdog_down` actually does. Ah, it uses `timer_delete`, and then checks the return, which tells us if the timer was pending at the time. `timer_delete` deletes the timer, and we're also releasing the tracker here.
Now, wait a minute... I need to consider `timer_delete`'s semantics. Is it synchronous? No, it's not the synchronous one, it's NOT `timer_delete_sync`. Okay, let me remind myself of the difference... `timer_delete` is really just an alias for `del_timer`, and therefore, it *doesn't* wait for the timer handler to finish if it's running on another CPU. That's a key distinction. The one that waits is `del_timer_sync`.
I'll check the header file to confirm my understanding of the timer functions. This confirms that `timer_delete` doesn't provide the same guarantees. This could definitely be where the race condition occurs if the watchdog timer is running on another CPU when `netdev_watchdog_down` is called.
|
| 19/2 |
2026/05/21 11:35 |
tool |
grepper |
0m
Error:
Args: map[Expression:define timer_delete]
Results:
|
| 20/2 |
2026/05/21 11:35 |
llm |
expert |
0m
Model:
Tokens:
input: 183850
output: 17
thoughts: 20
Thoughts: Okay, here's my summary, taking the perspective of someone familiar with kernel internals:
**Investigating the `timer_delete` Implementation**
Alright, time to get to the bottom of this. I need to understand how `timer_delete` works, so I'm going to start by going straight to the source. Specifically, I'm going to jump into the `include/linux/timer.h` file. I know that's where the primary declarations for timer-related functions and data structures reside within the kernel headers, which is the best place to start. I'll use `grep` to quickly search for the symbol `timer_delete` within that file. This should give me a quick look at the function declaration, any associated inline functions or macros, and potentially where the function is defined or at least where it's used. This should act as a useful first pass in forming an accurate mental model of how this function operates in the kernel. This is my preferred, and frankly, fastest method when digging into kernel code.
|
| 21/2 |
2026/05/21 11:35 |
tool |
grepper |
0m
Args: map[Expression:timer_delete]
Results: map[Output:
Full output is too long, showing 500 out of 9401 lines.
Use more precise expression if possible.
[Documentation/core-api/real-time/differences.rst=196=eventually succeed as that CPU makes progress.
Documentation/core-api/real-time/differences.rst-197-
Documentation/core-api/real-time/differences.rst:198:Some examples are hrtimer_cancel() or timer_delete_sync(). These functions
Documentation/core-api/real-time/differences.rst-199-cancel timers that execute with interrupts or soft interrupts disabled. If a
--
Documentation/kernel-hacking/locking.rst=962=you might do the following::
--
Documentation/kernel-hacking/locking.rst-969- struct foo *next = list->next;
Documentation/kernel-hacking/locking.rst:970: timer_delete(&list->timer);
Documentation/kernel-hacking/locking.rst-971- kfree(list);
--
Documentation/kernel-hacking/locking.rst=983=This can be avoided by checking the result of
Documentation/kernel-hacking/locking.rst:984:timer_delete(): if it returns 1, the timer has been deleted.
Documentation/kernel-hacking/locking.rst-985-If 0, it means (in this case) that it is currently running, so we can
Documentation/kernel-hacking/locking.rst=986=do::
--
Documentation/kernel-hacking/locking.rst-992- struct foo *next = list->next;
Documentation/kernel-hacking/locking.rst:993: if (!timer_delete(&list->timer)) {
Documentation/kernel-hacking/locking.rst-994- /* Give timer a chance to delete this */
--
Documentation/kernel-hacking/locking.rst=1007=Because this is a fairly common case which is prone to races, you should
Documentation/kernel-hacking/locking.rst:1008:use timer_delete_sync() (``include/linux/timer.h``) to handle this case.
Documentation/kernel-hacking/locking.rst-1009-
--
Documentation/kernel-hacking/locking.rst=1335=lock.
--
Documentation/kernel-hacking/locking.rst-1340-
Documentation/kernel-hacking/locking.rst:1341:- add_timer() and timer_delete()
Documentation/kernel-hacking/locking.rst-1342-
--
Documentation/timers/hrtimers.rst=120=hrtimer_try_to_cancel() and hrtimer_cancel() [which are roughly
Documentation/timers/hrtimers.rst:121:equivalent to timer_delete() and timer_delete_sync()] - so there's no direct
Documentation/timers/hrtimers.rst-122-1:1 mapping between them on the algorithmic level, and thus no real
--
Documentation/translations/it_IT/kernel-hacking/locking.rst=985=potreste fare come segue::
--
Documentation/translations/it_IT/kernel-hacking/locking.rst-992- struct foo *next = list->next;
Documentation/translations/it_IT/kernel-hacking/locking.rst:993: timer_delete(&list->timer);
Documentation/translations/it_IT/kernel-hacking/locking.rst-994- kfree(list);
--
Documentation/translations/it_IT/kernel-hacking/locking.rst=1005=Questo può essere evitato controllando il valore di ritorno di
Documentation/translations/it_IT/kernel-hacking/locking.rst:1006:timer_delete(): se ritorna 1, il temporizzatore è stato già
Documentation/translations/it_IT/kernel-hacking/locking.rst-1007-rimosso. Se 0, significa (in questo caso) che il temporizzatore è in
Documentation/translations/it_IT/kernel-hacking/locking.rst=1008=esecuzione, quindi possiamo fare come segue::
--
Documentation/translations/it_IT/kernel-hacking/locking.rst-1014- struct foo *next = list->next;
Documentation/translations/it_IT/kernel-hacking/locking.rst:1015: if (!timer_delete(&list->timer)) {
Documentation/translations/it_IT/kernel-hacking/locking.rst-1016- /* Give timer a chance to delete this */
--
Documentation/translations/it_IT/kernel-hacking/locking.rst=1028=Dato che questo è un problema abbastanza comune con una propensione
Documentation/translations/it_IT/kernel-hacking/locking.rst:1029:alle corse critiche, dovreste usare timer_delete_sync()
Documentation/translations/it_IT/kernel-hacking/locking.rst-1030-(``include/linux/timer.h``) per gestire questo caso.
--
Documentation/translations/it_IT/kernel-hacking/locking.rst=1374=contesto, o trattenendo un qualsiasi *lock*.
--
Documentation/translations/it_IT/kernel-hacking/locking.rst-1379-
Documentation/translations/it_IT/kernel-hacking/locking.rst:1380:- add_timer() e timer_delete()
Documentation/translations/it_IT/kernel-hacking/locking.rst-1381-
--
arch/alpha/kernel/srmcons.c=171=srmcons_close(struct tty_struct *tty, struct file *filp)
--
arch/alpha/kernel/srmcons.c-180- port->tty = NULL;
arch/alpha/kernel/srmcons.c:181: timer_delete(&srmconsp->timer);
arch/alpha/kernel/srmcons.c-182- }
--
arch/alpha/kernel/syscalls/syscall.tbl-348-417 common timer_getoverrun sys_timer_getoverrun
arch/alpha/kernel/syscalls/syscall.tbl:349:418 common timer_delete sys_timer_delete
arch/alpha/kernel/syscalls/syscall.tbl-350-419 common clock_settime sys_clock_settime
--
arch/arm/mach-footbridge/dc21285.c=136=static void dc21285_enable_error(struct timer_list *timer)
arch/arm/mach-footbridge/dc21285.c-137-{
arch/arm/mach-footbridge/dc21285.c:138: timer_delete(timer);
arch/arm/mach-footbridge/dc21285.c-139-
--
arch/arm/mach-pxa/sharpsl_pm.c=891=static void sharpsl_pm_remove(struct platform_device *pdev)
--
arch/arm/mach-pxa/sharpsl_pm.c-915-
arch/arm/mach-pxa/sharpsl_pm.c:916: timer_delete_sync(&sharpsl_pm.chrg_full_timer);
arch/arm/mach-pxa/sharpsl_pm.c:917: timer_delete_sync(&sharpsl_pm.ac_timer);
arch/arm/mach-pxa/sharpsl_pm.c-918-}
--
arch/arm/tools/syscall.tbl-278-260 common timer_getoverrun sys_timer_getoverrun
arch/arm/tools/syscall.tbl:279:261 common timer_delete sys_timer_delete
arch/arm/tools/syscall.tbl-280-262 common clock_settime sys_clock_settime32
--
arch/arm64/tools/syscall_32.tbl-275-260 common timer_getoverrun sys_timer_getoverrun
arch/arm64/tools/syscall_32.tbl:276:261 common timer_delete sys_timer_delete
arch/arm64/tools/syscall_32.tbl-277-262 common clock_settime sys_clock_settime32
--
arch/m68k/amiga/amisound.c=73=void amiga_mksound( unsigned int hz, unsigned int ticks )
--
arch/m68k/amiga/amisound.c-80- local_irq_save(flags);
arch/m68k/amiga/amisound.c:81: timer_delete(&sound_timer);
arch/m68k/amiga/amisound.c-82-
--
arch/m68k/kernel/syscalls/syscall.tbl-267-257 common timer_getoverrun sys_timer_getoverrun
arch/m68k/kernel/syscalls/syscall.tbl:268:258 common timer_delete sys_timer_delete
arch/m68k/kernel/syscalls/syscall.tbl-269-259 common clock_settime sys_clock_settime32
--
arch/m68k/mac/macboing.c=157=void mac_mksound( unsigned int freq, unsigned int length )
--
arch/m68k/mac/macboing.c-185-
arch/m68k/mac/macboing.c:186: timer_delete(&mac_sound_timer);
arch/m68k/mac/macboing.c-187-
--
arch/m68k/mac/macboing.c=266=static void mac_quadra_ring_bell(struct timer_list *unused)
--
arch/m68k/mac/macboing.c-279-
arch/m68k/mac/macboing.c:280: timer_delete(&mac_sound_timer);
arch/m68k/mac/macboing.c-281-
--
arch/microblaze/kernel/syscalls/syscall.tbl-272-262 common timer_getoverrun sys_timer_getoverrun
arch/microblaze/kernel/syscalls/syscall.tbl:273:263 common timer_delete sys_timer_delete
arch/microblaze/kernel/syscalls/syscall.tbl-274-264 common clock_settime sys_clock_settime32
--
arch/mips/kernel/syscalls/syscall_n32.tbl-234-223 n32 timer_getoverrun sys_timer_getoverrun
arch/mips/kernel/syscalls/syscall_n32.tbl:235:224 n32 timer_delete sys_timer_delete
arch/mips/kernel/syscalls/syscall_n32.tbl-236-225 n32 clock_settime sys_clock_settime32
--
arch/mips/kernel/syscalls/syscall_n64.tbl-230-219 n64 timer_getoverrun sys_timer_getoverrun
arch/mips/kernel/syscalls/syscall_n64.tbl:231:220 n64 timer_delete sys_timer_delete
arch/mips/kernel/syscalls/syscall_n64.tbl-232-221 n64 clock_settime sys_clock_settime
--
arch/mips/kernel/syscalls/syscall_o32.tbl-274-260 o32 timer_getoverrun sys_timer_getoverrun
arch/mips/kernel/syscalls/syscall_o32.tbl:275:261 o32 timer_delete sys_timer_delete
arch/mips/kernel/syscalls/syscall_o32.tbl-276-262 o32 clock_settime sys_clock_settime32
--
arch/mips/sgi-ip22/ip22-reset.c=99=static void debounce(struct timer_list *unused)
arch/mips/sgi-ip22/ip22-reset.c-100-{
arch/mips/sgi-ip22/ip22-reset.c:101: timer_delete(&debounce_timer);
arch/mips/sgi-ip22/ip22-reset.c-102- if (sgint->istat1 & SGINT_ISTAT1_PWR) {
--
arch/parisc/kernel/syscalls/syscall.tbl-284-253 common timer_getoverrun sys_timer_getoverrun
arch/parisc/kernel/syscalls/syscall.tbl:285:254 common timer_delete sys_timer_delete
arch/parisc/kernel/syscalls/syscall.tbl-286-255 32 clock_settime sys_clock_settime32
--
arch/powerpc/kernel/syscalls/syscall.tbl-313-243 common timer_getoverrun sys_timer_getoverrun
arch/powerpc/kernel/syscalls/syscall.tbl:314:244 common timer_delete sys_timer_delete
arch/powerpc/kernel/syscalls/syscall.tbl-315-245 32 clock_settime sys_clock_settime32
--
arch/powerpc/kvm/booke.c=604=static void arm_next_watchdog(struct kvm_vcpu *vcpu)
--
arch/powerpc/kvm/booke.c-624- else
arch/powerpc/kvm/booke.c:625: timer_delete(&vcpu->arch.wdt_timer);
arch/powerpc/kvm/booke.c-626- spin_unlock_irqrestore(&vcpu->arch.wdt_lock, flags);
--
arch/powerpc/kvm/booke.c=1442=void kvmppc_subarch_vcpu_uninit(struct kvm_vcpu *vcpu)
arch/powerpc/kvm/booke.c-1443-{
arch/powerpc/kvm/booke.c:1444: timer_delete_sync(&vcpu->arch.wdt_timer);
arch/powerpc/kvm/booke.c-1445-}
--
arch/powerpc/platforms/cell/spufs/sched.c=505=static void __spu_del_from_rq(struct spu_context *ctx)
--
arch/powerpc/platforms/cell/spufs/sched.c-510- if (!--spu_prio->nr_waiting)
arch/powerpc/platforms/cell/spufs/sched.c:511: timer_delete(&spusched_timer);
arch/powerpc/platforms/cell/spufs/sched.c-512- list_del_init(&ctx->rq);
--
arch/powerpc/platforms/cell/spufs/sched.c=1122=void spu_sched_exit(void)
--
arch/powerpc/platforms/cell/spufs/sched.c-1128-
arch/powerpc/platforms/cell/spufs/sched.c:1129: timer_delete_sync(&spusched_timer);
arch/powerpc/platforms/cell/spufs/sched.c:1130: timer_delete_sync(&spuloadavg_timer);
arch/powerpc/platforms/cell/spufs/sched.c-1131- kthread_stop(spusched_task);
--
arch/powerpc/platforms/powermac/low_i2c.c=344=static irqreturn_t kw_i2c_irq(int irq, void *dev_id)
--
arch/powerpc/platforms/powermac/low_i2c.c-349- spin_lock_irqsave(&host->lock, flags);
arch/powerpc/platforms/powermac/low_i2c.c:350: timer_delete(&host->timeout_timer);
arch/powerpc/platforms/powermac/low_i2c.c-351- kw_i2c_handle_interrupt(host, kw_read_reg(reg_isr));
--
arch/s390/kernel/syscalls/syscall.tbl-213-257 common timer_getoverrun sys_timer_getoverrun
arch/s390/kernel/syscalls/syscall.tbl:214:258 common timer_delete sys_timer_delete
arch/s390/kernel/syscalls/syscall.tbl-215-259 common clock_settime sys_clock_settime
--
arch/s390/kernel/time.c=563=static void stp_work_fn(struct work_struct *work)
--
arch/s390/kernel/time.c-572- chsc_sstpc(stp_page, STP_OP_CTRL, 0x0000, NULL);
arch/s390/kernel/time.c:573: timer_delete_sync(&stp_timer);
arch/s390/kernel/time.c-574- goto out_unlock;
--
arch/s390/mm/cmm.c=200=static void cmm_set_timer(void)
--
arch/s390/mm/cmm.c-203- if (timer_pending(&cmm_timer))
arch/s390/mm/cmm.c:204: timer_delete(&cmm_timer);
arch/s390/mm/cmm.c-205- return;
--
arch/s390/mm/cmm.c=393=static int __init cmm_init(void)
--
arch/s390/mm/cmm.c-426-out_sysctl:
arch/s390/mm/cmm.c:427: timer_delete_sync(&cmm_timer);
arch/s390/mm/cmm.c-428- return rc;
--
arch/s390/mm/cmm.c=432=static void __exit cmm_exit(void)
--
arch/s390/mm/cmm.c-439- kthread_stop(cmm_thread_ptr);
arch/s390/mm/cmm.c:440: timer_delete_sync(&cmm_timer);
arch/s390/mm/cmm.c-441- cmm_free_pages(cmm_pages, &cmm_pages, &cmm_page_list);
--
arch/sh/drivers/pci/common.c=89=static void pcibios_enable_err(struct timer_list *t)
--
arch/sh/drivers/pci/common.c-92-
arch/sh/drivers/pci/common.c:93: timer_delete(&hose->err_timer);
arch/sh/drivers/pci/common.c-94- printk(KERN_DEBUG "PCI: re-enabling error IRQ.\n");
--
arch/sh/drivers/pci/common.c=98=static void pcibios_enable_serr(struct timer_list *t)
--
arch/sh/drivers/pci/common.c-101-
arch/sh/drivers/pci/common.c:102: timer_delete(&hose->serr_timer);
arch/sh/drivers/pci/common.c-103- printk(KERN_DEBUG "PCI: re-enabling system error IRQ.\n");
--
arch/sh/kernel/syscalls/syscall.tbl-272-262 common timer_getoverrun sys_timer_getoverrun
arch/sh/kernel/syscalls/syscall.tbl:273:263 common timer_delete sys_timer_delete
arch/sh/kernel/syscalls/syscall.tbl-274-264 common clock_settime sys_clock_settime32
--
arch/sparc/kernel/led.c=68=static ssize_t led_proc_write(struct file *file, const char __user *buffer,
--
arch/sparc/kernel/led.c-86- */
arch/sparc/kernel/led.c:87: timer_delete_sync(&led_blink_timer);
arch/sparc/kernel/led.c-88-
--
arch/sparc/kernel/led.c=134=static void __exit led_exit(void)
--
arch/sparc/kernel/led.c-136- remove_proc_entry("led", NULL);
arch/sparc/kernel/led.c:137: timer_delete_sync(&led_blink_timer);
arch/sparc/kernel/led.c-138-}
--
arch/sparc/kernel/syscalls/syscall.tbl-323-264 common timer_getoverrun sys_timer_getoverrun
arch/sparc/kernel/syscalls/syscall.tbl:324:265 common timer_delete sys_timer_delete
arch/sparc/kernel/syscalls/syscall.tbl-325-266 common timer_create sys_timer_create compat_sys_timer_create
--
arch/um/drivers/vector_kern.c=1111=static int vector_net_close(struct net_device *dev)
--
arch/um/drivers/vector_kern.c-1115- netif_stop_queue(dev);
arch/um/drivers/vector_kern.c:1116: timer_delete(&vp->tl);
arch/um/drivers/vector_kern.c-1117-
--
arch/x86/entry/syscalls/syscall_32.tbl-277-262 i386 timer_getoverrun sys_timer_getoverrun
arch/x86/entry/syscalls/syscall_32.tbl:278:263 i386 timer_delete sys_timer_delete
arch/x86/entry/syscalls/syscall_32.tbl-279-264 i386 clock_settime sys_clock_settime32
--
arch/x86/entry/syscalls/syscall_64.tbl-237-225 common timer_getoverrun sys_timer_getoverrun
arch/x86/entry/syscalls/syscall_64.tbl:238:226 common timer_delete sys_timer_delete
arch/x86/entry/syscalls/syscall_64.tbl-239-227 common clock_settime sys_clock_settime
--
arch/x86/kernel/apic/vector.c=863=void lapic_offline(void)
--
arch/x86/kernel/apic/vector.c-872- irq_matrix_offline(vector_matrix);
arch/x86/kernel/apic/vector.c:873: WARN_ON_ONCE(timer_delete_sync_try(&cl->timer) < 0);
arch/x86/kernel/apic/vector.c-874- WARN_ON_ONCE(!hlist_empty(&cl->head));
--
arch/x86/kernel/cpu/mce/core.c=1825=void mce_timer_kick(bool storm)
--
arch/x86/kernel/cpu/mce/core.c-1836-
arch/x86/kernel/cpu/mce/core.c:1837:/* Must not be called in IRQ context where timer_delete_sync() can deadlock */
arch/x86/kernel/cpu/mce/core.c:1838:static void mce_timer_delete_all(void)
arch/x86/kernel/cpu/mce/core.c-1839-{
--
arch/x86/kernel/cpu/mce/core.c-1842- for_each_online_cpu(cpu)
arch/x86/kernel/cpu/mce/core.c:1843: timer_delete_sync(&per_cpu(mce_timer, cpu));
arch/x86/kernel/cpu/mce/core.c-1844-}
--
arch/x86/kernel/cpu/mce/core.c=2495=static void mce_restart(void)
arch/x86/kernel/cpu/mce/core.c-2496-{
arch/x86/kernel/cpu/mce/core.c:2497: mce_timer_delete_all();
arch/x86/kernel/cpu/mce/core.c-2498- on_each_cpu(mce_cpu_restart, NULL, 1);
--
arch/x86/kernel/cpu/mce/core.c=2575=static ssize_t set_ignore_ce(struct device *s,
--
arch/x86/kernel/cpu/mce/core.c-2587- /* disable ce features */
arch/x86/kernel/cpu/mce/core.c:2588: mce_timer_delete_all();
arch/x86/kernel/cpu/mce/core.c-2589- on_each_cpu(mce_disable_cmci, NULL, 1);
--
arch/x86/kernel/cpu/mce/core.c=2802=static int mce_cpu_pre_down(unsigned int cpu)
--
arch/x86/kernel/cpu/mce/core.c-2806- mce_disable_cpu();
arch/x86/kernel/cpu/mce/core.c:2807: timer_delete_sync(t);
arch/x86/kernel/cpu/mce/core.c-2808- mce_threshold_remove_device(cpu);
--
arch/x86/kvm/xen.c=1471=static bool kvm_xen_schedop_poll(struct kvm_vcpu *vcpu, bool longmode,
--
arch/x86/kvm/xen.c-1555- if (sched_poll.timeout)
arch/x86/kvm/xen.c:1556: timer_delete(&vcpu->arch.xen.poll_timer);
arch/x86/kvm/xen.c-1557-
--
arch/x86/kvm/xen.c=2313=void kvm_xen_destroy_vcpu(struct kvm_vcpu *vcpu)
--
arch/x86/kvm/xen.c-2322-
arch/x86/kvm/xen.c:2323: timer_delete_sync(&vcpu->arch.xen.poll_timer);
arch/x86/kvm/xen.c-2324-}
--
arch/xtensa/kernel/syscalls/syscall.tbl-269-248 common timer_create sys_timer_create
arch/xtensa/kernel/syscalls/syscall.tbl:270:249 common timer_delete sys_timer_delete
arch/xtensa/kernel/syscalls/syscall.tbl-271-250 common timer_settime sys_timer_settime32
--
arch/xtensa/platforms/iss/console.c=48=static void rs_close(struct tty_struct *tty, struct file * filp)
--
arch/xtensa/platforms/iss/console.c-50- if (tty->count == 1)
arch/xtensa/platforms/iss/console.c:51: timer_delete_sync(&serial_timer);
arch/xtensa/platforms/iss/console.c-52-}
--
arch/xtensa/platforms/iss/network.c=373=static int iss_net_close(struct net_device *dev)
--
arch/xtensa/platforms/iss/network.c-377- netif_stop_queue(dev);
arch/xtensa/platforms/iss/network.c:378: timer_delete_sync(&lp->timer);
arch/xtensa/platforms/iss/network.c-379- lp->tp.net_ops->close(lp);
--
block/blk-core.c=220=void blk_sync_queue(struct request_queue *q)
block/blk-core.c-221-{
block/blk-core.c:222: timer_delete_sync(&q->timeout);
block/blk-core.c-223- cancel_work_sync(&q->timeout_work);
--
block/blk-stat.c=154=void blk_stat_remove_callback(struct request_queue *q,
--
block/blk-stat.c-164-
block/blk-stat.c:165: timer_delete_sync(&cb->timer);
block/blk-stat.c-166-}
--
block/blk-stat.h=149=static inline void blk_stat_deactivate(struct blk_stat_callback *cb)
block/blk-stat.h-150-{
block/blk-stat.h:151: timer_delete_sync(&cb->timer);
block/blk-stat.h-152-}
--
block/blk-throttle.c=356=static void throtl_pd_free(struct blkg_policy_data *pd)
--
block/blk-throttle.c-359-
block/blk-throttle.c:360: timer_delete_sync(&tg->service_queue.pending_timer);
block/blk-throttle.c-361- blkg_rwstat_exit(&tg->stat_bytes);
--
block/blk-throttle.c=1824=void blk_throtl_exit(struct gendisk *disk)
--
block/blk-throttle.c-1834-
block/blk-throttle.c:1835: timer_delete_sync(&q->td->service_queue.pending_timer);
block/blk-throttle.c-1836- throtl_shutdown_wq(q);
--
drivers/accel/qaic/qaic_timesync.c=174=void qaic_mqts_ch_stop_timer(struct mhi_device *mhi_dev)
--
drivers/accel/qaic/qaic_timesync.c-177-
drivers/accel/qaic/qaic_timesync.c:178: timer_delete_sync(&mqtsdev->timer);
drivers/accel/qaic/qaic_timesync.c-179-}
--
drivers/accel/qaic/qaic_timesync.c=228=static void qaic_timesync_remove(struct mhi_device *mhi_dev)
--
drivers/accel/qaic/qaic_timesync.c-232- mqtsdev->qdev->mqts_ch = NULL;
drivers/accel/qaic/qaic_timesync.c:233: timer_delete_sync(&mqtsdev->timer);
drivers/accel/qaic/qaic_timesync.c-234- mhi_unprepare_from_transfer(mqtsdev->mhi_dev);
--
drivers/accessibility/speakup/main.c=1165=static void do_handle_shift(struct vc_data *vc, u_char value, char up_flag)
--
drivers/accessibility/speakup/main.c-1174- case KVAL(K_SHIFT):
drivers/accessibility/speakup/main.c:1175: timer_delete(&cursor_timer);
drivers/accessibility/speakup/main.c-1176- spk_shut_up &= 0xfe;
--
drivers/accessibility/speakup/main.c-1180- case KVAL(K_CTRL):
drivers/accessibility/speakup/main.c:1181: timer_delete(&cursor_timer);
drivers/accessibility/speakup/main.c-1182- cursor_track = prev_cursor_track;
--
drivers/accessibility/speakup/main.c=1400=static void kbd_fakekey2(struct vc_data *vc, enum read_all_command command)
drivers/accessibility/speakup/main.c-1401-{
drivers/accessibility/speakup/main.c:1402: timer_delete(&cursor_timer);
drivers/accessibility/speakup/main.c-1403- speakup_fake_down_arrow();
--
drivers/accessibility/speakup/main.c=1407=static void read_all_doc(struct vc_data *vc)
--
drivers/accessibility/speakup/main.c-1417- if (get_sentence_buf(vc, 0) == -1) {
drivers/accessibility/speakup/main.c:1418: timer_delete(&cursor_timer);
drivers/accessibility/speakup/main.c-1419- if (!in_keyboard_notifier)
--
drivers/accessibility/speakup/main.c=1429=static void stop_read_all(struct vc_data *vc)
drivers/accessibility/speakup/main.c-1430-{
drivers/accessibility/speakup/main.c:1431: timer_delete(&cursor_timer);
drivers/accessibility/speakup/main.c-1432- cursor_track = prev_cursor_track;
--
drivers/accessibility/speakup/main.c=1520=static int pre_handle_cursor(struct vc_data *vc, u_char value, char up_flag)
--
drivers/accessibility/speakup/main.c-1530- }
drivers/accessibility/speakup/main.c:1531: timer_delete(&cursor_timer);
drivers/accessibility/speakup/main.c-1532- spk_shut_up &= 0xfe;
--
drivers/accessibility/speakup/main.c=1690=static void cursor_done(struct timer_list *unused)
--
drivers/accessibility/speakup/main.c-1694-
drivers/accessibility/speakup/main.c:1695: timer_delete(&cursor_timer);
drivers/accessibility/speakup/main.c-1696- spin_lock_irqsave(&speakup_info.spinlock, flags);
--
drivers/accessibility/speakup/main.c=2327=static void __exit speakup_exit(void)
--
drivers/accessibility/speakup/main.c-2335- speakup_cancel_paste();
drivers/accessibility/speakup/main.c:2336: timer_delete_sync(&cursor_timer);
drivers/accessibility/speakup/main.c-2337- kthread_stop(speakup_task);
--
drivers/accessibility/speakup/main.c=2363=static int __init speakup_init(void)
--
drivers/accessibility/speakup/main.c-2439- unregister_keyboard_notifier(&keyboard_notifier_block);
drivers/accessibility/speakup/main.c:2440: timer_delete(&cursor_timer);
drivers/accessibility/speakup/main.c-2441-
--
drivers/accessibility/speakup/synth.c=514=void synth_release(void)
--
drivers/accessibility/speakup/synth.c-523- synth->alive = 0;
drivers/accessibility/speakup/synth.c:524: timer_delete(&thread_timer);
drivers/accessibility/speakup/synth.c-525- spin_unlock_irqrestore(&speakup_info.spinlock, flags);
--
drivers/ata/libata-eh.c=692=void ata_scsi_port_error_handler(struct Scsi_Host *host, struct ata_port *ap)
--
drivers/ata/libata-eh.c-700- /* kill fast drain timer */
drivers/ata/libata-eh.c:701: timer_delete_sync(&ap->fastdrain_timer);
drivers/ata/libata-eh.c-702-
--
drivers/atm/idt77105.c=366=static void __exit idt77105_exit(void)
--
drivers/atm/idt77105.c-368- /* turn off timers */
drivers/atm/idt77105.c:369: timer_delete_sync(&stats_timer);
drivers/atm/idt77105.c:370: timer_delete_sync(&restart_timer);
drivers/atm/idt77105.c-371-}
--
drivers/atm/iphase.c=3282=static void __exit ia_module_exit(void)
--
drivers/atm/iphase.c-3285-
drivers/atm/iphase.c:3286: timer_delete_sync(&ia_timer);
drivers/atm/iphase.c-3287-}
--
drivers/atm/lanai.c=1793=static inline void lanai_timed_poll_stop(struct lanai_dev *lanai)
drivers/atm/lanai.c-1794-{
drivers/atm/lanai.c:1795: timer_delete_sync(&lanai->timer);
drivers/atm/lanai.c-1796-}
--
drivers/atm/nicstar.c=299=static void __exit nicstar_cleanup(void)
--
drivers/atm/nicstar.c-302-
drivers/atm/nicstar.c:303: timer_delete_sync(&ns_timer);
drivers/atm/nicstar.c-304-
--
drivers/atm/suni.c=340=static int suni_stop(struct atm_dev *dev)
--
drivers/atm/suni.c-349- *walk = PRIV((*walk)->dev)->next;
drivers/atm/suni.c:350: if (!sunis) timer_delete_sync(&poll_timer);
drivers/atm/suni.c-351- spin_unlock_irqrestore(&sunis_lock,flags);
--
drivers/auxdisplay/line-display.c=164=static int linedisp_display(struct linedisp *linedisp, const char *msg,
--
drivers/auxdisplay/line-display.c-169- /* stop the scroll timer */
drivers/auxdisplay/line-display.c:170: timer_delete_sync(&linedisp->timer);
drivers/auxdisplay/line-display.c-171-
--
drivers/auxdisplay/line-display.c=273=static ssize_t scroll_step_ms_store(struct device *dev,
--
drivers/auxdisplay/line-display.c-284-
drivers/auxdisplay/line-display.c:285: timer_delete_sync(&linedisp->timer);
drivers/auxdisplay/line-display.c-286-
--
drivers/auxdisplay/line-display.c=438=int linedisp_attach(struct linedisp *linedisp, struct device *dev,
--
drivers/auxdisplay/line-display.c-478-out_del_timer:
drivers/auxdisplay/line-display.c:479: timer_delete_sync(&linedisp->timer);
drivers/auxdisplay/line-display.c-480-out_free_buf:
--
drivers/auxdisplay/line-display.c=491=void linedisp_detach(struct device *dev)
--
drivers/auxdisplay/line-display.c-498-
drivers/auxdisplay/line-display.c:499: timer_delete_sync(&linedisp->timer);
drivers/auxdisplay/line-display.c-500-
--
drivers/auxdisplay/line-display.c=523=int linedisp_register(struct linedisp *linedisp, struct device *parent,
--
drivers/auxdisplay/line-display.c-573-out_del_timer:
drivers/auxdisplay/line-display.c:574: timer_delete_sync(&linedisp->timer);
drivers/auxdisplay/line-display.c-575-out_put_device:
--
drivers/auxdisplay/line-display.c=586=void linedisp_unregister(struct linedisp *linedisp)
--
drivers/auxdisplay/line-display.c-589- delete_attachment(&linedisp->dev, false);
drivers/auxdisplay/line-display.c:590: timer_delete_sync(&linedisp->timer);
drivers/auxdisplay/line-display.c-591- put_device(&linedisp->dev);
--
drivers/auxdisplay/panel.c=1514=static void panel_attach(struct parport *port)
--
drivers/auxdisplay/panel.c-1656- if (scan_timer.function)
drivers/auxdisplay/panel.c:1657: timer_delete_sync(&scan_timer);
drivers/auxdisplay/panel.c-1658- if (lcd.enabled)
--
drivers/auxdisplay/panel.c=1667=static void panel_detach(struct parport *port)
--
drivers/auxdisplay/panel.c-1677- if (scan_timer.function)
drivers/auxdisplay/panel.c:1678: timer_delete_sync(&scan_timer);
drivers/auxdisplay/panel.c-1679-
--
drivers/base/power/main.c=586=static void dpm_watchdog_clear(struct dpm_watchdog *wd)
--
drivers/base/power/main.c-589-
drivers/base/power/main.c:590: timer_delete_sync(timer);
drivers/base/power/main.c-591- timer_destroy_on_stack(timer);
--
drivers/base/power/wakeup.c=600=void __pm_stay_awake(struct wakeup_source *ws)
--
drivers/base/power/wakeup.c-609- wakeup_source_report_event(ws, false);
drivers/base/power/wakeup.c:610: timer_delete(&ws->timer);
drivers/base/power/wakeup.c-611- ws->timer_expires = 0;
--
drivers/base/power/wakeup.c=660=static void wakeup_source_deactivate(struct wakeup_source *ws)
--
]
]
|
| 22/2 |
2026/05/21 11:35 |
llm |
expert |
0m
Model:
Tokens:
input: 194827
output: 19
thoughts: 1698
Thoughts: **Analyzing the Race Condition in `netdev_watchdog_down` and Timer Deletion**
Okay, so `timer_delete` is now the preferred method, replacing `del_timer`, and `timer_delete_sync` is for `del_timer_sync`. The first key observation is that `timer_delete` *doesn't* wait for the timer handler to finish. That's immediately relevant because `netdev_watchdog_down` uses it. The function `netdev_watchdog_down` is called within the context of shutting down a network device.
Now, if `timer_delete` returns 0, it means the timer either wasn't pending or it's already running. If it's running, especially on another CPU, `netdev_watchdog_down` unlocks the `tx_global_lock` and returns. Then `dev_shutdown` is called. It seems like the code doesn't wait for the watchdog timer and just assumes it will be shut down as part of deactivation.
`dev_shutdown` calls `WARN_ON(timer_pending(&dev->watchdog_timer))`. This is interesting: if the timer is *currently* running on another CPU, `timer_pending` will be false because it isn't "queued". This seems like a reasonable assumption, as a running timer won't be in the queue.
But what if the timer handler, `dev_watchdog`, re-arms the timer? It acquires `tx_global_lock`, and calls `mod_timer`. `netdev_watchdog_down` holds the lock, calls `timer_delete`, and *then* releases the lock. So `dev_watchdog` might be able to grab the lock and re-arm the timer.
The crucial question is whether the `netif_running(dev)` check will still be valid. I need to know when `IFF_UP` is cleared, because that's what `netif_running` tests. It looks like `__dev_change_flags` clears `IFF_UP`, then calls `__dev_close`, and so on, eventually calling `netdev_watchdog_down`. So, by the time `dev_watchdog` tries to re-arm, `IFF_UP` will already be cleared. Which means the timer won't be re-armed in most cases.
But wait, what about `unregister_netdevice_many_notify`? That's another potential entry point. The initial glance at the code says, no, it does not.
Let's re-examine that function. Okay, it goes through a list of devices, and calls a sequence of functions including `netif_close_many_and_unlock_cond` and then `netif_close_many`. Ah, so `netif_close_many` eventually calls `__dev_close_many`.
Does `netif_close_many` clear `IFF_UP`? No, the code doesn't show that. *However*, let's look at `__dev_close_many` directly: it calls `dev_deactivate_many`, which calls `netdev_watchdog_down`! And, within that function, there's a comment, "...just clear netif_running()." That comment is critical. Let's dig into that `netif_running()` definition to be sure. It looks like there's the clearing of `__LINK_STATE_START`. Yes, the key here is that clearing `__LINK_STATE_START` will make the `dev_watchdog` think the network interface is down.
|
| 23/2 |
2026/05/21 11:36 |
tool |
grepper |
0m
Error:
Args: map[Expression:define netif_running]
Results:
|
| 24/2 |
2026/05/21 11:36 |
llm |
expert |
1m
Model:
Tokens:
input: 197340
output: 18
thoughts: 22
Thoughts: Here's my thought process, as requested:
**Investigating Network Interface Status**
Okay, I need to get a handle on how network interfaces are being managed, specifically whether they're "running" or not within the Linux kernel. My first instinct, and the most direct approach, is to use `grep`. I know the header files are the best place to find these kinds of kernel-level definitions, and I'm pretty sure `netdevice.h` is the logical file to start with, given the subject matter. So, I'm going to quickly search for instances of `netif_running` within `include/linux/netdevice.h`. This should give me a good overview of how the kernel determines and uses this state, and where its definition lies. This should be a pretty straightforward way to dive in.
|
| 25/2 |
2026/05/21 11:37 |
tool |
grepper |
0m
Args: map[Expression:netif_running]
Results: map[Output:
Full output is too long, showing 500 out of 9289 lines.
Use more precise expression if possible.
[Documentation/networking/net_cachelines/net_device.rst=19=unsigned_long base_addr
Documentation/networking/net_cachelines/net_device.rst:20:unsigned_long state read_mostly read_mostly netif_running(dev)
Documentation/networking/net_cachelines/net_device.rst-21-struct list_head dev_list
--
Documentation/networking/netdevices.rst=217=ndo_stop:
--
Documentation/networking/netdevices.rst-220- Context: process
Documentation/networking/netdevices.rst:221: Note: netif_running() is guaranteed false
Documentation/networking/netdevices.rst-222-
--
arch/um/drivers/vector_kern.c=1078=static irqreturn_t vector_rx_interrupt(int irq, void *dev_id)
--
arch/um/drivers/vector_kern.c-1082-
arch/um/drivers/vector_kern.c:1083: if (!netif_running(dev))
arch/um/drivers/vector_kern.c-1084- return IRQ_NONE;
--
arch/um/drivers/vector_kern.c=1090=static irqreturn_t vector_tx_interrupt(int irq, void *dev_id)
--
arch/um/drivers/vector_kern.c-1094-
arch/um/drivers/vector_kern.c:1095: if (!netif_running(dev))
arch/um/drivers/vector_kern.c-1096- return IRQ_NONE;
--
arch/xtensa/platforms/iss/network.c=317=static int iss_net_poll(struct iss_net_private *lp)
--
arch/xtensa/platforms/iss/network.c-320-
arch/xtensa/platforms/iss/network.c:321: if (!netif_running(lp->dev))
arch/xtensa/platforms/iss/network.c-322- return 0;
--
drivers/hsi/clients/ssi_protocol.c=378=int ssip_slave_running(struct hsi_client *master)
--
drivers/hsi/clients/ssi_protocol.c-380- struct ssi_protocol *ssi = hsi_client_drvdata(master);
drivers/hsi/clients/ssi_protocol.c:381: return netif_running(ssi->netdev);
drivers/hsi/clients/ssi_protocol.c-382-}
--
drivers/hsi/clients/ssi_protocol.c=385=static void ssip_reset(struct hsi_client *cl)
--
drivers/hsi/clients/ssi_protocol.c-390-
drivers/hsi/clients/ssi_protocol.c:391: if (netif_running(ssi->netdev))
drivers/hsi/clients/ssi_protocol.c-392- netif_carrier_off(ssi->netdev);
--
drivers/hsi/clients/ssi_protocol.c=608=static void ssip_pn_rx(struct sk_buff *skb)
--
drivers/hsi/clients/ssi_protocol.c-611-
drivers/hsi/clients/ssi_protocol.c:612: if (unlikely(!netif_running(dev))) {
drivers/hsi/clients/ssi_protocol.c-613- dev_dbg(&dev->dev, "Drop RX packet\n");
--
drivers/infiniband/hw/bnxt_re/ib_verbs.c=273=int bnxt_re_query_port(struct ib_device *ibdev, u32 port_num,
--
drivers/infiniband/hw/bnxt_re/ib_verbs.c-281-
drivers/infiniband/hw/bnxt_re/ib_verbs.c:282: if (netif_running(rdev->netdev) && netif_carrier_ok(rdev->netdev)) {
drivers/infiniband/hw/bnxt_re/ib_verbs.c-283- port_attr->state = IB_PORT_ACTIVE;
--
drivers/infiniband/hw/bnxt_re/main.c=2012=static int bnxt_re_ib_init(struct bnxt_re_dev *rdev)
--
drivers/infiniband/hw/bnxt_re/main.c-2025-
drivers/infiniband/hw/bnxt_re/main.c:2026: event = netif_running(rdev->netdev) && netif_carrier_ok(rdev->netdev) ?
drivers/infiniband/hw/bnxt_re/main.c-2027- IB_EVENT_PORT_ACTIVE : IB_EVENT_PORT_ERR;
--
drivers/infiniband/hw/ionic/ionic_ibdev.c=75=static int ionic_query_port(struct ib_device *ibdev, u32 port,
--
drivers/infiniband/hw/ionic/ionic_ibdev.c-84-
drivers/infiniband/hw/ionic/ionic_ibdev.c:85: if (netif_running(ndev) && netif_carrier_ok(ndev)) {
drivers/infiniband/hw/ionic/ionic_ibdev.c-86- attr->state = IB_PORT_ACTIVE;
drivers/infiniband/hw/ionic/ionic_ibdev.c-87- attr->phys_state = IB_PORT_PHYS_STATE_LINK_UP;
drivers/infiniband/hw/ionic/ionic_ibdev.c:88: } else if (netif_running(ndev)) {
drivers/infiniband/hw/ionic/ionic_ibdev.c-89- attr->state = IB_PORT_DOWN;
--
drivers/infiniband/hw/irdma/verbs.c=86=static int irdma_query_port(struct ib_device *ibdev, u32 port,
--
drivers/infiniband/hw/irdma/verbs.c-99- props->sm_sl = 0;
drivers/infiniband/hw/irdma/verbs.c:100: if (netif_carrier_ok(netdev) && netif_running(netdev)) {
drivers/infiniband/hw/irdma/verbs.c-101- props->state = IB_PORT_ACTIVE;
--
drivers/infiniband/hw/mana/main.c=615=int mana_ib_query_port(struct ib_device *ibdev, u32 port,
--
drivers/infiniband/hw/mana/main.c-627-
drivers/infiniband/hw/mana/main.c:628: if (netif_carrier_ok(ndev) && netif_running(ndev)) {
drivers/infiniband/hw/mana/main.c-629- props->state = IB_PORT_ACTIVE;
--
drivers/infiniband/hw/mlx4/main.c=751=static int eth_link_query_port(struct ib_device *ibdev, u32 port,
--
drivers/infiniband/hw/mlx4/main.c-801-
drivers/infiniband/hw/mlx4/main.c:802: props->state = (netif_running(ndev) && netif_carrier_ok(ndev)) ?
drivers/infiniband/hw/mlx4/main.c-803- IB_PORT_ACTIVE : IB_PORT_DOWN;
--
drivers/infiniband/hw/mlx4/main.c=3132=static void handle_bonded_port_state_event(struct work_struct *work)
--
drivers/infiniband/hw/mlx4/main.c-3150- curr_port_state =
drivers/infiniband/hw/mlx4/main.c:3151: (netif_running(curr_netdev) &&
drivers/infiniband/hw/mlx4/main.c-3152- netif_carrier_ok(curr_netdev)) ?
--
drivers/infiniband/hw/mlx5/main.c=535=static int mlx5_query_port_roce(struct ib_device *device, u32 port_num,
--
drivers/infiniband/hw/mlx5/main.c-615-
drivers/infiniband/hw/mlx5/main.c:616: if (netif_running(ndev) && netif_carrier_ok(ndev)) {
drivers/infiniband/hw/mlx5/main.c-617- props->state = IB_PORT_ACTIVE;
--
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c=154=int ocrdma_query_port(struct ib_device *ibdev,
--
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c-163- netdev = dev->nic_info.netdev;
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c:164: if (netif_running(netdev) && netif_oper_up(netdev)) {
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c-165- port_state = IB_PORT_ACTIVE;
--
drivers/infiniband/ulp/ipoib/ipoib_main.c=1925=static void ipoib_parent_unregister_pre(struct net_device *ndev)
--
drivers/infiniband/ulp/ipoib/ipoib_main.c-1929- /*
drivers/infiniband/ulp/ipoib/ipoib_main.c:1930: * ipoib_set_mac checks netif_running before pushing work, clearing
drivers/infiniband/ulp/ipoib/ipoib_main.c-1931- * running ensures the it will not add more work.
--
drivers/infiniband/ulp/ipoib/ipoib_main.c=2472=static int ipoib_set_mac(struct net_device *dev, void *addr)
--
drivers/infiniband/ulp/ipoib/ipoib_main.c-2477-
drivers/infiniband/ulp/ipoib/ipoib_main.c:2478: if (!(dev->priv_flags & IFF_LIVE_ADDR_CHANGE) && netif_running(dev))
drivers/infiniband/ulp/ipoib/ipoib_main.c-2479- return -EBUSY;
--
drivers/media/dvb-core/dvb_net.c=1257=static void wq_restart_net_feed (struct work_struct *work)
--
drivers/media/dvb-core/dvb_net.c-1262-
drivers/media/dvb-core/dvb_net.c:1263: if (netif_running(dev)) {
drivers/media/dvb-core/dvb_net.c-1264- dvb_net_feed_stop(dev);
--
drivers/media/dvb-core/dvb_net.c=1270=static int dvb_net_set_mac (struct net_device *dev, void *p)
--
drivers/media/dvb-core/dvb_net.c-1276-
drivers/media/dvb-core/dvb_net.c:1277: if (netif_running(dev))
drivers/media/dvb-core/dvb_net.c-1278- schedule_work(&priv->restart_net_feed_wq);
--
drivers/net/amt.c=613=static void amt_send_discovery(struct amt_dev *amt)
--
drivers/net/amt.c-630-
drivers/net/amt.c:631: if (!netif_running(amt->stream_dev) || !netif_running(amt->dev))
drivers/net/amt.c-632- goto out;
--
drivers/net/amt.c=702=static void amt_send_request(struct amt_dev *amt, bool v6)
--
drivers/net/amt.c-719-
drivers/net/amt.c:720: if (!netif_running(amt->stream_dev) || !netif_running(amt->dev))
drivers/net/amt.c-721- goto out;
--
drivers/net/amt.c=2552=static void amt_send_advertisement(struct amt_dev *amt, __be32 nonce,
--
drivers/net/amt.c-2570-
drivers/net/amt.c:2571: if (!netif_running(amt->stream_dev) || !netif_running(amt->dev))
drivers/net/amt.c-2572- goto out;
--
drivers/net/amt.c=2663=static bool amt_request_handler(struct amt_dev *amt, struct sk_buff *skb)
--
drivers/net/amt.c-2728-
drivers/net/amt.c:2729: if (!netif_running(amt->dev) || !netif_running(amt->stream_dev))
drivers/net/amt.c-2730- return true;
--
drivers/net/arcnet/arcnet.c=398=static void reset_device_work(struct work_struct *work)
--
drivers/net/arcnet/arcnet.c-408- */
drivers/net/arcnet/arcnet.c:409: if (!netif_running(dev) || !lp->reset_in_progress)
drivers/net/arcnet/arcnet.c-410- return;
--
drivers/net/arcnet/arcnet.c-417- */
drivers/net/arcnet/arcnet.c:418: if (!netif_running(dev) || !lp->reset_in_progress)
drivers/net/arcnet/arcnet.c-419- goto out;
--
drivers/net/arcnet/arcnet.c=854=irqreturn_t arcnet_interrupt(int irq, void *dev_id)
--
drivers/net/arcnet/arcnet.c-876- */
drivers/net/arcnet/arcnet.c:877: if (!netif_running(dev)) {
drivers/net/arcnet/arcnet.c-878- if (lp->hw.status(dev) & RESETflag)
--
drivers/net/bonding/bond_3ad.c=1790=static int agg_device_up(const struct aggregator *agg)
--
drivers/net/bonding/bond_3ad.c-1798- port = port->next_port_in_aggregator) {
drivers/net/bonding/bond_3ad.c:1799: if (netif_running(port->slave->dev) &&
drivers/net/bonding/bond_3ad.c-1800- netif_carrier_ok(port->slave->dev))
--
drivers/net/bonding/bond_main.c=1213=void bond_change_active_slave(struct bonding *bond, struct slave *new_active)
--
drivers/net/bonding/bond_main.c-1281-
drivers/net/bonding/bond_main.c:1282: if (netif_running(bond->dev)) {
drivers/net/bonding/bond_main.c-1283- bond->send_peer_notif =
--
drivers/net/bonding/bond_main.c-1307- */
drivers/net/bonding/bond_main.c:1308: if (netif_running(bond->dev) && (bond->params.resend_igmp > 0) &&
drivers/net/bonding/bond_main.c-1309- ((bond_uses_primary(bond) && new_active) ||
--
drivers/net/bonding/bond_main.c=1799=int bond_enslave(struct net_device *bond_dev, struct net_device *slave_dev,
--
drivers/net/bonding/bond_main.c-2031- if (bond->params.miimon) {
drivers/net/bonding/bond_main.c:2032: if (netif_running(slave_dev) && netif_carrier_ok(slave_dev)) {
drivers/net/bonding/bond_main.c-2033- if (bond->params.updelay) {
--
drivers/net/bonding/bond_main.c=2556=static int bond_miimon_inspect(struct bonding *bond)
--
drivers/net/bonding/bond_main.c-2576-
drivers/net/bonding/bond_main.c:2577: link_state = netif_running(slave->dev) &&
drivers/net/bonding/bond_main.c-2578- netif_carrier_ok(slave->dev);
--
drivers/net/can/bxcan.c=1045=static int __maybe_unused bxcan_suspend(struct device *dev)
--
drivers/net/can/bxcan.c-1049-
drivers/net/can/bxcan.c:1050: if (!netif_running(ndev))
drivers/net/can/bxcan.c-1051- return 0;
--
drivers/net/can/bxcan.c=1062=static int __maybe_unused bxcan_resume(struct device *dev)
--
drivers/net/can/bxcan.c-1066-
drivers/net/can/bxcan.c:1067: if (!netif_running(ndev))
drivers/net/can/bxcan.c-1068- return 0;
--
drivers/net/can/c_can/c_can_platform.c=395=static int c_can_suspend(struct platform_device *pdev, pm_message_t state)
--
drivers/net/can/c_can/c_can_platform.c-405-
drivers/net/can/c_can/c_can_platform.c:406: if (netif_running(ndev)) {
drivers/net/can/c_can/c_can_platform.c-407- netif_stop_queue(ndev);
--
drivers/net/can/c_can/c_can_platform.c=422=static int c_can_resume(struct platform_device *pdev)
--
drivers/net/can/c_can/c_can_platform.c-440-
drivers/net/can/c_can/c_can_platform.c:441: if (netif_running(ndev)) {
drivers/net/can/c_can/c_can_platform.c-442- netif_device_attach(ndev);
--
drivers/net/can/can327.c=262=static void can327_feed_frame_to_netdev(struct can327 *elm, struct sk_buff *skb)
--
drivers/net/can/can327.c-265-
drivers/net/can/can327.c:266: if (!netif_running(elm->dev)) {
drivers/net/can/can327.c-267- kfree_skb(skb);
--
drivers/net/can/ctucanfd/ctucanfd_base.c=1310=int ctucan_suspend(struct device *dev)
--
drivers/net/can/ctucanfd/ctucanfd_base.c-1314-
drivers/net/can/ctucanfd/ctucanfd_base.c:1315: if (netif_running(ndev)) {
drivers/net/can/ctucanfd/ctucanfd_base.c-1316- netif_stop_queue(ndev);
--
drivers/net/can/ctucanfd/ctucanfd_base.c=1326=int ctucan_resume(struct device *dev)
--
drivers/net/can/ctucanfd/ctucanfd_base.c-1332-
drivers/net/can/ctucanfd/ctucanfd_base.c:1333: if (netif_running(ndev)) {
drivers/net/can/ctucanfd/ctucanfd_base.c-1334- netif_device_attach(ndev);
--
drivers/net/can/flexcan/flexcan-core.c=2288=static int __maybe_unused flexcan_suspend(struct device *device)
--
drivers/net/can/flexcan/flexcan-core.c-2293-
drivers/net/can/flexcan/flexcan-core.c:2294: if (netif_running(dev)) {
drivers/net/can/flexcan/flexcan-core.c-2295- /* if wakeup is enabled, enter stop mode
--
drivers/net/can/flexcan/flexcan-core.c=2327=static int __maybe_unused flexcan_resume(struct device *device)
--
drivers/net/can/flexcan/flexcan-core.c-2332-
drivers/net/can/flexcan/flexcan-core.c:2333: if (netif_running(dev)) {
drivers/net/can/flexcan/flexcan-core.c-2334- netif_device_attach(dev);
--
drivers/net/can/flexcan/flexcan-core.c=2383=static int __maybe_unused flexcan_noirq_suspend(struct device *device)
--
drivers/net/can/flexcan/flexcan-core.c-2387-
drivers/net/can/flexcan/flexcan-core.c:2388: if (netif_running(dev)) {
drivers/net/can/flexcan/flexcan-core.c-2389- int err;
--
drivers/net/can/flexcan/flexcan-core.c=2412=static int __maybe_unused flexcan_noirq_resume(struct device *device)
--
drivers/net/can/flexcan/flexcan-core.c-2416-
drivers/net/can/flexcan/flexcan-core.c:2417: if (netif_running(dev)) {
drivers/net/can/flexcan/flexcan-core.c-2418- int err;
--
drivers/net/can/flexcan/flexcan-ethtool.c=61=static int flexcan_set_priv_flags(struct net_device *ndev, u32 priv_flags)
--
drivers/net/can/flexcan/flexcan-ethtool.c-79-
drivers/net/can/flexcan/flexcan-ethtool.c:80: if (quirks != priv->devtype_data.quirks && netif_running(ndev))
drivers/net/can/flexcan/flexcan-ethtool.c-81- return -EBUSY;
--
drivers/net/can/m_can/m_can.c=2597=int m_can_class_suspend(struct device *dev)
--
drivers/net/can/m_can/m_can.c-2602-
drivers/net/can/m_can/m_can.c:2603: if (netif_running(ndev)) {
drivers/net/can/m_can/m_can.c-2604- netif_stop_queue(ndev);
--
drivers/net/can/m_can/m_can.c=2632=int m_can_class_resume(struct device *dev)
--
drivers/net/can/m_can/m_can.c-2640-
drivers/net/can/m_can/m_can.c:2641: if (netif_running(ndev)) {
drivers/net/can/m_can/m_can.c-2642- ret = m_can_clk_start(cdev);
--
drivers/net/can/rcar/rcar_can.c=842=static int rcar_can_suspend(struct device *dev)
--
drivers/net/can/rcar/rcar_can.c-847-
drivers/net/can/rcar/rcar_can.c:848: if (!netif_running(ndev))
drivers/net/can/rcar/rcar_can.c-849- return 0;
--
drivers/net/can/rcar/rcar_can.c=865=static int rcar_can_resume(struct device *dev)
--
drivers/net/can/rcar/rcar_can.c-869-
drivers/net/can/rcar/rcar_can.c:870: if (!netif_running(ndev))
drivers/net/can/rcar/rcar_can.c-871- return 0;
--
drivers/net/can/rcar/rcar_canfd.c=2267=static int rcar_canfd_suspend(struct device *dev)
--
drivers/net/can/rcar/rcar_canfd.c-2276-
drivers/net/can/rcar/rcar_canfd.c:2277: if (!netif_running(ndev))
drivers/net/can/rcar/rcar_canfd.c-2278- continue;
--
drivers/net/can/rcar/rcar_canfd.c=2298=static int rcar_canfd_resume(struct device *dev)
--
drivers/net/can/rcar/rcar_canfd.c-2313-
drivers/net/can/rcar/rcar_canfd.c:2314: if (!netif_running(ndev))
drivers/net/can/rcar/rcar_canfd.c-2315- continue;
--
drivers/net/can/slcan/slcan-core.c=126=int slcan_enable_err_rst_on_open(struct net_device *ndev, bool on)
--
drivers/net/can/slcan/slcan-core.c-129-
drivers/net/can/slcan/slcan-core.c:130: if (netif_running(ndev))
drivers/net/can/slcan/slcan-core.c-131- return -EBUSY;
--
drivers/net/can/slcan/slcan-core.c=557=static void slcan_transmit(struct work_struct *work)
--
drivers/net/can/slcan/slcan-core.c-563- /* First make sure we're connected. */
drivers/net/can/slcan/slcan-core.c:564: if (unlikely(!netif_running(sl->dev)) &&
drivers/net/can/slcan/slcan-core.c-565- likely(!test_bit(SLF_XCMD, &sl->flags))) {
--
drivers/net/can/slcan/slcan-core.c=606=static netdev_tx_t slcan_netdev_xmit(struct sk_buff *skb,
--
drivers/net/can/slcan/slcan-core.c-614- spin_lock(&sl->lock);
drivers/net/can/slcan/slcan-core.c:615: if (!netif_running(dev)) {
drivers/net/can/slcan/slcan-core.c-616- spin_unlock(&sl->lock);
--
drivers/net/can/slcan/slcan-core.c=790=static void slcan_receive_buf(struct tty_struct *tty, const u8 *cp,
--
drivers/net/can/slcan/slcan-core.c-794-
drivers/net/can/slcan/slcan-core.c:795: if (!netif_running(sl->dev))
drivers/net/can/slcan/slcan-core.c-796- return;
--
drivers/net/can/softing/softing_fw.c=417=int softing_startstop(struct net_device *dev, int up)
--
drivers/net/can/softing/softing_fw.c-451-
drivers/net/can/softing/softing_fw.c:452: if (netif_running(netdev)) {
drivers/net/can/softing/softing_fw.c-453- if (dev != netdev)
--
drivers/net/can/softing/softing_main.c=21=static inline int canif_is_active(struct net_device *netdev)
--
drivers/net/can/softing/softing_main.c-24-
drivers/net/can/softing/softing_main.c:25: if (!netif_running(netdev))
drivers/net/can/softing/softing_main.c-26- return 0;
--
drivers/net/can/softing/softing_main.c=569=static ssize_t store_output(struct device *dev, struct device_attribute *attr,
--
drivers/net/can/softing/softing_main.c-585- return -ERESTARTSYS;
drivers/net/can/softing/softing_main.c:586: if (netif_running(ndev)) {
drivers/net/can/softing/softing_main.c-587- mutex_unlock(&card->fw.lock);
--
drivers/net/can/spi/hi311x.c=973=static int __maybe_unused hi3110_can_suspend(struct device *dev)
--
drivers/net/can/spi/hi311x.c-984- */
drivers/net/can/spi/hi311x.c:985: if (netif_running(net)) {
drivers/net/can/spi/hi311x.c-986- netif_device_detach(net);
--
drivers/net/can/spi/mcp251x.c=1475=static int __maybe_unused mcp251x_can_suspend(struct device *dev)
--
drivers/net/can/spi/mcp251x.c-1485- */
drivers/net/can/spi/mcp251x.c:1486: if (netif_running(net)) {
drivers/net/can/spi/mcp251x.c-1487- netif_device_detach(net);
--
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c=33=mcp251xfd_ring_set_ringparam(struct net_device *ndev,
--
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c-44- layout.cur_tx != priv->tx->obj_num) &&
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c:45: netif_running(ndev))
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c-46- return -EBUSY;
--
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c=86=static int mcp251xfd_ring_set_coalesce(struct net_device *ndev,
--
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c-104- ec->tx_coalesce_usecs_irq != priv->tx_coalesce_usecs_irq) &&
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c:105: netif_running(ndev))
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c-106- return -EBUSY;
--
drivers/net/can/ti_hecc.c=977=static int ti_hecc_suspend(struct platform_device *pdev, pm_message_t state)
--
drivers/net/can/ti_hecc.c-981-
drivers/net/can/ti_hecc.c:982: if (netif_running(dev)) {
drivers/net/can/ti_hecc.c-983- netif_stop_queue(dev);
--
drivers/net/can/ti_hecc.c=995=static int ti_hecc_resume(struct platform_device *pdev)
--
drivers/net/can/ti_hecc.c-1007-
drivers/net/can/ti_hecc.c:1008: if (netif_running(dev)) {
drivers/net/can/ti_hecc.c-1009- netif_device_attach(dev);
--
drivers/net/can/usb/etas_es58x/es581_4.c=95=static int es581_4_rx_can_msg(struct es58x_device *es58x_dev,
--
drivers/net/can/usb/etas_es58x/es581_4.c-113-
drivers/net/can/usb/etas_es58x/es581_4.c:114: if (!netif_running(netdev)) {
drivers/net/can/usb/etas_es58x/es581_4.c-115- if (net_ratelimit())
--
drivers/net/can/usb/etas_es58x/es58x_core.c=382=int es58x_can_get_echo_skb(struct net_device *netdev, u32 rcv_packet_idx,
--
drivers/net/can/usb/etas_es58x/es58x_core.c-390-
drivers/net/can/usb/etas_es58x/es58x_core.c:391: if (!netif_running(netdev)) {
drivers/net/can/usb/etas_es58x/es58x_core.c-392- if (net_ratelimit())
--
drivers/net/can/usb/etas_es58x/es58x_core.c=658=int es58x_rx_err_msg(struct net_device *netdev, enum es58x_err error,
--
drivers/net/can/usb/etas_es58x/es58x_core.c-667-
drivers/net/can/usb/etas_es58x/es58x_core.c:668: if (!netif_running(netdev)) {
drivers/net/can/usb/etas_es58x/es58x_core.c-669- if (net_ratelimit())
--
drivers/net/can/usb/etas_es58x/es58x_fd.c=98=static int es58x_fd_rx_can_msg(struct net_device *netdev,
--
drivers/net/can/usb/etas_es58x/es58x_fd.c-136-
drivers/net/can/usb/etas_es58x/es58x_fd.c:137: if (netif_running(netdev)) {
drivers/net/can/usb/etas_es58x/es58x_fd.c-138- u64 tstamp = get_unaligned_le64(&rx_can_msg->timestamp);
--
drivers/net/can/usb/etas_es58x/es58x_fd.c-156-
drivers/net/can/usb/etas_es58x/es58x_fd.c:157: if (!netif_running(netdev)) {
drivers/net/can/usb/etas_es58x/es58x_fd.c-158- if (net_ratelimit())
--
drivers/net/can/usb/gs_usb.c=609=static void gs_usb_receive_bulk_callback(struct urb *urb)
--
drivers/net/can/usb/gs_usb.c-656-
drivers/net/can/usb/gs_usb.c:657: if (!netif_running(netdev))
drivers/net/can/usb/gs_usb.c-658- goto resubmit_urb;
--
drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c=1186=static void kvaser_usb_leaf_rx_error(const struct kvaser_usb *dev,
--
drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c-1206- /* Ignore e.g. state change to bus-off reported just after stopping */
drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c:1207: if (!netif_running(priv->netdev))
drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c-1208- return;
--
drivers/net/can/xilinx_can.c=1721=static int __maybe_unused xcan_suspend(struct device *dev)
--
drivers/net/can/xilinx_can.c-1724-
drivers/net/can/xilinx_can.c:1725: if (netif_running(ndev)) {
drivers/net/can/xilinx_can.c-1726- netif_stop_queue(ndev);
--
drivers/net/can/xilinx_can.c=1741=static int __maybe_unused xcan_resume(struct device *dev)
--
drivers/net/can/xilinx_can.c-1751-
drivers/net/can/xilinx_can.c:1752: if (netif_running(ndev)) {
drivers/net/can/xilinx_can.c-1753- ret = xcan_chip_start(ndev);
--
drivers/net/ethernet/3com/3c509.c=1315=el3_suspend(struct device *pdev, pm_message_t state)
--
drivers/net/ethernet/3com/3c509.c-1327-
drivers/net/ethernet/3com/3c509.c:1328: if (netif_running(dev))
drivers/net/ethernet/3com/3c509.c-1329- netif_device_detach(dev);
--
drivers/net/ethernet/3com/3c509.c=1339=el3_resume(struct device *pdev)
--
drivers/net/ethernet/3com/3c509.c-1355-
drivers/net/ethernet/3com/3c509.c:1356: if (netif_running(dev))
drivers/net/ethernet/3com/3c509.c-1357- netif_device_attach(dev);
--
drivers/net/ethernet/3com/3c515.c=1113=static irqreturn_t corkscrew_interrupt(int irq, void *dev_id)
--
drivers/net/ethernet/3com/3c515.c-1138- pr_err("%s: Bogus interrupt, bailing. Status %4.4x, start=%d.\n",
drivers/net/ethernet/3com/3c515.c:1139: dev->name, status, netif_running(dev));
drivers/net/ethernet/3com/3c515.c-1140- free_irq(dev->irq, dev);
--
drivers/net/ethernet/3com/3c515.c=1455=static struct net_device_stats *corkscrew_get_stats(struct net_device *dev)
--
drivers/net/ethernet/3com/3c515.c-1459-
drivers/net/ethernet/3com/3c515.c:1460: if (netif_running(dev)) {
drivers/net/ethernet/3com/3c515.c-1461- spin_lock_irqsave(&vp->lock, flags);
--
drivers/net/ethernet/3com/3c59x.c=848=static int vortex_suspend(struct device *dev)
--
drivers/net/ethernet/3com/3c59x.c-851-
drivers/net/ethernet/3com/3c59x.c:852: if (!ndev || !netif_running(ndev))
drivers/net/ethernet/3com/3c59x.c-853- return 0;
--
drivers/net/ethernet/3com/3c59x.c=861=static int vortex_resume(struct device *dev)
--
drivers/net/ethernet/3com/3c59x.c-865-
drivers/net/ethernet/3com/3c59x.c:866: if (!ndev || !netif_running(ndev))
drivers/net/ethernet/3com/3c59x.c-867- return 0;
--
drivers/net/ethernet/3com/3c59x.c=2824=static struct net_device_stats *vortex_get_stats(struct net_device *dev)
--
drivers/net/ethernet/3com/3c59x.c-2829-
drivers/net/ethernet/3com/3c59x.c:2830: if (netif_device_present(dev)) { /* AKPM: Used to be netif_running */
drivers/net/ethernet/3com/3c59x.c-2831- spin_lock_irqsave (&vp->lock, flags);
--
drivers/net/ethernet/3com/typhoon.c=1957=typhoon_stop_runtime(struct typhoon *tp, int wait_type)
--
drivers/net/ethernet/3com/typhoon.c-1965- /* Disable interrupts early, since we can't schedule a poll
drivers/net/ethernet/3com/typhoon.c:1966: * when called with !netif_running(). This will be posted
drivers/net/ethernet/3com/typhoon.c-1967- * when we force the posting of the command.
--
drivers/net/ethernet/3com/typhoon.c=2124=typhoon_resume(struct device *dev_d)
--
drivers/net/ethernet/3com/typhoon.c-2130- */
drivers/net/ethernet/3com/typhoon.c:2131: if (!netif_running(dev))
drivers/net/ethernet/3com/typhoon.c-2132- return 0;
--
drivers/net/ethernet/3com/typhoon.c=2153=typhoon_suspend(struct device *dev_d)
--
drivers/net/ethernet/3com/typhoon.c-2161- */
drivers/net/ethernet/3com/typhoon.c:2162: if (!netif_running(dev))
drivers/net/ethernet/3com/typhoon.c-2163- return 0;
--
drivers/net/ethernet/8390/ax88796.c=562=static int ax_ioctl(struct net_device *dev, struct ifreq *req, int cmd)
--
drivers/net/ethernet/8390/ax88796.c-565-
drivers/net/ethernet/8390/ax88796.c:566: if (!netif_running(dev))
drivers/net/ethernet/8390/ax88796.c-567- return -EINVAL;
--
drivers/net/ethernet/8390/axnet_cs.c=1088=static irqreturn_t ax_interrupt(int irq, void *dev_id)
]
]
|
| 26/2 |
2026/05/21 11:37 |
llm |
expert |
0m
Model:
Tokens:
input: 209074
output: 19
thoughts: 2878
Thoughts: **Analyzing the Network Device Shutdown Sequence**
Okay, so I'm diving into this network device shutdown flow. The core seems to be `netif_running(dev)` being equivalent to `dev->flags & IFF_UP`. My initial thought was that `dev_shutdown` happened *after* `netif_close_many`. If a device *wasn't* `IFF_UP`, `netif_close_many` wouldn't even call `__dev_close_many`, which avoids the call to `dev_deactivate_many`, and thus skips `netdev_watchdog_down`. That seems sensible. But then the nagging thought arose: if it *wasn't* up, why would the watchdog timer *be* pending in the first place? And after re-examining how `netdev_watchdog_up` gets called, it's ONLY under the condition of `IFF_UP`! This raises a red flag.
My focus then shifted to finding any scenario where the timer could be erroneously pending, even when `IFF_UP` is false. I reviewed the callers of `netdev_watchdog_up`. Two places: `netif_device_attach`, `netif_carrier_on`, and `dev_activate`. It confirmed my suspicions: they all depend on `netif_running(dev)` or similar, which boils down to the device being up. Crucially, `dev_activate` itself seems to be called from a function that *sets* `IFF_UP`.
The implication seems clear: if a device *is* `IFF_UP`, the shutdown *should* cascade through `netif_close_many`, `__dev_close_many`, `dev_deactivate_many`, to finally call `netdev_watchdog_down`, which makes sense. I had to look into scenarios where a race condition exists. The first one had to do with `netdev_need_ops_lock` and the use of the `unreg_list`.
During the `unregister_netdevice_many_notify` sequence, there's a loop that *doesn't* check `IFF_UP` before adding devices to a `close_head` list. This can lead to the addition of a device that is *not* `IFF_UP`. This then would be passed to `netif_close_many` which cleans up devices that are not `IFF_UP` from the `close_head` list. So then `__dev_close_many` is never called for these devices, meaning no `dev_deactivate_many` and no `netdev_watchdog_down`, which is correct.
However, if it's *not* up, the watchdog *shouldn't* be pending. That meant I needed to figure out if there were cases where `IFF_UP` could be cleared without a corresponding `__dev_close` call. The only place I saw `IFF_UP` being modified was through `__dev_change_flags`. My thought was also around how driver code could directly modify the flags, which seemed dangerous.
I shifted to consider the case where `netdev_watchdog_down` *is* called, but the timer is somehow still active. `timer_delete` should solve this, except if it returns zero, or the timer is running on *another* CPU. This case is handled in `dev_shutdown` with a `WARN_ON(timer_pending(...))`. However, the handler function `dev_watchdog` could potentially re-arm the timer. The `dev_watchdog` also acquires `tx_global_lock` after `netdev_watchdog_down` releases it. This can cause a race.
The analysis led me to look into `unregister_netdevice_many_notify` again, but this sequence specifically does *not* clear `IFF_UP` before calling `netif_close_many`.
So, the critical bit: `__dev_close_many` doesn't directly clear `IFF_UP`, but clears `__LINK_STATE_START` instead. This is something to be careful about because it means, essentially, there's a comment describing what "should" happen, but there's a disconnect.
|
| 27/2 |
2026/05/21 11:37 |
tool |
grepper |
0m
Error:
Args: map[Expression:define netif_running]
Results:
|
| 28/2 |
2026/05/21 11:37 |
llm |
expert |
0m
Model:
Tokens:
input: 212897
output: 18
thoughts: 22
Thoughts: Here's my thought process, as if I were thinking it:
**Investigating Network Interface State in the Kernel**
Okay, so I need to figure out how the kernel determines if a network interface is actually *up* and running. Seems like a good place to start would be the main header file dealing with network devices, `include/linux/netdevice.h`. Let's see... I'll just use `grep` to quickly search for a relevant term. Something like... `netif_running` sounds promising. That likely points to a macro or function used to check the operational state of the interface. This should give me a good jumping-off point to dig deeper into the actual implementation and usage. Time to get grepping!
|
| 29/2 |
2026/05/21 11:37 |
tool |
grepper |
0m
Args: map[Expression:netif_running]
Results: map[Output:
Full output is too long, showing 500 out of 9289 lines.
Use more precise expression if possible.
[Documentation/networking/net_cachelines/net_device.rst=19=unsigned_long base_addr
Documentation/networking/net_cachelines/net_device.rst:20:unsigned_long state read_mostly read_mostly netif_running(dev)
Documentation/networking/net_cachelines/net_device.rst-21-struct list_head dev_list
--
Documentation/networking/netdevices.rst=217=ndo_stop:
--
Documentation/networking/netdevices.rst-220- Context: process
Documentation/networking/netdevices.rst:221: Note: netif_running() is guaranteed false
Documentation/networking/netdevices.rst-222-
--
arch/um/drivers/vector_kern.c=1078=static irqreturn_t vector_rx_interrupt(int irq, void *dev_id)
--
arch/um/drivers/vector_kern.c-1082-
arch/um/drivers/vector_kern.c:1083: if (!netif_running(dev))
arch/um/drivers/vector_kern.c-1084- return IRQ_NONE;
--
arch/um/drivers/vector_kern.c=1090=static irqreturn_t vector_tx_interrupt(int irq, void *dev_id)
--
arch/um/drivers/vector_kern.c-1094-
arch/um/drivers/vector_kern.c:1095: if (!netif_running(dev))
arch/um/drivers/vector_kern.c-1096- return IRQ_NONE;
--
arch/xtensa/platforms/iss/network.c=317=static int iss_net_poll(struct iss_net_private *lp)
--
arch/xtensa/platforms/iss/network.c-320-
arch/xtensa/platforms/iss/network.c:321: if (!netif_running(lp->dev))
arch/xtensa/platforms/iss/network.c-322- return 0;
--
drivers/hsi/clients/ssi_protocol.c=378=int ssip_slave_running(struct hsi_client *master)
--
drivers/hsi/clients/ssi_protocol.c-380- struct ssi_protocol *ssi = hsi_client_drvdata(master);
drivers/hsi/clients/ssi_protocol.c:381: return netif_running(ssi->netdev);
drivers/hsi/clients/ssi_protocol.c-382-}
--
drivers/hsi/clients/ssi_protocol.c=385=static void ssip_reset(struct hsi_client *cl)
--
drivers/hsi/clients/ssi_protocol.c-390-
drivers/hsi/clients/ssi_protocol.c:391: if (netif_running(ssi->netdev))
drivers/hsi/clients/ssi_protocol.c-392- netif_carrier_off(ssi->netdev);
--
drivers/hsi/clients/ssi_protocol.c=608=static void ssip_pn_rx(struct sk_buff *skb)
--
drivers/hsi/clients/ssi_protocol.c-611-
drivers/hsi/clients/ssi_protocol.c:612: if (unlikely(!netif_running(dev))) {
drivers/hsi/clients/ssi_protocol.c-613- dev_dbg(&dev->dev, "Drop RX packet\n");
--
drivers/infiniband/hw/bnxt_re/ib_verbs.c=273=int bnxt_re_query_port(struct ib_device *ibdev, u32 port_num,
--
drivers/infiniband/hw/bnxt_re/ib_verbs.c-281-
drivers/infiniband/hw/bnxt_re/ib_verbs.c:282: if (netif_running(rdev->netdev) && netif_carrier_ok(rdev->netdev)) {
drivers/infiniband/hw/bnxt_re/ib_verbs.c-283- port_attr->state = IB_PORT_ACTIVE;
--
drivers/infiniband/hw/bnxt_re/main.c=2012=static int bnxt_re_ib_init(struct bnxt_re_dev *rdev)
--
drivers/infiniband/hw/bnxt_re/main.c-2025-
drivers/infiniband/hw/bnxt_re/main.c:2026: event = netif_running(rdev->netdev) && netif_carrier_ok(rdev->netdev) ?
drivers/infiniband/hw/bnxt_re/main.c-2027- IB_EVENT_PORT_ACTIVE : IB_EVENT_PORT_ERR;
--
drivers/infiniband/hw/ionic/ionic_ibdev.c=75=static int ionic_query_port(struct ib_device *ibdev, u32 port,
--
drivers/infiniband/hw/ionic/ionic_ibdev.c-84-
drivers/infiniband/hw/ionic/ionic_ibdev.c:85: if (netif_running(ndev) && netif_carrier_ok(ndev)) {
drivers/infiniband/hw/ionic/ionic_ibdev.c-86- attr->state = IB_PORT_ACTIVE;
drivers/infiniband/hw/ionic/ionic_ibdev.c-87- attr->phys_state = IB_PORT_PHYS_STATE_LINK_UP;
drivers/infiniband/hw/ionic/ionic_ibdev.c:88: } else if (netif_running(ndev)) {
drivers/infiniband/hw/ionic/ionic_ibdev.c-89- attr->state = IB_PORT_DOWN;
--
drivers/infiniband/hw/irdma/verbs.c=86=static int irdma_query_port(struct ib_device *ibdev, u32 port,
--
drivers/infiniband/hw/irdma/verbs.c-99- props->sm_sl = 0;
drivers/infiniband/hw/irdma/verbs.c:100: if (netif_carrier_ok(netdev) && netif_running(netdev)) {
drivers/infiniband/hw/irdma/verbs.c-101- props->state = IB_PORT_ACTIVE;
--
drivers/infiniband/hw/mana/main.c=615=int mana_ib_query_port(struct ib_device *ibdev, u32 port,
--
drivers/infiniband/hw/mana/main.c-627-
drivers/infiniband/hw/mana/main.c:628: if (netif_carrier_ok(ndev) && netif_running(ndev)) {
drivers/infiniband/hw/mana/main.c-629- props->state = IB_PORT_ACTIVE;
--
drivers/infiniband/hw/mlx4/main.c=751=static int eth_link_query_port(struct ib_device *ibdev, u32 port,
--
drivers/infiniband/hw/mlx4/main.c-801-
drivers/infiniband/hw/mlx4/main.c:802: props->state = (netif_running(ndev) && netif_carrier_ok(ndev)) ?
drivers/infiniband/hw/mlx4/main.c-803- IB_PORT_ACTIVE : IB_PORT_DOWN;
--
drivers/infiniband/hw/mlx4/main.c=3132=static void handle_bonded_port_state_event(struct work_struct *work)
--
drivers/infiniband/hw/mlx4/main.c-3150- curr_port_state =
drivers/infiniband/hw/mlx4/main.c:3151: (netif_running(curr_netdev) &&
drivers/infiniband/hw/mlx4/main.c-3152- netif_carrier_ok(curr_netdev)) ?
--
drivers/infiniband/hw/mlx5/main.c=535=static int mlx5_query_port_roce(struct ib_device *device, u32 port_num,
--
drivers/infiniband/hw/mlx5/main.c-615-
drivers/infiniband/hw/mlx5/main.c:616: if (netif_running(ndev) && netif_carrier_ok(ndev)) {
drivers/infiniband/hw/mlx5/main.c-617- props->state = IB_PORT_ACTIVE;
--
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c=154=int ocrdma_query_port(struct ib_device *ibdev,
--
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c-163- netdev = dev->nic_info.netdev;
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c:164: if (netif_running(netdev) && netif_oper_up(netdev)) {
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c-165- port_state = IB_PORT_ACTIVE;
--
drivers/infiniband/ulp/ipoib/ipoib_main.c=1925=static void ipoib_parent_unregister_pre(struct net_device *ndev)
--
drivers/infiniband/ulp/ipoib/ipoib_main.c-1929- /*
drivers/infiniband/ulp/ipoib/ipoib_main.c:1930: * ipoib_set_mac checks netif_running before pushing work, clearing
drivers/infiniband/ulp/ipoib/ipoib_main.c-1931- * running ensures the it will not add more work.
--
drivers/infiniband/ulp/ipoib/ipoib_main.c=2472=static int ipoib_set_mac(struct net_device *dev, void *addr)
--
drivers/infiniband/ulp/ipoib/ipoib_main.c-2477-
drivers/infiniband/ulp/ipoib/ipoib_main.c:2478: if (!(dev->priv_flags & IFF_LIVE_ADDR_CHANGE) && netif_running(dev))
drivers/infiniband/ulp/ipoib/ipoib_main.c-2479- return -EBUSY;
--
drivers/media/dvb-core/dvb_net.c=1257=static void wq_restart_net_feed (struct work_struct *work)
--
drivers/media/dvb-core/dvb_net.c-1262-
drivers/media/dvb-core/dvb_net.c:1263: if (netif_running(dev)) {
drivers/media/dvb-core/dvb_net.c-1264- dvb_net_feed_stop(dev);
--
drivers/media/dvb-core/dvb_net.c=1270=static int dvb_net_set_mac (struct net_device *dev, void *p)
--
drivers/media/dvb-core/dvb_net.c-1276-
drivers/media/dvb-core/dvb_net.c:1277: if (netif_running(dev))
drivers/media/dvb-core/dvb_net.c-1278- schedule_work(&priv->restart_net_feed_wq);
--
drivers/net/amt.c=613=static void amt_send_discovery(struct amt_dev *amt)
--
drivers/net/amt.c-630-
drivers/net/amt.c:631: if (!netif_running(amt->stream_dev) || !netif_running(amt->dev))
drivers/net/amt.c-632- goto out;
--
drivers/net/amt.c=702=static void amt_send_request(struct amt_dev *amt, bool v6)
--
drivers/net/amt.c-719-
drivers/net/amt.c:720: if (!netif_running(amt->stream_dev) || !netif_running(amt->dev))
drivers/net/amt.c-721- goto out;
--
drivers/net/amt.c=2552=static void amt_send_advertisement(struct amt_dev *amt, __be32 nonce,
--
drivers/net/amt.c-2570-
drivers/net/amt.c:2571: if (!netif_running(amt->stream_dev) || !netif_running(amt->dev))
drivers/net/amt.c-2572- goto out;
--
drivers/net/amt.c=2663=static bool amt_request_handler(struct amt_dev *amt, struct sk_buff *skb)
--
drivers/net/amt.c-2728-
drivers/net/amt.c:2729: if (!netif_running(amt->dev) || !netif_running(amt->stream_dev))
drivers/net/amt.c-2730- return true;
--
drivers/net/arcnet/arcnet.c=398=static void reset_device_work(struct work_struct *work)
--
drivers/net/arcnet/arcnet.c-408- */
drivers/net/arcnet/arcnet.c:409: if (!netif_running(dev) || !lp->reset_in_progress)
drivers/net/arcnet/arcnet.c-410- return;
--
drivers/net/arcnet/arcnet.c-417- */
drivers/net/arcnet/arcnet.c:418: if (!netif_running(dev) || !lp->reset_in_progress)
drivers/net/arcnet/arcnet.c-419- goto out;
--
drivers/net/arcnet/arcnet.c=854=irqreturn_t arcnet_interrupt(int irq, void *dev_id)
--
drivers/net/arcnet/arcnet.c-876- */
drivers/net/arcnet/arcnet.c:877: if (!netif_running(dev)) {
drivers/net/arcnet/arcnet.c-878- if (lp->hw.status(dev) & RESETflag)
--
drivers/net/bonding/bond_3ad.c=1790=static int agg_device_up(const struct aggregator *agg)
--
drivers/net/bonding/bond_3ad.c-1798- port = port->next_port_in_aggregator) {
drivers/net/bonding/bond_3ad.c:1799: if (netif_running(port->slave->dev) &&
drivers/net/bonding/bond_3ad.c-1800- netif_carrier_ok(port->slave->dev))
--
drivers/net/bonding/bond_main.c=1213=void bond_change_active_slave(struct bonding *bond, struct slave *new_active)
--
drivers/net/bonding/bond_main.c-1281-
drivers/net/bonding/bond_main.c:1282: if (netif_running(bond->dev)) {
drivers/net/bonding/bond_main.c-1283- bond->send_peer_notif =
--
drivers/net/bonding/bond_main.c-1307- */
drivers/net/bonding/bond_main.c:1308: if (netif_running(bond->dev) && (bond->params.resend_igmp > 0) &&
drivers/net/bonding/bond_main.c-1309- ((bond_uses_primary(bond) && new_active) ||
--
drivers/net/bonding/bond_main.c=1799=int bond_enslave(struct net_device *bond_dev, struct net_device *slave_dev,
--
drivers/net/bonding/bond_main.c-2031- if (bond->params.miimon) {
drivers/net/bonding/bond_main.c:2032: if (netif_running(slave_dev) && netif_carrier_ok(slave_dev)) {
drivers/net/bonding/bond_main.c-2033- if (bond->params.updelay) {
--
drivers/net/bonding/bond_main.c=2556=static int bond_miimon_inspect(struct bonding *bond)
--
drivers/net/bonding/bond_main.c-2576-
drivers/net/bonding/bond_main.c:2577: link_state = netif_running(slave->dev) &&
drivers/net/bonding/bond_main.c-2578- netif_carrier_ok(slave->dev);
--
drivers/net/can/bxcan.c=1045=static int __maybe_unused bxcan_suspend(struct device *dev)
--
drivers/net/can/bxcan.c-1049-
drivers/net/can/bxcan.c:1050: if (!netif_running(ndev))
drivers/net/can/bxcan.c-1051- return 0;
--
drivers/net/can/bxcan.c=1062=static int __maybe_unused bxcan_resume(struct device *dev)
--
drivers/net/can/bxcan.c-1066-
drivers/net/can/bxcan.c:1067: if (!netif_running(ndev))
drivers/net/can/bxcan.c-1068- return 0;
--
drivers/net/can/c_can/c_can_platform.c=395=static int c_can_suspend(struct platform_device *pdev, pm_message_t state)
--
drivers/net/can/c_can/c_can_platform.c-405-
drivers/net/can/c_can/c_can_platform.c:406: if (netif_running(ndev)) {
drivers/net/can/c_can/c_can_platform.c-407- netif_stop_queue(ndev);
--
drivers/net/can/c_can/c_can_platform.c=422=static int c_can_resume(struct platform_device *pdev)
--
drivers/net/can/c_can/c_can_platform.c-440-
drivers/net/can/c_can/c_can_platform.c:441: if (netif_running(ndev)) {
drivers/net/can/c_can/c_can_platform.c-442- netif_device_attach(ndev);
--
drivers/net/can/can327.c=262=static void can327_feed_frame_to_netdev(struct can327 *elm, struct sk_buff *skb)
--
drivers/net/can/can327.c-265-
drivers/net/can/can327.c:266: if (!netif_running(elm->dev)) {
drivers/net/can/can327.c-267- kfree_skb(skb);
--
drivers/net/can/ctucanfd/ctucanfd_base.c=1310=int ctucan_suspend(struct device *dev)
--
drivers/net/can/ctucanfd/ctucanfd_base.c-1314-
drivers/net/can/ctucanfd/ctucanfd_base.c:1315: if (netif_running(ndev)) {
drivers/net/can/ctucanfd/ctucanfd_base.c-1316- netif_stop_queue(ndev);
--
drivers/net/can/ctucanfd/ctucanfd_base.c=1326=int ctucan_resume(struct device *dev)
--
drivers/net/can/ctucanfd/ctucanfd_base.c-1332-
drivers/net/can/ctucanfd/ctucanfd_base.c:1333: if (netif_running(ndev)) {
drivers/net/can/ctucanfd/ctucanfd_base.c-1334- netif_device_attach(ndev);
--
drivers/net/can/flexcan/flexcan-core.c=2288=static int __maybe_unused flexcan_suspend(struct device *device)
--
drivers/net/can/flexcan/flexcan-core.c-2293-
drivers/net/can/flexcan/flexcan-core.c:2294: if (netif_running(dev)) {
drivers/net/can/flexcan/flexcan-core.c-2295- /* if wakeup is enabled, enter stop mode
--
drivers/net/can/flexcan/flexcan-core.c=2327=static int __maybe_unused flexcan_resume(struct device *device)
--
drivers/net/can/flexcan/flexcan-core.c-2332-
drivers/net/can/flexcan/flexcan-core.c:2333: if (netif_running(dev)) {
drivers/net/can/flexcan/flexcan-core.c-2334- netif_device_attach(dev);
--
drivers/net/can/flexcan/flexcan-core.c=2383=static int __maybe_unused flexcan_noirq_suspend(struct device *device)
--
drivers/net/can/flexcan/flexcan-core.c-2387-
drivers/net/can/flexcan/flexcan-core.c:2388: if (netif_running(dev)) {
drivers/net/can/flexcan/flexcan-core.c-2389- int err;
--
drivers/net/can/flexcan/flexcan-core.c=2412=static int __maybe_unused flexcan_noirq_resume(struct device *device)
--
drivers/net/can/flexcan/flexcan-core.c-2416-
drivers/net/can/flexcan/flexcan-core.c:2417: if (netif_running(dev)) {
drivers/net/can/flexcan/flexcan-core.c-2418- int err;
--
drivers/net/can/flexcan/flexcan-ethtool.c=61=static int flexcan_set_priv_flags(struct net_device *ndev, u32 priv_flags)
--
drivers/net/can/flexcan/flexcan-ethtool.c-79-
drivers/net/can/flexcan/flexcan-ethtool.c:80: if (quirks != priv->devtype_data.quirks && netif_running(ndev))
drivers/net/can/flexcan/flexcan-ethtool.c-81- return -EBUSY;
--
drivers/net/can/m_can/m_can.c=2597=int m_can_class_suspend(struct device *dev)
--
drivers/net/can/m_can/m_can.c-2602-
drivers/net/can/m_can/m_can.c:2603: if (netif_running(ndev)) {
drivers/net/can/m_can/m_can.c-2604- netif_stop_queue(ndev);
--
drivers/net/can/m_can/m_can.c=2632=int m_can_class_resume(struct device *dev)
--
drivers/net/can/m_can/m_can.c-2640-
drivers/net/can/m_can/m_can.c:2641: if (netif_running(ndev)) {
drivers/net/can/m_can/m_can.c-2642- ret = m_can_clk_start(cdev);
--
drivers/net/can/rcar/rcar_can.c=842=static int rcar_can_suspend(struct device *dev)
--
drivers/net/can/rcar/rcar_can.c-847-
drivers/net/can/rcar/rcar_can.c:848: if (!netif_running(ndev))
drivers/net/can/rcar/rcar_can.c-849- return 0;
--
drivers/net/can/rcar/rcar_can.c=865=static int rcar_can_resume(struct device *dev)
--
drivers/net/can/rcar/rcar_can.c-869-
drivers/net/can/rcar/rcar_can.c:870: if (!netif_running(ndev))
drivers/net/can/rcar/rcar_can.c-871- return 0;
--
drivers/net/can/rcar/rcar_canfd.c=2267=static int rcar_canfd_suspend(struct device *dev)
--
drivers/net/can/rcar/rcar_canfd.c-2276-
drivers/net/can/rcar/rcar_canfd.c:2277: if (!netif_running(ndev))
drivers/net/can/rcar/rcar_canfd.c-2278- continue;
--
drivers/net/can/rcar/rcar_canfd.c=2298=static int rcar_canfd_resume(struct device *dev)
--
drivers/net/can/rcar/rcar_canfd.c-2313-
drivers/net/can/rcar/rcar_canfd.c:2314: if (!netif_running(ndev))
drivers/net/can/rcar/rcar_canfd.c-2315- continue;
--
drivers/net/can/slcan/slcan-core.c=126=int slcan_enable_err_rst_on_open(struct net_device *ndev, bool on)
--
drivers/net/can/slcan/slcan-core.c-129-
drivers/net/can/slcan/slcan-core.c:130: if (netif_running(ndev))
drivers/net/can/slcan/slcan-core.c-131- return -EBUSY;
--
drivers/net/can/slcan/slcan-core.c=557=static void slcan_transmit(struct work_struct *work)
--
drivers/net/can/slcan/slcan-core.c-563- /* First make sure we're connected. */
drivers/net/can/slcan/slcan-core.c:564: if (unlikely(!netif_running(sl->dev)) &&
drivers/net/can/slcan/slcan-core.c-565- likely(!test_bit(SLF_XCMD, &sl->flags))) {
--
drivers/net/can/slcan/slcan-core.c=606=static netdev_tx_t slcan_netdev_xmit(struct sk_buff *skb,
--
drivers/net/can/slcan/slcan-core.c-614- spin_lock(&sl->lock);
drivers/net/can/slcan/slcan-core.c:615: if (!netif_running(dev)) {
drivers/net/can/slcan/slcan-core.c-616- spin_unlock(&sl->lock);
--
drivers/net/can/slcan/slcan-core.c=790=static void slcan_receive_buf(struct tty_struct *tty, const u8 *cp,
--
drivers/net/can/slcan/slcan-core.c-794-
drivers/net/can/slcan/slcan-core.c:795: if (!netif_running(sl->dev))
drivers/net/can/slcan/slcan-core.c-796- return;
--
drivers/net/can/softing/softing_fw.c=417=int softing_startstop(struct net_device *dev, int up)
--
drivers/net/can/softing/softing_fw.c-451-
drivers/net/can/softing/softing_fw.c:452: if (netif_running(netdev)) {
drivers/net/can/softing/softing_fw.c-453- if (dev != netdev)
--
drivers/net/can/softing/softing_main.c=21=static inline int canif_is_active(struct net_device *netdev)
--
drivers/net/can/softing/softing_main.c-24-
drivers/net/can/softing/softing_main.c:25: if (!netif_running(netdev))
drivers/net/can/softing/softing_main.c-26- return 0;
--
drivers/net/can/softing/softing_main.c=569=static ssize_t store_output(struct device *dev, struct device_attribute *attr,
--
drivers/net/can/softing/softing_main.c-585- return -ERESTARTSYS;
drivers/net/can/softing/softing_main.c:586: if (netif_running(ndev)) {
drivers/net/can/softing/softing_main.c-587- mutex_unlock(&card->fw.lock);
--
drivers/net/can/spi/hi311x.c=973=static int __maybe_unused hi3110_can_suspend(struct device *dev)
--
drivers/net/can/spi/hi311x.c-984- */
drivers/net/can/spi/hi311x.c:985: if (netif_running(net)) {
drivers/net/can/spi/hi311x.c-986- netif_device_detach(net);
--
drivers/net/can/spi/mcp251x.c=1475=static int __maybe_unused mcp251x_can_suspend(struct device *dev)
--
drivers/net/can/spi/mcp251x.c-1485- */
drivers/net/can/spi/mcp251x.c:1486: if (netif_running(net)) {
drivers/net/can/spi/mcp251x.c-1487- netif_device_detach(net);
--
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c=33=mcp251xfd_ring_set_ringparam(struct net_device *ndev,
--
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c-44- layout.cur_tx != priv->tx->obj_num) &&
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c:45: netif_running(ndev))
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c-46- return -EBUSY;
--
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c=86=static int mcp251xfd_ring_set_coalesce(struct net_device *ndev,
--
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c-104- ec->tx_coalesce_usecs_irq != priv->tx_coalesce_usecs_irq) &&
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c:105: netif_running(ndev))
drivers/net/can/spi/mcp251xfd/mcp251xfd-ethtool.c-106- return -EBUSY;
--
drivers/net/can/ti_hecc.c=977=static int ti_hecc_suspend(struct platform_device *pdev, pm_message_t state)
--
drivers/net/can/ti_hecc.c-981-
drivers/net/can/ti_hecc.c:982: if (netif_running(dev)) {
drivers/net/can/ti_hecc.c-983- netif_stop_queue(dev);
--
drivers/net/can/ti_hecc.c=995=static int ti_hecc_resume(struct platform_device *pdev)
--
drivers/net/can/ti_hecc.c-1007-
drivers/net/can/ti_hecc.c:1008: if (netif_running(dev)) {
drivers/net/can/ti_hecc.c-1009- netif_device_attach(dev);
--
drivers/net/can/usb/etas_es58x/es581_4.c=95=static int es581_4_rx_can_msg(struct es58x_device *es58x_dev,
--
drivers/net/can/usb/etas_es58x/es581_4.c-113-
drivers/net/can/usb/etas_es58x/es581_4.c:114: if (!netif_running(netdev)) {
drivers/net/can/usb/etas_es58x/es581_4.c-115- if (net_ratelimit())
--
drivers/net/can/usb/etas_es58x/es58x_core.c=382=int es58x_can_get_echo_skb(struct net_device *netdev, u32 rcv_packet_idx,
--
drivers/net/can/usb/etas_es58x/es58x_core.c-390-
drivers/net/can/usb/etas_es58x/es58x_core.c:391: if (!netif_running(netdev)) {
drivers/net/can/usb/etas_es58x/es58x_core.c-392- if (net_ratelimit())
--
drivers/net/can/usb/etas_es58x/es58x_core.c=658=int es58x_rx_err_msg(struct net_device *netdev, enum es58x_err error,
--
drivers/net/can/usb/etas_es58x/es58x_core.c-667-
drivers/net/can/usb/etas_es58x/es58x_core.c:668: if (!netif_running(netdev)) {
drivers/net/can/usb/etas_es58x/es58x_core.c-669- if (net_ratelimit())
--
drivers/net/can/usb/etas_es58x/es58x_fd.c=98=static int es58x_fd_rx_can_msg(struct net_device *netdev,
--
drivers/net/can/usb/etas_es58x/es58x_fd.c-136-
drivers/net/can/usb/etas_es58x/es58x_fd.c:137: if (netif_running(netdev)) {
drivers/net/can/usb/etas_es58x/es58x_fd.c-138- u64 tstamp = get_unaligned_le64(&rx_can_msg->timestamp);
--
drivers/net/can/usb/etas_es58x/es58x_fd.c-156-
drivers/net/can/usb/etas_es58x/es58x_fd.c:157: if (!netif_running(netdev)) {
drivers/net/can/usb/etas_es58x/es58x_fd.c-158- if (net_ratelimit())
--
drivers/net/can/usb/gs_usb.c=609=static void gs_usb_receive_bulk_callback(struct urb *urb)
--
drivers/net/can/usb/gs_usb.c-656-
drivers/net/can/usb/gs_usb.c:657: if (!netif_running(netdev))
drivers/net/can/usb/gs_usb.c-658- goto resubmit_urb;
--
drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c=1186=static void kvaser_usb_leaf_rx_error(const struct kvaser_usb *dev,
--
drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c-1206- /* Ignore e.g. state change to bus-off reported just after stopping */
drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c:1207: if (!netif_running(priv->netdev))
drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c-1208- return;
--
drivers/net/can/xilinx_can.c=1721=static int __maybe_unused xcan_suspend(struct device *dev)
--
drivers/net/can/xilinx_can.c-1724-
drivers/net/can/xilinx_can.c:1725: if (netif_running(ndev)) {
drivers/net/can/xilinx_can.c-1726- netif_stop_queue(ndev);
--
drivers/net/can/xilinx_can.c=1741=static int __maybe_unused xcan_resume(struct device *dev)
--
drivers/net/can/xilinx_can.c-1751-
drivers/net/can/xilinx_can.c:1752: if (netif_running(ndev)) {
drivers/net/can/xilinx_can.c-1753- ret = xcan_chip_start(ndev);
--
drivers/net/ethernet/3com/3c509.c=1315=el3_suspend(struct device *pdev, pm_message_t state)
--
drivers/net/ethernet/3com/3c509.c-1327-
drivers/net/ethernet/3com/3c509.c:1328: if (netif_running(dev))
drivers/net/ethernet/3com/3c509.c-1329- netif_device_detach(dev);
--
drivers/net/ethernet/3com/3c509.c=1339=el3_resume(struct device *pdev)
--
drivers/net/ethernet/3com/3c509.c-1355-
drivers/net/ethernet/3com/3c509.c:1356: if (netif_running(dev))
drivers/net/ethernet/3com/3c509.c-1357- netif_device_attach(dev);
--
drivers/net/ethernet/3com/3c515.c=1113=static irqreturn_t corkscrew_interrupt(int irq, void *dev_id)
--
drivers/net/ethernet/3com/3c515.c-1138- pr_err("%s: Bogus interrupt, bailing. Status %4.4x, start=%d.\n",
drivers/net/ethernet/3com/3c515.c:1139: dev->name, status, netif_running(dev));
drivers/net/ethernet/3com/3c515.c-1140- free_irq(dev->irq, dev);
--
drivers/net/ethernet/3com/3c515.c=1455=static struct net_device_stats *corkscrew_get_stats(struct net_device *dev)
--
drivers/net/ethernet/3com/3c515.c-1459-
drivers/net/ethernet/3com/3c515.c:1460: if (netif_running(dev)) {
drivers/net/ethernet/3com/3c515.c-1461- spin_lock_irqsave(&vp->lock, flags);
--
drivers/net/ethernet/3com/3c59x.c=848=static int vortex_suspend(struct device *dev)
--
drivers/net/ethernet/3com/3c59x.c-851-
drivers/net/ethernet/3com/3c59x.c:852: if (!ndev || !netif_running(ndev))
drivers/net/ethernet/3com/3c59x.c-853- return 0;
--
drivers/net/ethernet/3com/3c59x.c=861=static int vortex_resume(struct device *dev)
--
drivers/net/ethernet/3com/3c59x.c-865-
drivers/net/ethernet/3com/3c59x.c:866: if (!ndev || !netif_running(ndev))
drivers/net/ethernet/3com/3c59x.c-867- return 0;
--
drivers/net/ethernet/3com/3c59x.c=2824=static struct net_device_stats *vortex_get_stats(struct net_device *dev)
--
drivers/net/ethernet/3com/3c59x.c-2829-
drivers/net/ethernet/3com/3c59x.c:2830: if (netif_device_present(dev)) { /* AKPM: Used to be netif_running */
drivers/net/ethernet/3com/3c59x.c-2831- spin_lock_irqsave (&vp->lock, flags);
--
drivers/net/ethernet/3com/typhoon.c=1957=typhoon_stop_runtime(struct typhoon *tp, int wait_type)
--
drivers/net/ethernet/3com/typhoon.c-1965- /* Disable interrupts early, since we can't schedule a poll
drivers/net/ethernet/3com/typhoon.c:1966: * when called with !netif_running(). This will be posted
drivers/net/ethernet/3com/typhoon.c-1967- * when we force the posting of the command.
--
drivers/net/ethernet/3com/typhoon.c=2124=typhoon_resume(struct device *dev_d)
--
drivers/net/ethernet/3com/typhoon.c-2130- */
drivers/net/ethernet/3com/typhoon.c:2131: if (!netif_running(dev))
drivers/net/ethernet/3com/typhoon.c-2132- return 0;
--
drivers/net/ethernet/3com/typhoon.c=2153=typhoon_suspend(struct device *dev_d)
--
drivers/net/ethernet/3com/typhoon.c-2161- */
drivers/net/ethernet/3com/typhoon.c:2162: if (!netif_running(dev))
drivers/net/ethernet/3com/typhoon.c-2163- return 0;
--
drivers/net/ethernet/8390/ax88796.c=562=static int ax_ioctl(struct net_device *dev, struct ifreq *req, int cmd)
--
drivers/net/ethernet/8390/ax88796.c-565-
drivers/net/ethernet/8390/ax88796.c:566: if (!netif_running(dev))
drivers/net/ethernet/8390/ax88796.c-567- return -EINVAL;
--
drivers/net/ethernet/8390/axnet_cs.c=1088=static irqreturn_t ax_interrupt(int irq, void *dev_id)
]
]
|
| 30/2 |
2026/05/21 11:37 |
llm |
expert |
0m
Model:
Tokens:
input: 224613
output: 21
thoughts: 29
Thoughts: Here's my attempt at summarizing the thought process, tailored for an expert audience:
**Diving into the Kernel Network Interface Code**
Okay, so I need to understand this `netif_running(dev)` function. It's clearly a kernel-level construct since it's used in the code I'm analyzing. My immediate thought is, "Where's this defined? Let's check the header files." Given the context, my intuition tells me it's probably related to network device management. "Bingo! `include/linux/netdevice.h` seems like the likely location. Time to crack open that header and see what `netif_running(dev)` actually *does* behind the scenes. Need to refresh my memory on the underlying semantics."
|
| 31/2 |
2026/05/21 11:37 |
tool |
grepper |
0m
Args: map[Expression:static inline bool netif_running]
Results: map[Output:include/linux/netdevice.h=3936=static inline u16 netdev_cap_txqueue(struct net_device *dev, u16 queue_index)
--
include/linux/netdevice.h-3953- */
include/linux/netdevice.h:3954:static inline bool netif_running(const struct net_device *dev)
include/linux/netdevice.h-3955-{
]
|
| 32/2 |
2026/05/21 11:37 |
llm |
expert-compressor |
0m
Model:
Reply: Here is a comprehensive summary of the current state of the workspace and the investigation:
### 1. Approaches Tried and Results
* **Identified the WARNING:** The agent grepped for `dev_shutdown` and found it in `net/sched/sch_generic.c`. The warning at line 1496 is `WARN_ON(timer_pending(&dev->watchdog_timer));`, indicating that the network device's watchdog timer is still active when the device is being shut down.
* **Analyzed Timer Management:** The agent read `net/sched/sch_generic.c` to examine `dev_watchdog` (the timer callback), `netdev_watchdog_up` (arms the timer), and `netdev_watchdog_down` (disarms the timer). It noted that `netdev_watchdog_down` uses `timer_delete()` rather than `timer_delete_sync()`, but does so while holding the `netif_tx_lock_bh(dev)` (which acquires `dev->tx_global_lock`).
* **Analyzed Unregistration Flow:** The agent read `net/core/dev.c` to trace `unregister_netdevice_many_notify()`. The teardown sequence calls `netif_close_many()` (which eventually calls `dev_deactivate_many()` -> `netdev_watchdog_down()`) and later calls `dev_shutdown()`.
* **Searched for Timer Re-arming:** The agent grepped for `netdev_watchdog_up` to see where the timer might be re-armed. It found calls in `netif_device_attach()`, `netif_carrier_on()`, and `dev_activate()`. All of these check `netif_running(dev)` before arming the timer.
* **Searched for Definitions:** The agent attempted to grep for the definitions of `timer_delete` and `netif_running` to understand their exact semantics and check for potential race conditions.
### 2. Current Hypotheses and Active Lines of Investigation
* **Core Issue:** The watchdog timer is either failing to be deleted or is being re-armed between the time `netdev_watchdog_down()` is called and `dev_shutdown()` is executed during USB device disconnect.
* **Hypothesis 1 (Concurrency/Race Condition):** Because `netdev_watchdog_down()` uses `timer_delete()` instead of `timer_delete_sync()`, the timer callback might still be running on another CPU. However, because both `dev_watchdog()` and `netdev_watchdog_down()` acquire `dev->tx_global_lock`, they should be serialized. The agent is likely verifying if there's any edge case where this serialization fails.
* **Hypothesis 2 (Spurious Re-arming):** The timer is being re-armed by an asynchronous event (like a carrier change or device attach) after it was shut down. Since `netdev_watchdog_up()` is guarded by `netif_running(dev)`, the agent is currently investigating the exact definition of `netif_running()` to see if it could incorrectly evaluate to `true` during the unregistration phase (e.g., if `__LINK_STATE_START` is not properly cleared or is re-set).
### 3. Critical File Paths and Code Snippets
* **`net/sched/sch_generic.c`**:
* `dev_shutdown()`: Contains the trigger `WARN_ON(timer_pending(&dev->watchdog_timer));`.
* `netdev_watchdog_down()`: Uses `timer_delete(&dev->watchdog_timer)` under `netif_tx_lock_bh(dev)`.
* `dev_watchdog()`: The timer callback, which can re-arm itself via `mod_timer()`.
* **`net/core/dev.c`**:
* `unregister_netdevice_many_notify()`: The main function orchestrating the device teardown, dictating the order of `netif_close_many()` and `dev_shutdown()`.
* **`include/linux/netdevice.h`**: Contains the definition of `netif_running()` (which the agent is currently trying to locate).
### 4. Watch Out: Reasoning Loops and Repetitive Tool Calls
* **Repetitive Grepping:** The agent called the `grepper` tool with the exact same expression (`netdev_watchdog_up`) **three times in a row**, receiving the exact same output each time. You should avoid repeating identical tool calls.
* **Search Syntax:** The agent struggled slightly to find definitions, trying `define timer_delete` and `define netif_running` which yielded no matches, before falling back to broader searches. Remember that kernel macros/inlines might not match a strict `define <name>` regex.
Tokens:
input: 209952
output: 1075
thoughts: 3694
|
| 33/2 |
2026/05/21 11:38 |
llm |
expert |
0m
Model:
Tokens:
input: 16662
output: 41
thoughts: 126
Thoughts: Okay, here's my analysis of this warning, presented as my own thought process:
**Security Implications of a USB Network Device Disconnect Warning**
Alright, this looks like a potentially interesting issue. I see a warning related to a USB network device, specifically `net1080` being disconnected. The key here is the `WARN_ON(timer_pending(&dev->watchdog_timer));` line within `dev_shutdown()` in `net/sched/sch_generic.c`. My immediate thought is, what's the lifecycle of that watchdog timer? Why is it still pending when the device is shutting down? This could signal a resource leak or, even worse, a race condition.
I need to understand the function of `dev_shutdown()` within this specific context. If the watchdog timer is still active during the shutdown process, there's a chance a malicious actor could trigger a controlled disconnect to exploit timing vulnerabilities. We're looking for a possible denial-of-service, or perhaps something more subtle where unhandled interrupts or corrupted data could arise. I need to dig into the surrounding code to determine how this timer is set, how it's handled, and why it's not being properly cleared before the disconnect. I have to assess whether this unexpected state could be leveraged to gain unauthorized access, or to cause a kernel panic, or to create a path for information disclosure. The fact that it's a USB device adds another layer of complexity, because of its hot-pluggable nature. Is there a scenario where a carefully timed device removal could expose a vulnerability? Time to dive into the kernel source and figure out the full picture.
|
| 34/2 |
2026/05/21 11:38 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/sched/sch_generic.c IncludeLines:true Name:dev_shutdown]
Results: map[SourceCode:1488: void dev_shutdown(struct net_device *dev)
1489: {
1490: netdev_for_each_tx_queue(dev, shutdown_scheduler_queue, &noop_qdisc);
1491: if (dev_ingress_queue(dev))
1492: shutdown_scheduler_queue(dev, dev_ingress_queue(dev), &noop_qdisc);
1493: qdisc_put(rtnl_dereference(dev->qdisc));
1494: rcu_assign_pointer(dev->qdisc, &noop_qdisc);
1495:
1496: WARN_ON(timer_pending(&dev->watchdog_timer));
1497: }
SourceFile:net/sched/sch_generic.c]
|
| 35/2 |
2026/05/21 11:38 |
llm |
expert |
0m
Model:
Tokens:
input: 17396
output: 44
thoughts: 142
Thoughts: **Analyzing a Potential Use-After-Free Vulnerability**
Okay, so I'm looking at this warning: `WARN_ON(timer_pending(&dev->watchdog_timer));`. That's the key. If the watchdog timer is pending on a network device (`dev`, which is a `struct net_device`), what's the implication if the device is being unregistered?
The code is clearly concerned about what happens after `dev_shutdown()`. I know that function's primary role is to shut down the device, and eventually the driver will free the `net_device` struct. The problem is obvious: if the timer *is* pending and hasn't fired yet, it could fire *after* the device is freed. That would be a classic use-after-free scenario, leading to memory corruption, crashes, or potential security vulnerabilities – a nasty bug.
Ah, wait, now it clicks. The function calling `dev_shutdown()` is `unregister_netdevice_many_notify()`. That's the likely location for the issue. Let's delve into `net/core/dev.c` and take a closer look at `unregister_netdevice_many_notify()`. This is where the solution is going to be. Time to dig in!
|
| 36/2 |
2026/05/21 11:38 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:unregister_netdevice_many_notify]
Results: map[SourceCode:12305: void unregister_netdevice_many_notify(struct list_head *head,
12306: u32 portid, const struct nlmsghdr *nlh)
12307: {
12308: struct net_device *dev, *tmp;
12309: LIST_HEAD(close_head);
12310: int cnt = 0;
12311:
12312: BUG_ON(dev_boot_phase);
12313: ASSERT_RTNL();
12314:
12315: if (list_empty(head))
12316: return;
12317:
12318: list_for_each_entry_safe(dev, tmp, head, unreg_list) {
12319: /* Some devices call without registering
12320: * for initialization unwind. Remove those
12321: * devices and proceed with the remaining.
12322: */
12323: if (dev->reg_state == NETREG_UNINITIALIZED) {
12324: pr_debug("unregister_netdevice: device %s/%p never was registered\n",
12325: dev->name, dev);
12326:
12327: WARN_ON(1);
12328: list_del(&dev->unreg_list);
12329: continue;
12330: }
12331: dev->dismantle = true;
12332: BUG_ON(dev->reg_state != NETREG_REGISTERED);
12333: }
12334:
12335: /* If device is running, close it first. Start with ops locked... */
12336: list_for_each_entry(dev, head, unreg_list) {
12337: if (!(dev->flags & IFF_UP))
12338: continue;
12339: if (netdev_need_ops_lock(dev)) {
12340: list_add_tail(&dev->close_list, &close_head);
12341: netdev_lock(dev);
12342: }
12343: netif_close_many_and_unlock_cond(&close_head);
12344: }
12345: netif_close_many_and_unlock(&close_head);
12346: /* ... now go over the rest. */
12347: list_for_each_entry(dev, head, unreg_list) {
12348: if (!netdev_need_ops_lock(dev))
12349: list_add_tail(&dev->close_list, &close_head);
12350: }
12351: netif_close_many(&close_head, true);
12352:
12353: list_for_each_entry(dev, head, unreg_list) {
12354: /* And unlink it from device chain. */
12355: unlist_netdevice(dev);
12356: netdev_lock(dev);
12357: WRITE_ONCE(dev->reg_state, NETREG_UNREGISTERING);
12358: netdev_unlock(dev);
12359: }
12360: flush_all_backlogs();
12361:
12362: synchronize_net();
12363:
12364: list_for_each_entry(dev, head, unreg_list) {
12365: struct sk_buff *skb = NULL;
12366:
12367: /* Shutdown queueing discipline. */
12368: netdev_lock_ops(dev);
12369: dev_shutdown(dev);
12370: dev_tcx_uninstall(dev);
12371: dev_xdp_uninstall(dev);
12372: dev_memory_provider_uninstall(dev);
12373: netdev_unlock_ops(dev);
12374: bpf_dev_bound_netdev_unregister(dev);
12375:
12376: netdev_offload_xstats_disable_all(dev);
12377:
12378: /* Notify protocols, that we are about to destroy
12379: * this device. They should clean all the things.
12380: */
12381: call_netdevice_notifiers(NETDEV_UNREGISTER, dev);
12382:
12383: if (!(dev->rtnl_link_ops && dev->rtnl_link_initializing))
12384: skb = rtmsg_ifinfo_build_skb(RTM_DELLINK, dev, ~0U, 0,
12385: GFP_KERNEL, NULL, 0,
12386: portid, nlh);
12387:
12388: /*
12389: * Flush the unicast and multicast chains
12390: */
12391: dev_uc_flush(dev);
12392: dev_mc_flush(dev);
12393:
12394: netdev_name_node_alt_flush(dev);
12395: netdev_name_node_free(dev->name_node);
12396:
12397: netdev_rss_contexts_free(dev);
12398:
12399: call_netdevice_notifiers(NETDEV_PRE_UNINIT, dev);
12400:
12401: if (dev->netdev_ops->ndo_uninit)
12402: dev->netdev_ops->ndo_uninit(dev);
12403:
12404: mutex_destroy(&dev->ethtool->rss_lock);
12405:
12406: net_shaper_flush_netdev(dev);
12407:
12408: if (skb)
12409: rtmsg_ifinfo_send(skb, dev, GFP_KERNEL, portid, nlh);
12410:
12411: /* Notifier chain MUST detach us all upper devices. */
12412: WARN_ON(netdev_has_any_upper_dev(dev));
12413: WARN_ON(netdev_has_any_lower_dev(dev));
12414:
12415: /* Remove entries from kobject tree */
12416: netdev_unregister_kobject(dev);
12417: #ifdef CONFIG_XPS
12418: /* Remove XPS queueing entries */
12419: netif_reset_xps_queues_gt(dev, 0);
12420: #endif
12421: }
12422:
12423: synchronize_net();
12424:
12425: list_for_each_entry(dev, head, unreg_list) {
12426: netdev_put(dev, &dev->dev_registered_tracker);
12427: net_set_todo(dev);
12428: cnt++;
12429: }
12430: atomic_add(cnt, &dev_unreg_count);
12431:
12432: list_del(head);
12433: }
SourceFile:net/core/dev.c]
|
| 37/2 |
2026/05/21 11:38 |
llm |
expert |
0m
Model:
Tokens:
input: 19956
output: 42
thoughts: 152
Thoughts: **Analyzing a Potential Use-After-Free Vulnerability**
Okay, so I'm looking at this and the system is flagging a WARNING. My immediate thought is, "Can this be exploited?" This timer, if pending, could potentially fire *after* the device has been freed. That's a classic recipe for a use-after-free, isn't it? Let me think... wait a second, the `dev->watchdog_timer` is part of the `struct net_device`. Right, so if the timer is still pending, that implies it wasn't explicitly deleted or perhaps it was re-armed.
If the timer is allowed to trigger *after* `net_device` has been freed, we'd have a use-after-free condition in `dev_watchdog()`. The crucial question here is where is the freeing happening, and is the timer explicitly handled there? But hold on, the code does mention `free_netdev()` is called later. Right, let's look and see if `free_netdev()` itself, or something else called within that process, handles deleting the timer. Okay, looking at the code, it seems the `net_device` is being freed in `netdev_run_todo()`. Now to follow the code down to see what happens to the timer...
|
| 38/2 |
2026/05/21 11:39 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:netdev_run_todo]
Results: map[SourceCode:11613: /* The sequence is:
11614: *
11615: * rtnl_lock();
11616: * ...
11617: * register_netdevice(x1);
11618: * register_netdevice(x2);
11619: * ...
11620: * unregister_netdevice(y1);
11621: * unregister_netdevice(y2);
11622: * ...
11623: * rtnl_unlock();
11624: * free_netdev(y1);
11625: * free_netdev(y2);
11626: *
11627: * We are invoked by rtnl_unlock().
11628: * This allows us to deal with problems:
11629: * 1) We can delete sysfs objects which invoke hotplug
11630: * without deadlocking with linkwatch via keventd.
11631: * 2) Since we run with the RTNL semaphore not held, we can sleep
11632: * safely in order to wait for the netdev refcnt to drop to zero.
11633: *
11634: * We must not return until all unregister events added during
11635: * the interval the lock was held have been completed.
11636: */
11637: void netdev_run_todo(void)
11638: {
11639: struct net_device *dev, *tmp;
11640: struct list_head list;
11641: int cnt;
11642: #ifdef CONFIG_LOCKDEP
11643: struct list_head unlink_list;
11644:
11645: list_replace_init(&net_unlink_list, &unlink_list);
11646:
11647: while (!list_empty(&unlink_list)) {
11648: dev = list_first_entry(&unlink_list, struct net_device,
11649: unlink_list);
11650: list_del_init(&dev->unlink_list);
11651: dev->nested_level = dev->lower_level - 1;
11652: }
11653: #endif
11654:
11655: /* Snapshot list, allow later requests */
11656: list_replace_init(&net_todo_list, &list);
11657:
11658: __rtnl_unlock();
11659:
11660: /* Wait for rcu callbacks to finish before next phase */
11661: if (!list_empty(&list))
11662: rcu_barrier();
11663:
11664: list_for_each_entry_safe(dev, tmp, &list, todo_list) {
11665: if (unlikely(dev->reg_state != NETREG_UNREGISTERING)) {
11666: netdev_WARN(dev, "run_todo but not unregistering\n");
11667: list_del(&dev->todo_list);
11668: continue;
11669: }
11670:
11671: netdev_lock(dev);
11672: WRITE_ONCE(dev->reg_state, NETREG_UNREGISTERED);
11673: netdev_unlock(dev);
11674: linkwatch_sync_dev(dev);
11675: }
11676:
11677: cnt = 0;
11678: while (!list_empty(&list)) {
11679: dev = netdev_wait_allrefs_any(&list);
11680: list_del(&dev->todo_list);
11681:
11682: /* paranoia */
11683: BUG_ON(netdev_refcnt_read(dev) != 1);
11684: BUG_ON(!list_empty(&dev->ptype_all));
11685: BUG_ON(!list_empty(&dev->ptype_specific));
11686: WARN_ON(rcu_access_pointer(dev->ip_ptr));
11687: WARN_ON(rcu_access_pointer(dev->ip6_ptr));
11688:
11689: netdev_do_free_pcpu_stats(dev);
11690: if (dev->priv_destructor)
11691: dev->priv_destructor(dev);
11692: if (dev->needs_free_netdev)
11693: free_netdev(dev);
11694:
11695: cnt++;
11696:
11697: /* Free network device */
11698: kobject_put(&dev->dev.kobj);
11699: }
11700: if (cnt && atomic_sub_and_test(cnt, &dev_unreg_count))
11701: wake_up(&netdev_unregistering_wq);
11702: }
SourceFile:net/core/dev.c]
|
| 39/2 |
2026/05/21 11:39 |
llm |
expert |
0m
Model:
Tokens:
input: 21900
output: 40
thoughts: 80
Thoughts: Okay, let me break this down.
**Analyzing Potential Network Device Timer Issues**
Right, so I'm looking at a situation where a network device watchdog timer is causing some trouble. The first thing that jumps out is this: if the timer is still *pending*, that strongly suggests that `netdev_watchdog_down()` didn't successfully remove it. Or, perhaps, it was re-armed somehow. That's a critical clue.
Now, the real red flag is if the timer actually *fires* *after* `free_netdev()` is called. That immediately screams "use-after-free" vulnerability. The system has freed the memory, and a timer is trying to execute code that's no longer valid. Classic UAF.
Hold on, though. `free_netdev()` calls `kfree(dev)`. So, the entire `net_device` structure itself is being freed there. The question is: Does `free_netdev()` explicitly handle deleting the timer as part of its cleanup process? Or is that the responsibility of a different function or sequence? If it *doesn't* delete the timer, we've got a gaping hole, and a race condition is practically guaranteed. Time to dig into the `free_netdev()` implementation and see what it actually does. I need to know if the timer is cleaned up there.
|
| 40/2 |
2026/05/21 11:39 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:free_netdev]
Results: map[SourceCode:12124: /**
12125: * free_netdev - free network device
12126: * @dev: device
12127: *
12128: * This function does the last stage of destroying an allocated device
12129: * interface. The reference to the device object is released. If this
12130: * is the last reference then it will be freed.Must be called in process
12131: * context.
12132: */
12133: void free_netdev(struct net_device *dev)
12134: {
12135: might_sleep();
12136:
12137: /* When called immediately after register_netdevice() failed the unwind
12138: * handling may still be dismantling the device. Handle that case by
12139: * deferring the free.
12140: */
12141: if (dev->reg_state == NETREG_UNREGISTERING) {
12142: ASSERT_RTNL();
12143: dev->needs_free_netdev = true;
12144: return;
12145: }
12146:
12147: WARN_ON(dev->cfg != dev->cfg_pending);
12148: kfree(dev->cfg);
12149: kfree(dev->ethtool);
12150: netif_free_tx_queues(dev);
12151: netif_free_rx_queues(dev);
12152:
12153: kfree(rcu_dereference_protected(dev->ingress_queue, 1));
12154:
12155: /* Flush device addresses */
12156: dev_addr_flush(dev);
12157:
12158: netdev_napi_exit(dev);
12159:
12160: netif_del_cpu_rmap(dev);
12161:
12162: ref_tracker_dir_exit(&dev->refcnt_tracker);
12163: #ifdef CONFIG_PCPU_DEV_REFCNT
12164: free_percpu(dev->pcpu_refcnt);
12165: dev->pcpu_refcnt = NULL;
12166: #endif
12167: free_percpu(dev->core_stats);
12168: dev->core_stats = NULL;
12169: free_percpu(dev->xdp_bulkq);
12170: dev->xdp_bulkq = NULL;
12171:
12172: netdev_free_phy_link_topology(dev);
12173:
12174: mutex_destroy(&dev->lock);
12175:
12176: /* Compatibility with error handling in drivers */
12177: if (dev->reg_state == NETREG_UNINITIALIZED ||
12178: dev->reg_state == NETREG_DUMMY) {
12179: kvfree(dev);
12180: return;
12181: }
12182:
12183: BUG_ON(dev->reg_state != NETREG_UNREGISTERED);
12184: WRITE_ONCE(dev->reg_state, NETREG_RELEASED);
12185:
12186: /* will free via device release */
12187: put_device(&dev->dev);
12188: }
SourceFile:net/core/dev.c]
|
| 41/2 |
2026/05/21 11:39 |
llm |
expert |
0m
Model:
Tokens:
input: 23326
output: 40
thoughts: 17
Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience, presented in the first person:
**Navigating the Device Lifecycle**
Right, so I'm focusing on the call to `put_device(&dev->dev)`. That's the key here. What immediately jumps out is that this function will ultimately trigger `netdev_release()`. This placement is crucial. It’s a definite clue that the intent here is a controlled teardown of the network device. The `put_device()` call, and consequently `netdev_release()`, is the final step within the device lifecycle, ensuring all resources associated with the network device are correctly deallocated, or at least have a mechanism to do so. I need to keep in mind, of course, that the implication of this function call is that it's going to release, reference counted, and it means the device is finally being put to rest.
|
| 42/2 |
2026/05/21 11:39 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:netdev_release]
Results: map[SourceCode:2238: /*
2239: * netdev_release -- destroy and free a dead device.
2240: * Called when last reference to device kobject is gone.
2241: */
2242: static void netdev_release(struct device *d)
2243: {
2244: struct net_device *dev = to_net_dev(d);
2245:
2246: BUG_ON(dev->reg_state != NETREG_RELEASED);
2247:
2248: /* no need to wait for rcu grace period:
2249: * device is dead and about to be freed.
2250: */
2251: kfree(rcu_access_pointer(dev->ifalias));
2252: kvfree(dev);
2253: }
SourceFile:net/core/net-sysfs.c]
|
| 43/2 |
2026/05/21 11:39 |
llm |
expert |
0m
Model:
Tokens:
input: 23838
output: 45
thoughts: 152
Thoughts: **Investigating a Potential Use-After-Free in Network Device Handling**
Okay, so I'm looking at this code and something's bugging me about the network device cleanup. I see that the watchdog timer isn't explicitly deleted within `free_netdev()` or `netdev_release()`. That immediately sets off alarm bells. If this timer is pending when the device is freed, we're definitely looking at a use-after-free scenario when that timer eventually fires.
Wait a second… `dev_shutdown()` *does* have a `WARN_ON(timer_pending(&dev->watchdog_timer));`. Okay, so the kernel developers were aware of this potential problem. But, is the timer actually *deleted* after `dev_shutdown()`? I need to verify that. Let's see if either `netdev_run_todo()` or `unregister_netdevice_many_notify()` handles the timer deletion. Nope, looks like they don't. Hmmm, the intention *should* be that `netdev_watchdog_down()` does the deletion. But the timer is still pending, so I need to check where it is failing to be cleared.
Why is the timer still running? Something's not right. I need to take another look at the logic in `netdev_watchdog_down()` to figure out what's going on. Something is not being executed properly.
|
| 44/2 |
2026/05/21 11:39 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/sched/sch_generic.c IncludeLines:true Name:netdev_watchdog_down]
Results: map[SourceCode: 571: static void netdev_watchdog_down(struct net_device *dev)
572: {
573: netif_tx_lock_bh(dev);
574: if (timer_delete(&dev->watchdog_timer))
575: netdev_put(dev, &dev->watchdog_dev_tracker);
576: netif_tx_unlock_bh(dev);
577: }
SourceFile:net/sched/sch_generic.c]
|
| 45/2 |
2026/05/21 11:39 |
llm |
expert |
0m
Model:
Tokens:
input: 24488
output: 42
thoughts: 108
Thoughts: **Concerning Asynchronous Timer Deletion and Lock Contention**
Okay, so I'm looking at this code and something's just snagging my attention. I see they're using `timer_delete()` and not `timer_delete_sync()`. That immediately rings some alarm bells in my head, because I know `timer_delete()` is *asynchronous*. It doesn't wait for the timer to actually complete its execution if it's currently running on a different CPU. It just returns 0 immediately. The timer will finish *eventually*, but not right away.
The issue is that `netdev_watchdog_down()` calls `netif_tx_lock_bh(dev)`. That's a bottom-half lock, which means it disables interrupts. The timer callback, `dev_watchdog()`, also takes `netif_tx_lock(dev)`. Now I'm starting to visualize a potential race condition and some pretty nasty lock contention scenarios that could manifest when the timer is asynchronously deleted. It's time to dig deeper and determine exactly how these two are synchronized and whether we're going to experience any adverse effects.
|
| 46/2 |
2026/05/21 11:39 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/sched/sch_generic.c IncludeLines:true Name:dev_watchdog]
Results: map[SourceCode: 500: static void dev_watchdog(struct timer_list *t)
501: {
502: struct net_device *dev = timer_container_of(dev, t, watchdog_timer);
503: bool release = true;
504:
505: spin_lock(&dev->tx_global_lock);
506: if (!qdisc_tx_is_noop(dev)) {
507: if (netif_device_present(dev) &&
508: netif_running(dev) &&
509: netif_carrier_ok(dev)) {
510: unsigned int timedout_ms = 0;
511: unsigned int i;
512: unsigned long trans_start;
513: unsigned long oldest_start = jiffies;
514:
515: for (i = 0; i < dev->num_tx_queues; i++) {
516: struct netdev_queue *txq;
517:
518: txq = netdev_get_tx_queue(dev, i);
519: if (!netif_xmit_stopped(txq))
520: continue;
521:
522: /* Paired with WRITE_ONCE() + smp_mb...() in
523: * netdev_tx_sent_queue() and netif_tx_stop_queue().
524: */
525: smp_mb();
526: trans_start = READ_ONCE(txq->trans_start);
527:
528: if (time_after(jiffies, trans_start + dev->watchdog_timeo)) {
529: timedout_ms = jiffies_to_msecs(jiffies - trans_start);
530: atomic_long_inc(&txq->trans_timeout);
531: break;
532: }
533: if (time_after(oldest_start, trans_start))
534: oldest_start = trans_start;
535: }
536:
537: if (unlikely(timedout_ms)) {
538: trace_net_dev_xmit_timeout(dev, i);
539: netdev_crit(dev, "NETDEV WATCHDOG: CPU: %d: transmit queue %u timed out %u ms\n",
540: raw_smp_processor_id(),
541: i, timedout_ms);
542: netif_freeze_queues(dev);
543: dev->netdev_ops->ndo_tx_timeout(dev, i);
544: netif_unfreeze_queues(dev);
545: }
546: if (!mod_timer(&dev->watchdog_timer,
547: round_jiffies(oldest_start +
548: dev->watchdog_timeo)))
549: release = false;
550: }
551: }
552: spin_unlock(&dev->tx_global_lock);
553:
554: if (release)
555: netdev_put(dev, &dev->watchdog_dev_tracker);
556: }
SourceFile:net/sched/sch_generic.c]
|
| 47/2 |
2026/05/21 11:39 |
llm |
expert |
0m
Model:
Tokens:
input: 25836
output: 46
thoughts: 88
Thoughts: Okay, here's my interpretation of this scenario, focusing on the lock contention issue:
**Analyzing Potential Lock Contention in Network Device Driver**
Right, so I'm looking at a potential lock contention issue here. I see that the `dev_watchdog()` function uses `spin_lock(&dev->tx_global_lock)`. Okay, standard stuff, protecting critical sections. But then, I notice `netdev_watchdog_down()` also has `netif_tx_lock_bh(dev)`. That's where things get interesting. I know `netif_tx_lock_bh(dev)` can be tricky... Wait a second, it also *takes* `dev->tx_global_lock`? That's a red flag!
This looks like a classic double-lock scenario. If `dev_watchdog()` grabs `dev->tx_global_lock` first and then we end up in `netdev_watchdog_down()` trying to acquire `netif_tx_lock_bh(dev)` – which in turn attempts to grab the same `dev->tx_global_lock` – we're heading straight into a deadlock. Or, vice-versa, depending on the lock order, or even just high contention.
I need to investigate `netif_tx_lock_bh` further. I'll need to confirm exactly how it interacts with that `dev->tx_global_lock` to understand the full locking hierarchy and pinpoint how these functions are being called. I need to be sure that these locks are always acquired in the same order, to eliminate the potential for deadlocks. It seems like the architecture here is not ideal for proper protection, and I'll need to check the entire architecture to evaluate potential race conditions.
|
| 48/2 |
2026/05/21 11:39 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/sched/sch_generic.c IncludeLines:true Name:netif_tx_lock_bh]
Results: map[SourceCode:4792: static inline void netif_tx_lock_bh(struct net_device *dev)
4793: {
4794: local_bh_disable();
4795: netif_tx_lock(dev);
4796: }
SourceFile:include/linux/netdevice.h]
|
| 49/2 |
2026/05/21 11:39 |
llm |
expert |
0m
Model:
Tokens:
input: 26446
output: 43
thoughts: 9
Thoughts: Okay, here's my interpretation of that thought process, tailored for an expert audience, and written in the first person:
**My Perspective on `netif_tx_lock`**
Right, so I'm thinking about the role of `netif_tx_lock` in this context. It's a critical synchronization primitive, of course. My initial thoughts center on its core purpose: protecting the shared resources accessed during the network interface transmit path. I immediately consider the potential for contention and the implications for performance. We're talking about regulating access to the transmit queue, descriptors, and potentially even the hardware itself.
My mind immediately races through the possible implementations I've seen. Is this a spinlock? A mutex? What's the cost of acquisition and release? Is it a per-interface lock or a global one, and how does that design choice impact scalability, especially with multi-core systems and high-throughput network cards?
I need to quickly assess the lock's scope. What exactly *is* being protected? Is it only the queue manipulation, or does it extend into the hardware register writes for descriptor setup and DMA initiation? If it’s broader, I need to consider how this affects interrupt handling. Do we need to disable interrupts to avoid deadlocks? And are there any re-entrant scenarios I need to be acutely aware of, such as the NAPI path potentially accessing the same resources?
I then consider how this interacts with the rest of the kernel's networking stack, like the driver's transmit routines. The driver will be calling this lock. How is it used to make sure that transmit contexts don’t trample each other and corruption occurs? The more I reflect on the potential usage, the more I need to evaluate the implications for the transmit path’s performance characteristics, with the potential for blocking.
|
| 50/2 |
2026/05/21 11:39 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:include/linux/netdevice.h IncludeLines:true Name:netif_tx_lock]
Results: map[SourceCode: 470: void netif_tx_lock(struct net_device *dev)
471: {
472: spin_lock(&dev->tx_global_lock);
473: netif_freeze_queues(dev);
474: }
SourceFile:net/sched/sch_generic.c]
|
| 51/2 |
2026/05/21 11:39 |
llm |
expert |
0m
Model:
Tokens:
input: 26986
output: 44
thoughts: 382
Thoughts: **Race Conditions and Device Deactivation**
Okay, so I'm looking at this code and I see that these things are serialized by `dev->tx_global_lock`. That makes sense. But here's where it gets hairy. If `dev_watchdog` is running on, say, CPU A, it holds that lock. Now, `netdev_watchdog_down` on CPU B comes along and tries to acquire the same lock. It'll block, obviously.
Now, let's say `dev_watchdog` finishes its work and re-arms the timer using `mod_timer`. Then `netdev_watchdog_down` gets the lock and calls `timer_delete` to shut down the timer. Boom, no problem, it's all handled.
Or, if `netdev_watchdog_down` runs first, it grabs the lock, deletes the timer, and we are good.
But, wait a minute, here's a wrinkle... what if `dev_watchdog` is running on CPU A, but hasn't *yet* acquired the `tx_global_lock`? `netdev_watchdog_down` on CPU B comes along, acquires the lock, and calls `timer_delete`. Since the timer is *currently* running, `timer_delete` returns 0 – it *doesn't* delete the timer because it's in use. Then `netdev_watchdog_down` releases the lock.
Then, `dev_watchdog` on CPU A *finally* gets the lock. It checks a bunch of conditions: `qdisc_tx_is_noop(dev)`, `netif_device_present(dev)`, `netif_running(dev)`, and `netif_carrier_ok(dev)`. If they are all true, it *might* call `mod_timer` again and re-arm that timer, even though `netdev_watchdog_down` was *just* called. That looks like a race condition!
And now, a potentially more significant point. If `netdev_watchdog_down` was called, doesn't that mean the device is, in essence, being deactivated? I need to know if `dev_deactivate_many` actually sets `qdisc_tx_is_noop(dev)`. If it does, maybe this isn't a problem, or is at least handled with a different mechanism!
|
| 52/2 |
2026/05/21 11:39 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/sched/sch_generic.c IncludeLines:true Name:dev_deactivate_many]
Results: map[SourceCode:1347: /**
1348: * dev_deactivate_many - deactivate transmissions on several devices
1349: * @head: list of devices to deactivate
1350: *
1351: * This function returns only when all outstanding transmissions
1352: * have completed, unless all devices are in dismantle phase.
1353: */
1354: void dev_deactivate_many(struct list_head *head)
1355: {
1356: bool sync_needed = false;
1357: struct net_device *dev;
1358:
1359: list_for_each_entry(dev, head, close_list) {
1360: netdev_for_each_tx_queue(dev, dev_deactivate_queue,
1361: &sync_needed);
1362: if (dev_ingress_queue(dev))
1363: dev_deactivate_queue(dev, dev_ingress_queue(dev),
1364: &sync_needed);
1365:
1366: netdev_watchdog_down(dev);
1367: }
1368:
1369: /* Wait for outstanding qdisc enqueuing calls. */
1370: if (sync_needed)
1371: synchronize_net();
1372:
1373: list_for_each_entry(dev, head, close_list) {
1374: netdev_for_each_tx_queue(dev, dev_reset_queue, NULL);
1375:
1376: if (dev_ingress_queue(dev))
1377: dev_reset_queue(dev, dev_ingress_queue(dev), NULL);
1378: }
1379:
1380: /* Wait for outstanding qdisc_run calls. */
1381: list_for_each_entry(dev, head, close_list) {
1382: while (some_qdisc_is_busy(dev)) {
1383: /* wait_event() would avoid this sleep-loop but would
1384: * require expensive checks in the fast paths of packet
1385: * processing which isn't worth it.
1386: */
1387: schedule_timeout_uninterruptible(1);
1388: }
1389: }
1390: }
SourceFile:net/sched/sch_generic.c]
|
| 53/2 |
2026/05/21 11:39 |
llm |
expert |
0m
Model:
Tokens:
input: 28618
output: 44
thoughts: 34
Thoughts: Okay, here's my interpretation of that process, explained as if it were my thought process, keeping in mind my expertise:
**Deactivating Multiple Device Transmit Queues**
Alright, so the `dev_deactivate_many` function... right, I'm thinking, this is a wrapper, a higher-level function, yeah? It's tasked with deactivating, or essentially putting offline, a bunch of network device transmit queues. It's not doing the dirty work itself, no, no. Instead, it iterates through each of these transmit queues.
And for *each* queue, it calls this other function, `dev_deactivate_queue`. That's where the actual work happens. `dev_deactivate_queue` is the critical point, the workhorse.
My mental model of what's happening at that level, the immediate consequence, is that the queue's traffic shaping discipline, its `qdisc`, is being *set* to `noop_qdisc`. Ah, right, so `noop_qdisc`... that's the "no operation" queue discipline. Meaning that no packets are getting scheduled. It's essentially a kill-switch, a way to ensure that nothing gets transmitted from that queue. Perfect. It stops traffic cold, preventing any further transmissions on that particular queue. And it does this for *every* transmit queue being deactivated. So that makes sense. It's the simplest way to get the queues offline.
|
| 54/2 |
2026/05/21 11:39 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/sched/sch_generic.c IncludeLines:true Name:dev_deactivate_queue]
Results: map[SourceCode:1277: static void dev_deactivate_queue(struct net_device *dev,
1278: struct netdev_queue *dev_queue,
1279: void *_sync_needed)
1280: {
1281: bool *sync_needed = _sync_needed;
1282: struct Qdisc *qdisc;
1283:
1284: qdisc = rtnl_dereference(dev_queue->qdisc);
1285: if (qdisc) {
1286: if (qdisc->enqueue)
1287: *sync_needed = true;
1288: qdisc_deactivate(qdisc);
1289: rcu_assign_pointer(dev_queue->qdisc, &noop_qdisc);
1290: }
1291: }
SourceFile:net/sched/sch_generic.c]
|
| 55/2 |
2026/05/21 11:39 |
llm |
expert |
0m
Model:
Tokens:
input: 29260
output: 46
thoughts: 23
Thoughts: Okay, here's my interpretation of that thought process, assuming I'm an expert in network device driver internals, and rendering it as a first-person internal monologue:
**Investigating `dev_watchdog` and Its Dependencies**
Okay, so I'm digging into `dev_watchdog` again. My mind immediately goes to the crucial check it performs: `!qdisc_tx_is_noop(dev)`. That's the core of it, isn't it? That check is fundamental. It's essentially saying, "Only trigger the watchdog if the transmit queue isn't completely idle or doing nothing." I need to fully understand the implications of this condition. What exactly constitutes a "noop" Qdisc in this context? Is it simply an empty queue, or something more nuanced like a Qdisc that's configured but not actively forwarding traffic? Knowing the definition of a "noop" Qdisc is *absolutely* central to figuring out when the watchdog *should* and *shouldn't* fire. If I can understand what constitutes that "noop" then I can understand the logic and control flow of `dev_watchdog`. Right now, I can see how it is directly influenced by `qdisc_tx_is_noop(dev)`, and I need to start asking some detailed questions about that function.
|
| 56/2 |
2026/05/21 11:39 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/sched/sch_generic.c IncludeLines:true Name:qdisc_tx_is_noop]
Results: map[SourceCode: 834: /* Is the device using the noop qdisc on all queues? */
835: static inline bool qdisc_tx_is_noop(const struct net_device *dev)
836: {
837: unsigned int i;
838:
839: for (i = 0; i < dev->num_tx_queues; i++) {
840: struct netdev_queue *txq = netdev_get_tx_queue(dev, i);
841: if (rcu_access_pointer(txq->qdisc) != &noop_qdisc)
842: return false;
843: }
844: return true;
845: }
SourceFile:include/net/sch_generic.h]
|
| 57/2 |
2026/05/21 11:39 |
llm |
expert |
0m
Model:
Tokens:
input: 29838
output: 46
thoughts: 1032
Thoughts: **Analyzing a Potential Race Condition in the Network Device Watchdog**
Okay, so I'm digging into this interaction between `dev_deactivate_many`, `dev_watchdog`, and `netdev_watchdog_down`, and I think there's a possible race. It hinges on how the `noop_qdisc` is set and when the watchdog timer is manipulated.
If `dev_deactivate_many` sets all the transmit queues to the `noop_qdisc`, then `qdisc_tx_is_noop(dev)` will return `true`, right? And if `dev_watchdog` runs *after* this happens, it will see the `noop_qdisc` and not re-arm the timer. But `dev_deactivate_queue` is using `rcu_assign_pointer` to set the `dev_queue->qdisc` to `&noop_qdisc`, and `qdisc_tx_is_noop` uses `rcu_access_pointer`, so that's where things get interesting. Is there a way this can happen out of order?
Let me lay out the scenario: CPU A is running `dev_watchdog`. It hasn't grabbed the `tx_global_lock` yet. Then, CPU B runs `dev_deactivate_many`. That calls `dev_deactivate_queue`, and this *sets* `dev_queue->qdisc` to `noop_qdisc`. Now, CPU B calls `netdev_watchdog_down`, grabs `tx_global_lock`, and calls `timer_delete`. Since `dev_watchdog` is running on CPU A, `timer_delete` will return 0. CPU B releases `tx_global_lock`.
Now, CPU A acquires `tx_global_lock`. It checks `qdisc_tx_is_noop(dev)`. Since CPU B already set the `noop_qdisc`, `qdisc_tx_is_noop` *will* return true! This means `dev_watchdog` won't re-arm the timer, and we'll get into a problem.
But, wait... what if `dev_watchdog` checked `qdisc_tx_is_noop(dev)` *before* CPU B set `noop_qdisc`? Actually, it checks `qdisc_tx_is_noop(dev)` *inside* the `spin_lock(&dev->tx_global_lock)`. So, if CPU B acquired `tx_global_lock` in `netdev_watchdog_down`, CPU A must be either waiting for the lock or hasn't even reached it yet. If CPU A acquires the lock *after* CPU B releases it, then CPU A *will* see the updated `noop_qdisc`, because `spin_lock` provides a full memory barrier. So the timer won't be re-armed.
What if CPU A *gets* the lock *before* CPU B?
CPU A acquires `tx_global_lock`. Checks `qdisc_tx_is_noop(dev)`. It's false at this point. Checks all the `netif_` flags; everything is good. Re-arms the timer with `mod_timer`. Then releases `tx_global_lock`.
Then CPU B runs `dev_deactivate_many`. If CPU B *hasn't* run `dev_deactivate_queue` yet, it will now do so, setting the `noop_qdisc`. Now CPU B calls `netdev_watchdog_down` and grabs the `tx_global_lock`. `timer_delete` *succeeds* because the timer is not running. And the timer gets deleted.
Okay, what if `dev_watchdog` is running but is blocked on `tx_global_lock` because some *other* code is holding it? It doesn't matter. The only thing that matters is the order of `tx_global_lock` acquisition between `dev_watchdog` and `netdev_watchdog_down`.
But what if `dev_watchdog` re-arms the timer to fire *immediately*? Then `netdev_watchdog_down` calls `timer_delete`. If it's running again, `timer_delete` returns 0. But `netdev_watchdog_down` holds `tx_global_lock`, and the new `dev_watchdog` will block. When `netdev_watchdog_down` releases `tx_global_lock`, the new `dev_watchdog` will acquire it, see `noop_qdisc` and *not* re-arm. The timer will finish and not
|
| 58/2 |
2026/05/21 11:40 |
tool |
codesearch-find-references |
0m
Args: map[ContextFile:net/sched/sch_generic.c IncludeSnippetLines:0 Name:netdev_watchdog_up]
Results: map[References:[map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:dev_activate SourceFile:net/sched/sch_generic.c SourceLine:1264] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:netif_carrier_on SourceFile:net/sched/sch_generic.c SourceLine:593] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:netif_device_attach SourceFile:net/core/dev.c SourceLine:3481]] TruncatedOutput:false]
|
| 59/2 |
2026/05/21 11:40 |
llm |
expert |
0m
Model:
Tokens:
input: 32114
output: 43
thoughts: 247
Thoughts: **Investigating a Network Device Driver Issue**
Okay, so I'm looking at this code and trying to piece together a potential race condition. My initial thought process revolves around how the watchdog timer is triggered. I see `netif_device_attach` and `netif_carrier_on` both call `netdev_watchdog_up`. Furthermore, `dev_activate` is also triggering that watchdog function. Now, my question is: could there be a scenario where either `netif_carrier_on` or `netif_device_attach` are called *after* a `dev_deactivate_many` call? That seems like a prime area for a problem.
The bug report is illuminating. This is a USB disconnect scenario, and the logs are showing the disconnect and the subsequent `usbnet_disconnect` call. The crucial part is that the warning is originating from within `dev_shutdown`, which is being triggered when a USB device gets disconnected. The warning indicates some kind of problem within the scheduler. My instincts are telling me this disconnect sequence might be happening just at the moment of one of these calls, and that's potentially the source of the issue. I need to dig deeper into the timing and sequencing.
|
| 60/2 |
2026/05/21 11:40 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:drivers/net/usb/usbnet.c IncludeLines:true Name:usbnet_disconnect]
Results: map[SourceCode:1655: void usbnet_disconnect(struct usb_interface *intf)
1656: {
1657: struct usbnet *dev;
1658: struct usb_device *xdev;
1659: struct net_device *net;
1660: struct urb *urb;
1661:
1662: dev = usb_get_intfdata(intf);
1663: usb_set_intfdata(intf, NULL);
1664: if (!dev)
1665: return;
1666: usbnet_mark_going_away(dev);
1667:
1668: xdev = interface_to_usbdev(intf);
1669:
1670: netif_info(dev, probe, dev->net, "unregister '%s' usb-%s-%s, %s\n",
1671: intf->dev.driver->name,
1672: xdev->bus->bus_name, xdev->devpath,
1673: dev->driver_info->description);
1674:
1675: net = dev->net;
1676: unregister_netdev(net);
1677:
1678: cancel_work_sync(&dev->kevent);
1679:
1680: while ((urb = usb_get_from_anchor(&dev->deferred))) {
1681: dev_kfree_skb(urb->context);
1682: kfree(urb->sg);
1683: usb_free_urb(urb);
1684: }
1685:
1686: if (dev->driver_info->unbind)
1687: dev->driver_info->unbind(dev, intf);
1688:
1689: usb_kill_urb(dev->interrupt);
1690: usb_free_urb(dev->interrupt);
1691: kfree(dev->padding_pkt);
1692:
1693: free_netdev(net);
1694: }
SourceFile:drivers/net/usb/usbnet.c]
|
| 61/2 |
2026/05/21 11:40 |
llm |
expert |
0m
Model:
Tokens:
input: 33266
output: 41
thoughts: 132
Thoughts: **My Analysis of a Potential USBnet Disconnection Race Condition**
Okay, so I'm looking at the `usbnet_disconnect` function, and the first thing that jumps out is that it calls `unregister_netdev(net)`. Standard network device teardown. Then, that leads to `unregister_netdevice_queue`, and ultimately, `unregister_netdevice_many_notify`. Fine, seems reasonable so far.
Wait a second… I also see `cancel_work_sync(&dev->kevent)` in `usbnet_disconnect`. That’s a workqueue item, which is potentially running *concurrently* with the network device unregistration process! My spidey sense is tingling. Could `dev->kevent` be operating while `unregister_netdev` is in full swing?
Let's drill down... I remember that `unregister_netdev` will block until unregistration is *complete*. But if `dev->kevent` is scheduled on a *different CPU*, it's entirely possible that it could be running simultaneously. We could have a race condition on our hands.
Time to investigate what `dev->kevent` actually *does*. This could be the source of a nasty bug.
|
| 62/2 |
2026/05/21 11:40 |
tool |
codesearch-definition-source |
0m
Error:
Args: map[ContextFile:drivers/net/usb/usbnet.c IncludeLines:true Name:kevent]
Results: map[SourceCode: SourceFile:]
|
| 63/2 |
2026/05/21 11:40 |
llm |
expert |
0m
Model:
Tokens:
input: 33718
output: 20
thoughts: 22
Thoughts: Okay, here's my interpretation of that thought process, as if it were my own:
**Investigating a Network Driver's Event Handling**
Right, time to dive into this. The objective is clear: I need to understand how the `kevent` mechanism is used within the USB network driver. That means I need to start my search within the relevant source code – specifically, the `drivers/net/usb/usbnet.c` file. This is where the core logic of the USB network interface likely resides. The presence of `kevent` points towards the driver's event handling or notification implementation. My focus now needs to be on how `kevent` is integrated within the data flow of the driver. Is it used for incoming data, outgoing data, or something else entirely? I anticipate finding calls to `kevent` with specific arguments indicating event types, identifiers, and perhaps data associated with the network device's status. Let's get searching.
|
| 64/2 |
2026/05/21 11:40 |
tool |
grepper |
0m
Args: map[Expression:INIT_WORK.*kevent]
Results: map[Output:drivers/media/rc/mceusb.c=1651=static int mceusb_dev_probe(struct usb_interface *intf,
--
drivers/media/rc/mceusb.c-1763- */
drivers/media/rc/mceusb.c:1764: INIT_WORK(&ir->kevent, mceusb_deferred_kevent);
drivers/media/rc/mceusb.c-1765-
--
drivers/net/usb/sierra_net.c=661=static int sierra_net_bind(struct usbnet *dev, struct usb_interface *intf)
--
drivers/net/usb/sierra_net.c-736- /* Use the shared workqueue */
drivers/net/usb/sierra_net.c:737: INIT_WORK(&priv->sierra_net_kevent, sierra_net_kevent);
drivers/net/usb/sierra_net.c-738-
--
drivers/net/usb/usbnet.c=1721=usbnet_probe(struct usb_interface *udev, const struct usb_device_id *prod)
--
drivers/net/usb/usbnet.c-1776- INIT_WORK(&dev->bh_work, usbnet_bh_work);
drivers/net/usb/usbnet.c:1777: INIT_WORK(&dev->kevent, usbnet_deferred_kevent);
drivers/net/usb/usbnet.c-1778- init_usb_anchor(&dev->deferred);
]
|
| 65/2 |
2026/05/21 11:40 |
llm |
expert |
0m
Model:
Tokens:
input: 34352
output: 46
thoughts: 26
Thoughts: Here's my take on those thoughts, tailored for an expert audience:
**Diving into usbnet_deferred_kevent**
Alright, let's zero in on `usbnet_deferred_kevent`. I know this is in `drivers/net/usb/usbnet.c`, which immediately tells me we're dealing with USB network device drivers. Now, "deferred" in the function name is the key. That suggests this function is likely handling events related to the USB network interface that need to be processed asynchronously, perhaps due to interrupt context limitations or resource constraints. I need to understand how these events are being deferred and what specific events this function is responsible for. I'm guessing this involves packet handling, connection state changes, or possibly even power management related to the USB connection. I need to see if it's tied to workqueues, softirqs, or maybe even timers. I'll be looking for potential race conditions, locking issues, and performance bottlenecks, especially under heavy network load or with frequent device plug/unplug scenarios. Let's see what's what.
|
| 66/2 |
2026/05/21 11:40 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:drivers/net/usb/usbnet.c IncludeLines:true Name:usbnet_deferred_kevent]
Results: map[SourceCode:1171: /* work that cannot be done in interrupt context uses keventd.
1172: *
1173: * NOTE: with 2.5 we could do more of this using completion callbacks,
1174: * especially now that control transfers can be queued.
1175: */
1176: static void
1177: usbnet_deferred_kevent(struct work_struct *work)
1178: {
1179: struct usbnet *dev =
1180: container_of(work, struct usbnet, kevent);
1181: int status;
1182:
1183: /* usb_clear_halt() needs a thread context */
1184: if (test_bit(EVENT_TX_HALT, &dev->flags)) {
1185: unlink_urbs(dev, &dev->txq);
1186: status = usb_autopm_get_interface(dev->intf);
1187: if (status < 0)
1188: goto fail_pipe;
1189: status = usb_clear_halt(dev->udev, dev->out);
1190: usb_autopm_put_interface(dev->intf);
1191: if (status < 0 &&
1192: status != -EPIPE &&
1193: status != -ESHUTDOWN) {
1194: if (netif_msg_tx_err(dev))
1195: fail_pipe:
1196: netdev_err(dev->net, "can't clear tx halt, status %d\n",
1197: status);
1198: } else {
1199: clear_bit(EVENT_TX_HALT, &dev->flags);
1200: if (status != -ESHUTDOWN)
1201: netif_wake_queue(dev->net);
1202: }
1203: }
1204: if (test_bit(EVENT_RX_HALT, &dev->flags)) {
1205: unlink_urbs(dev, &dev->rxq);
1206: status = usb_autopm_get_interface(dev->intf);
1207: if (status < 0)
1208: goto fail_halt;
1209: status = usb_clear_halt(dev->udev, dev->in);
1210: usb_autopm_put_interface(dev->intf);
1211: if (status < 0 &&
1212: status != -EPIPE &&
1213: status != -ESHUTDOWN) {
1214: if (netif_msg_rx_err(dev))
1215: fail_halt:
1216: netdev_err(dev->net, "can't clear rx halt, status %d\n",
1217: status);
1218: } else {
1219: clear_bit(EVENT_RX_HALT, &dev->flags);
1220: if (!usbnet_going_away(dev))
1221: queue_work(system_bh_wq, &dev->bh_work);
1222: }
1223: }
1224:
1225: /* work could resubmit itself forever if memory is tight */
1226: if (test_bit(EVENT_RX_MEMORY, &dev->flags)) {
1227: struct urb *urb = NULL;
1228: int resched = 1;
1229:
1230: if (netif_running(dev->net))
1231: urb = usb_alloc_urb(0, GFP_KERNEL);
1232: else
1233: clear_bit(EVENT_RX_MEMORY, &dev->flags);
1234: if (urb != NULL) {
1235: clear_bit(EVENT_RX_MEMORY, &dev->flags);
1236: status = usb_autopm_get_interface(dev->intf);
1237: if (status < 0) {
1238: usb_free_urb(urb);
1239: goto fail_lowmem;
1240: }
1241: if (rx_submit(dev, urb, GFP_KERNEL) == -ENOLINK)
1242: resched = 0;
1243: usb_autopm_put_interface(dev->intf);
1244: fail_lowmem:
1245: if (resched)
1246: if (!usbnet_going_away(dev))
1247: queue_work(system_bh_wq, &dev->bh_work);
1248: }
1249: }
1250:
1251: if (test_bit (EVENT_LINK_RESET, &dev->flags)) {
1252: const struct driver_info *info = dev->driver_info;
1253: int retval = 0;
1254:
1255: clear_bit(EVENT_LINK_RESET, &dev->flags);
1256: status = usb_autopm_get_interface(dev->intf);
1257: if (status < 0)
1258: goto skip_reset;
1259: if(info->link_reset && (retval = info->link_reset(dev)) < 0) {
1260: usb_autopm_put_interface(dev->intf);
1261: skip_reset:
1262: netdev_info(dev->net, "link reset failed (%d) usbnet usb-%s-%s, %s\n",
1263: retval,
1264: dev->udev->bus->bus_name,
1265: dev->udev->devpath,
1266: info->description);
1267: } else {
1268: usb_autopm_put_interface(dev->intf);
1269: }
1270:
1271: /* handle link change from link resetting */
1272: __handle_link_change(dev);
1273: }
1274:
1275: if (test_bit(EVENT_LINK_CHANGE, &dev->flags))
1276: __handle_link_change(dev);
1277:
1278: if (test_bit(EVENT_SET_RX_MODE, &dev->flags))
1279: __handle_set_rx_mode(dev);
1280:
1281:
1282: if (dev->flags)
1283: netdev_dbg(dev->net, "kevent done, flags = 0x%lx\n", dev->flags);
1284: }
SourceFile:drivers/net/usb/usbnet.c]
|
| 67/2 |
2026/05/21 11:40 |
llm |
expert |
0m
Model:
Tokens:
input: 36518
output: 45
thoughts: 21
Thoughts: Okay, here's my interpretation of that thought process, tailored for an expert audience, and presented as if it were my own internal monologue:
**My Internal USB Networking Analysis**
Alright, so the `usbnet_deferred_kevent` function, that's the key here. It's handling deferred kernel events, and I need to keep in mind the potential race conditions and timing issues that can introduce. Specifically, it's calling `__handle_link_change(dev)`. That's the core. `__handle_link_change` is where the *real* logic for managing link state transitions in a USB network device resides. I immediately think: What's the context? What triggered this kevent? Is this a hotplug event, a disconnect, a state change, or a re-enumeration of sorts?
I need to understand what "dev" actually represents in this particular context. Is this a network_device pointer, or something else. I'll need to double check the struct definition, as well as ensure that a `netdev_notifier_call` is triggered. I'll need to confirm what information is being passed through this call, and ensure there aren't any potential race conditions with other drivers.
Essentially, I need to trace *from* this call to figure out:
* **What exactly triggers this `kevent`?** Is it a specific condition or timer?
* **What state changes does `__handle_link_change` perform?** Does it do anything with the underlying USB device or network stack? Are there locks involved?
* **What data structures are being modified?** Which locks might be necessary?
* **What is the impact of a fast or slow device?**
* **What are the failure modes I need to consider?** Will a failed `__handle_link_change` result in data loss, or just a stalled device?
I need to check for potential race conditions with the USB stack, any interrupt handlers, and any other parts of the network stack that might be interacting with this device. It's a quick call, but it's the gateway for understanding a critical part of how this USB networking device manages its link state and needs to be treated with care.
|
| 68/2 |
2026/05/21 11:40 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:drivers/net/usb/usbnet.c IncludeLines:true Name:__handle_link_change]
Results: map[SourceCode:1128: static void __handle_link_change(struct usbnet *dev)
1129: {
1130: if (!test_bit(EVENT_DEV_OPEN, &dev->flags))
1131: return;
1132:
1133: if (test_and_clear_bit(EVENT_LINK_CARRIER_ON, &dev->flags))
1134: netif_carrier_on(dev->net);
1135:
1136: if (!netif_carrier_ok(dev->net)) {
1137: /* kill URBs for reading packets to save bus bandwidth */
1138: unlink_urbs(dev, &dev->rxq);
1139:
1140: /*
1141: * tx_timeout will unlink URBs for sending packets and
1142: * tx queue is stopped by netcore after link becomes off
1143: */
1144: } else {
1145: /* submitting URBs for reading packets */
1146: queue_work(system_bh_wq, &dev->bh_work);
1147: }
1148:
1149: /* hard_mtu or rx_urb_size may change during link change */
1150: usbnet_update_max_qlen(dev);
1151:
1152: clear_bit(EVENT_LINK_CHANGE, &dev->flags);
1153: }
SourceFile:drivers/net/usb/usbnet.c]
|
| 69/2 |
2026/05/21 11:40 |
llm |
expert |
0m
Model:
Tokens:
input: 37483
output: 41
thoughts: 205
Thoughts: **Race Condition Concern in Network Device Shutdown**
Okay, so I'm looking at this driver's shutdown sequence and a potential race condition is starting to jump out at me. The `__handle_link_change` function, which I know is called when a link state change occurs, triggers `netif_carrier_on(dev->net)`. That, in turn, calls `netdev_watchdog_up(dev)`. That's the normal, expected behavior.
But then I see in `usbnet_disconnect` that the driver calls `unregister_netdev(net)` *before* `cancel_work_sync(&dev->kevent)`. This immediately raises a red flag. I know that `unregister_netdev` will eventually call `dev_shutdown`. Now, `dev_shutdown` specifically checks for a pending watchdog timer using `WARN_ON(timer_pending(&dev->watchdog_timer))`. This is an important check, but I'm worried it's not the complete picture.
The problem, as I see it, is that `dev->kevent` could be running concurrently with the shutdown process. If that happens, and `dev->kevent` is triggered while the device is in the process of being unregistered, it could still call `netif_carrier_on(dev->net)`. Now, `netif_carrier_on` does check `netif_running(dev)`.
The crucial question is: is `netif_running(dev)` still true during the execution of `unregister_netdev` or just after? If it remains true *during* the shutdown, then this could lead to a situation where the driver tries to update the net device state even after it's been unregistered, possibly leading to some serious issues. I need to dive deeper into how `netif_running` behaves during the `unregister_netdev` sequence. This feels like a classic race condition that I need to confirm with some code reviews and tracing.
|
| 70/2 |
2026/05/21 11:40 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:include/linux/netdevice.h IncludeLines:true Name:netif_running]
Results: map[SourceCode:3948: /**
3949: * netif_running - test if up
3950: * @dev: network device
3951: *
3952: * Test if the device has been brought up.
3953: */
3954: static inline bool netif_running(const struct net_device *dev)
3955: {
3956: return test_bit(__LINK_STATE_START, &dev->state);
3957: }
SourceFile:include/linux/netdevice.h]
|
| 71/2 |
2026/05/21 11:40 |
llm |
expert |
0m
Model:
Tokens:
input: 38316
output: 42
thoughts: 26
Thoughts: Okay, here's my take on those thoughts, framed as an expert's internal monologue:
**Investigating the Lifecycle of `__LINK_STATE_START`**
Right, so I'm trying to pin down the precise moment that `__LINK_STATE_START` gets cleared. Seems like a critical flag for link state management. The note says it's cleared in `__dev_close_many`. Hmm, that makes sense on a conceptual level. `__dev_close_many` handles the bulk device closing operations, so that's likely where the interface is being shut down and the link state would naturally be reset. Need to double-check the context though. I want to be certain that's the only place this crucial bit is being fiddled with. There might be some edge cases lurking, or other calls that inadvertently step on it during an interrupt. I should pull up `__dev_close_many` and its callers, just to make sure there aren't any gotchas in there. Time to dig into the source.
|
| 72/2 |
2026/05/21 11:40 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:__dev_close_many]
Results: map[SourceCode:1716: static void __dev_close_many(struct list_head *head)
1717: {
1718: struct net_device *dev;
1719:
1720: ASSERT_RTNL();
1721: might_sleep();
1722:
1723: list_for_each_entry(dev, head, close_list) {
1724: /* Temporarily disable netpoll until the interface is down */
1725: netpoll_poll_disable(dev);
1726:
1727: call_netdevice_notifiers(NETDEV_GOING_DOWN, dev);
1728:
1729: clear_bit(__LINK_STATE_START, &dev->state);
1730:
1731: /* Synchronize to scheduled poll. We cannot touch poll list, it
1732: * can be even on different cpu. So just clear netif_running().
1733: *
1734: * dev->stop() will invoke napi_disable() on all of it's
1735: * napi_struct instances on this device.
1736: */
1737: smp_mb__after_atomic(); /* Commit netif_running(). */
1738: }
1739:
1740: dev_deactivate_many(head);
1741:
1742: list_for_each_entry(dev, head, close_list) {
1743: const struct net_device_ops *ops = dev->netdev_ops;
1744:
1745: /*
1746: * Call the device specific close. This cannot fail.
1747: * Only if device is UP
1748: *
1749: * We allow it to be called even after a DETACH hot-plug
1750: * event.
1751: */
1752:
1753: netdev_ops_assert_locked(dev);
1754:
1755: if (ops->ndo_stop)
1756: ops->ndo_stop(dev);
1757:
1758: netif_set_up(dev, false);
1759: netpoll_poll_enable(dev);
1760: }
1761: }
SourceFile:net/core/dev.c]
|
| 73/2 |
2026/05/21 11:40 |
llm |
expert |
0m
Model:
Tokens:
input: 39270
output: 44
thoughts: 296
Thoughts: **Race Condition in Network Device Shutdown**
Okay, so I'm looking at this network device shutdown sequence, and I think I've spotted a potential race condition. Let me walk through it in my head. `__dev_close_many` first clears the `__LINK_STATE_START` flag. Then, it calls `dev_deactivate_many`, which, crucially, calls `netdev_watchdog_down`. So, the flag goes down *before* the watchdog timer is deactivated.
Now, here's where it gets interesting. I'm thinking about concurrent execution, specifically `netif_carrier_on`. If that function runs concurrently, it checks `netif_running(dev)`. If it gets there *before* the `__LINK_STATE_START` flag is cleared, it'll call `netdev_watchdog_up`, starting the timer. But the shutdown is still proceeding! `__dev_close_many` will *still* run, clear the flag, call `dev_deactivate_many`, which *still* calls `netdev_watchdog_down`, which *still* deletes the timer. This is a problem because we'd have a timer created, then deleted in quick succession.
But, if `netif_carrier_on` runs *after* the flag is cleared, `netif_running(dev)` will rightly return false, and we avoid calling `netdev_watchdog_up`. That's the expected and safe behavior.
Wait, hold on... what if `netif_carrier_on` checks `netif_running(dev)` *before* `__dev_close_many` clears the flag, *sees* that it's true, and THEN `__dev_close_many` clears the flag and deactivates the device? That's a real race! I need to dig into `netif_carrier_on` to see the exact sequence of events there. I need to understand how that function interacts with the `__LINK_STATE_START` flag and the `netif_running` check. This bears further investigation.
|
| 74/2 |
2026/05/21 11:41 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/sched/sch_generic.c IncludeLines:true Name:netif_carrier_on]
Results: map[SourceCode: 579: /**
580: * netif_carrier_on - set carrier
581: * @dev: network device
582: *
583: * Device has detected acquisition of carrier.
584: */
585: void netif_carrier_on(struct net_device *dev)
586: {
587: if (test_and_clear_bit(__LINK_STATE_NOCARRIER, &dev->state)) {
588: if (dev->reg_state == NETREG_UNINITIALIZED)
589: return;
590: atomic_inc(&dev->carrier_up_count);
591: linkwatch_fire_event(dev);
592: if (netif_running(dev))
593: netdev_watchdog_up(dev);
594: }
595: }
SourceFile:net/sched/sch_generic.c]
|
| 75/2 |
2026/05/21 11:41 |
llm |
expert |
0m
Model:
Tokens:
input: 40342
output: 42
thoughts: 855
Thoughts: **Analyzing the netdev Watchdog Timer Interactions**
Okay, so I'm digging into how the network device watchdog timer behaves, and I've spotted something potentially problematic. `netif_carrier_on` calls `netdev_watchdog_up` after checking if the device is running. `netdev_watchdog_up` is responsible for starting the watchdog timer, if the device isn't using a no-op qdisc. But here's where it gets interesting, or maybe concerning.
If the device *is* running, then `__dev_close_many` (or a similar function) clears `__LINK_STATE_START`, then calls `dev_deactivate_many`. This further calls `dev_deactivate_queue`, which sets a `noop_qdisc`. Then `dev_deactivate_many` calls `netdev_watchdog_down` which deletes the watchdog timer. Then, `netif_carrier_on` *re*calls `netdev_watchdog_up`. The problem is that `netdev_watchdog_up` then acquires `tx_global_lock`, and checks if `!qdisc_tx_is_noop(dev)`. However, `dev_deactivate_queue` just set `noop_qdisc`! So `qdisc_tx_is_noop(dev)` will return `true`, and the timer *won't* be armed. That seems straightforward and probably intended.
But wait, there's a race condition possibility! What if the `netdev_watchdog_up` call in `netif_carrier_on` happens *before* `dev_deactivate_queue` sets `noop_qdisc`? Then the timer *would* be armed. Then `dev_deactivate_many` would call `netdev_watchdog_down`, which would delete the newly armed timer! So the timer ends up deleted, which feels odd.
Now, is there another pathway to this condition? What if `dev_deactivate_many` *isn't* called? After some thought, I realize that `unregister_netdevice_many_notify` only calls `netif_close_many` if the device's `IFF_UP` flag is set. If the device is down, `netif_close_many` is skipped entirely! So that's important.
But wait! I'm getting a little mixed up. `netif_close_many` seems to be called for *all* devices, not just the UP ones. The first loop in that routine adds devices that have both `IFF_UP` and need the device ops lock. The second loop then adds the rest! So I need to go back and check the implementation of `netif_close_many` to get this straight. I have a hunch that this can be a serious issue, given the conditions.
Okay, let me go through that code...
|
| 76/2 |
2026/05/21 11:41 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:netif_close_many]
Results: map[SourceCode:1772: void netif_close_many(struct list_head *head, bool unlink)
1773: {
1774: struct net_device *dev, *tmp;
1775:
1776: /* Remove the devices that don't need to be closed */
1777: list_for_each_entry_safe(dev, tmp, head, close_list)
1778: if (!(dev->flags & IFF_UP))
1779: list_del_init(&dev->close_list);
1780:
1781: __dev_close_many(head);
1782:
1783: list_for_each_entry_safe(dev, tmp, head, close_list) {
1784: rtmsg_ifinfo(RTM_NEWLINK, dev, IFF_UP | IFF_RUNNING, GFP_KERNEL, 0, NULL);
1785: call_netdevice_notifiers(NETDEV_DOWN, dev);
1786: if (unlink)
1787: list_del_init(&dev->close_list);
1788: }
1789: }
SourceFile:net/core/dev.c]
|
| 77/2 |
2026/05/21 11:41 |
llm |
expert |
0m
Model:
Tokens:
input: 42202
output: 40
thoughts: 172
Thoughts: **My Analysis of `netif_close_many` and Device Deactivation**
Okay, so `netif_close_many` is specifically designed to handle the removal of network devices. The crucial point here is that it appears to *only* remove devices that are actually up. In other words, if the `IFF_UP` flag isn't set for a device – if `!(dev->flags & IFF_UP)` evaluates to true – then the subsequent functions within the close process are skipped.
This means if a device isn't up, `__dev_close_many` won't be called. Consequently, `dev_deactivate_many` will also be bypassed, and by extension, `netdev_watchdog_down` won't be invoked. And, crucially, it looks like the `noop_qdisc` won't be set either. This is a critical point to consider from a resource management perspective, and potentially opens up some edge cases.
Aha! This raises a question. If `!(dev->flags & IFF_UP)` is true, then `netif_running(dev)` should be false, since `__LINK_STATE_START` is typically set in `__dev_open`. I'd better double-check the `__dev_open` function and its interactions with the link state flags to confirm that this expected behavior is being implemented. Let's see...
|
| 78/2 |
2026/05/21 11:41 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:__dev_open]
Results: map[SourceCode:1648: static int __dev_open(struct net_device *dev, struct netlink_ext_ack *extack)
1649: {
1650: const struct net_device_ops *ops = dev->netdev_ops;
1651: int ret;
1652:
1653: ASSERT_RTNL();
1654: dev_addr_check(dev);
1655:
1656: if (!netif_device_present(dev)) {
1657: /* may be detached because parent is runtime-suspended */
1658: if (dev->dev.parent)
1659: pm_runtime_resume(dev->dev.parent);
1660: if (!netif_device_present(dev))
1661: return -ENODEV;
1662: }
1663:
1664: /* Block netpoll from trying to do any rx path servicing.
1665: * If we don't do this there is a chance ndo_poll_controller
1666: * or ndo_poll may be running while we open the device
1667: */
1668: netpoll_poll_disable(dev);
1669:
1670: ret = call_netdevice_notifiers_extack(NETDEV_PRE_UP, dev, extack);
1671: ret = notifier_to_errno(ret);
1672: if (ret)
1673: return ret;
1674:
1675: set_bit(__LINK_STATE_START, &dev->state);
1676:
1677: netdev_ops_assert_locked(dev);
1678:
1679: if (ops->ndo_validate_addr)
1680: ret = ops->ndo_validate_addr(dev);
1681:
1682: if (!ret && ops->ndo_open)
1683: ret = ops->ndo_open(dev);
1684:
1685: netpoll_poll_enable(dev);
1686:
1687: if (ret)
1688: clear_bit(__LINK_STATE_START, &dev->state);
1689: else {
1690: netif_set_up(dev, true);
1691: dev_set_rx_mode(dev);
1692: dev_activate(dev);
1693: add_device_randomness(dev->dev_addr, dev->addr_len);
1694: }
1695:
1696: return ret;
1697: }
SourceFile:net/core/dev.c]
|
| 79/2 |
2026/05/21 11:41 |
llm |
expert |
0m
Model:
Tokens:
input: 43462
output: 41
thoughts: 82
Thoughts: Okay, let me break this down.
**Network Interface Initialization Conundrum**
Alright, so I'm looking at this network interface initialization process and something isn't quite clicking. My initial thought is: if the `__dev_open` function throws an error, it clears `__LINK_STATE_START`, that seems straightforward enough. But if `__dev_open` is successful, the `IFF_UP` flag is then set on the device, and I know from prior experience that `dev_change_flags` handles that bit, which itself calls `__dev_open` at some point. However, and here's the rub, `dev_change_flags` sets `IFF_UP` *after* a successful `__dev_open` returns zero. The sequence seems slightly off, as the status of the flags and the `__dev_open` call appear out of order. I need to dig deeper into the control flow here because that's not quite lining up with how I'd expect this process to play out. Are there other flags involved that are dependent on the successful completion of the `__dev_open` call and, therefore, the `IFF_UP` flag? It’s not necessarily a bug, but it's making me wonder about the timing of the flag settings.
|
| 80/2 |
2026/05/21 11:41 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:dev_change_flags]
Results: map[SourceCode: 51: /**
52: * dev_change_flags() - change device settings
53: * @dev: device
54: * @flags: device state flags
55: * @extack: netlink extended ack
56: *
57: * Change settings on device based state flags. The flags are
58: * in the userspace exported format.
59: *
60: * Return: 0 on success, -errno on failure.
61: */
62: int dev_change_flags(struct net_device *dev, unsigned int flags,
63: struct netlink_ext_ack *extack)
64: {
65: int ret;
66:
67: netdev_lock_ops(dev);
68: ret = netif_change_flags(dev, flags, extack);
69: netdev_unlock_ops(dev);
70:
71: return ret;
72: }
SourceFile:net/core/dev_api.c]
|
| 81/2 |
2026/05/21 11:41 |
llm |
expert |
0m
Model:
Tokens:
input: 44167
output: 42
thoughts: 12
Thoughts: Here's my take on those thoughts:
**Analyzing `netif_change_flags`**
Okay, so I'm thinking we need to dive into `netif_change_flags`. Something in the current state of the network interface flags seems off, and this function is a key piece of the puzzle. I immediately want to understand *why* we're even considering this function. What triggered this inspection? Was there a specific symptom or reported issue? Knowing the context will help me frame my analysis. What are the possible call sites? Are there specific combinations of flags being changed that I need to be concerned about? I need to review it thoroughly and consider all possible dependencies. Let's get to work!
|
| 82/2 |
2026/05/21 11:41 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:netif_change_flags]
Results: map[SourceCode:9791: int netif_change_flags(struct net_device *dev, unsigned int flags,
9792: struct netlink_ext_ack *extack)
9793: {
9794: int ret;
9795: unsigned int changes, old_flags = dev->flags, old_gflags = dev->gflags;
9796:
9797: ret = __dev_change_flags(dev, flags, extack);
9798: if (ret < 0)
9799: return ret;
9800:
9801: changes = (old_flags ^ dev->flags) | (old_gflags ^ dev->gflags);
9802: __dev_notify_flags(dev, old_flags, changes, 0, NULL);
9803: return ret;
9804: }
SourceFile:net/core/dev.c]
|
| 83/2 |
2026/05/21 11:41 |
llm |
expert |
0m
Model:
Tokens:
input: 44628
output: 42
thoughts: 9
Thoughts: Okay, here's the summary, formatted as requested:
**Internal Reflection on `__dev_change_flags`**
Alright, let me get this straight in my head. I'm thinking about `__dev_change_flags`. It's a key piece, really, isn't it? It's not just some random variable. This is where the *important* stuff happens, the nuts and bolts of how devices tell the kernel that things have *changed*. Specifically, it's about what needs to be *done* in response to those changes.
My immediate focus is on how it's *used*, what bits are set, and, critically, the consequences of setting them. Am I looking at potential race conditions here? Does it handle interrupts correctly? What locking mechanisms are in place, if any? The nuances are always the issue. A simple flag can have ripple effects throughout the whole system if mishandled. So, I need to understand the architectural implications. Are we talking about a single device, or something that's propagated across a bus with multiple attached devices? That context matters, doesn't it?
Essentially, I'm thinking about the *semantic meaning* of each flag. What does it *mean* when a bit is set? Does this flag relate to some kind of device notification or is it associated with some kind of hardware interrupt? I need to know how the state transitions are handled. Are there any potential issues with atomicity? What are the implications if this call happens within an interrupt context?
More broadly, how does this fit into the bigger picture of the device driver model? Is this a standard interface, or is it driver-specific? Does it introduce any new complexities? And, ultimately, I'm considering potential areas for optimization or improvement. Are there any opportunities to streamline this process, reduce overhead, or enhance performance? This is all pretty crucial.
|
| 84/2 |
2026/05/21 11:41 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:__dev_change_flags]
Results: map[SourceCode:9696: int __dev_change_flags(struct net_device *dev, unsigned int flags,
9697: struct netlink_ext_ack *extack)
9698: {
9699: unsigned int old_flags = dev->flags;
9700: int ret;
9701:
9702: ASSERT_RTNL();
9703:
9704: /*
9705: * Set the flags on our device.
9706: */
9707:
9708: dev->flags = (flags & (IFF_DEBUG | IFF_NOTRAILERS | IFF_NOARP |
9709: IFF_DYNAMIC | IFF_MULTICAST | IFF_PORTSEL |
9710: IFF_AUTOMEDIA)) |
9711: (dev->flags & (IFF_UP | IFF_VOLATILE | IFF_PROMISC |
9712: IFF_ALLMULTI));
9713:
9714: /*
9715: * Load in the correct multicast list now the flags have changed.
9716: */
9717:
9718: if ((old_flags ^ flags) & IFF_MULTICAST)
9719: dev_change_rx_flags(dev, IFF_MULTICAST);
9720:
9721: dev_set_rx_mode(dev);
9722:
9723: /*
9724: * Have we downed the interface. We handle IFF_UP ourselves
9725: * according to user attempts to set it, rather than blindly
9726: * setting it.
9727: */
9728:
9729: ret = 0;
9730: if ((old_flags ^ flags) & IFF_UP) {
9731: if (old_flags & IFF_UP)
9732: __dev_close(dev);
9733: else
9734: ret = __dev_open(dev, extack);
9735: }
9736:
9737: if ((flags ^ dev->gflags) & IFF_PROMISC) {
9738: int inc = (flags & IFF_PROMISC) ? 1 : -1;
9739: old_flags = dev->flags;
9740:
9741: dev->gflags ^= IFF_PROMISC;
9742:
9743: if (__dev_set_promiscuity(dev, inc, false) >= 0)
9744: if (dev->flags != old_flags)
9745: dev_set_rx_mode(dev);
9746: }
9747:
9748: /* NOTE: order of synchronization of IFF_PROMISC and IFF_ALLMULTI
9749: * is important. Some (broken) drivers set IFF_PROMISC, when
9750: * IFF_ALLMULTI is requested not asking us and not reporting.
9751: */
9752: if ((flags ^ dev->gflags) & IFF_ALLMULTI) {
9753: int inc = (flags & IFF_ALLMULTI) ? 1 : -1;
9754:
9755: dev->gflags ^= IFF_ALLMULTI;
9756: netif_set_allmulti(dev, inc, false);
9757: }
9758:
9759: return ret;
9760: }
SourceFile:net/core/dev.c]
|
| 85/2 |
2026/05/21 11:41 |
llm |
expert |
0m
Model:
Tokens:
input: 46065
output: 42
thoughts: 89
Thoughts: Okay, let me break this down.
**Network Interface Flag Confusion**
Alright, I'm tracing the network interface setup here. It looks like `__dev_open` sets `__LINK_STATE_START`, which is expected. But I'm momentarily tripped up because I thought `dev->flags` gets updated with `IFF_UP` inside `__dev_change_flags`. Let me clarify... no, `__dev_open` itself *doesn't* actually set `IFF_UP`. That's a key distinction I need to keep in mind. So, the question remains: where does the `IFF_UP` update *actually* happen? Ah, got it. It's `__dev_notify_flags` that handles the update of `dev->flags`. Right. Keeping the order straight is critical here.
|
| 86/2 |
2026/05/21 11:41 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:__dev_notify_flags]
Results: map[SourceCode:9762: void __dev_notify_flags(struct net_device *dev, unsigned int old_flags,
9763: unsigned int gchanges, u32 portid,
9764: const struct nlmsghdr *nlh)
9765: {
9766: unsigned int changes = dev->flags ^ old_flags;
9767:
9768: if (gchanges)
9769: rtmsg_ifinfo(RTM_NEWLINK, dev, gchanges, GFP_ATOMIC, portid, nlh);
9770:
9771: if (changes & IFF_UP) {
9772: if (dev->flags & IFF_UP)
9773: call_netdevice_notifiers(NETDEV_UP, dev);
9774: else
9775: call_netdevice_notifiers(NETDEV_DOWN, dev);
9776: }
9777:
9778: if (dev->flags & IFF_UP &&
9779: (changes & ~(IFF_UP | IFF_PROMISC | IFF_ALLMULTI | IFF_VOLATILE))) {
9780: struct netdev_notifier_change_info change_info = {
9781: .info = {
9782: .dev = dev,
9783: },
9784: .flags_changed = changes,
9785: };
9786:
9787: call_netdevice_notifiers_info(NETDEV_CHANGE, &change_info.info);
9788: }
9789: }
SourceFile:net/core/dev.c]
|
| 87/2 |
2026/05/21 11:41 |
llm |
expert |
0m
Model:
Tokens:
input: 46838
output: 43
thoughts: 157
Thoughts: **Decoding the Network Interface Flag Update**
Okay, so I'm tracing the network interface setup and I've hit a snag. Where exactly is the `IFF_UP` flag being set within the `dev->flags` structure? I see the code in `__dev_change_flags`:
```c
if ((old_flags ^ flags) & IFF_UP) {
if (old_flags & IFF_UP)
__dev_close(dev);
else
ret = __dev_open(dev, extack);
}
```
... but hang on a minute. It looks like `dev->flags` isn't *directly* updated with `IFF_UP` here. It checks for the change, and then *calls* `__dev_open` if we're going from down to up. Okay, that makes sense - manage the open process which presumably, sets the interface up.
Aha! Now I see. Looking at `__dev_open`, it calls `netif_set_up(dev, true);`. That must be it! That should be the function which sets `IFF_UP` in the device's flags.
Wait... `netif_set_up` is an inline function? Right, that means the flag update is probably inlined directly into the calling code in a release build, improving performance.
|
| 88/2 |
2026/05/21 11:41 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:include/linux/netdevice.h IncludeLines:true Name:netif_set_up]
Results: map[SourceCode: 151: static inline void netif_set_up(struct net_device *dev, bool value)
152: {
153: if (value)
154: dev->flags |= IFF_UP;
155: else
156: dev->flags &= ~IFF_UP;
157:
158: if (!netdev_need_ops_lock(dev))
159: netdev_lock(dev);
160: dev->up = value;
161: if (!netdev_need_ops_lock(dev))
162: netdev_unlock(dev);
163: }
SourceFile:net/core/dev.h]
|
| 89/2 |
2026/05/21 11:41 |
llm |
expert |
0m
Model:
Tokens:
input: 47549
output: 46
thoughts: 2646
Thoughts: **Analyzing Network Device Unregistration and Watchdog Timer Interactions**
Okay, so I'm digging into how the network device unregistration process interacts with the watchdog timer, specifically considering potential race conditions. It seems that `netif_set_up` sets `IFF_UP`, and if `__dev_open` succeeds, that's also set. Now, if `IFF_UP` is set, `unregister_netdevice_many_notify` will be called. This calls `netif_close_many`, which then calls `__dev_close_many`, which clears `__LINK_STATE_START` and calls `dev_deactivate_many`.
I'm thinking, what if `__dev_open` and `unregister_netdevice_many_notify` are running at the same time? They both operate under the `RTNL` lock, so that's not a problem, they can't be concurrent.
Now, what about concurrent execution between `dev_watchdog` and `dev_shutdown`? The `dev_shutdown` function is called under `netdev_lock_ops(dev)`. `dev_shutdown` is called within `unregister_netdevice_many_notify`, which also includes `dev_tcx_uninstall`, `dev_xdp_uninstall`, and `dev_memory_provider_uninstall`. Inside `dev_shutdown`, there's a check for `WARN_ON(timer_pending(&dev->watchdog_timer))`. If that triggers, the timer is still pending.
The obvious question: why would the timer still be pending? Because the `netdev_watchdog_down` function failed to delete the timer? `netdev_watchdog_down` is supposed to call `timer_delete(&dev->watchdog_timer)`. If the timer is running on another CPU when `timer_delete` is called, it returns 0. But, `netdev_watchdog_down` holds the `netif_tx_lock_bh(dev)`, which then holds the `dev->tx_global_lock`. The `dev_watchdog` function also uses this same lock.
If the watchdog is running, it holds the lock and calls `mod_timer`. Now, if `netdev_watchdog_down` tries to delete the timer, but can't, because of `timer_delete` returning 0 due to concurrency, that's not a problem, since the `netdev_watchdog_down` function will block on the lock, waiting to acquire it. Once acquired, it calls `timer_delete`, and will succeed, since the timer has already either finished, or, was re-armed.
Now, the next thought is: What if `dev_watchdog` calls `mod_timer` without holding the lock? Let's check `dev_watchdog` again to make sure. Yes, it does call `mod_timer` *inside* the spin lock. This means if it's running, it does hold the lock.
Okay, what if `dev_watchdog` is running but hasn't acquired the lock yet, and another CPU calls `netdev_watchdog_down`? `netdev_watchdog_down` gets the lock and calls `timer_delete`. Because the watchdog is running, `timer_delete` returns 0. It then releases the lock, and the watchdog function acquires the lock. However, before the lock is acquired, the previous function calling `dev_deactivate_queue` will have set `noop_qdisc` within `dev_deactivate_many`. Thus the timer isn't re-armed, since the function checks the condition for the re-arm inside the lock. If `dev_watchdog` function does not re-arm the timer, then it is not pending. Why would `WARN_ON(timer_pending(&dev->watchdog_timer))` trigger?
Could someone *else* be calling `mod_timer`? I see `netdev_watchdog_up` calls `mod_timer`. If `netdev_watchdog_up` is called *after* `dev_deactivate_queue`, it will check if `qdisc_tx_is_noop` and do nothing. Wait, is there any other place that changes `qdisc`? No. The `dev_deactivate_many` and `dev_shutdown` functions also set `noop_qdisc`.
What if `netdev_watchdog_up` is called *before* `dev_deactivate_queue`, but *after* `netdev_watchdog_down`? `dev_deactivate_many` calls `dev_deactivate_queue` *before* `netdev_watchdog_down`, which means `noop_qdisc` is set before `
|
| 90/2 |
2026/05/21 11:42 |
tool |
codesearch-find-references |
0m
Args: map[ContextFile:drivers/net/usb/usbnet.c IncludeSnippetLines:0 Name:netif_device_attach]
Results: map[References:[map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:dsa_user_resume SourceFile:net/dsa/user.c SourceLine:2751] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:e1000_io_resume SourceFile:drivers/net/ethernet/intel/e1000/e1000_main.c SourceLine:5319] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:e1000_resume SourceFile:drivers/net/ethernet/intel/e1000/e1000_main.c SourceLine:5195] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:e1000e_pm_thaw SourceFile:drivers/net/ethernet/intel/e1000e/netdev.c SourceLine:6898] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:e100_io_resume SourceFile:drivers/net/ethernet/intel/e100.c SourceLine:3152] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:e100_resume SourceFile:drivers/net/ethernet/intel/e100.c SourceLine:3074] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:lan78xx_suspend SourceFile:drivers/net/usb/lan78xx.c SourceLine:5136] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:pegasus_resume SourceFile:drivers/net/usb/pegasus.c SourceLine:1255] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:rtl8150_resume SourceFile:drivers/net/usb/rtl8150.c SourceLine:544] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:rtl8152_system_resume SourceFile:drivers/net/usb/r8152.c SourceLine:8529] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:tbnet_resume SourceFile:drivers/net/thunderbolt/main.c SourceLine:1394] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:usbnet_probe SourceFile:drivers/net/usb/usbnet.c SourceLine:1893] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:usbnet_suspend SourceFile:drivers/net/usb/usbnet.c SourceLine:1956] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:virtnet_restore_up SourceFile:drivers/net/virtio_net.c SourceLine:5817] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:vmxnet3_resume SourceFile:drivers/net/vmxnet3/vmxnet3_drv.c SourceLine:4569]] TruncatedOutput:false]
|
| 91/2 |
2026/05/21 11:42 |
llm |
expert |
0m
Model:
Tokens:
input: 52165
output: 44
thoughts: 42
Thoughts: Okay, here's my interpretation of that thought process, assuming I'm deep in the weeds of USB networking driver code:
**USBnet Driver Conundrum: Resume vs. Suspend**
Alright, so `usbnet_resume`...is it really calling `netif_device_attach`? That seems like a strange place to put that. Let me double-check. Something feels off here. Wait a minute...the cross-reference is actually showing up in `usbnet_suspend` instead? That makes more sense, for some reason. Okay, let's just go straight to `usbnet_suspend` and see what's actually going on. I need to understand the relationship between those two functions and `netif_device_attach`.
|
| 92/2 |
2026/05/21 11:42 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:drivers/net/usb/usbnet.c IncludeLines:true Name:usbnet_suspend]
Results: map[SourceCode:1929: int usbnet_suspend(struct usb_interface *intf, pm_message_t message)
1930: {
1931: struct usbnet *dev = usb_get_intfdata(intf);
1932:
1933: if (!dev->suspend_count++) {
1934: spin_lock_irq(&dev->txq.lock);
1935: /* don't autosuspend while transmitting */
1936: if (dev->txq.qlen && PMSG_IS_AUTO(message)) {
1937: dev->suspend_count--;
1938: spin_unlock_irq(&dev->txq.lock);
1939: return -EBUSY;
1940: } else {
1941: set_bit(EVENT_DEV_ASLEEP, &dev->flags);
1942: spin_unlock_irq(&dev->txq.lock);
1943: }
1944: /*
1945: * accelerate emptying of the rx and queues, to avoid
1946: * having everything error out.
1947: */
1948: netif_device_detach(dev->net);
1949: usbnet_terminate_urbs(dev);
1950: __usbnet_status_stop_force(dev);
1951:
1952: /*
1953: * reattach so runtime management can use and
1954: * wake the device
1955: */
1956: netif_device_attach(dev->net);
1957: }
1958: return 0;
1959: }
SourceFile:drivers/net/usb/usbnet.c]
|
| 93/2 |
2026/05/21 11:42 |
llm |
expert |
0m
Model:
Tokens:
input: 52902
output: 43
thoughts: 139
Thoughts: Here's my thought process, summarized as you requested:
**Concurrency Concerns in USB Network Driver Suspend/Resume**
Okay, so I'm digging into this `usbnet` driver. My initial thought is about the `usbnet_suspend` function. I see that it's calling `netif_device_attach(dev->net)`. Hmm, that's interesting. The question that immediately pops into my head is, could `usbnet_suspend` be running at the *same time* as `unregister_netdevice_many_notify`? That would be a race condition I need to be aware of!
Now, I remember that `usbnet_suspend` is called by the USB core when the system is suspending. That makes sense. But then I have to think about `usbnet_disconnect` – which is obviously called during a disconnect event. Are those two – suspend/resume and disconnect – mutually exclusive? Could they overlap?
Ah, right, the USB core. It *does* serialize suspend/resume and disconnect. It uses the device lock, so that should prevent any concurrent issues I was initially concerned about. That's good.
But wait, hang on. `usbnet_suspend` is calling `netif_device_attach`... So, if it's being called during suspend, could `netif_device_attach` *also* be called from `usbnet_resume`? I need to look into the driver and see how that is handled.
|
| 94/2 |
2026/05/21 11:42 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:drivers/net/usb/usbnet.c IncludeLines:true Name:usbnet_resume]
Results: map[SourceCode:1962: int usbnet_resume(struct usb_interface *intf)
1963: {
1964: struct usbnet *dev = usb_get_intfdata(intf);
1965: struct sk_buff *skb;
1966: struct urb *res;
1967: int retval;
1968:
1969: if (!--dev->suspend_count) {
1970: /* resume interrupt URB if it was previously submitted */
1971: __usbnet_status_start_force(dev, GFP_NOIO);
1972:
1973: spin_lock_irq(&dev->txq.lock);
1974: while ((res = usb_get_from_anchor(&dev->deferred))) {
1975:
1976: skb = (struct sk_buff *)res->context;
1977: retval = usb_submit_urb(res, GFP_ATOMIC);
1978: if (retval < 0) {
1979: dev_kfree_skb_any(skb);
1980: kfree(res->sg);
1981: usb_free_urb(res);
1982: usb_autopm_put_interface_async(dev->intf);
1983: } else {
1984: netif_trans_update(dev->net);
1985: __skb_queue_tail(&dev->txq, skb);
1986: }
1987: }
1988:
1989: smp_mb();
1990: clear_bit(EVENT_DEV_ASLEEP, &dev->flags);
1991: spin_unlock_irq(&dev->txq.lock);
1992:
1993: if (test_bit(EVENT_DEV_OPEN, &dev->flags)) {
1994: /* handle remote wakeup ASAP
1995: * we cannot race against stop
1996: */
1997: if (netif_device_present(dev->net) &&
1998: !timer_pending(&dev->delay) &&
1999: !test_bit(EVENT_RX_HALT, &dev->flags))
2000: rx_alloc_submit(dev, GFP_NOIO);
2001:
2002: if (!(dev->txq.qlen >= TX_QLEN(dev)))
2003: netif_tx_wake_all_queues(dev->net);
2004: queue_work(system_bh_wq, &dev->bh_work);
2005: }
2006: }
2007:
2008: if (test_and_clear_bit(EVENT_DEVICE_REPORT_IDLE, &dev->flags))
2009: usb_autopm_get_interface_no_resume(intf);
2010:
2011: return 0;
2012: }
SourceFile:drivers/net/usb/usbnet.c]
|
| 95/2 |
2026/05/21 11:42 |
llm |
expert |
0m
Model:
Tokens:
input: 54244
output: 41
thoughts: 1241
Thoughts: **Debugging a Potential Netdev Watchdog Timer Issue**
Okay, so I'm trying to figure out why this watchdog timer warning is triggering. Initially, I was looking at `usbnet_resume`, but that clearly isn't the culprit, as it doesn't directly call `netif_device_attach`. Let's step through this logically. Could `netif_carrier_on` be called *after* `dev_shutdown` within `usbnet_deferred_kevent`? I see that `dev_shutdown` happens during `unregister_netdevice_many_notify`, and it explicitly checks for a pending watchdog timer with `WARN_ON(timer_pending(&dev->watchdog_timer))`.
If `netif_carrier_on` is called *before* `dev_shutdown` but *after* `dev_deactivate_many`, the timer *won't* be armed because the `noop_qdisc` is in place. And if it's called *after* `dev_shutdown`, the `noop_qdisc` setting would also prevent the timer from being armed. So, that can't be it.
Is there any situation where `qdisc_tx_is_noop(dev)` is FALSE after `dev_deactivate_many`? No, I don't think so. That's not the issue. Then why the warning?
Okay, let's examine the syzkaller reproducer. It's using `io_uring` and `usb_raw_gadget`, simulating a disconnected USB device. If I'm understanding this correctly, the device is disconnected, and the warning is triggered.
Could it be that `netdev_watchdog_down` is *not* being called? `netdev_watchdog_down` is called in `dev_deactivate_many`, which is called within `__dev_close_many`, which is called in `netif_close_many`, which is called in `unregister_netdevice_many_notify` *only if* `dev->flags & IFF_UP`.
Aha! What if `dev->flags & IFF_UP` is FALSE? If `IFF_UP` is not set, then `netif_close_many` is skipped! That means `__dev_close_many` is also skipped, so `dev_deactivate_many` is never called, and `netdev_watchdog_down` is *not* called. The `noop_qdisc` isn't set, then `dev_shutdown` *is* called. This still checks `WARN_ON(timer_pending(&dev->watchdog_timer))`, and if that timer is still pending, the warning will trigger!
But wait, if `IFF_UP` is FALSE, how could the timer *be* pending? That timer is armed by `netdev_watchdog_up`. `netdev_watchdog_up` is called by `dev_activate`, `netif_carrier_on`, or `netif_device_attach`. `dev_activate` itself is called by `__dev_open`. If `__dev_open` *succeeds*, `IFF_UP` is set. However, if `__dev_open` *fails*, `IFF_UP` is *not* set. If `__dev_open` fails, then `clear_bit(__LINK_STATE_START, &dev->state)` is called. But `dev_deactivate` is *not* called! `__dev_open` only calls `dev_activate` if it succeeds.
```c
if (ret)
clear_bit(__LINK_STATE_START, &dev->state);
else {
netif_set_up(dev, true);
dev_set_rx_mode(dev);
dev_activate(dev);
add_device_randomness(dev->dev_addr, dev->addr_len);
}
```
So, if `__dev_open` *fails*, `dev_activate` isn't even called. Okay, so what about `netif_carrier_on`? It checks `netif_running(dev)`. That is true if `__LINK_STATE_START` is set. `__LINK_STATE_START` is set within `__dev_open` *before* the `ops->ndo_open` call!
```c
set_bit(__LINK_STATE_START, &dev->state);
netdev_ops_assert_locked(dev);
if (ops->ndo_validate_addr)
ret = ops->ndo_validate_addr(dev);
if (!ret && ops->ndo
|
| 96/2 |
2026/05/21 11:42 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/sched/sch_generic.c IncludeLines:true Name:dev_activate]
Results: map[SourceCode:1241: void dev_activate(struct net_device *dev)
1242: {
1243: int need_watchdog;
1244:
1245: /* No queueing discipline is attached to device;
1246: * create default one for devices, which need queueing
1247: * and noqueue_qdisc for virtual interfaces
1248: */
1249:
1250: if (rtnl_dereference(dev->qdisc) == &noop_qdisc)
1251: attach_default_qdiscs(dev);
1252:
1253: if (!netif_carrier_ok(dev))
1254: /* Delay activation until next carrier-on event */
1255: return;
1256:
1257: need_watchdog = 0;
1258: netdev_for_each_tx_queue(dev, transition_one_qdisc, &need_watchdog);
1259: if (dev_ingress_queue(dev))
1260: transition_one_qdisc(dev, dev_ingress_queue(dev), NULL);
1261:
1262: if (need_watchdog) {
1263: netif_trans_update(dev);
1264: netdev_watchdog_up(dev);
1265: }
1266: }
SourceFile:net/sched/sch_generic.c]
|
| 97/2 |
2026/05/21 11:42 |
llm |
expert |
0m
Model:
Tokens:
input: 56961
output: 39
thoughts: 1012
Thoughts: **Analyzing the Network Device Watchdog Timer**
Okay, let's break this down. My initial thought is about the sequence of events affecting the network device watchdog timer, specifically around the `dev_activate` and `dev_deactivate_many` calls. I can see that `dev_activate` is critical because it changes the `qdisc` from a no-op queue (`noop_qdisc`) to the default queue. This is a key transition.
Before `dev_activate`, `qdisc_tx_is_noop(dev)` is definitely `TRUE`. This means that within `netdev_watchdog_up`, the timer won't be armed because that check is in place. If `netif_carrier_on` gets called *before* `dev_activate`, the watchdog timer remains inactive. That's the first condition to note.
Then I start considering other scenarios, especially the bringing up and down of a device. What if `dev_activate` *has* been called, so `IFF_UP` is set, but the device is then brought down? `__dev_close_many` calls `dev_deactivate_many`, which sets the `noop_qdisc` and calls `netdev_watchdog_down`. `IFF_UP` is cleared. Then, the device is brought back up, running `__dev_open` which sets `__LINK_STATE_START`. `ops->ndo_open` is called, and `netif_carrier_on` might happen during `ops->ndo_open`. But because `dev_deactivate_many` has already set `noop_qdisc`, `qdisc_tx_is_noop(dev)` is again `TRUE`. This means the watchdog is *still* not armed during this second up-cycle. That's the second condition to note.
Now, I think about a device that's *not* up, but the `qdisc_tx_is_noop(dev)` is `FALSE`. But how can that even be? As a device gets created, it starts with `noop_qdisc`. `dev_activate` attaches the default qdisc. `dev_deactivate_many` goes back to `noop_qdisc`. This pattern seems consistent. In all those scenarios, `qdisc_tx_is_noop(dev)` should *always* be `TRUE` when `IFF_UP` is `FALSE`. But I have to check, as I don't recall offhand any alternative path where `dev_deactivate_many` is skipped. It is called from `__dev_close_many`, so it is called when the device is being brought down.
Now, I'm shifting my focus to the warning I'm seeing: `WARN_ON(timer_pending(&dev->watchdog_timer));` in `dev_shutdown`. This indicates that the timer is *pending* at shutdown. The timer pending could mean the `netdev_watchdog_down` wasn't called, or that it was called but got re-armed, or that `timer_delete` failed. Since we know `dev_watchdog` checks `!qdisc_tx_is_noop(dev)`, if `dev_deactivate_many` was called, it would have set `noop_qdisc`, so the `dev_watchdog` wouldn't re-arm itself and that condition wouldn't apply, so it is unlikely that the timer would get pending again.
Here's the critical scenario I'm zeroing in on: What if `dev_shutdown` is called while the timer *is* running? Imagine the timer is running on CPU A. CPU B calls `dev_deactivate_many`. `noop_qdisc` gets set and `netdev_watchdog_down` is called. In `netdev_watchdog_down`, it acquires `tx_global_lock`. Then `timer_delete` is called. `timer_delete` can return 0 if the timer is still running, meaning it's still running, and thus "pending" on the other CPU. Then `tx_global_lock` gets released. CPU B continues and completes `__dev_close_many`, and then it goes on to call `dev_shutdown`. `dev_shutdown` then runs that check: `WARN_ON(timer_pending(&dev->watchdog_timer))`. This is the race: Is the timer still pending, even though we called `timer_delete` on another CPU? I need to check how `timer_pending` works when the timer is in the middle of execution. It is the timer that may be running on CPU A and then the `WARN_ON` may be triggered on CPU B. I need to dive into the definition of `timer_pending` and understand how
|
| 98/2 |
2026/05/21 11:43 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:include/linux/timer.h IncludeLines:true Name:timer_pending]
Results: map[SourceCode: 135: /**
136: * timer_pending - is a timer pending?
137: * @timer: the timer in question
138: *
139: * timer_pending will tell whether a given timer is currently pending,
140: * or not. Callers must ensure serialization wrt. other operations done
141: * to this timer, eg. interrupt contexts, or other CPUs on SMP.
142: *
143: * Returns: 1 if the timer is pending, 0 if not.
144: */
145: static inline int timer_pending(const struct timer_list * timer)
146: {
147: return !hlist_unhashed_lockless(&timer->entry);
148: }
SourceFile:include/linux/timer.h]
|
| 99/2 |
2026/05/21 11:43 |
llm |
expert |
0m
Model:
Tokens:
input: 59280
output: 43
thoughts: 270
Thoughts: **Analyzing Timer Re-arming Logic**
Okay, so I'm looking at this timer behavior. It seems critical. The first thing that jumps out is that when the timer is *running*, it's *removed* from the hash list. That means `timer_pending()` should return 0 while it's active. Right. Okay, so if `timer_pending()` is 0, the `WARN_ON(timer_pending(...))` should *not* trigger if the timer is just actively running. It will *only* trigger if the timer is *actually* pending, meaning it's been queued to run at some point in the future. Therefore, if the `WARN_ON` *doesn't* trigger, the timer must have been re-armed. That's the crux of it.
But how could the timer *be* re-armed? Let's trace it. If `dev_watchdog` re-arms it, then it *must* have seen `!qdisc_tx_is_noop(dev)`. But, hold on a second... `dev_deactivate_many` sets `noop_qdisc` *before* calling `netdev_watchdog_down`. So `dev_watchdog` *can't* re-arm it *after* `dev_deactivate_many`. That makes sense.
Could `netdev_watchdog_up` be the culprit? It *also* checks `!qdisc_tx_is_noop(dev)`. So, same problem, `netdev_watchdog_up` cannot re-arm it after `dev_deactivate_many`.
So, the question remains: *Is there ANY OTHER PLACE* that could possibly be re-arming this `watchdog_timer`? This is the core issue that has to be nailed down.
|
| 100/2 |
2026/05/21 11:43 |
tool |
codesearch-find-references |
0m
Error:
Args: map[ContextFile:net/sched/sch_generic.c IncludeSnippetLines:0 Name:watchdog_timer]
Results: map[References:<nil> TruncatedOutput:false]
|
| 101/2 |
2026/05/21 11:43 |
llm |
expert |
0m
Model:
Tokens:
input: 60006
output: 18
thoughts: 11
Thoughts: Okay, here's my interpretation of that thought process, tailored for an expert audience:
**Analyzing for Watchdog Timer Presence**
Alright, my initial instinct when faced with this is to go straight to the source – or, at least, the closest thing to it in this context. I need to understand the role of, and location of anything related to `watchdog_timer`. A grep is definitely the tool for the job here. It's the most efficient way to quickly scan across the filesystem and relevant source code to identify instances of this component. I'm assuming, given the context, that I'm looking for either a configuration file, a kernel module, systemd unit, or specific functions/macros. I will likely start with a recursive grep through the `/` directory, but I'll make sure to scope my search based on where I believe the relevant code or configuration is likely to reside, to ensure precision and prevent getting bogged down. The goal is pinpointing every instance – declarations, initializations, usages – to paint a comprehensive picture. I'll need to remember to account for variations, too. For instance, I might need to consider variations like `watchdog` or even less explicit forms. Let's see what we can find.
|
| 102/2 |
2026/05/21 11:43 |
tool |
grepper |
0m
Args: map[Expression:watchdog_timer]
Results: map[Output:
Full output is too long, showing 500 out of 1328 lines.
Use more precise expression if possible.
[Documentation/ABI/testing/sysfs-class-power-rt9756:1:What: /sys/class/power_supply/rt9756-*/watchdog_timer
Documentation/ABI/testing/sysfs-class-power-rt9756-2-Date: Dec 2025
--
Documentation/networking/net_cachelines/net_device.rst=125=struct hlist_head qdisc_hash[16]
Documentation/networking/net_cachelines/net_device.rst:126:struct timer_list watchdog_timer
Documentation/networking/net_cachelines/net_device.rst-127-int watchdog_timeo
--
arch/mips/bcm47xx/setup.c=62=static void bcm47xx_machine_restart(char *command)
--
arch/mips/bcm47xx/setup.c-71- write_c0_diag4(1 << 22);
arch/mips/bcm47xx/setup.c:72: ssb_watchdog_timer_set(&bcm47xx_bus.ssb, 1);
arch/mips/bcm47xx/setup.c-73- if (bcm47xx_bus.ssb.chip_id == 0x4785) {
--
arch/mips/bcm47xx/setup.c-83- case BCM47XX_BUS_TYPE_BCMA:
arch/mips/bcm47xx/setup.c:84: bcma_chipco_watchdog_timer_set(&bcm47xx_bus.bcma.bus.drv_cc, 1);
arch/mips/bcm47xx/setup.c-85- break;
--
arch/mips/bcm47xx/setup.c=92=static void bcm47xx_machine_halt(void)
--
arch/mips/bcm47xx/setup.c-98- case BCM47XX_BUS_TYPE_SSB:
arch/mips/bcm47xx/setup.c:99: ssb_watchdog_timer_set(&bcm47xx_bus.ssb, 0);
arch/mips/bcm47xx/setup.c-100- break;
--
arch/mips/bcm47xx/setup.c-103- case BCM47XX_BUS_TYPE_BCMA:
arch/mips/bcm47xx/setup.c:104: bcma_chipco_watchdog_timer_set(&bcm47xx_bus.bcma.bus.drv_cc, 0);
arch/mips/bcm47xx/setup.c-105- break;
--
arch/powerpc/kernel/watchdog.c=263=static void wd_smp_clear_cpu_pending(int cpu)
--
arch/powerpc/kernel/watchdog.c-345-
arch/powerpc/kernel/watchdog.c:346:static void watchdog_timer_interrupt(int cpu)
arch/powerpc/kernel/watchdog.c-347-{
--
arch/powerpc/kernel/watchdog.c=369=DEFINE_INTERRUPT_HANDLER_NMI(soft_nmi_interrupt)
--
arch/powerpc/kernel/watchdog.c-436-
arch/powerpc/kernel/watchdog.c:437:static enum hrtimer_restart watchdog_timer_fn(struct hrtimer *hrtimer)
arch/powerpc/kernel/watchdog.c-438-{
--
arch/powerpc/kernel/watchdog.c-446-
arch/powerpc/kernel/watchdog.c:447: watchdog_timer_interrupt(cpu);
arch/powerpc/kernel/watchdog.c-448-
--
arch/powerpc/kernel/watchdog.c=471=static void start_watchdog(void *arg)
--
arch/powerpc/kernel/watchdog.c-497-
arch/powerpc/kernel/watchdog.c:498: hrtimer_setup(hrtimer, watchdog_timer_fn, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
arch/powerpc/kernel/watchdog.c-499- hrtimer_start(hrtimer, ms_to_ktime(wd_timer_period_ms),
--
arch/powerpc/platforms/cell/spufs/switch.c=97=static inline void disable_interrupts(struct spu_state *csa, struct spu *spu)
--
arch/powerpc/platforms/cell/spufs/switch.c-133-
arch/powerpc/platforms/cell/spufs/switch.c:134:static inline void set_watchdog_timer(struct spu_state *csa, struct spu *spu)
arch/powerpc/platforms/cell/spufs/switch.c-135-{
--
arch/powerpc/platforms/cell/spufs/switch.c=1775=static int quiece_spu(struct spu_state *prev, struct spu *spu)
--
arch/powerpc/platforms/cell/spufs/switch.c-1790- disable_interrupts(prev, spu); /* Step 3. */
arch/powerpc/platforms/cell/spufs/switch.c:1791: set_watchdog_timer(prev, spu); /* Step 4. */
arch/powerpc/platforms/cell/spufs/switch.c-1792- inhibit_user_access(prev, spu); /* Step 5. */
--
arch/powerpc/platforms/cell/spufs/switch.c=1951=static void restore_lscsa(struct spu_state *next, struct spu *spu)
--
arch/powerpc/platforms/cell/spufs/switch.c-1958-
arch/powerpc/platforms/cell/spufs/switch.c:1959: set_watchdog_timer(next, spu); /* Step 26. */
arch/powerpc/platforms/cell/spufs/switch.c-1960- setup_spu_status_part1(next, spu); /* Step 27. */
--
drivers/bcma/driver_chipcommon.c=56=static u32 bcma_chipco_watchdog_get_max_timer(struct bcma_drv_cc *cc)
--
drivers/bcma/driver_chipcommon.c-76-
drivers/bcma/driver_chipcommon.c:77:static u32 bcma_chipco_watchdog_timer_set_wdt(struct bcm47xx_wdt *wdt,
drivers/bcma/driver_chipcommon.c-78- u32 ticks)
--
drivers/bcma/driver_chipcommon.c-81-
drivers/bcma/driver_chipcommon.c:82: return bcma_chipco_watchdog_timer_set(cc, ticks);
drivers/bcma/driver_chipcommon.c-83-}
drivers/bcma/driver_chipcommon.c-84-
drivers/bcma/driver_chipcommon.c:85:static u32 bcma_chipco_watchdog_timer_set_ms_wdt(struct bcm47xx_wdt *wdt,
drivers/bcma/driver_chipcommon.c-86- u32 ms)
--
drivers/bcma/driver_chipcommon.c-90-
drivers/bcma/driver_chipcommon.c:91: ticks = bcma_chipco_watchdog_timer_set(cc, cc->ticks_per_ms * ms);
drivers/bcma/driver_chipcommon.c-92- return ticks / cc->ticks_per_ms;
--
drivers/bcma/driver_chipcommon.c=113=int bcma_chipco_watchdog_register(struct bcma_drv_cc *cc)
--
drivers/bcma/driver_chipcommon.c-125- wdt.driver_data = cc;
drivers/bcma/driver_chipcommon.c:126: wdt.timer_set = bcma_chipco_watchdog_timer_set_wdt;
drivers/bcma/driver_chipcommon.c:127: wdt.timer_set_ms = bcma_chipco_watchdog_timer_set_ms_wdt;
drivers/bcma/driver_chipcommon.c-128- wdt.max_timer_ms =
--
drivers/bcma/driver_chipcommon.c=193=void bcma_core_chipcommon_init(struct bcma_drv_cc *cc)
--
drivers/bcma/driver_chipcommon.c-235-/* Set chip watchdog reset timer to fire in 'ticks' backplane cycles */
drivers/bcma/driver_chipcommon.c:236:u32 bcma_chipco_watchdog_timer_set(struct bcma_drv_cc *cc, u32 ticks)
drivers/bcma/driver_chipcommon.c-237-{
--
drivers/firmware/efi/libstub/efistub.h=255=union efi_boot_services {
--
drivers/firmware/efi/libstub/efistub.h-310- efi_status_t (__efiapi *stall)(unsigned long);
drivers/firmware/efi/libstub/efistub.h:311: void *set_watchdog_timer;
drivers/firmware/efi/libstub/efistub.h-312- void *connect_controller;
--
drivers/firmware/efi/libstub/efistub.h-362- u32 stall;
drivers/firmware/efi/libstub/efistub.h:363: u32 set_watchdog_timer;
drivers/firmware/efi/libstub/efistub.h-364- u32 connect_controller;
--
drivers/gpu/drm/amd/amdgpu/amdgpu.h=147=struct amdgpu_hwip_reg_entry {
--
drivers/gpu/drm/amd/amdgpu/amdgpu.h-154-
drivers/gpu/drm/amd/amdgpu/amdgpu.h:155:struct amdgpu_watchdog_timer {
drivers/gpu/drm/amd/amdgpu/amdgpu.h-156- bool timeout_fatal_disable;
--
drivers/gpu/drm/amd/amdgpu/amdgpu.h=220=extern bool amdgpu_ignore_bad_page_threshold;
drivers/gpu/drm/amd/amdgpu/amdgpu.h:221:extern struct amdgpu_watchdog_timer amdgpu_watchdog_timer;
drivers/gpu/drm/amd/amdgpu/amdgpu.h-222-extern int amdgpu_async_gfx_ring;
--
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c=267=int amdgpu_bad_page_threshold = -1;
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c:268:struct amdgpu_watchdog_timer amdgpu_watchdog_timer = {
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c-269- .timeout_fatal_disable = false,
--
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c=609=MODULE_PARM_DESC(timeout_fatal_disable, "disable watchdog timeout fatal error (false = default)");
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c:610:module_param_named(timeout_fatal_disable, amdgpu_watchdog_timer.timeout_fatal_disable, bool, 0644);
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c-611-
--
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c=616=MODULE_PARM_DESC(timeout_period, "watchdog timeout period (0 = timeout disabled, 1 ~ 0x23 = timeout maxcycles = (1 << period)");
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c:617:module_param_named(timeout_period, amdgpu_watchdog_timer.period, uint, 0644);
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c-618-
--
drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h=316=struct amdgpu_gfx_ras {
drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h-317- struct amdgpu_ras_block_object ras_block;
drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h:318: void (*enable_watchdog_timer)(struct amdgpu_device *adev);
drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h-319- int (*rlc_gc_fed_irq)(struct amdgpu_device *adev,
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c=4818=static int gfx_v9_0_ecc_late_init(struct amdgpu_ip_block *ip_block)
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c-4845- if (adev->gfx.ras &&
drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c:4846: adev->gfx.ras->enable_watchdog_timer)
drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c:4847: adev->gfx.ras->enable_watchdog_timer(adev);
drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c-4848-
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c=1779=static void gfx_v9_4_2_reset_ras_error_status(struct amdgpu_device *adev)
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c-1788-
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c:1789:static void gfx_v9_4_2_enable_watchdog_timer(struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c-1790-{
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c-1794- data = REG_SET_FIELD(0, SQ_TIMEOUT_CONFIG, TIMEOUT_FATAL_DISABLE,
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c:1795: amdgpu_watchdog_timer.timeout_fatal_disable ? 1 :
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c-1796- 0);
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c-1797-
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c:1798: if (amdgpu_watchdog_timer.timeout_fatal_disable &&
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c:1799: (amdgpu_watchdog_timer.period < 1 ||
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c:1800: amdgpu_watchdog_timer.period > 0x23)) {
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c-1801- dev_warn(adev->dev, "Watchdog period range is 1 to 0x23\n");
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c:1802: amdgpu_watchdog_timer.period = 0x23;
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c-1803- }
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c-1804- data = REG_SET_FIELD(data, SQ_TIMEOUT_CONFIG, PERIOD_SEL,
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c:1805: amdgpu_watchdog_timer.period);
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c-1806-
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c=1933=struct amdgpu_gfx_ras gfx_v9_4_2_ras = {
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c-1936- },
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c:1937: .enable_watchdog_timer = &gfx_v9_4_2_enable_watchdog_timer,
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c-1938-};
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c=2530=static int gfx_v9_4_3_late_init(struct amdgpu_ip_block *ip_block)
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-2547- if (adev->gfx.ras &&
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c:2548: adev->gfx.ras->enable_watchdog_timer)
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c:2549: adev->gfx.ras->enable_watchdog_timer(adev);
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-2550-
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c=4455=static void gfx_v9_4_3_inst_reset_ras_err_count(struct amdgpu_device *adev,
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-4504-
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c:4505:static void gfx_v9_4_3_inst_enable_watchdog_timer(struct amdgpu_device *adev,
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-4506- void *ras_error_status, int xcc_id)
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-4515- data = REG_SET_FIELD(data, SQ_TIMEOUT_CONFIG, TIMEOUT_FATAL_DISABLE,
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c:4516: amdgpu_watchdog_timer.timeout_fatal_disable ? 1 : 0);
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-4517-
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c:4518: if (amdgpu_watchdog_timer.timeout_fatal_disable &&
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c:4519: (amdgpu_watchdog_timer.period < 1 ||
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c:4520: amdgpu_watchdog_timer.period > 0x23)) {
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-4521- dev_warn(adev->dev, "Watchdog period range is 1 to 0x23\n");
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c:4522: amdgpu_watchdog_timer.period = 0x23;
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-4523- }
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-4524- data = REG_SET_FIELD(data, SQ_TIMEOUT_CONFIG, PERIOD_SEL,
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c:4525: amdgpu_watchdog_timer.period);
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-4526-
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c=4544=static void gfx_v9_4_3_reset_ras_error_count(struct amdgpu_device *adev)
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-4548-
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c:4549:static void gfx_v9_4_3_enable_watchdog_timer(struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-4550-{
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c:4551: amdgpu_gfx_ras_error_func(adev, NULL, gfx_v9_4_3_inst_enable_watchdog_timer);
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-4552-}
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c=5056=struct amdgpu_gfx_ras gfx_v9_4_3_ras = {
--
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-5060- },
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c:5061: .enable_watchdog_timer = &gfx_v9_4_3_enable_watchdog_timer,
drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c-5062-};
--
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c=161=static void process_output(struct hdcp_workqueue *hdcp_work)
--
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c-171-
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c:172: if (output.watchdog_timer_stop)
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c:173: cancel_delayed_work(&hdcp_work->watchdog_timer_dwork);
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c-174-
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c:175: if (output.watchdog_timer_needed)
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c:176: schedule_delayed_work(&hdcp_work->watchdog_timer_dwork,
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c:177: msecs_to_jiffies(output.watchdog_timer_delay));
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c-178-
--
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c=400=static void event_property_validate(struct work_struct *work)
--
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c-449-
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c:450:static void event_watchdog_timer(struct work_struct *work)
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c-451-{
--
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c-455- struct hdcp_workqueue,
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c:456: watchdog_timer_dwork);
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c-457-
--
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c-459-
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c:460: cancel_delayed_work(&hdcp_work->watchdog_timer_dwork);
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c-461-
--
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c=482=void hdcp_destroy(struct kobject *kobj, struct hdcp_workqueue *hdcp_work)
--
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c-487- cancel_delayed_work_sync(&hdcp_work[i].callback_dwork);
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c:488: cancel_delayed_work_sync(&hdcp_work[i].watchdog_timer_dwork);
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c-489- cancel_delayed_work_sync(&hdcp_work[i].property_validate_dwork);
--
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c=740=struct hdcp_workqueue *hdcp_create_workqueue(struct amdgpu_device *adev,
--
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c-770- INIT_DELAYED_WORK(&hdcp_work[i].callback_dwork, event_callback);
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c:771: INIT_DELAYED_WORK(&hdcp_work[i].watchdog_timer_dwork, event_watchdog_timer);
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c-772- INIT_DELAYED_WORK(&hdcp_work[i].property_validate_dwork, event_property_validate);
--
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.h=41=struct hdcp_workqueue {
--
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.h-44- struct delayed_work callback_dwork;
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.h:45: struct delayed_work watchdog_timer_dwork;
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.h-46- struct delayed_work property_validate_dwork;
--
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.c=189=static enum mod_hdcp_status reset_authentication(struct mod_hdcp *hdcp,
--
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.c-211- output->callback_needed = 0;
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.c:212: output->watchdog_timer_needed = 0;
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.c-213- goto out;
--
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.c-229- /* stop callback and watchdog requests from previous authentication*/
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.c:230: output->watchdog_timer_stop = 1;
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.c-231- output->callback_stop = 1;
--
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h=415=static inline void set_state_id(struct mod_hdcp *hdcp,
--
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h-421- output->callback_stop = 1;
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h:422: output->watchdog_timer_stop = 1;
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h-423- HDCP_NEXT_STATE_TRACE(hdcp, id, output);
--
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h=484=static inline void fail_and_restart_in_ms(uint16_t time,
--
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h-489- output->callback_delay = time;
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h:490: output->watchdog_timer_needed = 0;
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h:491: output->watchdog_timer_delay = 0;
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h-492- *status = MOD_HDCP_STATUS_RESET_NEEDED;
--
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h=501=static inline void set_watchdog_in_ms(struct mod_hdcp *hdcp, uint16_t time,
--
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h-503-{
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h:504: output->watchdog_timer_needed = 1;
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h:505: output->watchdog_timer_delay = time;
drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h-506-}
--
drivers/gpu/drm/amd/display/modules/hdcp/hdcp_log.h-74-#define HDCP_NEXT_STATE_TRACE(hdcp, id, output) do { \
drivers/gpu/drm/amd/display/modules/hdcp/hdcp_log.h:75: if (output->watchdog_timer_needed) \
drivers/gpu/drm/amd/display/modules/hdcp/hdcp_log.h-76- HDCP_LOG_FSM(hdcp, \
--
drivers/gpu/drm/amd/display/modules/hdcp/hdcp_log.h-78- hdcp->config.index, \
drivers/gpu/drm/amd/display/modules/hdcp/hdcp_log.h:79: mod_hdcp_state_id_to_str(id), output->watchdog_timer_delay); \
drivers/gpu/drm/amd/display/modules/hdcp/hdcp_log.h-80- else \
--
drivers/gpu/drm/amd/display/modules/inc/mod_hdcp.h=274=struct mod_hdcp_output {
--
drivers/gpu/drm/amd/display/modules/inc/mod_hdcp.h-276- uint8_t callback_stop;
drivers/gpu/drm/amd/display/modules/inc/mod_hdcp.h:277: uint8_t watchdog_timer_needed;
drivers/gpu/drm/amd/display/modules/inc/mod_hdcp.h:278: uint8_t watchdog_timer_stop;
drivers/gpu/drm/amd/display/modules/inc/mod_hdcp.h-279- uint16_t callback_delay;
drivers/gpu/drm/amd/display/modules/inc/mod_hdcp.h:280: uint16_t watchdog_timer_delay;
drivers/gpu/drm/amd/display/modules/inc/mod_hdcp.h-281- uint8_t auth_complete;
--
drivers/gpu/drm/ci/xfails/msm-sm8350-hdk-skips.txt=20=msm/msm_mapping@ring
--
drivers/gpu/drm/ci/xfails/msm-sm8350-hdk-skips.txt-161-# [ 228.506639] panic+0x3bc/0x41c
drivers/gpu/drm/ci/xfails/msm-sm8350-hdk-skips.txt:162:# [ 228.509826] watchdog_timer_fn+0x254/0x2e4
drivers/gpu/drm/ci/xfails/msm-sm8350-hdk-skips.txt-163-# [ 228.514087] __hrtimer_run_queues+0x3b0/0x40c
--
drivers/gpu/drm/imagination/pvr_device.c=269=static void pvr_device_safety_irq_init(struct pvr_device *pvr_dev)
--
drivers/gpu/drm/imagination/pvr_device.c-287- pvr_dev->has_safety_events =
drivers/gpu/drm/imagination/pvr_device.c:288: num_ecc_rams > 0 || PVR_HAS_FEATURE(pvr_dev, watchdog_timer);
drivers/gpu/drm/imagination/pvr_device.c-289-}
--
drivers/gpu/drm/imagination/pvr_device_info.c=125=static const struct {
--
drivers/gpu/drm/imagination/pvr_device_info.c-189- FEATURE_MAPPING_VALUE(VIRTUAL_ADDRESS_SPACE_BITS, virtual_address_space_bits),
drivers/gpu/drm/imagination/pvr_device_info.c:190: FEATURE_MAPPING(WATCHDOG_TIMER, watchdog_timer),
drivers/gpu/drm/imagination/pvr_device_info.c-191- FEATURE_MAPPING(WORKGROUP_PROTECTION, workgroup_protection),
--
drivers/gpu/drm/imagination/pvr_device_info.h=14=struct pvr_device_features {
--
drivers/gpu/drm/imagination/pvr_device_info.h-75- bool has_virtual_address_space_bits;
drivers/gpu/drm/imagination/pvr_device_info.h:76: bool has_watchdog_timer;
drivers/gpu/drm/imagination/pvr_device_info.h-77- bool has_workgroup_protection;
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c=144=static void s5p_mfc_watchdog(struct timer_list *t)
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-145-{
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c:146: struct s5p_mfc_dev *dev = timer_container_of(dev, t, watchdog_timer);
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-147-
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-160- }
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c:161: dev->watchdog_timer.expires = jiffies +
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-162- msecs_to_jiffies(MFC_WATCHDOG_INTERVAL);
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c:163: add_timer(&dev->watchdog_timer);
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-164-}
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c=782=static int s5p_mfc_open(struct file *file)
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-855- if (dev->num_inst == 1) {
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c:856: dev->watchdog_timer.expires = jiffies +
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-857- msecs_to_jiffies(MFC_WATCHDOG_INTERVAL);
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c:858: add_timer(&dev->watchdog_timer);
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-859- ret = s5p_mfc_power_on(dev);
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-950- mfc_err("power off failed\n");
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c:951: timer_delete_sync(&dev->watchdog_timer);
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-952- }
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c=970=static int s5p_mfc_release(struct file *file)
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-1000- s5p_mfc_deinit_hw(dev);
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c:1001: timer_delete_sync(&dev->watchdog_timer);
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-1002- s5p_mfc_clock_off(dev);
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c=1301=static int s5p_mfc_probe(struct platform_device *pdev)
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-1363- atomic_set(&dev->watchdog_cnt, 0);
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c:1364: timer_setup(&dev->watchdog_timer, s5p_mfc_watchdog, 0);
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-1365-
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c=1454=static void s5p_mfc_remove(struct platform_device *pdev)
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-1476-
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c:1477: timer_delete_sync(&dev->watchdog_timer);
drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c-1478- flush_work(&dev->watchdog_work);
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc_common.h=248=struct s5p_mfc_priv_buf {
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc_common.h-289- * @watchdog_cnt: counter for the watchdog
drivers/media/platform/samsung/s5p-mfc/s5p_mfc_common.h:290: * @watchdog_timer: timer for the watchdog
drivers/media/platform/samsung/s5p-mfc/s5p_mfc_common.h-291- * @watchdog_workqueue: workqueue for the watchdog
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc_common.h=305=struct s5p_mfc_dev {
--
drivers/media/platform/samsung/s5p-mfc/s5p_mfc_common.h-336- atomic_t watchdog_cnt;
drivers/media/platform/samsung/s5p-mfc/s5p_mfc_common.h:337: struct timer_list watchdog_timer;
drivers/media/platform/samsung/s5p-mfc/s5p_mfc_common.h-338- struct workqueue_struct *watchdog_workqueue;
--
drivers/net/ethernet/agere/et131x.c=1495=static void et131x_configure_global_regs(struct et131x_adapter *adapter)
--
drivers/net/ethernet/agere/et131x.c-1531- */
drivers/net/ethernet/agere/et131x.c:1532: writel(0, ®s->watchdog_timer);
drivers/net/ethernet/agere/et131x.c-1533-}
--
drivers/net/ethernet/agere/et131x.c=2292=static int et131x_handle_recv_pkts(struct et131x_adapter *adapter, int budget)
--
drivers/net/ethernet/agere/et131x.c-2336- writel(PARM_TX_TIME_INT_DEF * NANO_IN_A_MICRO,
drivers/net/ethernet/agere/et131x.c:2337: &adapter->regs->global.watchdog_timer);
drivers/net/ethernet/agere/et131x.c-2338- } else {
--
drivers/net/ethernet/agere/et131x.c=2419=static int nic_send_packet(struct et131x_adapter *adapter, struct tcb *tcb)
--
drivers/net/ethernet/agere/et131x.c-2592- writel(PARM_TX_TIME_INT_DEF * NANO_IN_A_MICRO,
drivers/net/ethernet/agere/et131x.c:2593: &adapter->regs->global.watchdog_timer);
drivers/net/ethernet/agere/et131x.c-2594- }
--
drivers/net/ethernet/agere/et131x.c=2832=static void et131x_get_regs(struct net_device *netdev,
--
drivers/net/ethernet/agere/et131x.c-2923- regs_buff[num++] = readl(&aregs->global.loopback);
drivers/net/ethernet/agere/et131x.c:2924: regs_buff[num++] = readl(&aregs->global.watchdog_timer);
drivers/net/ethernet/agere/et131x.c-2925-
--
drivers/net/ethernet/agere/et131x.c=3393=static irqreturn_t et131x_isr(int irq, void *dev_id)
--
drivers/net/ethernet/agere/et131x.c-3436- else if (tcb == NULL)
drivers/net/ethernet/agere/et131x.c:3437: writel(0, &adapter->regs->global.watchdog_timer);
drivers/net/ethernet/agere/et131x.c-3438-
--
drivers/net/ethernet/agere/et131x.h=155=struct global_regs { /* Location: */
--
drivers/net/ethernet/agere/et131x.h-169- u32 loopback; /* 0x0034 */
drivers/net/ethernet/agere/et131x.h:170: u32 watchdog_timer; /* 0x0038 */
drivers/net/ethernet/agere/et131x.h-171-};
--
drivers/net/ethernet/amd/pcnet32.c=255=struct pcnet32_private {
--
drivers/net/ethernet/amd/pcnet32.c-295- struct mii_if_info mii_if;
drivers/net/ethernet/amd/pcnet32.c:296: struct timer_list watchdog_timer;
drivers/net/ethernet/amd/pcnet32.c-297- u32 msg_enable; /* debug message level */
--
drivers/net/ethernet/amd/pcnet32.c=1596=pcnet32_probe1(unsigned long ioaddr, int shared, struct pci_dev *pdev)
--
drivers/net/ethernet/amd/pcnet32.c-1978-
drivers/net/ethernet/amd/pcnet32.c:1979: timer_setup(&lp->watchdog_timer, pcnet32_watchdog, 0);
drivers/net/ethernet/amd/pcnet32.c-1980-
--
drivers/net/ethernet/amd/pcnet32.c=2092=static int pcnet32_open(struct net_device *dev)
--
drivers/net/ethernet/amd/pcnet32.c-2287- pcnet32_check_media(dev, 1);
drivers/net/ethernet/amd/pcnet32.c:2288: mod_timer(&lp->watchdog_timer, PCNET32_WATCHDOG_TIMEOUT);
drivers/net/ethernet/amd/pcnet32.c-2289- }
--
drivers/net/ethernet/amd/pcnet32.c=2627=static int pcnet32_close(struct net_device *dev)
--
drivers/net/ethernet/amd/pcnet32.c-2632-
drivers/net/ethernet/amd/pcnet32.c:2633: timer_delete_sync(&lp->watchdog_timer);
drivers/net/ethernet/amd/pcnet32.c-2634-
--
drivers/net/ethernet/amd/pcnet32.c=2906=static void pcnet32_watchdog(struct timer_list *t)
drivers/net/ethernet/amd/pcnet32.c-2907-{
drivers/net/ethernet/amd/pcnet32.c:2908: struct pcnet32_private *lp = timer_container_of(lp, t, watchdog_timer);
drivers/net/ethernet/amd/pcnet32.c-2909- struct net_device *dev = lp->dev;
--
drivers/net/ethernet/amd/pcnet32.c-2916-
drivers/net/ethernet/amd/pcnet32.c:2917: mod_timer(&lp->watchdog_timer, round_jiffies(PCNET32_WATCHDOG_TIMEOUT));
drivers/net/ethernet/amd/pcnet32.c-2918-}
--
drivers/net/ethernet/atheros/atl1c/atl1c.h=510=struct atl1c_adapter {
--
drivers/net/ethernet/atheros/atl1c/atl1c.h-537- struct work_struct common_task;
drivers/net/ethernet/atheros/atl1c/atl1c.h:538: struct timer_list watchdog_timer;
drivers/net/ethernet/atheros/atl1c/atl1c.h-539- struct timer_list phy_config_timer;
--
drivers/net/ethernet/atheros/atl1e/atl1e.h=419=struct atl1e_adapter {
--
drivers/net/ethernet/atheros/atl1e/atl1e.h-435- struct work_struct link_chg_task;
drivers/net/ethernet/atheros/atl1e/atl1e.h:436: struct timer_list watchdog_timer;
drivers/net/ethernet/atheros/atl1e/atl1e.h-437- struct timer_list phy_config_timer;
--
drivers/net/ethernet/atheros/atlx/atl2.c=682=static int atl2_open(struct net_device *netdev)
--
drivers/net/ethernet/atheros/atlx/atl2.c-719-
drivers/net/ethernet/atheros/atlx/atl2.c:720: mod_timer(&adapter->watchdog_timer, round_jiffies(jiffies + 4*HZ));
drivers/net/ethernet/atheros/atlx/atl2.c-721-
--
drivers/net/ethernet/atheros/atlx/atl2.c=739=static void atl2_down(struct atl2_adapter *adapter)
--
drivers/net/ethernet/atheros/atlx/atl2.c-754-
drivers/net/ethernet/atheros/atlx/atl2.c:755: timer_delete_sync(&adapter->watchdog_timer);
drivers/net/ethernet/atheros/atlx/atl2.c-756- timer_delete_sync(&adapter->phy_config_timer);
--
drivers/net/ethernet/atheros/atlx/atl2.c=1011=static void atl2_watchdog(struct timer_list *t)
--
drivers/net/ethernet/atheros/atlx/atl2.c-1013- struct atl2_adapter *adapter = timer_container_of(adapter, t,
drivers/net/ethernet/atheros/atlx/atl2.c:1014: watchdog_timer);
drivers/net/ethernet/atheros/atlx/atl2.c-1015-
--
drivers/net/ethernet/atheros/atlx/atl2.c-1027- /* Reset the timer */
drivers/net/ethernet/atheros/atlx/atl2.c:1028: mod_timer(&adapter->watchdog_timer,
drivers/net/ethernet/atheros/atlx/atl2.c-1029- round_jiffies(jiffies + 4 * HZ));
--
drivers/net/ethernet/atheros/atlx/atl2.c=1315=static int atl2_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
--
drivers/net/ethernet/atheros/atlx/atl2.c-1416-
drivers/net/ethernet/atheros/atlx/atl2.c:1417: timer_setup(&adapter->watchdog_timer, atl2_watchdog, 0);
drivers/net/ethernet/atheros/atlx/atl2.c-1418-
--
drivers/net/ethernet/atheros/atlx/atl2.c=1463=static void atl2_remove(struct pci_dev *pdev)
--
drivers/net/ethernet/atheros/atlx/atl2.c-1471-
drivers/net/ethernet/atheros/atlx/atl2.c:1472: timer_delete_sync(&adapter->watchdog_timer);
drivers/net/ethernet/atheros/atlx/atl2.c-1473- timer_delete_sync(&adapter->phy_config_timer);
--
drivers/net/ethernet/atheros/atlx/atl2.h=435=struct atl2_adapter {
--
drivers/net/ethernet/atheros/atlx/atl2.h-446- struct work_struct link_chg_task;
drivers/net/ethernet/atheros/atlx/atl2.h:447: struct timer_list watchdog_timer;
drivers/net/ethernet/atheros/atlx/atl2.h-448- struct timer_list phy_config_timer;
--
drivers/net/ethernet/intel/e1000e/e1000.h=191=struct e1000_adapter {
drivers/net/ethernet/intel/e1000e/e1000.h:192: struct timer_list watchdog_timer;
drivers/net/ethernet/intel/e1000e/e1000.h-193- struct timer_list phy_info_timer;
--
drivers/net/ethernet/intel/e1000e/netdev.c=1750=static irqreturn_t e1000_intr_msi(int __always_unused irq, void *data)
]
]
|
| 103/2 |
2026/05/21 11:43 |
llm |
expert |
0m
Model:
Tokens:
input: 74321
output: 40
thoughts: 38
Thoughts: Okay, here's my take on those thoughts, framed as my own mental process, given my expertise:
**Checking the Watchdog Timer Definition**
Hmm, so `dev->watchdog_timer`... that feels like a *netdevice* specific thing, doesn't it? Let's confirm that. My gut says it's in the network device header. It's a key element of network device drivers, particularly in this context. Let's quickly pull up `include/linux/netdevice.h`. Need to make absolutely sure. Can't trust guesses when troubleshooting kernel internals! Time to check the source.
|
| 104/2 |
2026/05/21 11:43 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:include/linux/netdevice.h IncludeLines:true Name:net_device]
Results: map[SourceCode:2105: struct net_device {
2106: /* Cacheline organization can be found documented in
2107: * Documentation/networking/net_cachelines/net_device.rst.
2108: * Please update the document when adding new fields.
2109: */
2110:
2111: /* TX read-mostly hotpath */
2112: __cacheline_group_begin(net_device_read_tx);
2113: struct_group(priv_flags_fast,
2114: unsigned long priv_flags:32;
2115: unsigned long lltx:1;
2116: unsigned long netmem_tx:1;
2117: );
2118: const struct net_device_ops *netdev_ops;
2119: const struct header_ops *header_ops;
2120: struct netdev_queue *_tx;
2121: netdev_features_t gso_partial_features;
2122: unsigned int real_num_tx_queues;
2123: unsigned int gso_max_size;
2124: unsigned int gso_ipv4_max_size;
2125: u16 gso_max_segs;
2126: s16 num_tc;
2127: /* Note : dev->mtu is often read without holding a lock.
2128: * Writers usually hold RTNL.
2129: * It is recommended to use READ_ONCE() to annotate the reads,
2130: * and to use WRITE_ONCE() to annotate the writes.
2131: */
2132: unsigned int mtu;
2133: unsigned short needed_headroom;
2134: struct netdev_tc_txq tc_to_txq[TC_MAX_QUEUE];
2135: #ifdef CONFIG_XPS
2136: struct xps_dev_maps __rcu *xps_maps[XPS_MAPS_MAX];
2137: #endif
2138: #ifdef CONFIG_NETFILTER_EGRESS
2139: struct nf_hook_entries __rcu *nf_hooks_egress;
2140: #endif
2141: #ifdef CONFIG_NET_XGRESS
2142: struct bpf_mprog_entry __rcu *tcx_egress;
2143: #endif
2144: __cacheline_group_end(net_device_read_tx);
2145:
2146: /* TXRX read-mostly hotpath */
2147: __cacheline_group_begin(net_device_read_txrx);
2148: union {
2149: struct pcpu_lstats __percpu *lstats;
2150: struct pcpu_sw_netstats __percpu *tstats;
2151: struct pcpu_dstats __percpu *dstats;
2152: };
2153: unsigned long state;
2154: unsigned int flags;
2155: unsigned short hard_header_len;
2156: netdev_features_t features;
2157: struct inet6_dev __rcu *ip6_ptr;
2158: __cacheline_group_end(net_device_read_txrx);
2159:
2160: /* RX read-mostly hotpath */
2161: __cacheline_group_begin(net_device_read_rx);
2162: struct bpf_prog __rcu *xdp_prog;
2163: struct list_head ptype_specific;
2164: int ifindex;
2165: unsigned int real_num_rx_queues;
2166: struct netdev_rx_queue *_rx;
2167: unsigned int gro_max_size;
2168: unsigned int gro_ipv4_max_size;
2169: rx_handler_func_t __rcu *rx_handler;
2170: void __rcu *rx_handler_data;
2171: possible_net_t nd_net;
2172: #ifdef CONFIG_NETPOLL
2173: struct netpoll_info __rcu *npinfo;
2174: #endif
2175: #ifdef CONFIG_NET_XGRESS
2176: struct bpf_mprog_entry __rcu *tcx_ingress;
2177: #endif
2178: __cacheline_group_end(net_device_read_rx);
2179:
2180: char name[IFNAMSIZ];
2181: struct netdev_name_node *name_node;
2182: struct dev_ifalias __rcu *ifalias;
2183: /*
2184: * I/O specific fields
2185: * FIXME: Merge these and struct ifmap into one
2186: */
2187: unsigned long mem_end;
2188: unsigned long mem_start;
2189: unsigned long base_addr;
2190:
2191: /*
2192: * Some hardware also needs these fields (state,dev_list,
2193: * napi_list,unreg_list,close_list) but they are not
2194: * part of the usual set specified in Space.c.
2195: */
2196:
2197:
2198: struct list_head dev_list;
2199: struct list_head napi_list;
2200: struct list_head unreg_list;
2201: struct list_head close_list;
2202: struct list_head ptype_all;
2203:
2204: struct {
2205: struct list_head upper;
2206: struct list_head lower;
2207: } adj_list;
2208:
2209: /* Read-mostly cache-line for fast-path access */
2210: xdp_features_t xdp_features;
2211: const struct xdp_metadata_ops *xdp_metadata_ops;
2212: const struct xsk_tx_metadata_ops *xsk_tx_metadata_ops;
2213: unsigned short gflags;
2214:
2215: unsigned short needed_tailroom;
2216:
2217: netdev_features_t hw_features;
2218: netdev_features_t wanted_features;
2219: netdev_features_t vlan_features;
2220: netdev_features_t hw_enc_features;
2221: netdev_features_t mpls_features;
2222:
2223: unsigned int min_mtu;
2224: unsigned int max_mtu;
2225: unsigned short type;
2226: unsigned char min_header_len;
2227: unsigned char name_assign_type;
2228:
2229: int group;
2230:
2231: struct net_device_stats stats; /* not used by modern drivers */
2232:
2233: struct net_device_core_stats __percpu *core_stats;
2234:
2235: /* Stats to monitor link on/off, flapping */
2236: atomic_t carrier_up_count;
2237: atomic_t carrier_down_count;
2238:
2239: #ifdef CONFIG_WIRELESS_EXT
2240: const struct iw_handler_def *wireless_handlers;
2241: #endif
2242: const struct ethtool_ops *ethtool_ops;
2243: #ifdef CONFIG_NET_L3_MASTER_DEV
2244: const struct l3mdev_ops *l3mdev_ops;
2245: #endif
2246: #if IS_ENABLED(CONFIG_IPV6)
2247: const struct ndisc_ops *ndisc_ops;
2248: #endif
2249:
2250: #ifdef CONFIG_XFRM_OFFLOAD
2251: const struct xfrmdev_ops *xfrmdev_ops;
2252: #endif
2253:
2254: #if IS_ENABLED(CONFIG_TLS_DEVICE)
2255: const struct tlsdev_ops *tlsdev_ops;
2256: #endif
2257:
2258: unsigned int operstate;
2259: unsigned char link_mode;
2260:
2261: unsigned char if_port;
2262: unsigned char dma;
2263:
2264: /* Interface address info. */
2265: unsigned char perm_addr[MAX_ADDR_LEN];
2266: unsigned char addr_assign_type;
2267: unsigned char addr_len;
2268: unsigned char upper_level;
2269: unsigned char lower_level;
2270: u8 threaded;
2271:
2272: unsigned short neigh_priv_len;
2273: unsigned short dev_id;
2274: unsigned short dev_port;
2275: int irq;
2276: u32 priv_len;
2277:
2278: spinlock_t addr_list_lock;
2279:
2280: struct netdev_hw_addr_list uc;
2281: struct netdev_hw_addr_list mc;
2282: struct netdev_hw_addr_list dev_addrs;
2283:
2284: #ifdef CONFIG_SYSFS
2285: struct kset *queues_kset;
2286: #endif
2287: #ifdef CONFIG_LOCKDEP
2288: struct list_head unlink_list;
2289: #endif
2290: unsigned int promiscuity;
2291: unsigned int allmulti;
2292: bool uc_promisc;
2293: #ifdef CONFIG_LOCKDEP
2294: unsigned char nested_level;
2295: #endif
2296:
2297:
2298: /* Protocol-specific pointers */
2299: struct in_device __rcu *ip_ptr;
2300: /** @fib_nh_head: nexthops associated with this netdev */
2301: struct hlist_head fib_nh_head;
2302:
2303: #if IS_ENABLED(CONFIG_VLAN_8021Q)
2304: struct vlan_info __rcu *vlan_info;
2305: #endif
2306: #if IS_ENABLED(CONFIG_NET_DSA)
2307: struct dsa_port *dsa_ptr;
2308: #endif
2309: #if IS_ENABLED(CONFIG_TIPC)
2310: struct tipc_bearer __rcu *tipc_ptr;
2311: #endif
2312: #if IS_ENABLED(CONFIG_ATALK)
2313: void *atalk_ptr;
2314: #endif
2315: #if IS_ENABLED(CONFIG_AX25)
2316: struct ax25_dev __rcu *ax25_ptr;
2317: #endif
2318: #if IS_ENABLED(CONFIG_CFG80211)
2319: struct wireless_dev *ieee80211_ptr;
2320: #endif
2321: #if IS_ENABLED(CONFIG_IEEE802154) || IS_ENABLED(CONFIG_6LOWPAN)
2322: struct wpan_dev *ieee802154_ptr;
2323: #endif
2324: #if IS_ENABLED(CONFIG_MPLS_ROUTING)
2325: struct mpls_dev __rcu *mpls_ptr;
2326: #endif
2327: #if IS_ENABLED(CONFIG_MCTP)
2328: struct mctp_dev __rcu *mctp_ptr;
2329: #endif
2330: #if IS_ENABLED(CONFIG_INET_PSP)
2331: struct psp_dev __rcu *psp_dev;
2332: #endif
2333:
2334: /*
2335: * Cache lines mostly used on receive path (including eth_type_trans())
2336: */
2337: /* Interface address info used in eth_type_trans() */
2338: const unsigned char *dev_addr;
2339:
2340: unsigned int num_rx_queues;
2341: #define GRO_LEGACY_MAX_SIZE 65536u
2342: /* TCP minimal MSS is 8 (TCP_MIN_GSO_SIZE),
2343: * and shinfo->gso_segs is a 16bit field.
2344: */
2345: #define GRO_MAX_SIZE (8 * 65535u)
2346: unsigned int xdp_zc_max_segs;
2347: struct netdev_queue __rcu *ingress_queue;
2348: #ifdef CONFIG_NETFILTER_INGRESS
2349: struct nf_hook_entries __rcu *nf_hooks_ingress;
2350: #endif
2351:
2352: unsigned char broadcast[MAX_ADDR_LEN];
2353: #ifdef CONFIG_RFS_ACCEL
2354: struct cpu_rmap *rx_cpu_rmap;
2355: #endif
2356: struct hlist_node index_hlist;
2357:
2358: /*
2359: * Cache lines mostly used on transmit path
2360: */
2361: unsigned int num_tx_queues;
2362: struct Qdisc __rcu *qdisc;
2363: unsigned int tx_queue_len;
2364: spinlock_t tx_global_lock;
2365:
2366: struct xdp_dev_bulk_queue __percpu *xdp_bulkq;
2367:
2368: #ifdef CONFIG_NET_SCHED
2369: DECLARE_HASHTABLE (qdisc_hash, 4);
2370: #endif
2371: /* These may be needed for future network-power-down code. */
2372: struct timer_list watchdog_timer;
2373: int watchdog_timeo;
2374:
2375: u32 proto_down_reason;
2376:
2377: struct list_head todo_list;
2378:
2379: #ifdef CONFIG_PCPU_DEV_REFCNT
2380: int __percpu *pcpu_refcnt;
2381: #else
2382: refcount_t dev_refcnt;
2383: #endif
2384: struct ref_tracker_dir refcnt_tracker;
2385:
2386: struct list_head link_watch_list;
2387:
2388: u8 reg_state;
2389:
2390: bool dismantle;
2391:
2392: /** @moving_ns: device is changing netns, protected by @lock */
2393: bool moving_ns;
2394: /** @rtnl_link_initializing: Device being created, suppress events */
2395: bool rtnl_link_initializing;
2396:
2397: bool needs_free_netdev;
2398: void (*priv_destructor)(struct net_device *dev);
2399:
2400: /* mid-layer private */
2401: void *ml_priv;
2402: enum netdev_ml_priv_type ml_priv_type;
2403:
2404: enum netdev_stat_type pcpu_stat_type:8;
2405:
2406: #if IS_ENABLED(CONFIG_GARP)
2407: struct garp_port __rcu *garp_port;
2408: #endif
2409: #if IS_ENABLED(CONFIG_MRP)
2410: struct mrp_port __rcu *mrp_port;
2411: #endif
2412: #if IS_ENABLED(CONFIG_NET_DROP_MONITOR)
2413: struct dm_hw_stat_delta __rcu *dm_private;
2414: #endif
2415: struct device dev;
2416: const struct attribute_group *sysfs_groups[5];
2417: const struct attribute_group *sysfs_rx_queue_group;
2418:
2419: const struct rtnl_link_ops *rtnl_link_ops;
2420:
2421: const struct netdev_stat_ops *stat_ops;
2422:
2423: const struct netdev_queue_mgmt_ops *queue_mgmt_ops;
2424:
2425: /* for setting kernel sock attribute on TCP connection setup */
2426: #define GSO_MAX_SEGS 65535u
2427: #define GSO_LEGACY_MAX_SIZE 65536u
2428: /* TCP minimal MSS is 8 (TCP_MIN_GSO_SIZE),
2429: * and shinfo->gso_segs is a 16bit field.
2430: */
2431: #define GSO_MAX_SIZE (8 * GSO_MAX_SEGS)
2432:
2433: #define TSO_LEGACY_MAX_SIZE 65536
2434: #define TSO_MAX_SIZE UINT_MAX
2435: unsigned int tso_max_size;
2436: #define TSO_MAX_SEGS U16_MAX
2437: u16 tso_max_segs;
2438:
2439: #ifdef CONFIG_DCB
2440: const struct dcbnl_rtnl_ops *dcbnl_ops;
2441: #endif
2442: u8 prio_tc_map[TC_BITMASK + 1];
2443:
2444: #if IS_ENABLED(CONFIG_FCOE)
2445: unsigned int fcoe_ddp_xid;
2446: #endif
2447: #if IS_ENABLED(CONFIG_CGROUP_NET_PRIO)
2448: struct netprio_map __rcu *priomap;
2449: #endif
2450: struct phy_link_topology *link_topo;
2451: struct phy_device *phydev;
2452: struct sfp_bus *sfp_bus;
2453: struct lock_class_key *qdisc_tx_busylock;
2454: bool proto_down;
2455: bool irq_affinity_auto;
2456: bool rx_cpu_rmap_auto;
2457:
2458: /* priv_flags_slow, ungrouped to save space */
2459: unsigned long see_all_hwtstamp_requests:1;
2460: unsigned long change_proto_down:1;
2461: unsigned long netns_immutable:1;
2462: unsigned long fcoe_mtu:1;
2463:
2464: struct list_head net_notifier_list;
2465:
2466: #if IS_ENABLED(CONFIG_MACSEC)
2467: /* MACsec management functions */
2468: const struct macsec_ops *macsec_ops;
2469: #endif
2470: const struct udp_tunnel_nic_info *udp_tunnel_nic_info;
2471: struct udp_tunnel_nic *udp_tunnel_nic;
2472:
2473: /** @cfg: net_device queue-related configuration */
2474: struct netdev_config *cfg;
2475: /**
2476: * @cfg_pending: same as @cfg but when device is being actively
2477: * reconfigured includes any changes to the configuration
2478: * requested by the user, but which may or may not be rejected.
2479: */
2480: struct netdev_config *cfg_pending;
2481: struct ethtool_netdev_state *ethtool;
2482:
2483: /* protected by rtnl_lock */
2484: struct bpf_xdp_entity xdp_state[__MAX_XDP_MODE];
2485:
2486: u8 dev_addr_shadow[MAX_ADDR_LEN];
2487: netdevice_tracker linkwatch_dev_tracker;
2488: netdevice_tracker watchdog_dev_tracker;
2489: netdevice_tracker dev_registered_tracker;
2490: struct rtnl_hw_stats64 *offload_xstats_l3;
2491:
2492: struct devlink_port *devlink_port;
2493:
2494: #if IS_ENABLED(CONFIG_DPLL)
2495: struct dpll_pin __rcu *dpll_pin;
2496: #endif
2497: #if IS_ENABLED(CONFIG_PAGE_POOL)
2498: /** @page_pools: page pools created for this netdevice */
2499: struct hlist_head page_pools;
2500: #endif
2501:
2502: /** @irq_moder: dim parameters used if IS_ENABLED(CONFIG_DIMLIB). */
2503: struct dim_irq_moder *irq_moder;
2504:
2505: u64 max_pacing_offload_horizon;
2506: struct napi_config *napi_config;
2507: u32 num_napi_configs;
2508: u32 napi_defer_hard_irqs;
2509: unsigned long gro_flush_timeout;
2510:
2511: /**
2512: * @up: copy of @state's IFF_UP, but safe to read with just @lock.
2513: * May report false negatives while the device is being opened
2514: * or closed (@lock does not protect .ndo_open, or .ndo_close).
2515: */
2516: bool up;
2517:
2518: /**
2519: * @request_ops_lock: request the core to run all @netdev_ops and
2520: * @ethtool_ops under the @lock.
2521: */
2522: bool request_ops_lock;
2523:
2524: /**
2525: * @lock: netdev-scope lock, protects a small selection of fields.
2526: * Should always be taken using netdev_lock() / netdev_unlock() helpers.
2527: * Drivers are free to use it for other protection.
2528: *
2529: * For the drivers that implement shaper or queue API, the scope
2530: * of this lock is expanded to cover most ndo/queue/ethtool/sysfs
2531: * operations. Drivers may opt-in to this behavior by setting
2532: * @request_ops_lock.
2533: *
2534: * @lock protection mixes with rtnl_lock in multiple ways, fields are
2535: * either:
2536: *
2537: * - simply protected by the instance @lock;
2538: *
2539: * - double protected - writers hold both locks, readers hold either;
2540: *
2541: * - ops protected - protected by the lock held around the NDOs
2542: * and other callbacks, that is the instance lock on devices for
2543: * which netdev_need_ops_lock() returns true, otherwise by rtnl_lock;
2544: *
2545: * - double ops protected - always protected by rtnl_lock but for
2546: * devices for which netdev_need_ops_lock() returns true - also
2547: * the instance lock.
2548: *
2549: * Simply protects:
2550: * @gro_flush_timeout, @napi_defer_hard_irqs, @napi_list,
2551: * @net_shaper_hierarchy, @reg_state, @threaded
2552: *
2553: * Double protects:
2554: * @up, @moving_ns, @nd_net, @xdp_features
2555: *
2556: * Double ops protects:
2557: * @real_num_rx_queues, @real_num_tx_queues
2558: *
2559: * Also protects some fields in:
2560: * struct napi_struct, struct netdev_queue, struct netdev_rx_queue
2561: *
2562: * Ordering: take after rtnl_lock.
2563: */
2564: struct mutex lock;
2565:
2566: #if IS_ENABLED(CONFIG_NET_SHAPER)
2567: /**
2568: * @net_shaper_hierarchy: data tracking the current shaper status
2569: * see include/net/net_shapers.h
2570: */
2571: struct net_shaper_hierarchy *net_shaper_hierarchy;
2572: #endif
2573:
2574: struct hlist_head neighbours[NEIGH_NR_TABLES];
2575:
2576: struct hwtstamp_provider __rcu *hwprov;
2577:
2578: u8 priv[] ____cacheline_aligned
2579: __counted_by(priv_len);
2580: } ____cacheline_aligned;
SourceFile:include/linux/netdevice.h]
|
| 105/2 |
2026/05/21 11:43 |
llm |
expert |
0m
Model:
Tokens:
input: 81970
output: 44
thoughts: 2008
Thoughts: **Analyzing the Watchdog Timer: A Deep Dive**
Okay, so the `watchdog_timer` is a `timer_list`, and the `timer_pending` function tells us if it's currently queued. If that warning triggers, it's because the timer is unexpectedly queued. My first thought is: why would it be queued if the interface isn't up? The `netdev_watchdog_up` function is responsible for arming it, but `netdev_watchdog_down` is supposed to disarm it. The key here is that if the device isn't up (IFF_UP is FALSE), `netif_close_many` and therefore `dev_deactivate_many`, isn't called, so `netdev_watchdog_down` isn't called.
I recall that if `__dev_open` is called (during the interface bringup), it sets the `__LINK_STATE_START` bit. It then calls the device's `ndo_open` method. If `ndo_open` fails, the `__LINK_STATE_START` bit is cleared, and `IFF_UP` is never set. Even though `netif_running(dev)` is TRUE during `ndo_open`, `netdev_watchdog_up` might be called through `netif_carrier_on`. However, `netdev_watchdog_up` checks `!qdisc_tx_is_noop(dev)`. Since the qdisc isn't set until `dev_activate` which is called AFTER `ndo_open` is successful, the watchdog timer will never be armed during that process.
Now, if the interface *is* brought down, the process is: `__dev_close` calls `__dev_close_many`, which calls `dev_deactivate_many`. `dev_deactivate_many` sets the qdisc to `noop_qdisc` and calls `netdev_watchdog_down`, *then* `IFF_UP` is cleared. So, bringing the interface up and down should disarm it correctly.
But wait, what about the sequence? Is there a case where `netdev_watchdog_up` might be called AFTER `dev_activate` but BEFORE `IFF_UP` is set? `netif_set_up(dev, true)` is called before `dev_activate(dev)`, and `netif_set_up` sets `IFF_UP`. Therefore `IFF_UP` is set *before* the qdisc could possibly be changed from `noop_qdisc`.
Okay, back to the bring-down process: In `__dev_close_many`, the code calls `netif_set_up(dev, false)`. This clears `IFF_UP` and sets the qdisc to `noop_qdisc` *after* calling `dev_deactivate_many`. This means that `noop_qdisc` is ALWAYS set before `IFF_UP` is cleared during a normal bring-down.
What if the device is unregistered via `unregister_netdevice_many_notify`?
If `IFF_UP` is true, `netif_close_many` calls `__dev_close_many`, which calls `dev_deactivate_many`. This is the same logic as the bring-down, disarming the timer, THEN it clears `IFF_UP`.
Then the code calls `dev_shutdown` and checks the timer. If `netdev_watchdog_down` has been called, the timer *should* be deleted!
Could `netdev_watchdog_up` be called *after* `netdev_watchdog_down` but *before* `dev_shutdown`? The check in `netdev_watchdog_up` relies on `!qdisc_tx_is_noop(dev)`. However, `dev_deactivate_many` *does* set `noop_qdisc`, so `netdev_watchdog_up` will not arm the timer.
And the question is: How could `qdisc_tx_is_noop(dev)` be FALSE? It checks if *all* tx queues have `noop_qdisc`. `dev_deactivate_many` calls `dev_deactivate_queue` for each queue. The key is in `dev_deactivate_queue`: If `qdisc` is NULL in this routine, then the routine will NOT set `noop_qdisc`! That seems impossible, since the qdisc is initialized to `&noop_qdisc` when a device queue is initialized.
|
| 106/2 |
2026/05/21 11:43 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:netdev_init_one_queue]
Results: map[SourceCode:11160: static void netdev_init_one_queue(struct net_device *dev,
11161: struct netdev_queue *queue, void *_unused)
11162: {
11163: /* Initialize queue lock */
11164: spin_lock_init(&queue->_xmit_lock);
11165: netdev_set_xmit_lockdep_class(&queue->_xmit_lock, dev->type);
11166: queue->xmit_lock_owner = -1;
11167: netdev_queue_numa_node_write(queue, NUMA_NO_NODE);
11168: queue->dev = dev;
11169: #ifdef CONFIG_BQL
11170: dql_init(&queue->dql, HZ);
11171: #endif
11172: }
SourceFile:net/core/dev.c]
|
| 107/2 |
2026/05/21 11:43 |
llm |
expert |
0m
Model:
Tokens:
input: 85281
output: 41
thoughts: 33
Thoughts: Okay, here's my interpretation of those thoughts, crafted with the perspective of someone deeply familiar with networking internals:
**Investigating Queue Discipline Initialization**
Hmm, hold on a second. I was under the impression that `netdev_init_one_queue` handled setting the default `qdisc` to `noop_qdisc`. But looking back at the code, it's clear that it **doesn't** explicitly do that. This changes things. I need to figure out *where* that initial `qdisc` assignment is actually happening. It's a crucial initialization step, so it's essential that I understand the call chain and the specific logic responsible. I'll need to trace the initialization sequence more closely now. This is a potential oversight on my part; I had assumed that function did more than it actually did. I need to find the correct call that sets it.
|
| 108/2 |
2026/05/21 11:43 |
tool |
codesearch-find-references |
0m
Args: map[ContextFile:net/core/dev.c IncludeSnippetLines:0 Name:noop_qdisc]
Results: map[References:<nil> TruncatedOutput:false]
|
| 109/2 |
2026/05/21 11:43 |
llm |
expert |
0m
Model:
Tokens:
input: 85558
output: 18
thoughts: 11
Thoughts: Okay, here's my interpretation of that thought process, tailored for an expert audience, and written in the first person:
**Initial Diagnostic: Hunting Down a Suspect**
Alright, I'm thinking, "Okay, we've got a networking performance issue, or at least a hunch that something isn't quite right." My immediate reaction is to delve into the qdisc configuration, because that's often where you find bottlenecks or unexpected behavior. So, the brain kicks into gear and I'm mentally parsing the possibilities. Given the context (implied, of course, but that's how my mind works!), my thought process jumps to identifying anything related to `noop_qdisc`. It’s a pretty basic starting point, but hey, you have to establish a baseline. It implies a quick diagnostic check to see if the current qdisc in use is the one I expect, or something unexpected. A quick grep is always a good starting point. I can then develop a plan from this first assessment.
|
| 110/2 |
2026/05/21 11:43 |
tool |
grepper |
0m
Args: map[Expression:noop_qdisc]
Results: map[Output:drivers/net/ethernet/netronome/nfp/abm/qdisc.c=698=nfp_abm_red_replace(struct net_device *netdev, struct nfp_abm_link *alink,
--
drivers/net/ethernet/netronome/nfp/abm/qdisc.c-714- } else {
drivers/net/ethernet/netronome/nfp/abm/qdisc.c:715: /* Qdisc was just allocated without a limit will use noop_qdisc,
drivers/net/ethernet/netronome/nfp/abm/qdisc.c-716- * i.e. a block hole.
--
include/net/sch_generic.h=605=static inline void sch_tree_unlock(struct Qdisc *q)
--
include/net/sch_generic.h-612-
include/net/sch_generic.h:613:extern struct Qdisc noop_qdisc;
include/net/sch_generic.h:614:extern struct Qdisc_ops noop_qdisc_ops;
include/net/sch_generic.h-615-extern struct Qdisc_ops pfifo_fast_ops;
--
include/net/sch_generic.h=835=static inline bool qdisc_tx_is_noop(const struct net_device *dev)
--
include/net/sch_generic.h-840- struct netdev_queue *txq = netdev_get_tx_queue(dev, i);
include/net/sch_generic.h:841: if (rcu_access_pointer(txq->qdisc) != &noop_qdisc)
include/net/sch_generic.h-842- return false;
--
net/core/dev.c=11915=struct netdev_queue *dev_ingress_queue_create(struct net_device *dev)
--
net/core/dev.c-11925- netdev_init_one_queue(dev, queue, NULL);
net/core/dev.c:11926: RCU_INIT_POINTER(queue->qdisc, &noop_qdisc);
net/core/dev.c:11927: RCU_INIT_POINTER(queue->qdisc_sleeping, &noop_qdisc);
net/core/dev.c-11928- rcu_assign_pointer(dev->ingress_queue, queue);
--
net/sched/cls_api.c=1180=static int __tcf_qdisc_find(struct net *net, struct Qdisc **q,
--
net/sched/cls_api.c-1234-errout_rcu:
net/sched/cls_api.c:1235: /* At this point we know that qdisc is not noop_qdisc,
net/sched/cls_api.c-1236- * which means that qdisc holds a reference to net_device
--
net/sched/sch_api.c=133=int register_qdisc(struct Qdisc_ops *qops)
--
net/sched/sch_api.c-143- if (qops->enqueue == NULL)
net/sched/sch_api.c:144: qops->enqueue = noop_qdisc_ops.enqueue;
net/sched/sch_api.c-145- if (qops->peek == NULL) {
net/sched/sch_api.c-146- if (qops->dequeue == NULL)
net/sched/sch_api.c:147: qops->peek = noop_qdisc_ops.peek;
net/sched/sch_api.c-148- else
--
net/sched/sch_api.c-151- if (qops->dequeue == NULL)
net/sched/sch_api.c:152: qops->dequeue = noop_qdisc_ops.dequeue;
net/sched/sch_api.c-153-
--
net/sched/sch_api.c=837=void qdisc_offload_graft_helper(struct net_device *dev, struct Qdisc *sch,
--
net/sched/sch_api.c-850- /* Don't report error if the graft is part of destroy operation. */
net/sched/sch_api.c:851: if (!err || !new || new == &noop_qdisc)
net/sched/sch_api.c-852- return;
--
net/sched/sch_api.c=1087=static int qdisc_graft(struct net_device *dev, struct Qdisc *parent,
--
net/sched/sch_api.c-1156- qdisc_refcount_inc(new);
net/sched/sch_api.c:1157: rcu_assign_pointer(dev->qdisc, new ? : &noop_qdisc);
net/sched/sch_api.c-1158-
--
net/sched/sch_cbs.c=492=static int cbs_graft(struct Qdisc *sch, unsigned long arg, struct Qdisc *new,
--
net/sched/sch_cbs.c-500- if (!new)
net/sched/sch_cbs.c:501: new = &noop_qdisc;
net/sched/sch_cbs.c-502- }
--
net/sched/sch_drr.c=58=static int drr_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
--
net/sched/sch_drr.c-119- if (cl->qdisc == NULL)
net/sched/sch_drr.c:120: cl->qdisc = &noop_qdisc;
net/sched/sch_drr.c-121- else
--
net/sched/sch_drr.c=210=static int drr_graft_class(struct Qdisc *sch, unsigned long arg,
--
net/sched/sch_drr.c-219- if (new == NULL)
net/sched/sch_drr.c:220: new = &noop_qdisc;
net/sched/sch_drr.c-221- }
--
net/sched/sch_ets.c=254=static int ets_class_graft(struct Qdisc *sch, unsigned long arg,
--
net/sched/sch_ets.c-263- if (!new)
net/sched/sch_ets.c:264: new = &noop_qdisc;
net/sched/sch_ets.c-265- else
--
net/sched/sch_ets.c=581=static int ets_qdisc_change(struct Qdisc *sch, struct nlattr *opt,
--
net/sched/sch_ets.c-674- q->classes[i].qdisc = queues[i];
net/sched/sch_ets.c:675: if (q->classes[i].qdisc != &noop_qdisc)
net/sched/sch_ets.c-676- qdisc_hash_add(q->classes[i].qdisc, true);
--
net/sched/sch_generic.c=646=static struct sk_buff *noop_dequeue(struct Qdisc *qdisc)
--
net/sched/sch_generic.c-650-
net/sched/sch_generic.c:651:struct Qdisc_ops noop_qdisc_ops __read_mostly = {
net/sched/sch_generic.c-652- .id = "noop",
--
net/sched/sch_generic.c=660=static struct netdev_queue noop_netdev_queue = {
net/sched/sch_generic.c:661: RCU_POINTER_INITIALIZER(qdisc, &noop_qdisc),
net/sched/sch_generic.c:662: RCU_POINTER_INITIALIZER(qdisc_sleeping, &noop_qdisc),
net/sched/sch_generic.c-663-};
net/sched/sch_generic.c-664-
net/sched/sch_generic.c:665:struct Qdisc noop_qdisc = {
net/sched/sch_generic.c-666- .enqueue = noop_enqueue,
--
net/sched/sch_generic.c-668- .flags = TCQ_F_BUILTIN,
net/sched/sch_generic.c:669: .ops = &noop_qdisc_ops,
net/sched/sch_generic.c:670: .q.lock = __SPIN_LOCK_UNLOCKED(noop_qdisc.q.lock),
net/sched/sch_generic.c-671- .dev_queue = &noop_netdev_queue,
net/sched/sch_generic.c-672- .gso_skb = {
net/sched/sch_generic.c:673: .next = (struct sk_buff *)&noop_qdisc.gso_skb,
net/sched/sch_generic.c:674: .prev = (struct sk_buff *)&noop_qdisc.gso_skb,
net/sched/sch_generic.c-675- .qlen = 0,
net/sched/sch_generic.c:676: .lock = __SPIN_LOCK_UNLOCKED(noop_qdisc.gso_skb.lock),
net/sched/sch_generic.c-677- },
net/sched/sch_generic.c-678- .skb_bad_txq = {
net/sched/sch_generic.c:679: .next = (struct sk_buff *)&noop_qdisc.skb_bad_txq,
net/sched/sch_generic.c:680: .prev = (struct sk_buff *)&noop_qdisc.skb_bad_txq,
net/sched/sch_generic.c-681- .qlen = 0,
net/sched/sch_generic.c:682: .lock = __SPIN_LOCK_UNLOCKED(noop_qdisc.skb_bad_txq.lock),
net/sched/sch_generic.c-683- },
net/sched/sch_generic.c-684-};
net/sched/sch_generic.c:685:EXPORT_SYMBOL(noop_qdisc);
net/sched/sch_generic.c-686-
--
net/sched/sch_generic.c=1126=struct Qdisc *dev_graft_qdisc(struct netdev_queue *dev_queue,
--
net/sched/sch_generic.c-1136- if (qdisc == NULL)
net/sched/sch_generic.c:1137: qdisc = &noop_qdisc;
net/sched/sch_generic.c-1138- rcu_assign_pointer(dev_queue->qdisc_sleeping, qdisc);
net/sched/sch_generic.c:1139: rcu_assign_pointer(dev_queue->qdisc, &noop_qdisc);
net/sched/sch_generic.c-1140-
--
net/sched/sch_generic.c=1183=static void attach_default_qdiscs(struct net_device *dev)
--
net/sched/sch_generic.c-1205- /* Detect default qdisc setup/init failed and fallback to "noqueue" */
net/sched/sch_generic.c:1206: if (qdisc == &noop_qdisc) {
net/sched/sch_generic.c-1207- netdev_warn(dev, "default qdisc (%s) fail, fallback to %s\n",
net/sched/sch_generic.c-1208- default_qdisc_ops->id, noqueue_qdisc_ops.id);
net/sched/sch_generic.c:1209: netdev_for_each_tx_queue(dev, shutdown_scheduler_queue, &noop_qdisc);
net/sched/sch_generic.c-1210- dev->priv_flags |= IFF_NO_QUEUE;
--
net/sched/sch_generic.c-1218-#ifdef CONFIG_NET_SCHED
net/sched/sch_generic.c:1219: if (qdisc != &noop_qdisc)
net/sched/sch_generic.c-1220- qdisc_hash_add(qdisc, false);
--
net/sched/sch_generic.c=1241=void dev_activate(struct net_device *dev)
--
net/sched/sch_generic.c-1249-
net/sched/sch_generic.c:1250: if (rtnl_dereference(dev->qdisc) == &noop_qdisc)
net/sched/sch_generic.c-1251- attach_default_qdiscs(dev);
--
net/sched/sch_generic.c=1277=static void dev_deactivate_queue(struct net_device *dev,
--
net/sched/sch_generic.c-1288- qdisc_deactivate(qdisc);
net/sched/sch_generic.c:1289: rcu_assign_pointer(dev_queue->qdisc, &noop_qdisc);
net/sched/sch_generic.c-1290- }
--
net/sched/sch_generic.c=1422=void mq_change_real_num_tx(struct Qdisc *sch, unsigned int new_real_tx)
--
net/sched/sch_generic.c-1433- */
net/sched/sch_generic.c:1434: if (qdisc != &noop_qdisc && !qdisc->handle)
net/sched/sch_generic.c-1435- qdisc_hash_del(qdisc);
--
net/sched/sch_generic.c-1438- qdisc = rtnl_dereference(netdev_get_tx_queue(dev, i)->qdisc_sleeping);
net/sched/sch_generic.c:1439: if (qdisc != &noop_qdisc && !qdisc->handle)
net/sched/sch_generic.c-1440- qdisc_hash_add(qdisc, false);
--
net/sched/sch_generic.c=1478=void dev_init_scheduler(struct net_device *dev)
net/sched/sch_generic.c-1479-{
net/sched/sch_generic.c:1480: rcu_assign_pointer(dev->qdisc, &noop_qdisc);
net/sched/sch_generic.c:1481: netdev_for_each_tx_queue(dev, dev_init_scheduler_queue, &noop_qdisc);
net/sched/sch_generic.c-1482- if (dev_ingress_queue(dev))
net/sched/sch_generic.c:1483: dev_init_scheduler_queue(dev, dev_ingress_queue(dev), &noop_qdisc);
net/sched/sch_generic.c-1484-
--
net/sched/sch_generic.c=1488=void dev_shutdown(struct net_device *dev)
net/sched/sch_generic.c-1489-{
net/sched/sch_generic.c:1490: netdev_for_each_tx_queue(dev, shutdown_scheduler_queue, &noop_qdisc);
net/sched/sch_generic.c-1491- if (dev_ingress_queue(dev))
net/sched/sch_generic.c:1492: shutdown_scheduler_queue(dev, dev_ingress_queue(dev), &noop_qdisc);
net/sched/sch_generic.c-1493- qdisc_put(rtnl_dereference(dev->qdisc));
net/sched/sch_generic.c:1494: rcu_assign_pointer(dev->qdisc, &noop_qdisc);
net/sched/sch_generic.c-1495-
--
net/sched/sch_hfsc.c=912=hfsc_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
--
net/sched/sch_hfsc.c-1062- if (cl->qdisc == NULL)
net/sched/sch_hfsc.c:1063: cl->qdisc = &noop_qdisc;
net/sched/sch_hfsc.c-1064- else
--
net/sched/sch_hfsc.c=1185=hfsc_graft_class(struct Qdisc *sch, unsigned long arg, struct Qdisc *new,
--
net/sched/sch_hfsc.c-1195- if (new == NULL)
net/sched/sch_hfsc.c:1196: new = &noop_qdisc;
net/sched/sch_hfsc.c-1197- }
--
net/sched/sch_hfsc.c=1394=hfsc_init_qdisc(struct Qdisc *sch, struct nlattr *opt,
--
net/sched/sch_hfsc.c-1422- if (q->root.qdisc == NULL)
net/sched/sch_hfsc.c:1423: q->root.qdisc = &noop_qdisc;
net/sched/sch_hfsc.c-1424- else
--
net/sched/sch_htb.c=1505=static void htb_parent_to_leaf(struct Qdisc *sch, struct htb_class *cl,
--
net/sched/sch_htb.c-1518- memset(&parent->inner, 0, sizeof(parent->inner));
net/sched/sch_htb.c:1519: parent->leaf.q = new_q ? new_q : &noop_qdisc;
net/sched/sch_htb.c-1520- parent->tokens = parent->buffer;
--
net/sched/sch_htb.c=1759=static int htb_change_class(struct Qdisc *sch, u32 classid,
--
net/sched/sch_htb.c-1964- /* leaf (we) needs elementary qdisc */
net/sched/sch_htb.c:1965: cl->leaf.q = new_q ? new_q : &noop_qdisc;
net/sched/sch_htb.c-1966- if (q->offload)
--
net/sched/sch_htb.c-1981- parent->children++;
net/sched/sch_htb.c:1982: if (cl->leaf.q != &noop_qdisc)
net/sched/sch_htb.c-1983- qdisc_hash_add(cl->leaf.q, true);
--
net/sched/sch_multiq.c=171=static int multiq_tune(struct Qdisc *sch, struct nlattr *opt,
--
net/sched/sch_multiq.c-195- for (i = q->bands; i < q->max_bands; i++) {
net/sched/sch_multiq.c:196: if (q->queues[i] != &noop_qdisc) {
net/sched/sch_multiq.c-197- struct Qdisc *child = q->queues[i];
net/sched/sch_multiq.c-198-
net/sched/sch_multiq.c:199: q->queues[i] = &noop_qdisc;
net/sched/sch_multiq.c-200- qdisc_purge_queue(child);
--
net/sched/sch_multiq.c-211- for (i = 0; i < q->bands; i++) {
net/sched/sch_multiq.c:212: if (q->queues[i] == &noop_qdisc) {
net/sched/sch_multiq.c-213- struct Qdisc *child, *old;
--
net/sched/sch_multiq.c-221- q->queues[i] = child;
net/sched/sch_multiq.c:222: if (child != &noop_qdisc)
net/sched/sch_multiq.c-223- qdisc_hash_add(child, true);
net/sched/sch_multiq.c-224-
net/sched/sch_multiq.c:225: if (old != &noop_qdisc)
net/sched/sch_multiq.c-226- qdisc_purge_queue(old);
--
net/sched/sch_multiq.c=235=static int multiq_init(struct Qdisc *sch, struct nlattr *opt,
--
net/sched/sch_multiq.c-255- for (i = 0; i < q->max_bands; i++)
net/sched/sch_multiq.c:256: q->queues[i] = &noop_qdisc;
net/sched/sch_multiq.c-257-
--
net/sched/sch_multiq.c=280=static int multiq_graft(struct Qdisc *sch, unsigned long arg, struct Qdisc *new,
--
net/sched/sch_multiq.c-286- if (new == NULL)
net/sched/sch_multiq.c:287: new = &noop_qdisc;
net/sched/sch_multiq.c-288-
--
net/sched/sch_prio.c=176=static int prio_tune(struct Qdisc *sch, struct nlattr *opt,
--
net/sched/sch_prio.c-217- q->queues[i] = queues[i];
net/sched/sch_prio.c:218: if (q->queues[i] != &noop_qdisc)
net/sched/sch_prio.c-219- qdisc_hash_add(q->queues[i], true);
--
net/sched/sch_prio.c=286=static int prio_graft(struct Qdisc *sch, unsigned long arg, struct Qdisc *new,
--
net/sched/sch_prio.c-296- if (!new)
net/sched/sch_prio.c:297: new = &noop_qdisc;
net/sched/sch_prio.c-298- else
--
net/sched/sch_qfq.c=406=static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
--
net/sched/sch_qfq.c-490- if (cl->qdisc == NULL)
net/sched/sch_qfq.c:491: cl->qdisc = &noop_qdisc;
net/sched/sch_qfq.c-492-
--
net/sched/sch_qfq.c-502-
net/sched/sch_qfq.c:503: if (cl->qdisc != &noop_qdisc)
net/sched/sch_qfq.c-504- qdisc_hash_add(cl->qdisc, true);
--
net/sched/sch_qfq.c=601=static int qfq_graft_class(struct Qdisc *sch, unsigned long arg,
--
net/sched/sch_qfq.c-610- if (new == NULL)
net/sched/sch_qfq.c:611: new = &noop_qdisc;
net/sched/sch_qfq.c-612- }
--
net/sched/sch_red.c=236=static int __red_change(struct Qdisc *sch, struct nlattr **tb,
--
net/sched/sch_red.c-272-
net/sched/sch_red.c:273: /* child is fifo, no need to check for noop_qdisc */
net/sched/sch_red.c-274- qdisc_hash_add(child, true);
--
net/sched/sch_red.c=337=static int red_init(struct Qdisc *sch, struct nlattr *opt,
--
net/sched/sch_red.c-343-
net/sched/sch_red.c:344: q->qdisc = &noop_qdisc;
net/sched/sch_red.c-345- q->sch = sch;
--
net/sched/sch_red.c=498=static int red_graft(struct Qdisc *sch, unsigned long arg, struct Qdisc *new,
--
net/sched/sch_red.c-503- if (new == NULL)
net/sched/sch_red.c:504: new = &noop_qdisc;
net/sched/sch_red.c-505-
--
net/sched/sch_sfb.c=492=static int sfb_change(struct Qdisc *sch, struct nlattr *opt,
--
net/sched/sch_sfb.c-521-
net/sched/sch_sfb.c:522: if (child != &noop_qdisc)
net/sched/sch_sfb.c-523- qdisc_hash_add(child, true);
--
net/sched/sch_sfb.c=555=static int sfb_init(struct Qdisc *sch, struct nlattr *opt,
--
net/sched/sch_sfb.c-564-
net/sched/sch_sfb.c:565: q->qdisc = &noop_qdisc;
net/sched/sch_sfb.c-566- return sfb_change(sch, opt, extack);
--
net/sched/sch_sfb.c=621=static int sfb_graft(struct Qdisc *sch, unsigned long arg, struct Qdisc *new,
--
net/sched/sch_sfb.c-626- if (new == NULL)
net/sched/sch_sfb.c:627: new = &noop_qdisc;
net/sched/sch_sfb.c-628-
--
net/sched/sch_tbf.c=357=static int tbf_change(struct Qdisc *sch, struct nlattr *opt,
--
net/sched/sch_tbf.c-437-
net/sched/sch_tbf.c:438: if (q->qdisc != &noop_qdisc) {
net/sched/sch_tbf.c-439- err = fifo_set_limit(q->qdisc, qopt->limit);
--
net/sched/sch_tbf.c-449-
net/sched/sch_tbf.c:450: /* child is fifo, no need to check for noop_qdisc */
net/sched/sch_tbf.c-451- qdisc_hash_add(child, true);
--
net/sched/sch_tbf.c=485=static int tbf_init(struct Qdisc *sch, struct nlattr *opt,
--
net/sched/sch_tbf.c-490- qdisc_watchdog_init(&q->watchdog, sch);
net/sched/sch_tbf.c:491: q->qdisc = &noop_qdisc;
net/sched/sch_tbf.c-492-
--
net/sched/sch_tbf.c=563=static int tbf_graft(struct Qdisc *sch, unsigned long arg, struct Qdisc *new,
--
net/sched/sch_tbf.c-568- if (new == NULL)
net/sched/sch_tbf.c:569: new = &noop_qdisc;
net/sched/sch_tbf.c-570-
--
net/sched/sch_teql.c=256=static inline int teql_resolve(struct sk_buff *skb,
--
net/sched/sch_teql.c-263-
net/sched/sch_teql.c:264: if (rcu_access_pointer(txq->qdisc) == &noop_qdisc)
net/sched/sch_teql.c-265- return -ENODEV;
]
|
| 111/2 |
2026/05/21 11:43 |
llm |
expert |
0m
Model:
Tokens:
input: 93012
output: 43
thoughts: 87
Thoughts: **Investigating Queue Discipline Configuration**
Okay, so I'm digging into this network device initialization. I noticed `dev_ingress_queue_create` specifically sets the `noop_qdisc`. That's straightforward enough for ingress. But what about the *transmit* side? That's where things get interesting. I know `alloc_netdev_mqs` is responsible for initializing those tx queues. It calls `netdev_init_one_queue` for each one, which seems like the right place to look. The question then becomes: does `alloc_netdev_mqs` *also* set `noop_qdisc` for the tx queues, or is something else handling it? Hmm, I'll need to check the `alloc_netdev_mqs` source code directly to confirm. Better get reading.
|
| 112/2 |
2026/05/21 11:43 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:alloc_netdev_mqs]
Results: map[SourceCode:11961: /**
11962: * alloc_netdev_mqs - allocate network device
11963: * @sizeof_priv: size of private data to allocate space for
11964: * @name: device name format string
11965: * @name_assign_type: origin of device name
11966: * @setup: callback to initialize device
11967: * @txqs: the number of TX subqueues to allocate
11968: * @rxqs: the number of RX subqueues to allocate
11969: *
11970: * Allocates a struct net_device with private data area for driver use
11971: * and performs basic initialization. Also allocates subqueue structs
11972: * for each queue on the device.
11973: */
11974: struct net_device *alloc_netdev_mqs(int sizeof_priv, const char *name,
11975: unsigned char name_assign_type,
11976: void (*setup)(struct net_device *),
11977: unsigned int txqs, unsigned int rxqs)
11978: {
11979: struct net_device *dev;
11980: size_t napi_config_sz;
11981: unsigned int maxqs;
11982:
11983: BUG_ON(strlen(name) >= sizeof(dev->name));
11984:
11985: if (txqs < 1) {
11986: pr_err("alloc_netdev: Unable to allocate device with zero queues\n");
11987: return NULL;
11988: }
11989:
11990: if (rxqs < 1) {
11991: pr_err("alloc_netdev: Unable to allocate device with zero RX queues\n");
11992: return NULL;
11993: }
11994:
11995: maxqs = max(txqs, rxqs);
11996:
11997: dev = kvzalloc(struct_size(dev, priv, sizeof_priv),
11998: GFP_KERNEL_ACCOUNT | __GFP_RETRY_MAYFAIL);
11999: if (!dev)
12000: return NULL;
12001:
12002: dev->priv_len = sizeof_priv;
12003:
12004: ref_tracker_dir_init(&dev->refcnt_tracker, 128, "netdev");
12005: #ifdef CONFIG_PCPU_DEV_REFCNT
12006: dev->pcpu_refcnt = alloc_percpu(int);
12007: if (!dev->pcpu_refcnt)
12008: goto free_dev;
12009: __dev_hold(dev);
12010: #else
12011: refcount_set(&dev->dev_refcnt, 1);
12012: #endif
12013:
12014: if (dev_addr_init(dev))
12015: goto free_pcpu;
12016:
12017: dev_mc_init(dev);
12018: dev_uc_init(dev);
12019:
12020: dev_net_set(dev, &init_net);
12021:
12022: dev->gso_max_size = GSO_LEGACY_MAX_SIZE;
12023: dev->xdp_zc_max_segs = 1;
12024: dev->gso_max_segs = GSO_MAX_SEGS;
12025: dev->gro_max_size = GRO_LEGACY_MAX_SIZE;
12026: dev->gso_ipv4_max_size = GSO_LEGACY_MAX_SIZE;
12027: dev->gro_ipv4_max_size = GRO_LEGACY_MAX_SIZE;
12028: dev->tso_max_size = TSO_LEGACY_MAX_SIZE;
12029: dev->tso_max_segs = TSO_MAX_SEGS;
12030: dev->upper_level = 1;
12031: dev->lower_level = 1;
12032: #ifdef CONFIG_LOCKDEP
12033: dev->nested_level = 0;
12034: INIT_LIST_HEAD(&dev->unlink_list);
12035: #endif
12036:
12037: INIT_LIST_HEAD(&dev->napi_list);
12038: INIT_LIST_HEAD(&dev->unreg_list);
12039: INIT_LIST_HEAD(&dev->close_list);
12040: INIT_LIST_HEAD(&dev->link_watch_list);
12041: INIT_LIST_HEAD(&dev->adj_list.upper);
12042: INIT_LIST_HEAD(&dev->adj_list.lower);
12043: INIT_LIST_HEAD(&dev->ptype_all);
12044: INIT_LIST_HEAD(&dev->ptype_specific);
12045: INIT_LIST_HEAD(&dev->net_notifier_list);
12046: #ifdef CONFIG_NET_SCHED
12047: hash_init(dev->qdisc_hash);
12048: #endif
12049:
12050: mutex_init(&dev->lock);
12051:
12052: dev->priv_flags = IFF_XMIT_DST_RELEASE | IFF_XMIT_DST_RELEASE_PERM;
12053: setup(dev);
12054:
12055: if (!dev->tx_queue_len) {
12056: dev->priv_flags |= IFF_NO_QUEUE;
12057: dev->tx_queue_len = DEFAULT_TX_QUEUE_LEN;
12058: }
12059:
12060: dev->num_tx_queues = txqs;
12061: dev->real_num_tx_queues = txqs;
12062: if (netif_alloc_netdev_queues(dev))
12063: goto free_all;
12064:
12065: dev->num_rx_queues = rxqs;
12066: dev->real_num_rx_queues = rxqs;
12067: if (netif_alloc_rx_queues(dev))
12068: goto free_all;
12069: dev->ethtool = kzalloc(sizeof(*dev->ethtool), GFP_KERNEL_ACCOUNT);
12070: if (!dev->ethtool)
12071: goto free_all;
12072:
12073: dev->cfg = kzalloc(sizeof(*dev->cfg), GFP_KERNEL_ACCOUNT);
12074: if (!dev->cfg)
12075: goto free_all;
12076: dev->cfg_pending = dev->cfg;
12077:
12078: dev->num_napi_configs = maxqs;
12079: napi_config_sz = array_size(maxqs, sizeof(*dev->napi_config));
12080: dev->napi_config = kvzalloc(napi_config_sz, GFP_KERNEL_ACCOUNT);
12081: if (!dev->napi_config)
12082: goto free_all;
12083:
12084: strscpy(dev->name, name);
12085: dev->name_assign_type = name_assign_type;
12086: dev->group = INIT_NETDEV_GROUP;
12087: if (!dev->ethtool_ops)
12088: dev->ethtool_ops = &default_ethtool_ops;
12089:
12090: nf_hook_netdev_init(dev);
12091:
12092: return dev;
12093:
12094: free_all:
12095: free_netdev(dev);
12096: return NULL;
12097:
12098: free_pcpu:
12099: #ifdef CONFIG_PCPU_DEV_REFCNT
12100: free_percpu(dev->pcpu_refcnt);
12101: free_dev:
12102: #endif
12103: kvfree(dev);
12104: return NULL;
12105: }
SourceFile:net/core/dev.c]
|
| 113/2 |
2026/05/21 11:43 |
llm |
expert |
0m
Model:
Tokens:
input: 95771
output: 46
thoughts: 18
Thoughts: Here's my attempt at summarizing the thought process, tailored for an expert audience:
**Queue Allocation Refresher**
Okay, so I'm digging back into this networking code... Right, the `netif_alloc_netdev_queues` function is the key here. It’s the one actually responsible for allocating the individual transmit and receive queues associated with a network device. So, it's not some implicit process happening elsewhere; this specific function is the gatekeeper for those hardware-facing data structures. This means any issue with queue assignment or configuration fundamentally ties back to that call. I need to keep that in mind as I track the allocation and initialization flow. Time to re-examine how it interacts with the underlying hardware drivers and any potential resource limits.
|
| 114/2 |
2026/05/21 11:44 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:netif_alloc_netdev_queues]
Results: map[SourceCode:11179: static int netif_alloc_netdev_queues(struct net_device *dev)
11180: {
11181: unsigned int count = dev->num_tx_queues;
11182: struct netdev_queue *tx;
11183: size_t sz = count * sizeof(*tx);
11184:
11185: if (count < 1 || count > 0xffff)
11186: return -EINVAL;
11187:
11188: tx = kvzalloc(sz, GFP_KERNEL_ACCOUNT | __GFP_RETRY_MAYFAIL);
11189: if (!tx)
11190: return -ENOMEM;
11191:
11192: dev->_tx = tx;
11193:
11194: netdev_for_each_tx_queue(dev, netdev_init_one_queue, NULL);
11195: spin_lock_init(&dev->tx_global_lock);
11196:
11197: return 0;
11198: }
SourceFile:net/core/dev.c]
|
| 115/2 |
2026/05/21 11:44 |
llm |
expert |
0m
Model:
Tokens:
input: 96337
output: 43
thoughts: 84
Thoughts: Okay, let's break this down.
**Debugging a Network Queue Disc**
Alright, I'm staring at this network code, and it looks like `txq->qdisc` is coming up as NULL. That's a red flag. Now, if `txq->qdisc` is NULL, then `qdisc_tx_is_noop(dev)` should be returning FALSE, right? Because the noop qdisc isn't NULL, it's a specific address - `&noop_qdisc`. So, the check should pass if the qdisc isn't the noop qdisc. Hold on... when exactly *is* `txq->qdisc` being set to `noop_qdisc`? I need to trace this back. Ah, here it is, it's happening during the `dev_init_scheduler` function. Okay, now I know where to look to understand why it is currently NULL. Time to dive deeper into that function.
|
| 116/2 |
2026/05/21 11:44 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/sched/sch_generic.c IncludeLines:true Name:dev_init_scheduler]
Results: map[SourceCode:1478: void dev_init_scheduler(struct net_device *dev)
1479: {
1480: rcu_assign_pointer(dev->qdisc, &noop_qdisc);
1481: netdev_for_each_tx_queue(dev, dev_init_scheduler_queue, &noop_qdisc);
1482: if (dev_ingress_queue(dev))
1483: dev_init_scheduler_queue(dev, dev_ingress_queue(dev), &noop_qdisc);
1484:
1485: timer_setup(&dev->watchdog_timer, dev_watchdog, 0);
1486: }
SourceFile:net/sched/sch_generic.c]
|
| 117/2 |
2026/05/21 11:44 |
llm |
expert |
0m
Model:
Tokens:
input: 96881
output: 40
thoughts: 52
Thoughts: Okay, here's my interpretation of those thoughts, tailored for an expert audience:
**Analyzing Network Device Initialization**
Alright, so `dev_init_scheduler` is clearly the key here. It's initializing the queues with `noop_qdisc`, which is the default for most simple configurations. That's good to know – default, minimal overhead, straightforward. And hey, it's also setting up the `watchdog_timer`, that's an important piece for error detection and recovery at the device level. Keeping an eye out to make sure everything's running smoothly. Now, the real question is, *when* does this vital function get called? Let's trace this... ah, yes, it's embedded in `register_netdevice`! That makes sense, because you would want to establish the basic settings of a device as soon as it's registered with the networking stack. This is fundamental infrastructure. Good design, good design.
|
| 118/2 |
2026/05/21 11:44 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:register_netdevice]
Results: map[SourceCode:11271: /**
11272: * register_netdevice() - register a network device
11273: * @dev: device to register
11274: *
11275: * Take a prepared network device structure and make it externally accessible.
11276: * A %NETDEV_REGISTER message is sent to the netdev notifier chain.
11277: * Callers must hold the rtnl lock - you may want register_netdev()
11278: * instead of this.
11279: */
11280: int register_netdevice(struct net_device *dev)
11281: {
11282: int ret;
11283: struct net *net = dev_net(dev);
11284:
11285: BUILD_BUG_ON(sizeof(netdev_features_t) * BITS_PER_BYTE <
11286: NETDEV_FEATURE_COUNT);
11287: BUG_ON(dev_boot_phase);
11288: ASSERT_RTNL();
11289:
11290: might_sleep();
11291:
11292: /* When net_device's are persistent, this will be fatal. */
11293: BUG_ON(dev->reg_state != NETREG_UNINITIALIZED);
11294: BUG_ON(!net);
11295:
11296: ret = ethtool_check_ops(dev->ethtool_ops);
11297: if (ret)
11298: return ret;
11299:
11300: /* rss ctx ID 0 is reserved for the default context, start from 1 */
11301: xa_init_flags(&dev->ethtool->rss_ctx, XA_FLAGS_ALLOC1);
11302: mutex_init(&dev->ethtool->rss_lock);
11303:
11304: spin_lock_init(&dev->addr_list_lock);
11305: netdev_set_addr_lockdep_class(dev);
11306:
11307: ret = dev_get_valid_name(net, dev, dev->name);
11308: if (ret < 0)
11309: goto out;
11310:
11311: ret = -ENOMEM;
11312: dev->name_node = netdev_name_node_head_alloc(dev);
11313: if (!dev->name_node)
11314: goto out;
11315:
11316: /* Init, if this function is available */
11317: if (dev->netdev_ops->ndo_init) {
11318: ret = dev->netdev_ops->ndo_init(dev);
11319: if (ret) {
11320: if (ret > 0)
11321: ret = -EIO;
11322: goto err_free_name;
11323: }
11324: }
11325:
11326: if (((dev->hw_features | dev->features) &
11327: NETIF_F_HW_VLAN_CTAG_FILTER) &&
11328: (!dev->netdev_ops->ndo_vlan_rx_add_vid ||
11329: !dev->netdev_ops->ndo_vlan_rx_kill_vid)) {
11330: netdev_WARN(dev, "Buggy VLAN acceleration in driver!\n");
11331: ret = -EINVAL;
11332: goto err_uninit;
11333: }
11334:
11335: ret = netdev_do_alloc_pcpu_stats(dev);
11336: if (ret)
11337: goto err_uninit;
11338:
11339: ret = dev_index_reserve(net, dev->ifindex);
11340: if (ret < 0)
11341: goto err_free_pcpu;
11342: dev->ifindex = ret;
11343:
11344: /* Transfer changeable features to wanted_features and enable
11345: * software offloads (GSO and GRO).
11346: */
11347: dev->hw_features |= (NETIF_F_SOFT_FEATURES | NETIF_F_SOFT_FEATURES_OFF);
11348: dev->features |= NETIF_F_SOFT_FEATURES;
11349:
11350: if (dev->udp_tunnel_nic_info) {
11351: dev->features |= NETIF_F_RX_UDP_TUNNEL_PORT;
11352: dev->hw_features |= NETIF_F_RX_UDP_TUNNEL_PORT;
11353: }
11354:
11355: dev->wanted_features = dev->features & dev->hw_features;
11356:
11357: if (!(dev->flags & IFF_LOOPBACK))
11358: dev->hw_features |= NETIF_F_NOCACHE_COPY;
11359:
11360: /* If IPv4 TCP segmentation offload is supported we should also
11361: * allow the device to enable segmenting the frame with the option
11362: * of ignoring a static IP ID value. This doesn't enable the
11363: * feature itself but allows the user to enable it later.
11364: */
11365: if (dev->hw_features & NETIF_F_TSO)
11366: dev->hw_features |= NETIF_F_TSO_MANGLEID;
11367: if (dev->vlan_features & NETIF_F_TSO)
11368: dev->vlan_features |= NETIF_F_TSO_MANGLEID;
11369: if (dev->mpls_features & NETIF_F_TSO)
11370: dev->mpls_features |= NETIF_F_TSO_MANGLEID;
11371: if (dev->hw_enc_features & NETIF_F_TSO)
11372: dev->hw_enc_features |= NETIF_F_TSO_MANGLEID;
11373:
11374: /* Make NETIF_F_HIGHDMA inheritable to VLAN devices.
11375: */
11376: dev->vlan_features |= NETIF_F_HIGHDMA;
11377:
11378: /* Make NETIF_F_SG inheritable to tunnel devices.
11379: */
11380: dev->hw_enc_features |= NETIF_F_SG | NETIF_F_GSO_PARTIAL;
11381:
11382: /* Make NETIF_F_SG inheritable to MPLS.
11383: */
11384: dev->mpls_features |= NETIF_F_SG;
11385:
11386: ret = call_netdevice_notifiers(NETDEV_POST_INIT, dev);
11387: ret = notifier_to_errno(ret);
11388: if (ret)
11389: goto err_ifindex_release;
11390:
11391: ret = netdev_register_kobject(dev);
11392:
11393: netdev_lock(dev);
11394: WRITE_ONCE(dev->reg_state, ret ? NETREG_UNREGISTERED : NETREG_REGISTERED);
11395: netdev_unlock(dev);
11396:
11397: if (ret)
11398: goto err_uninit_notify;
11399:
11400: netdev_lock_ops(dev);
11401: __netdev_update_features(dev);
11402: netdev_unlock_ops(dev);
11403:
11404: /*
11405: * Default initial state at registry is that the
11406: * device is present.
11407: */
11408:
11409: set_bit(__LINK_STATE_PRESENT, &dev->state);
11410:
11411: linkwatch_init_dev(dev);
11412:
11413: dev_init_scheduler(dev);
11414:
11415: netdev_hold(dev, &dev->dev_registered_tracker, GFP_KERNEL);
11416: list_netdevice(dev);
11417:
11418: add_device_randomness(dev->dev_addr, dev->addr_len);
11419:
11420: /* If the device has permanent device address, driver should
11421: * set dev_addr and also addr_assign_type should be set to
11422: * NET_ADDR_PERM (default value).
11423: */
11424: if (dev->addr_assign_type == NET_ADDR_PERM)
11425: memcpy(dev->perm_addr, dev->dev_addr, dev->addr_len);
11426:
11427: /* Notify protocols, that a new device appeared. */
11428: netdev_lock_ops(dev);
11429: ret = call_netdevice_notifiers(NETDEV_REGISTER, dev);
11430: netdev_unlock_ops(dev);
11431: ret = notifier_to_errno(ret);
11432: if (ret) {
11433: /* Expect explicit free_netdev() on failure */
11434: dev->needs_free_netdev = false;
11435: unregister_netdevice_queue(dev, NULL);
11436: goto out;
11437: }
11438: /*
11439: * Prevent userspace races by waiting until the network
11440: * device is fully setup before sending notifications.
11441: */
11442: if (!(dev->rtnl_link_ops && dev->rtnl_link_initializing))
11443: rtmsg_ifinfo(RTM_NEWLINK, dev, ~0U, GFP_KERNEL, 0, NULL);
11444:
11445: out:
11446: return ret;
11447:
11448: err_uninit_notify:
11449: call_netdevice_notifiers(NETDEV_PRE_UNINIT, dev);
11450: err_ifindex_release:
11451: dev_index_release(net, dev->ifindex);
11452: err_free_pcpu:
11453: netdev_do_free_pcpu_stats(dev);
11454: err_uninit:
11455: if (dev->netdev_ops->ndo_uninit)
11456: dev->netdev_ops->ndo_uninit(dev);
11457: if (dev->priv_destructor)
11458: dev->priv_destructor(dev);
11459: err_free_name:
11460: netdev_name_node_free(dev->name_node);
11461: goto out;
11462: }
SourceFile:net/core/dev.c]
|
| 119/2 |
2026/05/21 11:44 |
llm |
expert |
0m
Model:
Tokens:
input: 100415
output: 39
thoughts: 967
Thoughts: **Analyzing the Network Device Watchdog Timer Issue**
Okay, let's break this down. My initial thought was that `dev_init_scheduler` sets the `noop_qdisc` for all queues, and `qdisc_tx_is_noop(dev)` would then return TRUE. This would prevent `netdev_watchdog_up` from arming the timer. But the bug report suggests something else.
First, I need to confirm that `dev_init_scheduler` was actually called. It *should* be called in `register_netdevice`, and the error specifically mentions `usb0`, a registered device. So, yes, it seems `dev_init_scheduler` *was* called. That's a key point.
If `dev_init_scheduler` was called and the timer is still pending, it must be that `dev_activate` was also called. This would have switched the `qdisc` to the default, making `qdisc_tx_is_noop(dev)` FALSE and allowing `netdev_watchdog_up` to arm the timer. Then `dev_deactivate_many` is called, setting the `noop_qdisc` again and calling `netdev_watchdog_down`, which calls `timer_delete`. Now, here's where it gets interesting:
`netdev_watchdog_down` holds `tx_global_lock`. `dev_watchdog` also holds this same lock. If `dev_watchdog` is running and *hasn't* acquired the lock yet, then `netdev_watchdog_down` could acquire the lock, call `timer_delete` (which *should* return 0 because the timer is running), and release the lock. Then, `dev_watchdog` acquires the lock, checks `!qdisc_tx_is_noop(dev)` (which is now TRUE because `dev_deactivate_many` set `noop_qdisc`), and *doesn't* re-arm the timer. In this scenario, the timer finishes and is *not* pending.
But the question is, is there *any* case where `dev_watchdog` re-arms the timer *after* `dev_deactivate_many`? I need to consider all possibilities. What if `dev_watchdog` checks `!qdisc_tx_is_noop(dev)` *before* `dev_deactivate_many` sets `noop_qdisc`?
For this to happen, `dev_watchdog` *must* have acquired `tx_global_lock` before `netdev_watchdog_down`. If `dev_watchdog` acquires the lock first, it checks `!qdisc_tx_is_noop(dev)` (which is TRUE), arms the timer, and releases the lock. Then `netdev_watchdog_down` acquires the lock and calls `timer_delete`. Since the timer is not running, `timer_delete` *should* successfully delete the timer. So, the timer *should* be deleted in that case.
However, consider another edge case. What if `timer_delete` is called *while* the timer is running on *another* CPU? If `dev_watchdog` is running on CPU A, it acquires the lock, sees `!qdisc_tx_is_noop(dev)` is true, arms the timer, and releases the lock. The timer is queued. But `dev_watchdog` is *still* running. `netdev_watchdog_down` then runs on CPU B, acquires the lock, and calls `timer_delete`. If the timer is still running on CPU A, `timer_delete` *should* return 0, which *doesn't* actually delete the timer! I need to double-check that `timer_delete` implementation.
|
| 120/2 |
2026/05/21 11:44 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:kernel/time/timer.c IncludeLines:true Name:timer_delete]
Results: map[SourceCode:1390: /**
1391: * timer_delete - Deactivate a timer
1392: * @timer: The timer to be deactivated
1393: *
1394: * The function only deactivates a pending timer, but contrary to
1395: * timer_delete_sync() it does not take into account whether the timer's
1396: * callback function is concurrently executed on a different CPU or not.
1397: * It neither prevents rearming of the timer. If @timer can be rearmed
1398: * concurrently then the return value of this function is meaningless.
1399: *
1400: * Return:
1401: * * %0 - The timer was not pending
1402: * * %1 - The timer was pending and deactivated
1403: */
1404: int timer_delete(struct timer_list *timer)
1405: {
1406: return __timer_delete(timer, false);
1407: }
SourceFile:kernel/time/timer.c]
|
| 121/2 |
2026/05/21 11:44 |
llm |
expert |
0m
Model:
Tokens:
input: 102548
output: 40
thoughts: 171
Thoughts: **Analyzing the Timer Deletion Process**
Okay, so `timer_delete` *only* deactivates timers that are currently *pending*. That's the key. But wait, if the timer is *running*, is it considered "pending"? No, `timer_pending` returns 0 when the timer is active. Right, that makes sense.
Now, here's where it gets interesting. If `dev_watchdog` calls `mod_timer`, the timer *becomes* pending! Got it. So, following that logic: if `dev_watchdog` calls `mod_timer`, the timer is *now* in a pending state. Good.
Then, `dev_watchdog` releases the lock. And *then* `netdev_watchdog_down` acquires the lock. And crucially, it calls `timer_delete`. Since the timer, thanks to that `mod_timer` call in `dev_watchdog`, *is* pending, `timer_delete` *will* effectively deactivate it! That's the chain of events.
But, hang on a second. `timer_delete` calls `__timer_delete(timer, false)`. Right, I'm going to have to dive into that function to verify the final behavior, just to be sure. Let's dig into `__timer_delete`.
|
| 122/2 |
2026/05/21 11:44 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:kernel/time/timer.c IncludeLines:true Name:__timer_delete]
Results: map[SourceCode:1345: /**
1346: * __timer_delete - Internal function: Deactivate a timer
1347: * @timer: The timer to be deactivated
1348: * @shutdown: If true, this indicates that the timer is about to be
1349: * shutdown permanently.
1350: *
1351: * If @shutdown is true then @timer->function is set to NULL under the
1352: * timer base lock which prevents further rearming of the time. In that
1353: * case any attempt to rearm @timer after this function returns will be
1354: * silently ignored.
1355: *
1356: * Return:
1357: * * %0 - The timer was not pending
1358: * * %1 - The timer was pending and deactivated
1359: */
1360: static int __timer_delete(struct timer_list *timer, bool shutdown)
1361: {
1362: struct timer_base *base;
1363: unsigned long flags;
1364: int ret = 0;
1365:
1366: debug_assert_init(timer);
1367:
1368: /*
1369: * If @shutdown is set then the lock has to be taken whether the
1370: * timer is pending or not to protect against a concurrent rearm
1371: * which might hit between the lockless pending check and the lock
1372: * acquisition. By taking the lock it is ensured that such a newly
1373: * enqueued timer is dequeued and cannot end up with
1374: * timer->function == NULL in the expiry code.
1375: *
1376: * If timer->function is currently executed, then this makes sure
1377: * that the callback cannot requeue the timer.
1378: */
1379: if (timer_pending(timer) || shutdown) {
1380: base = lock_timer_base(timer, &flags);
1381: ret = detach_if_pending(timer, base, true);
1382: if (shutdown)
1383: timer->function = NULL;
1384: raw_spin_unlock_irqrestore(&base->lock, flags);
1385: }
1386:
1387: return ret;
1388: }
SourceFile:kernel/time/timer.c]
|
| 123/2 |
2026/05/21 11:44 |
llm |
expert |
0m
Model:
Tokens:
input: 103725
output: 22
thoughts: 2295
Thoughts: **Analyzing the Watchdog Timer Deletion**
Okay, here's the breakdown of my thought process regarding the watchdog timer:
Initially, I thought: If the timer is pending and the `__timer_delete` function runs, then the timer is deleted, which seems correct. This would be the expected behavior when `netdev_watchdog_down` is called. So, the timer *should* be deleted.
However, a potential race condition arises if `dev_watchdog` calls `mod_timer` *after* `netdev_watchdog_down` is triggered. I had to determine if that's possible. I remembered `dev_watchdog` checks if the qdisc is a no-op *before* calling `mod_timer`. Since `netdev_watchdog_down` calls `dev_deactivate_many` and that sets the no-op qdisc, `dev_watchdog` *shouldn't* call `mod_timer` after `netdev_watchdog_down`.
My next question was: Are there *other* ways `mod_timer` could be called after `netdev_watchdog_down`? What about `netdev_watchdog_up`? But again, it checks the qdisc. Therefore, that can't be the issue.
Now, I considered a preemption scenario. What if `dev_watchdog` *starts* to arm the timer, but gets preempted *before* completing the call to `mod_timer`? In this situation, CPU B could run `netdev_watchdog_down`, which would set the no-op qdisc. This could be a problem, until I reviewed the code and saw that `dev_watchdog` and `netdev_watchdog_up` both hold the spinlock `tx_global_lock` to protect the state of the device and `mod_timer` when checking the qdisc. Spinlocks disable preemption, meaning the timer cannot be started and the state changed at the same time. This is a crucial safety mechanism to prevent the race condition.
I then considered what would happen if `netdev_watchdog_down` was *never* called at all. Where is it called from? It's called by `dev_deactivate_many`, which is called by `__dev_close_many`, which is called by `netif_close_many`, and which is called by `unregister_netdevice_many_notify` but *only* if `dev->flags & IFF_UP` is set. If the device isn't up, the watchdog down won't be called.
But then, if `IFF_UP` is false, the qdisc is a no-op, which means the timer cannot be armed. Thus, if it's false, the timer is not pending. Unless I overlooked something.
So I worked through every way to set the up flag. `dev_activate` is called in `__dev_open`, which is called when the device is first opened and which *always* sets the up flag, so I had to make sure there weren't ways it could be missed. Even if `dev_change_flags` were called and tried to clear it, there's a check, and it won't succeed if the device is running. There is no way for this flag to be missed.
What if `unregister_netdevice_many_notify` is called while `__dev_open` is running? But the entire function is called under a global lock.
I dug further, realizing that `netif_close_many_and_unlock_cond` can call `netif_close_many` with a list of devices to close. I had to determine how this list is created. If `dev->flags & IFF_UP` is true, the device is added to the list, unless the device has an ops lock. If it doesn't need the ops lock, then the device is *always* added to the list, regardless of the up flag. But then `netif_close_many` *removes* devices without `IFF_UP` set, meaning it's still not a problem.
Therefore, if `IFF_UP` is FALSE, `__dev_close_many` is NOT called. So, `netdev_watchdog_down` is NOT called, and if the timer *is* pending, a warning would be triggered.
The final question: Is there *any* other place that arms the watchdog timer *without* checking the qdisc? A thorough check for callers of `mod_timer` did not reveal any other code paths.
Thus, the potential issue causing the warning is limited to the case where the timer is armed *before*
|
| 124/2 |
2026/05/21 11:45 |
tool |
grepper |
0m
Args: map[Expression:mod_timer.*watchdog_timer]
Results: map[Output:drivers/net/ethernet/amd/pcnet32.c=2092=static int pcnet32_open(struct net_device *dev)
--
drivers/net/ethernet/amd/pcnet32.c-2287- pcnet32_check_media(dev, 1);
drivers/net/ethernet/amd/pcnet32.c:2288: mod_timer(&lp->watchdog_timer, PCNET32_WATCHDOG_TIMEOUT);
drivers/net/ethernet/amd/pcnet32.c-2289- }
--
drivers/net/ethernet/amd/pcnet32.c=2906=static void pcnet32_watchdog(struct timer_list *t)
--
drivers/net/ethernet/amd/pcnet32.c-2916-
drivers/net/ethernet/amd/pcnet32.c:2917: mod_timer(&lp->watchdog_timer, round_jiffies(PCNET32_WATCHDOG_TIMEOUT));
drivers/net/ethernet/amd/pcnet32.c-2918-}
--
drivers/net/ethernet/atheros/atlx/atl2.c=682=static int atl2_open(struct net_device *netdev)
--
drivers/net/ethernet/atheros/atlx/atl2.c-719-
drivers/net/ethernet/atheros/atlx/atl2.c:720: mod_timer(&adapter->watchdog_timer, round_jiffies(jiffies + 4*HZ));
drivers/net/ethernet/atheros/atlx/atl2.c-721-
--
drivers/net/ethernet/atheros/atlx/atl2.c=1011=static void atl2_watchdog(struct timer_list *t)
--
drivers/net/ethernet/atheros/atlx/atl2.c-1027- /* Reset the timer */
drivers/net/ethernet/atheros/atlx/atl2.c:1028: mod_timer(&adapter->watchdog_timer,
drivers/net/ethernet/atheros/atlx/atl2.c-1029- round_jiffies(jiffies + 4 * HZ));
--
drivers/net/ethernet/intel/e1000e/netdev.c=1750=static irqreturn_t e1000_intr_msi(int __always_unused irq, void *data)
--
drivers/net/ethernet/intel/e1000e/netdev.c-1780- if (!test_bit(__E1000_DOWN, &adapter->state))
drivers/net/ethernet/intel/e1000e/netdev.c:1781: mod_timer(&adapter->watchdog_timer, jiffies + 1);
drivers/net/ethernet/intel/e1000e/netdev.c-1782- }
--
drivers/net/ethernet/intel/e1000e/netdev.c=1816=static irqreturn_t e1000_intr(int __always_unused irq, void *data)
--
drivers/net/ethernet/intel/e1000e/netdev.c-1859- if (!test_bit(__E1000_DOWN, &adapter->state))
drivers/net/ethernet/intel/e1000e/netdev.c:1860: mod_timer(&adapter->watchdog_timer, jiffies + 1);
drivers/net/ethernet/intel/e1000e/netdev.c-1861- }
--
drivers/net/ethernet/intel/e1000e/netdev.c=1890=static irqreturn_t e1000_msix_other(int __always_unused irq, void *data)
--
drivers/net/ethernet/intel/e1000e/netdev.c-1903- if (!test_bit(__E1000_DOWN, &adapter->state))
drivers/net/ethernet/intel/e1000e/netdev.c:1904: mod_timer(&adapter->watchdog_timer, jiffies + 1);
drivers/net/ethernet/intel/e1000e/netdev.c-1905- }
--
drivers/net/ethernet/intel/e1000e/netdev.c=5210=static void e1000_watchdog_task(struct work_struct *work)
--
drivers/net/ethernet/intel/e1000e/netdev.c-5442- if (!test_bit(__E1000_DOWN, &adapter->state))
drivers/net/ethernet/intel/e1000e/netdev.c:5443: mod_timer(&adapter->watchdog_timer,
drivers/net/ethernet/intel/e1000e/netdev.c-5444- round_jiffies(jiffies + 2 * HZ));
--
drivers/net/ethernet/intel/igb/igb_main.c=5565=static void igb_watchdog_task(struct work_struct *work)
--
drivers/net/ethernet/intel/igb/igb_main.c-5796- if (adapter->flags & IGB_FLAG_NEED_LINK_UPDATE)
drivers/net/ethernet/intel/igb/igb_main.c:5797: mod_timer(&adapter->watchdog_timer,
drivers/net/ethernet/intel/igb/igb_main.c-5798- round_jiffies(jiffies + HZ));
drivers/net/ethernet/intel/igb/igb_main.c-5799- else
drivers/net/ethernet/intel/igb/igb_main.c:5800: mod_timer(&adapter->watchdog_timer,
drivers/net/ethernet/intel/igb/igb_main.c-5801- round_jiffies(jiffies + 2 * HZ));
--
drivers/net/ethernet/intel/igb/igb_main.c=7097=static irqreturn_t igb_msix_other(int irq, void *data)
--
drivers/net/ethernet/intel/igb/igb_main.c-7124- if (!test_bit(__IGB_DOWN, &adapter->state))
drivers/net/ethernet/intel/igb/igb_main.c:7125: mod_timer(&adapter->watchdog_timer, jiffies + 1);
drivers/net/ethernet/intel/igb/igb_main.c-7126- }
--
drivers/net/ethernet/intel/igb/igb_main.c=8177=static irqreturn_t igb_intr_msi(int irq, void *data)
--
drivers/net/ethernet/intel/igb/igb_main.c-8197- if (!test_bit(__IGB_DOWN, &adapter->state))
drivers/net/ethernet/intel/igb/igb_main.c:8198: mod_timer(&adapter->watchdog_timer, jiffies + 1);
drivers/net/ethernet/intel/igb/igb_main.c-8199- }
--
drivers/net/ethernet/intel/igb/igb_main.c=8214=static irqreturn_t igb_intr(int irq, void *data)
--
drivers/net/ethernet/intel/igb/igb_main.c-8243- if (!test_bit(__IGB_DOWN, &adapter->state))
drivers/net/ethernet/intel/igb/igb_main.c:8244: mod_timer(&adapter->watchdog_timer, jiffies + 1);
drivers/net/ethernet/intel/igb/igb_main.c-8245- }
--
drivers/net/ethernet/intel/igbvf/netdev.c=852=static irqreturn_t igbvf_msix_other(int irq, void *data)
--
drivers/net/ethernet/intel/igbvf/netdev.c-859- if (!test_bit(__IGBVF_DOWN, &adapter->state))
drivers/net/ethernet/intel/igbvf/netdev.c:860: mod_timer(&adapter->watchdog_timer, jiffies + 1);
drivers/net/ethernet/intel/igbvf/netdev.c-861-
--
drivers/net/ethernet/intel/igbvf/netdev.c=1537=int igbvf_up(struct igbvf_adapter *adapter)
--
drivers/net/ethernet/intel/igbvf/netdev.c-1555- hw->mac.get_link_status = 1;
drivers/net/ethernet/intel/igbvf/netdev.c:1556: mod_timer(&adapter->watchdog_timer, jiffies + 1);
drivers/net/ethernet/intel/igbvf/netdev.c-1557-
--
drivers/net/ethernet/intel/igbvf/netdev.c=1697=static int igbvf_open(struct net_device *netdev)
--
drivers/net/ethernet/intel/igbvf/netdev.c-1739- hw->mac.get_link_status = 1;
drivers/net/ethernet/intel/igbvf/netdev.c:1740: mod_timer(&adapter->watchdog_timer, jiffies + 1);
drivers/net/ethernet/intel/igbvf/netdev.c-1741-
--
drivers/net/ethernet/intel/igbvf/netdev.c=1902=static void igbvf_watchdog_task(struct work_struct *work)
--
drivers/net/ethernet/intel/igbvf/netdev.c-1956- if (!test_bit(__IGBVF_DOWN, &adapter->state))
drivers/net/ethernet/intel/igbvf/netdev.c:1957: mod_timer(&adapter->watchdog_timer,
drivers/net/ethernet/intel/igbvf/netdev.c-1958- round_jiffies(jiffies + (2 * HZ)));
--
drivers/net/ethernet/intel/igc/igc_main.c=5636=static irqreturn_t igc_msix_other(int irq, void *data)
--
drivers/net/ethernet/intel/igc/igc_main.c-5654- if (!test_bit(__IGC_DOWN, &adapter->state))
drivers/net/ethernet/intel/igc/igc_main.c:5655: mod_timer(&adapter->watchdog_timer, jiffies + 1);
drivers/net/ethernet/intel/igc/igc_main.c-5656- }
--
drivers/net/ethernet/intel/igc/igc_main.c=5830=static void igc_watchdog_task(struct work_struct *work)
--
drivers/net/ethernet/intel/igc/igc_main.c-6019- if (adapter->flags & IGC_FLAG_NEED_LINK_UPDATE)
drivers/net/ethernet/intel/igc/igc_main.c:6020: mod_timer(&adapter->watchdog_timer,
drivers/net/ethernet/intel/igc/igc_main.c-6021- round_jiffies(jiffies + HZ));
drivers/net/ethernet/intel/igc/igc_main.c-6022- else
drivers/net/ethernet/intel/igc/igc_main.c:6023: mod_timer(&adapter->watchdog_timer,
drivers/net/ethernet/intel/igc/igc_main.c-6024- round_jiffies(jiffies + 2 * HZ));
--
drivers/net/ethernet/intel/igc/igc_main.c=6033=static irqreturn_t igc_intr_msi(int irq, void *data)
--
drivers/net/ethernet/intel/igc/igc_main.c-6053- if (!test_bit(__IGC_DOWN, &adapter->state))
drivers/net/ethernet/intel/igc/igc_main.c:6054: mod_timer(&adapter->watchdog_timer, jiffies + 1);
drivers/net/ethernet/intel/igc/igc_main.c-6055- }
--
drivers/net/ethernet/intel/igc/igc_main.c=6070=static irqreturn_t igc_intr(int irq, void *data)
--
drivers/net/ethernet/intel/igc/igc_main.c-6099- if (!test_bit(__IGC_DOWN, &adapter->state))
drivers/net/ethernet/intel/igc/igc_main.c:6100: mod_timer(&adapter->watchdog_timer, jiffies + 1);
drivers/net/ethernet/intel/igc/igc_main.c-6101- }
--
drivers/net/ethernet/marvell/sky2.c=2173=static void sky2_link_up(struct sky2_port *sky2)
--
drivers/net/ethernet/marvell/sky2.c-2191-
drivers/net/ethernet/marvell/sky2.c:2192: mod_timer(&hw->watchdog_timer, jiffies + 1);
drivers/net/ethernet/marvell/sky2.c-2193-
--
drivers/net/ethernet/marvell/sky2.c=2962=static void sky2_watchdog(struct timer_list *t)
--
drivers/net/ethernet/marvell/sky2.c-2990-
drivers/net/ethernet/marvell/sky2.c:2991: mod_timer(&hw->watchdog_timer, round_jiffies(jiffies + HZ));
drivers/net/ethernet/marvell/sky2.c-2992-}
--
drivers/net/ethernet/myricom/myri10ge/myri10ge.c=3469=static void myri10ge_watchdog_timer(struct timer_list *t)
--
drivers/net/ethernet/myricom/myri10ge/myri10ge.c-3517- /* rearm timer */
drivers/net/ethernet/myricom/myri10ge/myri10ge.c:3518: mod_timer(&mgp->watchdog_timer,
drivers/net/ethernet/myricom/myri10ge/myri10ge.c-3519- jiffies + myri10ge_watchdog_timeout * HZ);
--
drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c=1024=static void pch_gbe_watchdog(struct timer_list *t)
--
drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c-1060- }
drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c:1061: mod_timer(&adapter->watchdog_timer,
drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c-1062- round_jiffies(jiffies + PCH_GBE_WATCHDOG_PERIOD));
--
drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c=1857=int pch_gbe_up(struct pch_gbe_adapter *adapter)
--
drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c-1895-
drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c:1896: mod_timer(&adapter->watchdog_timer, jiffies);
drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c-1897-
--
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c=308=static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
--
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c-382-
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c:383: mod_timer(&ionic->watchdog_timer,
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c-384- round_jiffies(jiffies + ionic->watchdog_period));
--
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c=468=static void ionic_reset_done(struct pci_dev *pdev)
--
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c-484-
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c:485: mod_timer(&ionic->watchdog_timer, jiffies + 1);
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c-486-
--
drivers/net/ethernet/pensando/ionic/ionic_dev.c=14=static void ionic_watchdog_cb(struct timer_list *t)
--
drivers/net/ethernet/pensando/ionic/ionic_dev.c-20-
drivers/net/ethernet/pensando/ionic/ionic_dev.c:21: mod_timer(&ionic->watchdog_timer,
drivers/net/ethernet/pensando/ionic/ionic_dev.c-22- round_jiffies(jiffies + ionic->watchdog_period));
--
drivers/net/ethernet/pensando/ionic/ionic_lif.c=86=static void ionic_lif_deferred_work(struct work_struct *work)
--
drivers/net/ethernet/pensando/ionic/ionic_lif.c-120- */
drivers/net/ethernet/pensando/ionic/ionic_lif.c:121: mod_timer(&lif->ionic->watchdog_timer, jiffies + 1);
drivers/net/ethernet/pensando/ionic/ionic_lif.c-122- }
--
drivers/net/wireless/realtek/rtlwifi/base.c=2221=void rtl_watch_dog_timer_callback(struct timer_list *t)
--
drivers/net/wireless/realtek/rtlwifi/base.c-2228-
drivers/net/wireless/realtek/rtlwifi/base.c:2229: mod_timer(&rtlpriv->works.watchdog_timer,
drivers/net/wireless/realtek/rtlwifi/base.c-2230- jiffies + MSECS(RTL_WATCH_DOG_TIME));
--
kernel/workqueue.c=7581=static void wq_watchdog_timer_fn(struct timer_list *unused)
--
kernel/workqueue.c-7643- wq_watchdog_reset_touched();
kernel/workqueue.c:7644: mod_timer(&wq_watchdog_timer, jiffies + thresh);
kernel/workqueue.c-7645-}
--
kernel/workqueue.c=7663=static void wq_watchdog_set_thresh(unsigned long thresh)
--
kernel/workqueue.c-7670- wq_watchdog_reset_touched();
kernel/workqueue.c:7671: mod_timer(&wq_watchdog_timer, jiffies + thresh * HZ);
kernel/workqueue.c-7672- }
--
net/sched/sch_generic.c=500=static void dev_watchdog(struct timer_list *t)
--
net/sched/sch_generic.c-545- }
net/sched/sch_generic.c:546: if (!mod_timer(&dev->watchdog_timer,
net/sched/sch_generic.c-547- round_jiffies(oldest_start +
--
net/sched/sch_generic.c=558=void netdev_watchdog_up(struct net_device *dev)
--
net/sched/sch_generic.c-563- dev->watchdog_timeo = 5*HZ;
net/sched/sch_generic.c:564: if (!mod_timer(&dev->watchdog_timer,
net/sched/sch_generic.c-565- round_jiffies(jiffies + dev->watchdog_timeo)))
]
|
| 125/2 |
2026/05/21 11:45 |
llm |
expert |
0m
Model:
Tokens:
input: 112484
output: 42
thoughts: 927
Thoughts: **Analyzing the Network Device Watchdog Timer**
Okay, so I've been tracing the logic around the network device watchdog timer, `dev->watchdog_timer`. It looks like `dev_watchdog` and `netdev_watchdog_up` are the *only* functions directly responsible for arming this timer, specifically using `mod_timer(&dev->watchdog_timer, ...)`. That's the critical starting point. Let's dig deeper into those two functions.
`dev_watchdog` is pretty straightforward: it arms the timer within a spinlock if the device is present, running, and the carrier is okay, *and* if the qdisc isn't noop. The key conditional being `!qdisc_tx_is_noop(dev)`.
`netdev_watchdog_up` also checks `!qdisc_tx_is_noop(dev)`. If `qdisc_tx_is_noop(dev)` returns true, the timer isn't armed! This is important because it means we can prevent timer arming. This noop status is a critical control path.
So, when *is* `qdisc_tx_is_noop(dev)` true? The obvious answer is when the device's Quality of Service (QoS) scheduler (`qdisc`) is set to `noop_qdisc`. This happens under two circumstances:
1. After `dev_init_scheduler` during device registration (`register_netdevice`).
2. After `dev_deactivate_many` during device closure (`__dev_close_many`).
Now, the connection! If `IFF_UP` (Interface Flags - Up) is *false*, `__dev_close_many` is *not* called during device unregistration. But, and this is the crux of it, if `IFF_UP` is false, the `qdisc` *must* be `noop_qdisc`! It was initialized to `noop_qdisc` and never changed! But I need to be sure.
My next line of thinking, could `dev_activate` be called, but `IFF_UP` be cleared *without* calling `__dev_close_many`?
`dev_change_flags` can clear `IFF_UP`, but only by calling `__dev_close` if it was previously set. `__dev_close` *then* calls `__dev_close_many`, which calls `dev_deactivate_many`, setting the qdisc to `noop_qdisc`. So clearing `IFF_UP` *always* means `noop_qdisc` will be set. Good!
Another question, What if `dev_activate` is called, but `IFF_UP` is NOT set?
`dev_activate` is called in `__dev_open`, but within that function, `netif_set_up(dev, true)` SETS `IFF_UP` before it calls `dev_activate`. Thus, IFF_UP *is* set.
*But* what if `dev_activate` is called from *somewhere else*? I need to check the call graph for `dev_activate`...
|
| 126/2 |
2026/05/21 11:45 |
tool |
codesearch-find-references |
0m
Args: map[ContextFile:net/sched/sch_generic.c IncludeSnippetLines:0 Name:dev_activate]
Results: map[References:[map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:__dev_open SourceFile:net/core/dev.c SourceLine:1692] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:blackhole_netdev_init SourceFile:drivers/net/loopback.c SourceLine:284] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:dev_qdisc_change_tx_queue_len SourceFile:net/sched/sch_generic.c SourceLine:1464] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:htb_graft_helper SourceFile:net/sched/sch_htb.c SourceLine:1396] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:htb_offload_move_qdisc SourceFile:net/sched/sch_htb.c SourceLine:1439] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:linkwatch_do_dev SourceFile:net/core/link_watch.c SourceLine:182] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:mq_graft SourceFile:net/sched/sch_mq.c SourceLine:189] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:mqprio_graft SourceFile:net/sched/sch_mqprio.c SourceLine:481] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:qdisc_graft SourceFile:net/sched/sch_api.c SourceLine:1166] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:taprio_graft SourceFile:net/sched/sch_taprio.c SourceLine:2212]] TruncatedOutput:false]
|
| 127/2 |
2026/05/21 11:45 |
llm |
expert |
0m
Model:
Tokens:
input: 114686
output: 44
thoughts: 30
Thoughts: Here's my thought process, as if I were analyzing that code:
**Diving into the Device Activation Sequence**
Okay, so `linkwatch_do_dev` is the crucial function here, and it's calling `dev_activate`. That's the key connection. This tells me that `linkwatch_do_dev` is likely responsible for initiating some sort of device activation process, which makes sense given the name. The call to `dev_activate` suggests this might be the core function for bringing a network interface or other hardware device online, or at least setting up some essential properties. Now, I need to understand what triggers `linkwatch_do_dev`. Is it a periodic check? A hardware interrupt? A user command? Understanding that will give me a better grasp of the context. Let's delve into `linkwatch_do_dev` to see what it's all about. I want to see what triggers it, and what else it does besides calling `dev_activate`.
|
| 128/2 |
2026/05/21 11:45 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/link_watch.c IncludeLines:true Name:linkwatch_do_dev]
Results: map[SourceCode: 166: static void linkwatch_do_dev(struct net_device *dev)
167: {
168: /*
169: * Make sure the above read is complete since it can be
170: * rewritten as soon as we clear the bit below.
171: */
172: smp_mb__before_atomic();
173:
174: /* We are about to handle this device,
175: * so new events can be accepted
176: */
177: clear_bit(__LINK_STATE_LINKWATCH_PENDING, &dev->state);
178:
179: rfc2863_policy(dev);
180: if (dev->flags & IFF_UP) {
181: if (netif_carrier_ok(dev))
182: dev_activate(dev);
183: else
184: dev_deactivate(dev);
185:
186: netif_state_change(dev);
187: }
188: /* Note: our callers are responsible for calling netdev_tracker_free().
189: * This is the reason we use __dev_put() instead of dev_put().
190: */
191: __dev_put(dev);
192: }
SourceFile:net/core/link_watch.c]
|
| 129/2 |
2026/05/21 11:45 |
llm |
expert |
0m
Model:
Tokens:
input: 115379
output: 43
thoughts: 52
Thoughts: Okay, here's my interpretation of those thoughts, considering my expertise:
**Checking the Activation Logic of `linkwatch_do_dev`**
Alright, let's break this down. `linkwatch_do_dev` is essentially the routine that handles network device state changes, right? And I see it starts by checking the device's operational state using `if (dev->flags & IFF_UP)`. So, it's explicitly ensuring the interface is *administratively* up before doing anything else. It's a fundamental safety check, a critical first step. It is only activating if the device's `IFF_UP` flag is set to `TRUE`.
But here's where my mind jumps: what about the queuing discipline grafting? Specifically, what happens with `qdisc_graft` in this scenario? Does `qdisc_graft` get called *after* this check? Does it bypass `IFF_UP` somehow? It can't, right? It should still respect the operational state. I'd need to trace the call paths to be 100% certain, but intuitively, `qdisc_graft` should also be conditionally dependent on `IFF_UP`, or else we have a potential race condition and undesirable behavior. I need to make sure the queueing discipline doesn't try to send packets on a downed interface. This calls for a code audit.
|
| 130/2 |
2026/05/21 11:45 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/sched/sch_api.c IncludeLines:true Name:qdisc_graft]
Results: map[SourceCode:1087: static int qdisc_graft(struct net_device *dev, struct Qdisc *parent,
1088: struct sk_buff *skb, struct nlmsghdr *n, u32 classid,
1089: struct Qdisc *new, struct Qdisc *old,
1090: struct netlink_ext_ack *extack)
1091: {
1092: struct Qdisc *q = old;
1093: struct net *net = dev_net(dev);
1094:
1095: if (parent == NULL) {
1096: unsigned int i, num_q, ingress;
1097: struct netdev_queue *dev_queue;
1098:
1099: ingress = 0;
1100: num_q = dev->num_tx_queues;
1101: if ((q && q->flags & TCQ_F_INGRESS) ||
1102: (new && new->flags & TCQ_F_INGRESS)) {
1103: ingress = 1;
1104: dev_queue = dev_ingress_queue(dev);
1105: if (!dev_queue) {
1106: NL_SET_ERR_MSG(extack, "Device does not have an ingress queue");
1107: return -ENOENT;
1108: }
1109:
1110: q = rtnl_dereference(dev_queue->qdisc_sleeping);
1111:
1112: /* This is the counterpart of that qdisc_refcount_inc_nz() call in
1113: * __tcf_qdisc_find() for filter requests.
1114: */
1115: if (!qdisc_refcount_dec_if_one(q)) {
1116: NL_SET_ERR_MSG(extack,
1117: "Current ingress or clsact Qdisc has ongoing filter requests");
1118: return -EBUSY;
1119: }
1120: }
1121:
1122: if (dev->flags & IFF_UP)
1123: dev_deactivate(dev);
1124:
1125: qdisc_offload_graft_root(dev, new, old, extack);
1126:
1127: if (new && new->ops->attach && !ingress)
1128: goto skip;
1129:
1130: if (!ingress) {
1131: for (i = 0; i < num_q; i++) {
1132: dev_queue = netdev_get_tx_queue(dev, i);
1133: old = dev_graft_qdisc(dev_queue, new);
1134:
1135: if (new && i > 0)
1136: qdisc_refcount_inc(new);
1137: qdisc_put(old);
1138: }
1139: } else {
1140: old = dev_graft_qdisc(dev_queue, NULL);
1141:
1142: /* {ingress,clsact}_destroy() @old before grafting @new to avoid
1143: * unprotected concurrent accesses to net_device::miniq_{in,e}gress
1144: * pointer(s) in mini_qdisc_pair_swap().
1145: */
1146: qdisc_notify(net, skb, n, classid, old, new, extack);
1147: qdisc_destroy(old);
1148:
1149: dev_graft_qdisc(dev_queue, new);
1150: }
1151:
1152: skip:
1153: if (!ingress) {
1154: old = rtnl_dereference(dev->qdisc);
1155: if (new && !new->ops->attach)
1156: qdisc_refcount_inc(new);
1157: rcu_assign_pointer(dev->qdisc, new ? : &noop_qdisc);
1158:
1159: notify_and_destroy(net, skb, n, classid, old, new, extack);
1160:
1161: if (new && new->ops->attach)
1162: new->ops->attach(new);
1163: }
1164:
1165: if (dev->flags & IFF_UP)
1166: dev_activate(dev);
1167: } else {
1168: const struct Qdisc_class_ops *cops = parent->ops->cl_ops;
1169: unsigned long cl;
1170: int err;
1171:
1172: /* Only support running class lockless if parent is lockless */
1173: if (new && (new->flags & TCQ_F_NOLOCK) && !(parent->flags & TCQ_F_NOLOCK))
1174: qdisc_clear_nolock(new);
1175:
1176: if (!cops || !cops->graft)
1177: return -EOPNOTSUPP;
1178:
1179: cl = cops->find(parent, classid);
1180: if (!cl) {
1181: NL_SET_ERR_MSG(extack, "Specified class not found");
1182: return -ENOENT;
1183: }
1184:
1185: if (new && new->ops == &noqueue_qdisc_ops) {
1186: NL_SET_ERR_MSG(extack, "Cannot assign noqueue to a class");
1187: return -EINVAL;
1188: }
1189:
1190: if (new &&
1191: !(parent->flags & TCQ_F_MQROOT) &&
1192: rcu_access_pointer(new->stab)) {
1193: NL_SET_ERR_MSG(extack, "STAB not supported on a non root");
1194: return -EINVAL;
1195: }
1196: err = cops->graft(parent, cl, new, &old, extack);
1197: if (err)
1198: return err;
1199: notify_and_destroy(net, skb, n, classid, old, new, extack);
1200: }
1201: return 0;
1202: }
SourceFile:net/sched/sch_api.c]
|
| 131/2 |
2026/05/21 11:45 |
llm |
expert |
0m
Model:
Tokens:
input: 117606
output: 21
thoughts: 3582
Thoughts: **Analyzing the Mysterious Pending Timer**
Okay, let's break this down. My initial thought process revolves around understanding why a watchdog timer might be pending after a network device unregisters, even though the expected behavior is for the timer to be deleted. The core issue seems to hinge on the relationship between device activation (`IFF_UP`), queueing disciplines (`qdisc`), and the watchdog timer itself.
First, I observe that `dev_activate` is only called when `IFF_UP` is true. Conversely, if `IFF_UP` is false, `dev_activate` isn't called, and the `qdisc` defaults to `noop_qdisc`, as I'd expect. However, what if `IFF_UP` *is* true, and a device is unregistered? This is where things get interesting. The unregistration process calls `netif_close_many`, which eventually calls `dev_deactivate_many`. This function is responsible for setting the `qdisc` to `noop_qdisc` and calling `netdev_watchdog_down`, which *should* delete the timer. `dev_shutdown` then warns if the timer is *still* pending.
So, why the warning? The bug report seems to indicate a scenario where `timer_delete` doesn't actually detach the timer. `timer_delete` returns 0 if the timer isn't pending, but even if the timer is *running*, `__timer_delete` should detach it. However, I realize that `timer_pending` returns false while the timer is actually running. Thus, `__timer_delete` does nothing if the timer is running. So, the timer will still be active.
My next line of thought is to consider concurrency. Could there be an interleaving of calls where the timer is re-armed *after* the deletion attempt? I note that `dev_watchdog` checks if the `qdisc` is not `noop_qdisc`. `dev_deactivate_many` is supposed to set the qdisc to `noop_qdisc` *before* calling `netdev_watchdog_down`. If a race condition exists, and `dev_watchdog` runs before `dev_deactivate_many` does its job, then the watchdog timer might get re-armed before the deletion happens! I realize that both `dev_watchdog_up` and `netdev_watchdog_down` are protected by `tx_global_lock`, which implies a race condition is possible in this scenario.
I start to analyze where else the timer could have been re-armed, but I can't find anything, the conditions are tightly coupled to the qdisc and `IFF_UP` status.
I start to think about the cases that might cause the function to be skipped, the first being that if `IFF_UP` is FALSE, then the critical steps of deactivating queues, setting `noop_qdisc`, and deleting the timer via `netif_close_many` are skipped! It's because the deactivation, qdisc reset, and timer removal steps are only triggered when the device is "closed" by unregistering. If `IFF_UP` is false, these functions are not called. Thus, if `IFF_UP` is set to false, but the driver never "closes" the device, the `noop_qdisc` won't be set, and the timer will remain.
Now I'm focusing on that case, is there any situation when `IFF_UP` is FALSE and the `qdisc` is *not* `noop_qdisc`? What if some driver *manually* modified `dev->flags` after the activation? If a driver does `dev->flags &= ~IFF_UP`, this will occur. The system won't "know" to deactivate everything. The timer would be running, and the next unregister would warn that the timer is pending.
So, ultimately, my suspicion is that there is a driver that is modifying `dev->flags` directly and preventing `dev_deactivate_many` from properly deactivating the device. This leads to the timer potentially remaining pending when the device is unregistered. The problem is that the system relies on the driver to set the flags correctly, but nothing explicitly enforces this. I think this is where the bug report and investigation should focus: on identifying drivers that might be directly manipulating the device flags in a way that bypasses the normal deactivation process.
|
| 132/2 |
2026/05/21 11:46 |
tool |
grepper |
0m
Args: map[Expression:dev->flags.*IFF_UP]
Results: map[Output:
Full output is too long, showing 500 out of 1679 lines.
Use more precise expression if possible.
[drivers/infiniband/hw/irdma/cm.c=1656=static int irdma_add_mqh_6(struct irdma_device *iwdev,
--
drivers/infiniband/hw/irdma/cm.c-1668- for_each_netdev(&init_net, ip_dev) {
drivers/infiniband/hw/irdma/cm.c:1669: if (!(ip_dev->flags & IFF_UP))
drivers/infiniband/hw/irdma/cm.c-1670- continue;
--
drivers/infiniband/hw/irdma/cm.c=1744=static int irdma_add_mqh_4(struct irdma_device *iwdev,
--
drivers/infiniband/hw/irdma/cm.c-1756- for_each_netdev(&init_net, ip_dev) {
drivers/infiniband/hw/irdma/cm.c:1757: if (!(ip_dev->flags & IFF_UP))
drivers/infiniband/hw/irdma/cm.c-1758- continue;
--
drivers/infiniband/hw/irdma/utils.c=335=static void irdma_add_ipv6_addr(struct irdma_device *iwdev)
--
drivers/infiniband/hw/irdma/utils.c-346- ip_dev == iwdev->netdev) &&
drivers/infiniband/hw/irdma/utils.c:347: (READ_ONCE(ip_dev->flags) & IFF_UP)) {
drivers/infiniband/hw/irdma/utils.c-348- idev = __in6_dev_get(ip_dev);
--
drivers/infiniband/hw/irdma/utils.c=377=static void irdma_add_ipv4_addr(struct irdma_device *iwdev)
--
drivers/infiniband/hw/irdma/utils.c-386- rdma_vlan_dev_real_dev(dev) == iwdev->netdev) ||
drivers/infiniband/hw/irdma/utils.c:387: dev == iwdev->netdev) && (READ_ONCE(dev->flags) & IFF_UP)) {
drivers/infiniband/hw/irdma/utils.c-388- const struct in_ifaddr *ifa;
--
drivers/infiniband/ulp/ipoib/ipoib_main.c=164=static void ipoib_schedule_ifupdown_task(struct net_device *dev, bool up)
--
drivers/infiniband/ulp/ipoib/ipoib_main.c-167-
drivers/infiniband/ulp/ipoib/ipoib_main.c:168: if ((up && (dev->flags & IFF_UP)) ||
drivers/infiniband/ulp/ipoib/ipoib_main.c:169: (!up && !(dev->flags & IFF_UP)))
drivers/infiniband/ulp/ipoib/ipoib_main.c-170- return;
--
drivers/infiniband/ulp/ipoib/ipoib_main.c=1853=static int ipoib_dev_init(struct net_device *dev)
--
drivers/infiniband/ulp/ipoib/ipoib_main.c-1888-
drivers/infiniband/ulp/ipoib/ipoib_main.c:1889: if (dev->flags & IFF_UP) {
drivers/infiniband/ulp/ipoib/ipoib_main.c-1890- if (ipoib_ib_dev_open(dev)) {
--
drivers/infiniband/ulp/ipoib/ipoib_main.c=1925=static void ipoib_parent_unregister_pre(struct net_device *ndev)
--
drivers/infiniband/ulp/ipoib/ipoib_main.c-1933- rtnl_lock();
drivers/infiniband/ulp/ipoib/ipoib_main.c:1934: dev_change_flags(priv->dev, priv->dev->flags & ~IFF_UP, NULL);
drivers/infiniband/ulp/ipoib/ipoib_main.c-1935- rtnl_unlock();
--
drivers/infiniband/ulp/opa_vnic/opa_vnic_netdev.c=134=void opa_vnic_process_vema_config(struct opa_vnic_adapter *adapter)
--
drivers/infiniband/ulp/opa_vnic/opa_vnic_netdev.c-185- /* update state */
drivers/infiniband/ulp/opa_vnic/opa_vnic_netdev.c:186: opa_vnic_update_state(adapter, !!(netdev->flags & IFF_UP));
drivers/infiniband/ulp/opa_vnic/opa_vnic_netdev.c-187-}
--
drivers/net/arcnet/com20020.c=372=static void com20020_set_mc_list(struct net_device *dev)
--
drivers/net/arcnet/com20020.c-376-
drivers/net/arcnet/com20020.c:377: if ((dev->flags & IFF_PROMISC) && (dev->flags & IFF_UP)) {
drivers/net/arcnet/com20020.c-378- /* Enable promiscuous mode */
--
drivers/net/bonding/bond_main.c=922=static void bond_hw_addr_swap(struct bonding *bond, struct slave *new_active,
--
drivers/net/bonding/bond_main.c-931-
drivers/net/bonding/bond_main.c:932: if (bond->dev->flags & IFF_UP)
drivers/net/bonding/bond_main.c-933- bond_hw_addr_flush(bond->dev, old_active->dev);
--
drivers/net/bonding/bond_main.c-945-
drivers/net/bonding/bond_main.c:946: if (bond->dev->flags & IFF_UP) {
drivers/net/bonding/bond_main.c-947- netif_addr_lock_bh(bond->dev);
--
drivers/net/bonding/bond_main.c=1471=static void bond_setup_by_slave(struct net_device *bond_dev,
--
drivers/net/bonding/bond_main.c-1473-{
drivers/net/bonding/bond_main.c:1474: bool was_up = !!(bond_dev->flags & IFF_UP);
drivers/net/bonding/bond_main.c-1475-
--
drivers/net/bonding/bond_main.c=1767=static void bond_ether_setup(struct net_device *bond_dev)
drivers/net/bonding/bond_main.c-1768-{
drivers/net/bonding/bond_main.c:1769: unsigned int flags = bond_dev->flags & (IFF_SLAVE | IFF_UP);
drivers/net/bonding/bond_main.c-1770-
--
drivers/net/bonding/bond_main.c=1799=int bond_enslave(struct net_device *bond_dev, struct net_device *slave_dev,
--
drivers/net/bonding/bond_main.c-1849- */
drivers/net/bonding/bond_main.c:1850: if (slave_dev->flags & IFF_UP) {
drivers/net/bonding/bond_main.c-1851- SLAVE_NL_ERR(bond_dev, slave_dev, extack,
--
drivers/net/bonding/bond_main.c-2172-
drivers/net/bonding/bond_main.c:2173: if (bond_dev->flags & IFF_UP) {
drivers/net/bonding/bond_main.c-2174- netif_addr_lock_bh(bond_dev);
--
drivers/net/bonding/bond_options.c=679=static int bond_opt_check_deps(struct bonding *bond,
--
drivers/net/bonding/bond_options.c-687- return -ENOTEMPTY;
drivers/net/bonding/bond_options.c:688: if ((opt->flags & BOND_OPTFLAG_IFDOWN) && (bond->dev->flags & IFF_UP))
drivers/net/bonding/bond_options.c-689- return -EBUSY;
--
drivers/net/bonding/bond_options.c=1007=static int bond_option_miimon_set(struct bonding *bond,
--
drivers/net/bonding/bond_options.c-1027- }
drivers/net/bonding/bond_options.c:1028: if (bond->dev->flags & IFF_UP) {
drivers/net/bonding/bond_options.c-1029- /* If the interface is up, we may need to fire off
--
drivers/net/bonding/bond_options.c=1110=static int bond_option_arp_interval_set(struct bonding *bond,
--
drivers/net/bonding/bond_options.c-1123- }
drivers/net/bonding/bond_options.c:1124: if (bond->dev->flags & IFF_UP) {
drivers/net/bonding/bond_options.c-1125- /* If the interface is up, we may need to fire off
--
drivers/net/bonding/bond_options.c=1893=static int bond_option_broadcast_neigh_set(struct bonding *bond,
--
drivers/net/bonding/bond_options.c-1899- bond->params.broadcast_neighbor = newval->value;
drivers/net/bonding/bond_options.c:1900: if (bond->dev->flags & IFF_UP) {
drivers/net/bonding/bond_options.c-1901- if (bond->params.broadcast_neighbor)
--
drivers/net/can/at91_can.c=968=static ssize_t mb0_id_store(struct device *dev,
--
drivers/net/can/at91_can.c-979-
drivers/net/can/at91_can.c:980: if (ndev->flags & IFF_UP) {
drivers/net/can/at91_can.c-981- ret = -EBUSY;
--
drivers/net/can/c_can/c_can_main.c=1267=int c_can_power_down(struct net_device *dev)
--
drivers/net/can/c_can/c_can_main.c-1272-
drivers/net/can/c_can/c_can_main.c:1273: if (!(dev->flags & IFF_UP))
drivers/net/can/c_can/c_can_main.c-1274- return 0;
--
drivers/net/can/c_can/c_can_main.c=1301=int c_can_power_up(struct net_device *dev)
--
drivers/net/can/c_can/c_can_main.c-1307-
drivers/net/can/c_can/c_can_main.c:1308: if (!(dev->flags & IFF_UP))
drivers/net/can/c_can/c_can_main.c-1309- return 0;
--
drivers/net/can/dev/netlink.c=304=static int can_ctrlmode_changelink(struct net_device *dev,
--
drivers/net/can/dev/netlink.c-315- /* Do not allow changing controller mode while running */
drivers/net/can/dev/netlink.c:316: if (dev->flags & IFF_UP)
drivers/net/can/dev/netlink.c-317- return -EBUSY;
--
drivers/net/can/dev/netlink.c=435=static int can_dbt_changelink(struct net_device *dev, struct nlattr *data[],
--
drivers/net/can/dev/netlink.c-461- /* Do not allow changing bittiming while running */
drivers/net/can/dev/netlink.c:462: if (dev->flags & IFF_UP)
drivers/net/can/dev/netlink.c-463- return -EBUSY;
--
drivers/net/can/dev/netlink.c=593=static int can_changelink(struct net_device *dev, struct nlattr *tb[],
--
drivers/net/can/dev/netlink.c-608- /* Do not allow changing bittiming while running */
drivers/net/can/dev/netlink.c:609: if (dev->flags & IFF_UP)
drivers/net/can/dev/netlink.c-610- return -EBUSY;
--
drivers/net/can/dev/netlink.c-656- /* Do not allow changing restart delay while running */
drivers/net/can/dev/netlink.c:657: if (dev->flags & IFF_UP)
drivers/net/can/dev/netlink.c-658- return -EBUSY;
--
drivers/net/can/dev/netlink.c-669- /* Do not allow a restart while not running */
drivers/net/can/dev/netlink.c:670: if (!(dev->flags & IFF_UP))
drivers/net/can/dev/netlink.c-671- return -EINVAL;
--
drivers/net/can/grcan.c=1335=static netdev_tx_t grcan_start_xmit(struct sk_buff *skb,
--
drivers/net/can/grcan.c-1481- int ret; \
drivers/net/can/grcan.c:1482: if (dev->flags & IFF_UP) \
drivers/net/can/grcan.c-1483- return -EBUSY; \
--
drivers/net/can/m_can/m_can.c=788=static int m_can_get_berr_counter(const struct net_device *dev,
--
drivers/net/can/m_can/m_can.c-794- /* Avoid waking up the controller if the interface is down */
drivers/net/can/m_can/m_can.c:795: if (!(dev->flags & IFF_UP))
drivers/net/can/m_can/m_can.c-796- return 0;
--
drivers/net/can/sja1000/peak_pci.c=287=static void peak_pciec_led_work(struct work_struct *work)
--
drivers/net/can/sja1000/peak_pci.c-301- netdev = card->channel[i].netdev;
drivers/net/can/sja1000/peak_pci.c:302: if (!netdev || !(netdev->flags & IFF_UP))
drivers/net/can/sja1000/peak_pci.c-303- continue;
--
drivers/net/can/sja1000/peak_pcmcia.c=375=static void pcan_led_timer(struct timer_list *t)
--
drivers/net/can/sja1000/peak_pcmcia.c-388- netdev = card->channel[i].netdev;
drivers/net/can/sja1000/peak_pcmcia.c:389: if (!netdev || !(netdev->flags & IFF_UP))
drivers/net/can/sja1000/peak_pcmcia.c-390- continue;
--
drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c=859=static int mcp251xfd_get_berr_counter(const struct net_device *ndev,
--
drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c-864- /* Avoid waking up the controller if the interface is down */
drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c:865: if (!(ndev->flags & IFF_UP))
drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c-866- return 0;
--
drivers/net/can/usb/peak_usb/pcan_usb.c=403=static int pcan_usb_set_can_channel_id(struct peak_usb_device *dev, u32 can_ch_id)
--
drivers/net/can/usb/peak_usb/pcan_usb.c-413- */
drivers/net/can/usb/peak_usb/pcan_usb.c:414: if (dev->netdev->flags & IFF_UP)
drivers/net/can/usb/peak_usb/pcan_usb.c-415- return -EBUSY;
--
drivers/net/can/vcan.c=133=static int vcan_change_mtu(struct net_device *dev, int new_mtu)
--
drivers/net/can/vcan.c-135- /* Do not allow changing the MTU while running */
drivers/net/can/vcan.c:136: if (dev->flags & IFF_UP)
drivers/net/can/vcan.c-137- return -EBUSY;
--
drivers/net/can/vxcan.c=128=static int vxcan_change_mtu(struct net_device *dev, int new_mtu)
--
drivers/net/can/vxcan.c-130- /* Do not allow changing the MTU while running */
drivers/net/can/vxcan.c:131: if (dev->flags & IFF_UP)
drivers/net/can/vxcan.c-132- return -EBUSY;
--
drivers/net/eql.c=144=static void eql_timer(struct timer_list *t)
--
drivers/net/eql.c-153-
drivers/net/eql.c:154: if ((slave->dev->flags & IFF_UP) == IFF_UP) {
drivers/net/eql.c-155- slave->bytes_queued -= slave->priority_Bps;
--
drivers/net/eql.c=302=static slave_t *__eql_schedule_slaves(slave_queue_t *queue)
--
drivers/net/eql.c-320- priority_Bps = slave->priority_Bps;
drivers/net/eql.c:321: if ((slave->dev->flags & IFF_UP) == IFF_UP) {
drivers/net/eql.c-322- slave_load = (~0UL - (~0UL / 2)) -
--
drivers/net/eql.c=413=static int eql_enslave(struct net_device *master_dev, slaving_request_t __user *srqp)
--
drivers/net/eql.c-424-
drivers/net/eql.c:425: if ((master_dev->flags & IFF_UP) == IFF_UP) {
drivers/net/eql.c-426- /* slave is not a master & not already a slave: */
--
drivers/net/ethernet/freescale/gianfar.c=2002=static int gfar_change_mtu(struct net_device *dev, int new_mtu)
--
drivers/net/ethernet/freescale/gianfar.c-2008-
drivers/net/ethernet/freescale/gianfar.c:2009: if (dev->flags & IFF_UP)
drivers/net/ethernet/freescale/gianfar.c-2010- stop_gfar(dev);
--
drivers/net/ethernet/freescale/gianfar.c-2013-
drivers/net/ethernet/freescale/gianfar.c:2014: if (dev->flags & IFF_UP)
drivers/net/ethernet/freescale/gianfar.c-2015- startup_gfar(dev);
--
drivers/net/ethernet/freescale/gianfar_ethtool.c=288=static int gfar_scoalesce(struct net_device *dev,
--
drivers/net/ethernet/freescale/gianfar_ethtool.c-362-
drivers/net/ethernet/freescale/gianfar_ethtool.c:363: if (dev->flags & IFF_UP) {
drivers/net/ethernet/freescale/gianfar_ethtool.c-364- stop_gfar(dev);
--
drivers/net/ethernet/freescale/gianfar_ethtool.c=407=static int gfar_sringparam(struct net_device *dev,
--
drivers/net/ethernet/freescale/gianfar_ethtool.c-433-
drivers/net/ethernet/freescale/gianfar_ethtool.c:434: if (dev->flags & IFF_UP)
drivers/net/ethernet/freescale/gianfar_ethtool.c-435- stop_gfar(dev);
--
drivers/net/ethernet/freescale/gianfar_ethtool.c-444- /* Rebuild the rings with the new size */
drivers/net/ethernet/freescale/gianfar_ethtool.c:445: if (dev->flags & IFF_UP)
drivers/net/ethernet/freescale/gianfar_ethtool.c-446- err = startup_gfar(dev);
--
drivers/net/ethernet/freescale/gianfar_ethtool.c=512=int gfar_set_features(struct net_device *dev, netdev_features_t features)
--
drivers/net/ethernet/freescale/gianfar_ethtool.c-526-
drivers/net/ethernet/freescale/gianfar_ethtool.c:527: if (dev->flags & IFF_UP) {
drivers/net/ethernet/freescale/gianfar_ethtool.c-528- /* Now we take down the rings to rebuild them */
--
drivers/net/ethernet/freescale/ucc_geth.c=3236=static void ucc_geth_timeout_work(struct work_struct *work)
--
drivers/net/ethernet/freescale/ucc_geth.c-3249-
drivers/net/ethernet/freescale/ucc_geth.c:3250: if (dev->flags & IFF_UP) {
drivers/net/ethernet/freescale/ucc_geth.c-3251- /*
--
drivers/net/ethernet/ibm/ehea/ehea_main.c=2714=static void ehea_rereg_mrs(void)
--
drivers/net/ethernet/ibm/ehea/ehea_main.c-2732-
drivers/net/ethernet/ibm/ehea/ehea_main.c:2733: if (dev->flags & IFF_UP) {
drivers/net/ethernet/ibm/ehea/ehea_main.c-2734- mutex_lock(&port->port_lock);
--
drivers/net/ethernet/ibm/ehea/ehea_main.c-2773-
drivers/net/ethernet/ibm/ehea/ehea_main.c:2774: if (dev->flags & IFF_UP) {
drivers/net/ethernet/ibm/ehea/ehea_main.c-2775- mutex_lock(&port->port_lock);
--
drivers/net/ethernet/ibm/ibmveth.c=1117=static int ibmveth_set_channels(struct net_device *netdev,
--
drivers/net/ethernet/ibm/ibmveth.c-1127- */
drivers/net/ethernet/ibm/ibmveth.c:1128: if (!(netdev->flags & IFF_UP))
drivers/net/ethernet/ibm/ibmveth.c-1129- return netif_set_real_num_tx_queues(netdev, goal);
--
drivers/net/ethernet/intel/e1000e/netdev.c=7031=static int e1000e_pm_runtime_resume(struct device *dev)
--
drivers/net/ethernet/intel/e1000e/netdev.c-7043-
drivers/net/ethernet/intel/e1000e/netdev.c:7044: if (netdev->flags & IFF_UP)
drivers/net/ethernet/intel/e1000e/netdev.c-7045- e1000e_up(adapter);
--
drivers/net/ethernet/intel/e1000e/netdev.c=7050=static int e1000e_pm_runtime_suspend(struct device *dev)
--
drivers/net/ethernet/intel/e1000e/netdev.c-7055-
drivers/net/ethernet/intel/e1000e/netdev.c:7056: if (netdev->flags & IFF_UP) {
drivers/net/ethernet/intel/e1000e/netdev.c-7057- int count = E1000_CHECK_RESET_COUNT;
--
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c=971=static int fm10k_set_mac(struct net_device *dev, void *p)
--
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c-980-
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c:981: if (dev->flags & IFF_UP) {
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c-982- /* setting MAC address requires mailbox */
--
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c=1036=static void fm10k_set_rx_mode(struct net_device *dev)
--
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c-1042- /* no need to update the harwdare if we are not running */
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c:1043: if (!(dev->flags & IFF_UP))
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c-1044- return;
--
drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c=1845=static void otx2_do_set_rx_mode(struct otx2_nic *pf)
--
drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c-1850-
drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c:1851: if (!(netdev->flags & IFF_UP))
drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c-1852- return;
--
drivers/net/ethernet/marvell/octeontx2/nic/qos.c=992=static int otx2_qos_push_txschq_cfg(struct otx2_nic *pfvf,
--
drivers/net/ethernet/marvell/octeontx2/nic/qos.c-1005-
drivers/net/ethernet/marvell/octeontx2/nic/qos.c:1006: if (!(pfvf->netdev->flags & IFF_UP)) {
drivers/net/ethernet/marvell/octeontx2/nic/qos.c-1007- otx2_qos_txschq_fill_cfg(pfvf, node, cfg);
--
drivers/net/ethernet/marvell/octeontx2/nic/qos.c=1030=static int otx2_qos_root_add(struct otx2_nic *pfvf, u16 htb_maj_id, u16 htb_defcls,
--
drivers/net/ethernet/marvell/octeontx2/nic/qos.c-1068-
drivers/net/ethernet/marvell/octeontx2/nic/qos.c:1069: if (!(pfvf->netdev->flags & IFF_UP) ||
drivers/net/ethernet/marvell/octeontx2/nic/qos.c-1070- root->level == NIX_TXSCH_LVL_TL1) {
--
drivers/net/ethernet/mediatek/mtk_eth_soc.c=4961=void mtk_eth_set_dma_device(struct mtk_eth *eth, struct device *dma_dev)
--
drivers/net/ethernet/mediatek/mtk_eth_soc.c-4971-
drivers/net/ethernet/mediatek/mtk_eth_soc.c:4972: if (!dev || !(dev->flags & IFF_UP))
drivers/net/ethernet/mediatek/mtk_eth_soc.c-4973- continue;
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c=1787=int __mlxsw_sp_ipip_entry_update_tunnel(struct mlxsw_sp *mlxsw_sp,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-1814-
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c:1815: if (ipip_entry->ol_dev->flags & IFF_UP)
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-1816- mlxsw_sp_ipip_entry_ol_up_event(mlxsw_sp, ipip_entry);
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c=1913=void mlxsw_sp_ipip_entry_demote_tunnel(struct mlxsw_sp *mlxsw_sp,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-1917-
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c:1918: if (ol_dev->flags & IFF_UP)
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-1919- mlxsw_sp_ipip_entry_ol_down_event(mlxsw_sp, ipip_entry);
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c=4479=static bool mlxsw_sp_ipip_netdev_ul_up(struct net_device *ol_dev)
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-4485- ul_dev = mlxsw_sp_ipip_netdev_ul_dev_get(ol_dev);
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c:4486: is_up = ul_dev ? (ul_dev->flags & IFF_UP) : true;
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-4487- rcu_read_unlock();
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c=6397=mlxsw_sp_fib4_entry_type_set(struct mlxsw_sp *mlxsw_sp,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-6411- MLXSW_SP_L3_PROTO_IPV4, dip);
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c:6412: if (ipip_entry && ipip_entry->ol_dev->flags & IFF_UP) {
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-6413- fib_entry->type = MLXSW_SP_FIB_ENTRY_TYPE_IPIP_DECAP;
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c=7303=mlxsw_sp_fib6_entry_type_set_local(struct mlxsw_sp *mlxsw_sp,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-7318-
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c:7319: if (ipip_entry && ipip_entry->ol_dev->flags & IFF_UP) {
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-7320- fib_entry->type = MLXSW_SP_FIB_ENTRY_TYPE_IPIP_DECAP;
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c=354=mlxsw_sp_span_entry_tunnel_parms_common(struct net_device *edev,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-380- if (is_vlan_dev(edev)) {
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c:381: if (vid || !(edev->flags & IFF_UP))
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-382- goto unoffloadable;
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-386- if (netif_is_lag_master(edev)) {
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c:387: if (!(edev->flags & IFF_UP))
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-388- goto unoffloadable;
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c=449=mlxsw_sp_span_entry_gretap4_parms(struct mlxsw_sp *mlxsw_sp,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-460-
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c:461: if (!(to_dev->flags & IFF_UP) ||
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-462- /* Reject tunnels with GRE keys, checksums, etc. */
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c=554=mlxsw_sp_span_entry_gretap6_parms(struct mlxsw_sp *mlxsw_sp,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-565-
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c:566: if (!(to_dev->flags & IFF_UP) ||
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-567- /* Reject tunnels with GRE keys, checksums, etc. */
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c=632=mlxsw_sp_span_entry_vlan_parms(struct mlxsw_sp *mlxsw_sp,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-638-
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c:639: if (!(to_dev->flags & IFF_UP))
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-640- return mlxsw_sp_span_entry_unoffloadable(sparmsp);
--
drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c=1357=int nfp_tunnel_mac_event_handler(struct nfp_app *app,
--
drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c-1376- /* Only offload addr change if netdev is already up. */
drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c:1377: if (!(netdev->flags & IFF_UP))
drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c-1378- return NOTIFY_OK;
--
drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c-1415-
drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c:1416: if (!(netdev->flags & IFF_UP))
drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c-1417- return NOTIFY_OK;
--
drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c=689=static int nfp_test_link(struct net_device *netdev)
drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c-690-{
drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c:691: if (!netif_carrier_ok(netdev) || !(netdev->flags & IFF_UP))
drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c-692- return 1;
--
drivers/net/ethernet/pensando/ionic/ionic_lif.c=141=static void ionic_link_status_check(struct ionic_lif *lif)
--
drivers/net/ethernet/pensando/ionic/ionic_lif.c-161-
drivers/net/ethernet/pensando/ionic/ionic_lif.c:162: if (netdev->flags & IFF_UP && netif_running(netdev)) {
drivers/net/ethernet/pensando/ionic/ionic_lif.c-163- mutex_lock(&lif->queue_lock);
--
drivers/net/ethernet/pensando/ionic/ionic_lif.c-186-
drivers/net/ethernet/pensando/ionic/ionic_lif.c:187: if (netdev->flags & IFF_UP && netif_running(netdev)) {
drivers/net/ethernet/pensando/ionic/ionic_lif.c-188- mutex_lock(&lif->queue_lock);
--
drivers/net/ethernet/qualcomm/emac/emac.c=747=static void emac_shutdown(struct platform_device *pdev)
--
drivers/net/ethernet/qualcomm/emac/emac.c-751-
drivers/net/ethernet/qualcomm/emac/emac.c:752: if (netdev->flags & IFF_UP) {
drivers/net/ethernet/qualcomm/emac/emac.c-753- /* Closing the SGMII turns off its interrupts */
--
drivers/net/ethernet/rocker/rocker_ofdpa.c=2606=static int ofdpa_port_bridge_leave(struct ofdpa_port *ofdpa_port)
--
drivers/net/ethernet/rocker/rocker_ofdpa.c-2628-
drivers/net/ethernet/rocker/rocker_ofdpa.c:2629: if (ofdpa_port->dev->flags & IFF_UP)
drivers/net/ethernet/rocker/rocker_ofdpa.c-2630- err = ofdpa_port_fwd_enable(ofdpa_port, 0);
--
drivers/net/ethernet/sfc/ef100_rx.c=56=void __ef100_rx_packet(struct efx_channel *channel)
--
drivers/net/ethernet/sfc/ef100_rx.c-100- if (efv) {
drivers/net/ethernet/sfc/ef100_rx.c:101: if (efv->net_dev->flags & IFF_UP)
drivers/net/ethernet/sfc/ef100_rx.c-102- efx_ef100_rep_rx_packet(efv, rx_buf);
--
drivers/net/ethernet/sfc/ethtool_common.c=128=void efx_ethtool_self_test(struct net_device *net_dev,
--
drivers/net/ethernet/sfc/ethtool_common.c-148- /* We need rx buffers and interrupts. */
drivers/net/ethernet/sfc/ethtool_common.c:149: already_up = (efx->net_dev->flags & IFF_UP);
drivers/net/ethernet/sfc/ethtool_common.c-150- if (!already_up) {
--
drivers/net/ethernet/sfc/falcon/ethtool.c=488=static void ef4_ethtool_self_test(struct net_device *net_dev,
--
drivers/net/ethernet/sfc/falcon/ethtool.c-508- /* We need rx buffers and interrupts. */
drivers/net/ethernet/sfc/falcon/ethtool.c:509: already_up = (efx->net_dev->flags & IFF_UP);
drivers/net/ethernet/sfc/falcon/ethtool.c-510- if (!already_up) {
--
drivers/net/ethernet/sfc/nic.c=410=void efx_nic_fix_nodesc_drop_stat(struct efx_nic *efx, u64 *rx_nodesc_drops)
--
drivers/net/ethernet/sfc/nic.c-412- /* if down, or this is the first update after coming up */
drivers/net/ethernet/sfc/nic.c:413: if (!(efx->net_dev->flags & IFF_UP) || !efx->rx_nodesc_drops_prev_state)
drivers/net/ethernet/sfc/nic.c-414- efx->rx_nodesc_drops_while_down +=
--
drivers/net/ethernet/sfc/nic.c-416- efx->rx_nodesc_drops_total = *rx_nodesc_drops;
drivers/net/ethernet/sfc/nic.c:417: efx->rx_nodesc_drops_prev_state = !!(efx->net_dev->flags & IFF_UP);
drivers/net/ethernet/sfc/nic.c-418- *rx_nodesc_drops -= efx->rx_nodesc_drops_while_down;
--
drivers/net/ethernet/sfc/siena/ethtool_common.c=350=void efx_siena_ethtool_self_test(struct net_device *net_dev,
--
drivers/net/ethernet/sfc/siena/ethtool_common.c-370- /* We need rx buffers and interrupts. */
drivers/net/ethernet/sfc/siena/ethtool_common.c:371: already_up = (efx->net_dev->flags & IFF_UP);
drivers/net/ethernet/sfc/siena/ethtool_common.c-372- if (!already_up) {
--
drivers/net/ethernet/sfc/siena/nic.c=521=void efx_siena_fix_nodesc_drop_stat(struct efx_nic *efx, u64 *rx_nodesc_drops)
--
drivers/net/ethernet/sfc/siena/nic.c-523- /* if down, or this is the first update after coming up */
drivers/net/ethernet/sfc/siena/nic.c:524: if (!(efx->net_dev->flags & IFF_UP) || !efx->rx_nodesc_drops_prev_state)
drivers/net/ethernet/sfc/siena/nic.c-525- efx->rx_nodesc_drops_while_down +=
--
drivers/net/ethernet/sfc/siena/nic.c-527- efx->rx_nodesc_drops_total = *rx_nodesc_drops;
drivers/net/ethernet/sfc/siena/nic.c:528: efx->rx_nodesc_drops_prev_state = !!(efx->net_dev->flags & IFF_UP);
drivers/net/ethernet/sfc/siena/nic.c-529- *rx_nodesc_drops -= efx->rx_nodesc_drops_while_down;
--
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c=6404=static int stmmac_rings_status_show(struct seq_file *seq, void *v)
--
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c-6411-
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c:6412: if ((dev->flags & IFF_UP) == 0)
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c-6413- return 0;
--
drivers/net/ethernet/ti/cpsw.c=203=static int cpsw_update_vlan_mc(struct net_device *vdev, int vid, void *ctx)
--
drivers/net/ethernet/ti/cpsw.c-208-
drivers/net/ethernet/ti/cpsw.c:209: if (!vdev || !(vdev->flags & IFF_UP))
drivers/net/ethernet/ti/cpsw.c-210- return 0;
--
drivers/net/ethernet/ti/cpsw.c=268=static int cpsw_purge_vlan_mc(struct net_device *vdev, int vid, void *ctx)
--
drivers/net/ethernet/ti/cpsw.c-273-
drivers/net/ethernet/ti/cpsw.c:274: if (!vdev || !(vdev->flags & IFF_UP))
drivers/net/ethernet/ti/cpsw.c-275- return 0;
--
drivers/net/ethernet/ti/cpsw_new.c=146=static int cpsw_update_vlan_mc(struct net_device *vdev, int vid, void *ctx)
--
drivers/net/ethernet/ti/cpsw_new.c-151-
drivers/net/ethernet/ti/cpsw_new.c:152: if (!vdev || !(vdev->flags & IFF_UP))
drivers/net/ethernet/ti/cpsw_new.c-153- return 0;
--
drivers/net/ethernet/ti/cpsw_new.c=211=static int cpsw_purge_vlan_mc(struct net_device *vdev, int vid, void *ctx)
--
drivers/net/ethernet/ti/cpsw_new.c-216-
drivers/net/ethernet/ti/cpsw_new.c:217: if (!vdev || !(vdev->flags & IFF_UP))
drivers/net/ethernet/ti/cpsw_new.c-218- return 0;
--
drivers/net/ethernet/toshiba/ps3_gelic_net.c=1419=static void gelic_net_tx_timeout_task(struct work_struct *work)
--
drivers/net/ethernet/toshiba/ps3_gelic_net.c-1426-
drivers/net/ethernet/toshiba/ps3_gelic_net.c:1427: if (!(netdev->flags & IFF_UP))
drivers/net/ethernet/toshiba/ps3_gelic_net.c-1428- goto out;
--
drivers/net/ethernet/toshiba/ps3_gelic_net.c=1447=void gelic_net_tx_timeout(struct net_device *netdev, unsigned int txqueue)
--
drivers/net/ethernet/toshiba/ps3_gelic_net.c-1452- atomic_inc(&card->tx_timeout_task_counter);
]
]
|
| 133/2 |
2026/05/21 11:46 |
llm |
expert |
0m
Model:
Tokens:
input: 134794
output: 46
thoughts: 3331
Thoughts: **Analyzing Network Device Shutdown and Watchdog Timer Interactions**
Okay, so I'm trying to wrap my head around this shutdown sequence involving network devices and their watchdog timers. It seems like there's a potential race condition and I'm chasing down how the timer behaves with multiple CPUs and queue deactivation.
First, I confirmed that `usbnet` doesn't directly mess with `dev->flags`. Now, I'm back to scrutinizing `dev_deactivate_many`. This function seems key, since it iterates through each transmit queue and calls `dev_deactivate_queue`. `dev_deactivate_queue` sets the queue's Qdisc to `noop_qdisc` and then calls `netdev_watchdog_down`, which deletes the watchdog timer. The trouble is, if the timer is *currently* running, `timer_delete` doesn't wait!
If CPU A's running the timer, CPU B's `netdev_watchdog_down` will return immediately after calling `timer_delete`, potentially before the timer on CPU A is actually finished. Then, after all this, the system tries to shut down the device, and `dev_shutdown` checks `timer_pending`. If the timer is still running when this check happens, then the `WARN_ON` won't trigger!
My initial thoughts were that the timer wouldn't re-arm because CPU B would have set `noop_qdisc`. That's not right. CPU A *has* to hold the `tx_global_lock` while checking if the Qdisc is noop. If it does not, CPU B's `netdev_watchdog_down` would block until CPU A releases the lock, after re-arming the timer. In that scenario, `timer_delete` will delete it correctly.
Okay, let's zoom in on `netdev_watchdog_down` and the use of `netdev_put`. It calls `timer_delete`. If it deletes the timer successfully, it calls `netdev_put`. I also see that the watchdog timer, when it runs, *also* calls `netdev_put`. So, if the timer re-arms itself, it should *not* call `netdev_put`. This appears to be correct and handles the refcounting properly. However, if the `mod_timer` is called from the timer callback itself, it must always return 0, since it's not pending. I confirmed this to be true.
Alright, what about concurrent calls to `netdev_watchdog_up`? If `netdev_watchdog_up` runs *concurrently* with `dev_watchdog`? The upshot is that `netdev_watchdog_up` *also* holds the `tx_global_lock`. So, this will serialize access to the timer.
Now, I'm thinking about the final `dev_shutdown` check. What if the `netdev_watchdog_down` was *never* called? I already know if the `IFF_UP` flag is cleared, `netif_close_many` doesn't call `__dev_close_many`, which calls `netdev_watchdog_down`. If that happens, the timer *could* still be pending! But if the `IFF_UP` flag is cleared, then the qdisc *must* be the `noop_qdisc`! Therefore the timer cannot be armed!
Okay, let's step back and look at the whole picture. When does the system activate the device? It looks like `dev_activate` is called after the `IFF_UP` flag is set. That means the system should set the flag, and activate the device, before bringing it down.
What about `dev_deactivate_queue`? It deactivates the queues. I realized if the number of tx queues is zero, `dev_deactivate_queue` is *not* called. Is that the problem? No. The system refuses to allocate a device with zero queues.
Lastly, could `netdev_watchdog_up` be called from `netif_device_attach`? Yes. But, I checked `__netdev_watchdog_up`, and it looks like it's properly handled!
|
| 134/2 |
2026/05/21 11:46 |
tool |
codesearch-definition-source |
0m
Error:
Args: map[ContextFile:net/sched/sch_generic.c IncludeLines:true Name:__netdev_watchdog_up]
Results: map[SourceCode: SourceFile:]
|
| 135/2 |
2026/05/21 11:46 |
llm |
expert |
0m
Model:
Tokens:
input: 139106
output: 42
thoughts: 14
Thoughts: Okay, here's the summary, rendered as a first-person thought process assuming the user is a networking expert:
**Hmm, It's That Old Friend, `netdev_watchdog_up`**
Right, so I'm looking at this... and immediately my brain hones in on that familiar function, `netdev_watchdog_up`. Classic. *That* indicates the potential for a network interface watchdog event. I've seen this a thousand times. Instantly, I'm thinking: is this a stalled transmit queue? Driver issue? Resource contention? Or, is it perhaps a genuine hardware problem manifesting now? I need to quickly evaluate the context. What's the interface, what's been happening on the system recently, have there been any significant configuration changes? I need to dig into the surrounding logs and metrics... the usual suspects, right? Check the interface statistics for dropped packets, errors, and of course, the Tx queue length. I'll need to confirm if the watchdog is triggering persistently, or if this is a one-off. Classic... Let's see what's happening.
|
| 136/2 |
2026/05/21 11:47 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:net/core/dev.c IncludeLines:true Name:netif_device_attach]
Results: map[SourceCode:3470: /**
3471: * netif_device_attach - mark device as attached
3472: * @dev: network device
3473: *
3474: * Mark device as attached from system and restart if needed.
3475: */
3476: void netif_device_attach(struct net_device *dev)
3477: {
3478: if (!test_and_set_bit(__LINK_STATE_PRESENT, &dev->state) &&
3479: netif_running(dev)) {
3480: netif_tx_wake_all_queues(dev);
3481: netdev_watchdog_up(dev);
3482: }
3483: }
SourceFile:net/core/dev.c]
|
| 137/2 |
2026/05/21 11:47 |
llm |
expert |
0m
Model:
Tokens:
input: 139632
output: 25
thoughts: 6668
Thoughts: **Analyzing the Potential for a Pending Watchdog Timer**
Okay, let's break this down. We have this watchdog timer, and we're seeing warnings about it still being pending during shutdown. This shouldn't happen. `netif_device_attach` starts things, and then there's a dance with `netdev_watchdog_up`, and `netdev_watchdog_down` when things close. We also have `dev_deactivate_many` and `__dev_close_many` in the mix which also sets `noop_qdisc`.
My initial thought: could there be an ordering issue between `netif_device_attach` and `dev_deactivate_many` where the timer gets set *before* being cleared? But that assumes `netif_running(dev)` is set to TRUE, and that depends on the `__LINK_STATE_START` flag. This flag gets cleared during close. Let's trace it carefully. If things happen in the right order (attach *before* close), we have a pending timer. This requires that the timer is armed, and then deactivated. `netdev_watchdog_down` should then delete the timer. So we must ensure it is deleted.
Then we have `netif_carrier_on` which does the same things.
So, how can we *ensure* the timer is always deleted? We must look at the interactions with the `tx_global_lock`, and possible race conditions. `dev_watchdog` can start and call `mod_timer`. Now, if `dev_watchdog` is delayed and then `dev_deactivate_many` sets `noop_qdisc` and calls `netdev_watchdog_down`, which acquires `tx_global_lock` and deletes the timer. The question comes down to how and where the `tx_global_lock` is taken. If `dev_watchdog` is blocked on the lock, the timer is deleted.
So, `dev_deactivate_many` should always synchronize. Now, the problem I'm trying to get at here is: is the spinlock *always* held when `netdev_watchdog_down` is called? The answer is yes. `dev_deactivate_many` and `netdev_watchdog_down` synchronizes correctly.
Then, there could be a race condition when `timer_delete` is called, but the timer is running on another CPU. But the spinlock should protect against that, since the `dev_watchdog` MUST be blocked on it in its critical section, and therefore, it can't be in a state where it is setting the timer.
The more troubling case is: is the timer pending when it starts and has a race with the timer operation. It can't, because once `dev_deactivate_many` has finished then the timer will *not* have been set. If it is running it is already in the lock, it will have checked that `!qdisc_tx_is_noop(dev)` and returned false, and therefore, it won't set a pending timer.
Now, a critical thought, where does this warning come from? It's from `dev_shutdown`, which happens after the device is closed, when things are getting torn down and unhooked. This function should *not* see a pending timer. The key question is: is `dev_shutdown` always called *after* `dev_deactivate_many` and other parts of the close process? This is where it gets interesting... The answer is NO.
If `IFF_UP` is false, `unregister_netdevice_many_notify` will skip calling `netif_close_many`.
So, the warning we're seeing *must* be because `netdev_watchdog_down` and therefore `dev_deactivate_many` and potentially other relevant functions are *not* called at all because `IFF_UP` is false. This leaves the timer running.
But... if `IFF_UP` is false, how could the timer be pending? With `IFF_UP` false, the `qdisc` is `noop_qdisc`. `netdev_watchdog_up` or `dev_watchdog` will *not* arm the timer, but we already covered that. The problem, as is often the case, lies in an *invariant* which isn't always true.
So, *how* can we get into the situation where `IFF_UP` is false, but there's a pending timer? If `IFF_UP` is false, the close process doesn't run, leaving the timer pending. Is there some path where the device comes up, then shuts down without going through the complete close-down sequence that would delete the timer? That must
|
| 138/2 |
2026/05/21 11:47 |
tool |
grepper |
0m
Error:
Args: map[Expression:rcu_assign_pointer.*qdisc.*NULL]
Results:
|
| 139/2 |
2026/05/21 11:47 |
llm |
expert |
0m
Model:
Tokens:
input: 147364
output: 18
thoughts: 11
Thoughts: Okay, here's my summarized thought process, assuming I'm an expert and this is a deep dive into network configuration or kernel internals:
**Investigating a Network Queueing Issue**
Alright, I'm diving into this network performance issue. My initial thought is to quickly identify where a queueing discipline might be uninitialized, or perhaps unexpectedly cleared. The most efficient way to do that is to grep for "qdisc = NULL". That's a classic sign of a problem – a queueing discipline, which is fundamental to how packets are shaped and prioritized, shouldn't be null unless something went terribly wrong during initialization or a deallocation process. The `grep` should provide a targeted search through the relevant code, configuration files, or log files. I'm expecting this will narrow down the scope of the problem considerably, or point me in the right direction. From there, I can analyze the surrounding code to understand *why* `qdisc` is NULL and formulate hypotheses for testing.
|
| 140/2 |
2026/05/21 11:48 |
tool |
grepper |
0m
Args: map[Expression:qdisc = NULL]
Results: map[Output:drivers/net/ethernet/netronome/nfp/abm/qdisc.c=404=nfp_abm_qdisc_destroy(struct net_device *netdev, struct nfp_abm_link *alink,
--
drivers/net/ethernet/netronome/nfp/abm/qdisc.c-420- if (alink->root_qdisc == qdisc) {
drivers/net/ethernet/netronome/nfp/abm/qdisc.c:421: alink->root_qdisc = NULL;
drivers/net/ethernet/netronome/nfp/abm/qdisc.c-422- /* Only root change matters, other changes are acted upon on
--
net/sched/sch_ets.c=581=static int ets_qdisc_change(struct Qdisc *sch, struct nlattr *opt,
--
net/sched/sch_ets.c-683- qdisc_put(q->classes[i].qdisc);
net/sched/sch_ets.c:684: q->classes[i].qdisc = NULL;
net/sched/sch_ets.c-685- WRITE_ONCE(q->classes[i].quantum, 0);
--
net/sched/sch_htb.c=1759=static int htb_change_class(struct Qdisc *sch, u32 classid,
--
net/sched/sch_htb.c-1768- struct nlattr *tb[TCA_HTB_MAX + 1];
net/sched/sch_htb.c:1769: struct Qdisc *parent_qdisc = NULL;
net/sched/sch_htb.c-1770- struct netdev_queue *dev_queue;
]
|
| 141/2 |
2026/05/21 11:48 |
llm |
expert |
0m
Model:
Tokens:
input: 148044
output: 21
thoughts: 555
Thoughts: **Analyzing Potential Network Device Queue State Issues**
Okay, so I'm digging into this network device queue (`dev_queue->qdisc`) state, and my initial assumption is that it should *never* be null. But then I started to think, what if we have a scenario where `dev_deactivate_many` *isn't* called, but `dev_shutdown` is? Specifically, if `IFF_UP` is false, `dev_deactivate_many` doesn't run. In that situation, `qdisc` *should* be set to `noop_qdisc`.
The core question then becomes: is there *any* edge case where `IFF_UP` is false, and yet the associated `qdisc` is *not* `noop_qdisc`? What about `dev_activate` being called, but `IFF_UP` not being set? But, hold on... the code ensures that `IFF_UP` is set *before* `dev_activate`. Okay, that's reassuring.
Now, what about `dev_change_flags`? What if it clears `IFF_UP` but `__dev_close` *isn't* called? But the code shows that `__dev_close` *is* called. That's good.
Next, what if `dev_change_flags` sets `IFF_UP`, but `__dev_open` *fails*? If `__dev_open` fails, it returns an error. Does it then clear `IFF_UP`? No, it doesn't. `__dev_open` doesn't set `IFF_UP` if it fails, which means that the `IFF_UP` flag will not be set.
But here's a potential weak point: what if a device driver modifies `dev->flags` directly, bypassing the usual control paths? I need to check for that. I ran a grep for `dev->flags.*IFF_UP`. The results show that the `usbnet` driver doesn't seem to touch it directly. But... wait! The reproducer uses `usb_raw_gadget`, which means it's simulating a USB device. The driver in question is either `net1080` or `usbnet`. I need to specifically check if `net1080` modifies `dev->flags` and `IFF_UP` directly.
|
| 142/2 |
2026/05/21 11:48 |
tool |
grepper |
0m
Args: map[Expression:dev->flags.*IFF_UP]
Results: map[Output:
Full output is too long, showing 500 out of 1679 lines.
Use more precise expression if possible.
[drivers/infiniband/hw/irdma/cm.c=1656=static int irdma_add_mqh_6(struct irdma_device *iwdev,
--
drivers/infiniband/hw/irdma/cm.c-1668- for_each_netdev(&init_net, ip_dev) {
drivers/infiniband/hw/irdma/cm.c:1669: if (!(ip_dev->flags & IFF_UP))
drivers/infiniband/hw/irdma/cm.c-1670- continue;
--
drivers/infiniband/hw/irdma/cm.c=1744=static int irdma_add_mqh_4(struct irdma_device *iwdev,
--
drivers/infiniband/hw/irdma/cm.c-1756- for_each_netdev(&init_net, ip_dev) {
drivers/infiniband/hw/irdma/cm.c:1757: if (!(ip_dev->flags & IFF_UP))
drivers/infiniband/hw/irdma/cm.c-1758- continue;
--
drivers/infiniband/hw/irdma/utils.c=335=static void irdma_add_ipv6_addr(struct irdma_device *iwdev)
--
drivers/infiniband/hw/irdma/utils.c-346- ip_dev == iwdev->netdev) &&
drivers/infiniband/hw/irdma/utils.c:347: (READ_ONCE(ip_dev->flags) & IFF_UP)) {
drivers/infiniband/hw/irdma/utils.c-348- idev = __in6_dev_get(ip_dev);
--
drivers/infiniband/hw/irdma/utils.c=377=static void irdma_add_ipv4_addr(struct irdma_device *iwdev)
--
drivers/infiniband/hw/irdma/utils.c-386- rdma_vlan_dev_real_dev(dev) == iwdev->netdev) ||
drivers/infiniband/hw/irdma/utils.c:387: dev == iwdev->netdev) && (READ_ONCE(dev->flags) & IFF_UP)) {
drivers/infiniband/hw/irdma/utils.c-388- const struct in_ifaddr *ifa;
--
drivers/infiniband/ulp/ipoib/ipoib_main.c=164=static void ipoib_schedule_ifupdown_task(struct net_device *dev, bool up)
--
drivers/infiniband/ulp/ipoib/ipoib_main.c-167-
drivers/infiniband/ulp/ipoib/ipoib_main.c:168: if ((up && (dev->flags & IFF_UP)) ||
drivers/infiniband/ulp/ipoib/ipoib_main.c:169: (!up && !(dev->flags & IFF_UP)))
drivers/infiniband/ulp/ipoib/ipoib_main.c-170- return;
--
drivers/infiniband/ulp/ipoib/ipoib_main.c=1853=static int ipoib_dev_init(struct net_device *dev)
--
drivers/infiniband/ulp/ipoib/ipoib_main.c-1888-
drivers/infiniband/ulp/ipoib/ipoib_main.c:1889: if (dev->flags & IFF_UP) {
drivers/infiniband/ulp/ipoib/ipoib_main.c-1890- if (ipoib_ib_dev_open(dev)) {
--
drivers/infiniband/ulp/ipoib/ipoib_main.c=1925=static void ipoib_parent_unregister_pre(struct net_device *ndev)
--
drivers/infiniband/ulp/ipoib/ipoib_main.c-1933- rtnl_lock();
drivers/infiniband/ulp/ipoib/ipoib_main.c:1934: dev_change_flags(priv->dev, priv->dev->flags & ~IFF_UP, NULL);
drivers/infiniband/ulp/ipoib/ipoib_main.c-1935- rtnl_unlock();
--
drivers/infiniband/ulp/opa_vnic/opa_vnic_netdev.c=134=void opa_vnic_process_vema_config(struct opa_vnic_adapter *adapter)
--
drivers/infiniband/ulp/opa_vnic/opa_vnic_netdev.c-185- /* update state */
drivers/infiniband/ulp/opa_vnic/opa_vnic_netdev.c:186: opa_vnic_update_state(adapter, !!(netdev->flags & IFF_UP));
drivers/infiniband/ulp/opa_vnic/opa_vnic_netdev.c-187-}
--
drivers/net/arcnet/com20020.c=372=static void com20020_set_mc_list(struct net_device *dev)
--
drivers/net/arcnet/com20020.c-376-
drivers/net/arcnet/com20020.c:377: if ((dev->flags & IFF_PROMISC) && (dev->flags & IFF_UP)) {
drivers/net/arcnet/com20020.c-378- /* Enable promiscuous mode */
--
drivers/net/bonding/bond_main.c=922=static void bond_hw_addr_swap(struct bonding *bond, struct slave *new_active,
--
drivers/net/bonding/bond_main.c-931-
drivers/net/bonding/bond_main.c:932: if (bond->dev->flags & IFF_UP)
drivers/net/bonding/bond_main.c-933- bond_hw_addr_flush(bond->dev, old_active->dev);
--
drivers/net/bonding/bond_main.c-945-
drivers/net/bonding/bond_main.c:946: if (bond->dev->flags & IFF_UP) {
drivers/net/bonding/bond_main.c-947- netif_addr_lock_bh(bond->dev);
--
drivers/net/bonding/bond_main.c=1471=static void bond_setup_by_slave(struct net_device *bond_dev,
--
drivers/net/bonding/bond_main.c-1473-{
drivers/net/bonding/bond_main.c:1474: bool was_up = !!(bond_dev->flags & IFF_UP);
drivers/net/bonding/bond_main.c-1475-
--
drivers/net/bonding/bond_main.c=1767=static void bond_ether_setup(struct net_device *bond_dev)
drivers/net/bonding/bond_main.c-1768-{
drivers/net/bonding/bond_main.c:1769: unsigned int flags = bond_dev->flags & (IFF_SLAVE | IFF_UP);
drivers/net/bonding/bond_main.c-1770-
--
drivers/net/bonding/bond_main.c=1799=int bond_enslave(struct net_device *bond_dev, struct net_device *slave_dev,
--
drivers/net/bonding/bond_main.c-1849- */
drivers/net/bonding/bond_main.c:1850: if (slave_dev->flags & IFF_UP) {
drivers/net/bonding/bond_main.c-1851- SLAVE_NL_ERR(bond_dev, slave_dev, extack,
--
drivers/net/bonding/bond_main.c-2172-
drivers/net/bonding/bond_main.c:2173: if (bond_dev->flags & IFF_UP) {
drivers/net/bonding/bond_main.c-2174- netif_addr_lock_bh(bond_dev);
--
drivers/net/bonding/bond_options.c=679=static int bond_opt_check_deps(struct bonding *bond,
--
drivers/net/bonding/bond_options.c-687- return -ENOTEMPTY;
drivers/net/bonding/bond_options.c:688: if ((opt->flags & BOND_OPTFLAG_IFDOWN) && (bond->dev->flags & IFF_UP))
drivers/net/bonding/bond_options.c-689- return -EBUSY;
--
drivers/net/bonding/bond_options.c=1007=static int bond_option_miimon_set(struct bonding *bond,
--
drivers/net/bonding/bond_options.c-1027- }
drivers/net/bonding/bond_options.c:1028: if (bond->dev->flags & IFF_UP) {
drivers/net/bonding/bond_options.c-1029- /* If the interface is up, we may need to fire off
--
drivers/net/bonding/bond_options.c=1110=static int bond_option_arp_interval_set(struct bonding *bond,
--
drivers/net/bonding/bond_options.c-1123- }
drivers/net/bonding/bond_options.c:1124: if (bond->dev->flags & IFF_UP) {
drivers/net/bonding/bond_options.c-1125- /* If the interface is up, we may need to fire off
--
drivers/net/bonding/bond_options.c=1893=static int bond_option_broadcast_neigh_set(struct bonding *bond,
--
drivers/net/bonding/bond_options.c-1899- bond->params.broadcast_neighbor = newval->value;
drivers/net/bonding/bond_options.c:1900: if (bond->dev->flags & IFF_UP) {
drivers/net/bonding/bond_options.c-1901- if (bond->params.broadcast_neighbor)
--
drivers/net/can/at91_can.c=968=static ssize_t mb0_id_store(struct device *dev,
--
drivers/net/can/at91_can.c-979-
drivers/net/can/at91_can.c:980: if (ndev->flags & IFF_UP) {
drivers/net/can/at91_can.c-981- ret = -EBUSY;
--
drivers/net/can/c_can/c_can_main.c=1267=int c_can_power_down(struct net_device *dev)
--
drivers/net/can/c_can/c_can_main.c-1272-
drivers/net/can/c_can/c_can_main.c:1273: if (!(dev->flags & IFF_UP))
drivers/net/can/c_can/c_can_main.c-1274- return 0;
--
drivers/net/can/c_can/c_can_main.c=1301=int c_can_power_up(struct net_device *dev)
--
drivers/net/can/c_can/c_can_main.c-1307-
drivers/net/can/c_can/c_can_main.c:1308: if (!(dev->flags & IFF_UP))
drivers/net/can/c_can/c_can_main.c-1309- return 0;
--
drivers/net/can/dev/netlink.c=304=static int can_ctrlmode_changelink(struct net_device *dev,
--
drivers/net/can/dev/netlink.c-315- /* Do not allow changing controller mode while running */
drivers/net/can/dev/netlink.c:316: if (dev->flags & IFF_UP)
drivers/net/can/dev/netlink.c-317- return -EBUSY;
--
drivers/net/can/dev/netlink.c=435=static int can_dbt_changelink(struct net_device *dev, struct nlattr *data[],
--
drivers/net/can/dev/netlink.c-461- /* Do not allow changing bittiming while running */
drivers/net/can/dev/netlink.c:462: if (dev->flags & IFF_UP)
drivers/net/can/dev/netlink.c-463- return -EBUSY;
--
drivers/net/can/dev/netlink.c=593=static int can_changelink(struct net_device *dev, struct nlattr *tb[],
--
drivers/net/can/dev/netlink.c-608- /* Do not allow changing bittiming while running */
drivers/net/can/dev/netlink.c:609: if (dev->flags & IFF_UP)
drivers/net/can/dev/netlink.c-610- return -EBUSY;
--
drivers/net/can/dev/netlink.c-656- /* Do not allow changing restart delay while running */
drivers/net/can/dev/netlink.c:657: if (dev->flags & IFF_UP)
drivers/net/can/dev/netlink.c-658- return -EBUSY;
--
drivers/net/can/dev/netlink.c-669- /* Do not allow a restart while not running */
drivers/net/can/dev/netlink.c:670: if (!(dev->flags & IFF_UP))
drivers/net/can/dev/netlink.c-671- return -EINVAL;
--
drivers/net/can/grcan.c=1335=static netdev_tx_t grcan_start_xmit(struct sk_buff *skb,
--
drivers/net/can/grcan.c-1481- int ret; \
drivers/net/can/grcan.c:1482: if (dev->flags & IFF_UP) \
drivers/net/can/grcan.c-1483- return -EBUSY; \
--
drivers/net/can/m_can/m_can.c=788=static int m_can_get_berr_counter(const struct net_device *dev,
--
drivers/net/can/m_can/m_can.c-794- /* Avoid waking up the controller if the interface is down */
drivers/net/can/m_can/m_can.c:795: if (!(dev->flags & IFF_UP))
drivers/net/can/m_can/m_can.c-796- return 0;
--
drivers/net/can/sja1000/peak_pci.c=287=static void peak_pciec_led_work(struct work_struct *work)
--
drivers/net/can/sja1000/peak_pci.c-301- netdev = card->channel[i].netdev;
drivers/net/can/sja1000/peak_pci.c:302: if (!netdev || !(netdev->flags & IFF_UP))
drivers/net/can/sja1000/peak_pci.c-303- continue;
--
drivers/net/can/sja1000/peak_pcmcia.c=375=static void pcan_led_timer(struct timer_list *t)
--
drivers/net/can/sja1000/peak_pcmcia.c-388- netdev = card->channel[i].netdev;
drivers/net/can/sja1000/peak_pcmcia.c:389: if (!netdev || !(netdev->flags & IFF_UP))
drivers/net/can/sja1000/peak_pcmcia.c-390- continue;
--
drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c=859=static int mcp251xfd_get_berr_counter(const struct net_device *ndev,
--
drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c-864- /* Avoid waking up the controller if the interface is down */
drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c:865: if (!(ndev->flags & IFF_UP))
drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c-866- return 0;
--
drivers/net/can/usb/peak_usb/pcan_usb.c=403=static int pcan_usb_set_can_channel_id(struct peak_usb_device *dev, u32 can_ch_id)
--
drivers/net/can/usb/peak_usb/pcan_usb.c-413- */
drivers/net/can/usb/peak_usb/pcan_usb.c:414: if (dev->netdev->flags & IFF_UP)
drivers/net/can/usb/peak_usb/pcan_usb.c-415- return -EBUSY;
--
drivers/net/can/vcan.c=133=static int vcan_change_mtu(struct net_device *dev, int new_mtu)
--
drivers/net/can/vcan.c-135- /* Do not allow changing the MTU while running */
drivers/net/can/vcan.c:136: if (dev->flags & IFF_UP)
drivers/net/can/vcan.c-137- return -EBUSY;
--
drivers/net/can/vxcan.c=128=static int vxcan_change_mtu(struct net_device *dev, int new_mtu)
--
drivers/net/can/vxcan.c-130- /* Do not allow changing the MTU while running */
drivers/net/can/vxcan.c:131: if (dev->flags & IFF_UP)
drivers/net/can/vxcan.c-132- return -EBUSY;
--
drivers/net/eql.c=144=static void eql_timer(struct timer_list *t)
--
drivers/net/eql.c-153-
drivers/net/eql.c:154: if ((slave->dev->flags & IFF_UP) == IFF_UP) {
drivers/net/eql.c-155- slave->bytes_queued -= slave->priority_Bps;
--
drivers/net/eql.c=302=static slave_t *__eql_schedule_slaves(slave_queue_t *queue)
--
drivers/net/eql.c-320- priority_Bps = slave->priority_Bps;
drivers/net/eql.c:321: if ((slave->dev->flags & IFF_UP) == IFF_UP) {
drivers/net/eql.c-322- slave_load = (~0UL - (~0UL / 2)) -
--
drivers/net/eql.c=413=static int eql_enslave(struct net_device *master_dev, slaving_request_t __user *srqp)
--
drivers/net/eql.c-424-
drivers/net/eql.c:425: if ((master_dev->flags & IFF_UP) == IFF_UP) {
drivers/net/eql.c-426- /* slave is not a master & not already a slave: */
--
drivers/net/ethernet/freescale/gianfar.c=2002=static int gfar_change_mtu(struct net_device *dev, int new_mtu)
--
drivers/net/ethernet/freescale/gianfar.c-2008-
drivers/net/ethernet/freescale/gianfar.c:2009: if (dev->flags & IFF_UP)
drivers/net/ethernet/freescale/gianfar.c-2010- stop_gfar(dev);
--
drivers/net/ethernet/freescale/gianfar.c-2013-
drivers/net/ethernet/freescale/gianfar.c:2014: if (dev->flags & IFF_UP)
drivers/net/ethernet/freescale/gianfar.c-2015- startup_gfar(dev);
--
drivers/net/ethernet/freescale/gianfar_ethtool.c=288=static int gfar_scoalesce(struct net_device *dev,
--
drivers/net/ethernet/freescale/gianfar_ethtool.c-362-
drivers/net/ethernet/freescale/gianfar_ethtool.c:363: if (dev->flags & IFF_UP) {
drivers/net/ethernet/freescale/gianfar_ethtool.c-364- stop_gfar(dev);
--
drivers/net/ethernet/freescale/gianfar_ethtool.c=407=static int gfar_sringparam(struct net_device *dev,
--
drivers/net/ethernet/freescale/gianfar_ethtool.c-433-
drivers/net/ethernet/freescale/gianfar_ethtool.c:434: if (dev->flags & IFF_UP)
drivers/net/ethernet/freescale/gianfar_ethtool.c-435- stop_gfar(dev);
--
drivers/net/ethernet/freescale/gianfar_ethtool.c-444- /* Rebuild the rings with the new size */
drivers/net/ethernet/freescale/gianfar_ethtool.c:445: if (dev->flags & IFF_UP)
drivers/net/ethernet/freescale/gianfar_ethtool.c-446- err = startup_gfar(dev);
--
drivers/net/ethernet/freescale/gianfar_ethtool.c=512=int gfar_set_features(struct net_device *dev, netdev_features_t features)
--
drivers/net/ethernet/freescale/gianfar_ethtool.c-526-
drivers/net/ethernet/freescale/gianfar_ethtool.c:527: if (dev->flags & IFF_UP) {
drivers/net/ethernet/freescale/gianfar_ethtool.c-528- /* Now we take down the rings to rebuild them */
--
drivers/net/ethernet/freescale/ucc_geth.c=3236=static void ucc_geth_timeout_work(struct work_struct *work)
--
drivers/net/ethernet/freescale/ucc_geth.c-3249-
drivers/net/ethernet/freescale/ucc_geth.c:3250: if (dev->flags & IFF_UP) {
drivers/net/ethernet/freescale/ucc_geth.c-3251- /*
--
drivers/net/ethernet/ibm/ehea/ehea_main.c=2714=static void ehea_rereg_mrs(void)
--
drivers/net/ethernet/ibm/ehea/ehea_main.c-2732-
drivers/net/ethernet/ibm/ehea/ehea_main.c:2733: if (dev->flags & IFF_UP) {
drivers/net/ethernet/ibm/ehea/ehea_main.c-2734- mutex_lock(&port->port_lock);
--
drivers/net/ethernet/ibm/ehea/ehea_main.c-2773-
drivers/net/ethernet/ibm/ehea/ehea_main.c:2774: if (dev->flags & IFF_UP) {
drivers/net/ethernet/ibm/ehea/ehea_main.c-2775- mutex_lock(&port->port_lock);
--
drivers/net/ethernet/ibm/ibmveth.c=1117=static int ibmveth_set_channels(struct net_device *netdev,
--
drivers/net/ethernet/ibm/ibmveth.c-1127- */
drivers/net/ethernet/ibm/ibmveth.c:1128: if (!(netdev->flags & IFF_UP))
drivers/net/ethernet/ibm/ibmveth.c-1129- return netif_set_real_num_tx_queues(netdev, goal);
--
drivers/net/ethernet/intel/e1000e/netdev.c=7031=static int e1000e_pm_runtime_resume(struct device *dev)
--
drivers/net/ethernet/intel/e1000e/netdev.c-7043-
drivers/net/ethernet/intel/e1000e/netdev.c:7044: if (netdev->flags & IFF_UP)
drivers/net/ethernet/intel/e1000e/netdev.c-7045- e1000e_up(adapter);
--
drivers/net/ethernet/intel/e1000e/netdev.c=7050=static int e1000e_pm_runtime_suspend(struct device *dev)
--
drivers/net/ethernet/intel/e1000e/netdev.c-7055-
drivers/net/ethernet/intel/e1000e/netdev.c:7056: if (netdev->flags & IFF_UP) {
drivers/net/ethernet/intel/e1000e/netdev.c-7057- int count = E1000_CHECK_RESET_COUNT;
--
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c=971=static int fm10k_set_mac(struct net_device *dev, void *p)
--
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c-980-
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c:981: if (dev->flags & IFF_UP) {
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c-982- /* setting MAC address requires mailbox */
--
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c=1036=static void fm10k_set_rx_mode(struct net_device *dev)
--
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c-1042- /* no need to update the harwdare if we are not running */
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c:1043: if (!(dev->flags & IFF_UP))
drivers/net/ethernet/intel/fm10k/fm10k_netdev.c-1044- return;
--
drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c=1845=static void otx2_do_set_rx_mode(struct otx2_nic *pf)
--
drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c-1850-
drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c:1851: if (!(netdev->flags & IFF_UP))
drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c-1852- return;
--
drivers/net/ethernet/marvell/octeontx2/nic/qos.c=992=static int otx2_qos_push_txschq_cfg(struct otx2_nic *pfvf,
--
drivers/net/ethernet/marvell/octeontx2/nic/qos.c-1005-
drivers/net/ethernet/marvell/octeontx2/nic/qos.c:1006: if (!(pfvf->netdev->flags & IFF_UP)) {
drivers/net/ethernet/marvell/octeontx2/nic/qos.c-1007- otx2_qos_txschq_fill_cfg(pfvf, node, cfg);
--
drivers/net/ethernet/marvell/octeontx2/nic/qos.c=1030=static int otx2_qos_root_add(struct otx2_nic *pfvf, u16 htb_maj_id, u16 htb_defcls,
--
drivers/net/ethernet/marvell/octeontx2/nic/qos.c-1068-
drivers/net/ethernet/marvell/octeontx2/nic/qos.c:1069: if (!(pfvf->netdev->flags & IFF_UP) ||
drivers/net/ethernet/marvell/octeontx2/nic/qos.c-1070- root->level == NIX_TXSCH_LVL_TL1) {
--
drivers/net/ethernet/mediatek/mtk_eth_soc.c=4961=void mtk_eth_set_dma_device(struct mtk_eth *eth, struct device *dma_dev)
--
drivers/net/ethernet/mediatek/mtk_eth_soc.c-4971-
drivers/net/ethernet/mediatek/mtk_eth_soc.c:4972: if (!dev || !(dev->flags & IFF_UP))
drivers/net/ethernet/mediatek/mtk_eth_soc.c-4973- continue;
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c=1787=int __mlxsw_sp_ipip_entry_update_tunnel(struct mlxsw_sp *mlxsw_sp,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-1814-
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c:1815: if (ipip_entry->ol_dev->flags & IFF_UP)
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-1816- mlxsw_sp_ipip_entry_ol_up_event(mlxsw_sp, ipip_entry);
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c=1913=void mlxsw_sp_ipip_entry_demote_tunnel(struct mlxsw_sp *mlxsw_sp,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-1917-
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c:1918: if (ol_dev->flags & IFF_UP)
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-1919- mlxsw_sp_ipip_entry_ol_down_event(mlxsw_sp, ipip_entry);
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c=4479=static bool mlxsw_sp_ipip_netdev_ul_up(struct net_device *ol_dev)
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-4485- ul_dev = mlxsw_sp_ipip_netdev_ul_dev_get(ol_dev);
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c:4486: is_up = ul_dev ? (ul_dev->flags & IFF_UP) : true;
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-4487- rcu_read_unlock();
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c=6397=mlxsw_sp_fib4_entry_type_set(struct mlxsw_sp *mlxsw_sp,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-6411- MLXSW_SP_L3_PROTO_IPV4, dip);
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c:6412: if (ipip_entry && ipip_entry->ol_dev->flags & IFF_UP) {
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-6413- fib_entry->type = MLXSW_SP_FIB_ENTRY_TYPE_IPIP_DECAP;
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c=7303=mlxsw_sp_fib6_entry_type_set_local(struct mlxsw_sp *mlxsw_sp,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-7318-
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c:7319: if (ipip_entry && ipip_entry->ol_dev->flags & IFF_UP) {
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c-7320- fib_entry->type = MLXSW_SP_FIB_ENTRY_TYPE_IPIP_DECAP;
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c=354=mlxsw_sp_span_entry_tunnel_parms_common(struct net_device *edev,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-380- if (is_vlan_dev(edev)) {
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c:381: if (vid || !(edev->flags & IFF_UP))
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-382- goto unoffloadable;
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-386- if (netif_is_lag_master(edev)) {
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c:387: if (!(edev->flags & IFF_UP))
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-388- goto unoffloadable;
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c=449=mlxsw_sp_span_entry_gretap4_parms(struct mlxsw_sp *mlxsw_sp,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-460-
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c:461: if (!(to_dev->flags & IFF_UP) ||
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-462- /* Reject tunnels with GRE keys, checksums, etc. */
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c=554=mlxsw_sp_span_entry_gretap6_parms(struct mlxsw_sp *mlxsw_sp,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-565-
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c:566: if (!(to_dev->flags & IFF_UP) ||
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-567- /* Reject tunnels with GRE keys, checksums, etc. */
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c=632=mlxsw_sp_span_entry_vlan_parms(struct mlxsw_sp *mlxsw_sp,
--
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-638-
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c:639: if (!(to_dev->flags & IFF_UP))
drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c-640- return mlxsw_sp_span_entry_unoffloadable(sparmsp);
--
drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c=1357=int nfp_tunnel_mac_event_handler(struct nfp_app *app,
--
drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c-1376- /* Only offload addr change if netdev is already up. */
drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c:1377: if (!(netdev->flags & IFF_UP))
drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c-1378- return NOTIFY_OK;
--
drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c-1415-
drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c:1416: if (!(netdev->flags & IFF_UP))
drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c-1417- return NOTIFY_OK;
--
drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c=689=static int nfp_test_link(struct net_device *netdev)
drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c-690-{
drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c:691: if (!netif_carrier_ok(netdev) || !(netdev->flags & IFF_UP))
drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c-692- return 1;
--
drivers/net/ethernet/pensando/ionic/ionic_lif.c=141=static void ionic_link_status_check(struct ionic_lif *lif)
--
drivers/net/ethernet/pensando/ionic/ionic_lif.c-161-
drivers/net/ethernet/pensando/ionic/ionic_lif.c:162: if (netdev->flags & IFF_UP && netif_running(netdev)) {
drivers/net/ethernet/pensando/ionic/ionic_lif.c-163- mutex_lock(&lif->queue_lock);
--
drivers/net/ethernet/pensando/ionic/ionic_lif.c-186-
drivers/net/ethernet/pensando/ionic/ionic_lif.c:187: if (netdev->flags & IFF_UP && netif_running(netdev)) {
drivers/net/ethernet/pensando/ionic/ionic_lif.c-188- mutex_lock(&lif->queue_lock);
--
drivers/net/ethernet/qualcomm/emac/emac.c=747=static void emac_shutdown(struct platform_device *pdev)
--
drivers/net/ethernet/qualcomm/emac/emac.c-751-
drivers/net/ethernet/qualcomm/emac/emac.c:752: if (netdev->flags & IFF_UP) {
drivers/net/ethernet/qualcomm/emac/emac.c-753- /* Closing the SGMII turns off its interrupts */
--
drivers/net/ethernet/rocker/rocker_ofdpa.c=2606=static int ofdpa_port_bridge_leave(struct ofdpa_port *ofdpa_port)
--
drivers/net/ethernet/rocker/rocker_ofdpa.c-2628-
drivers/net/ethernet/rocker/rocker_ofdpa.c:2629: if (ofdpa_port->dev->flags & IFF_UP)
drivers/net/ethernet/rocker/rocker_ofdpa.c-2630- err = ofdpa_port_fwd_enable(ofdpa_port, 0);
--
drivers/net/ethernet/sfc/ef100_rx.c=56=void __ef100_rx_packet(struct efx_channel *channel)
--
drivers/net/ethernet/sfc/ef100_rx.c-100- if (efv) {
drivers/net/ethernet/sfc/ef100_rx.c:101: if (efv->net_dev->flags & IFF_UP)
drivers/net/ethernet/sfc/ef100_rx.c-102- efx_ef100_rep_rx_packet(efv, rx_buf);
--
drivers/net/ethernet/sfc/ethtool_common.c=128=void efx_ethtool_self_test(struct net_device *net_dev,
--
drivers/net/ethernet/sfc/ethtool_common.c-148- /* We need rx buffers and interrupts. */
drivers/net/ethernet/sfc/ethtool_common.c:149: already_up = (efx->net_dev->flags & IFF_UP);
drivers/net/ethernet/sfc/ethtool_common.c-150- if (!already_up) {
--
drivers/net/ethernet/sfc/falcon/ethtool.c=488=static void ef4_ethtool_self_test(struct net_device *net_dev,
--
drivers/net/ethernet/sfc/falcon/ethtool.c-508- /* We need rx buffers and interrupts. */
drivers/net/ethernet/sfc/falcon/ethtool.c:509: already_up = (efx->net_dev->flags & IFF_UP);
drivers/net/ethernet/sfc/falcon/ethtool.c-510- if (!already_up) {
--
drivers/net/ethernet/sfc/nic.c=410=void efx_nic_fix_nodesc_drop_stat(struct efx_nic *efx, u64 *rx_nodesc_drops)
--
drivers/net/ethernet/sfc/nic.c-412- /* if down, or this is the first update after coming up */
drivers/net/ethernet/sfc/nic.c:413: if (!(efx->net_dev->flags & IFF_UP) || !efx->rx_nodesc_drops_prev_state)
drivers/net/ethernet/sfc/nic.c-414- efx->rx_nodesc_drops_while_down +=
--
drivers/net/ethernet/sfc/nic.c-416- efx->rx_nodesc_drops_total = *rx_nodesc_drops;
drivers/net/ethernet/sfc/nic.c:417: efx->rx_nodesc_drops_prev_state = !!(efx->net_dev->flags & IFF_UP);
drivers/net/ethernet/sfc/nic.c-418- *rx_nodesc_drops -= efx->rx_nodesc_drops_while_down;
--
drivers/net/ethernet/sfc/siena/ethtool_common.c=350=void efx_siena_ethtool_self_test(struct net_device *net_dev,
--
drivers/net/ethernet/sfc/siena/ethtool_common.c-370- /* We need rx buffers and interrupts. */
drivers/net/ethernet/sfc/siena/ethtool_common.c:371: already_up = (efx->net_dev->flags & IFF_UP);
drivers/net/ethernet/sfc/siena/ethtool_common.c-372- if (!already_up) {
--
drivers/net/ethernet/sfc/siena/nic.c=521=void efx_siena_fix_nodesc_drop_stat(struct efx_nic *efx, u64 *rx_nodesc_drops)
--
drivers/net/ethernet/sfc/siena/nic.c-523- /* if down, or this is the first update after coming up */
drivers/net/ethernet/sfc/siena/nic.c:524: if (!(efx->net_dev->flags & IFF_UP) || !efx->rx_nodesc_drops_prev_state)
drivers/net/ethernet/sfc/siena/nic.c-525- efx->rx_nodesc_drops_while_down +=
--
drivers/net/ethernet/sfc/siena/nic.c-527- efx->rx_nodesc_drops_total = *rx_nodesc_drops;
drivers/net/ethernet/sfc/siena/nic.c:528: efx->rx_nodesc_drops_prev_state = !!(efx->net_dev->flags & IFF_UP);
drivers/net/ethernet/sfc/siena/nic.c-529- *rx_nodesc_drops -= efx->rx_nodesc_drops_while_down;
--
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c=6404=static int stmmac_rings_status_show(struct seq_file *seq, void *v)
--
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c-6411-
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c:6412: if ((dev->flags & IFF_UP) == 0)
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c-6413- return 0;
--
drivers/net/ethernet/ti/cpsw.c=203=static int cpsw_update_vlan_mc(struct net_device *vdev, int vid, void *ctx)
--
drivers/net/ethernet/ti/cpsw.c-208-
drivers/net/ethernet/ti/cpsw.c:209: if (!vdev || !(vdev->flags & IFF_UP))
drivers/net/ethernet/ti/cpsw.c-210- return 0;
--
drivers/net/ethernet/ti/cpsw.c=268=static int cpsw_purge_vlan_mc(struct net_device *vdev, int vid, void *ctx)
--
drivers/net/ethernet/ti/cpsw.c-273-
drivers/net/ethernet/ti/cpsw.c:274: if (!vdev || !(vdev->flags & IFF_UP))
drivers/net/ethernet/ti/cpsw.c-275- return 0;
--
drivers/net/ethernet/ti/cpsw_new.c=146=static int cpsw_update_vlan_mc(struct net_device *vdev, int vid, void *ctx)
--
drivers/net/ethernet/ti/cpsw_new.c-151-
drivers/net/ethernet/ti/cpsw_new.c:152: if (!vdev || !(vdev->flags & IFF_UP))
drivers/net/ethernet/ti/cpsw_new.c-153- return 0;
--
drivers/net/ethernet/ti/cpsw_new.c=211=static int cpsw_purge_vlan_mc(struct net_device *vdev, int vid, void *ctx)
--
drivers/net/ethernet/ti/cpsw_new.c-216-
drivers/net/ethernet/ti/cpsw_new.c:217: if (!vdev || !(vdev->flags & IFF_UP))
drivers/net/ethernet/ti/cpsw_new.c-218- return 0;
--
drivers/net/ethernet/toshiba/ps3_gelic_net.c=1419=static void gelic_net_tx_timeout_task(struct work_struct *work)
--
drivers/net/ethernet/toshiba/ps3_gelic_net.c-1426-
drivers/net/ethernet/toshiba/ps3_gelic_net.c:1427: if (!(netdev->flags & IFF_UP))
drivers/net/ethernet/toshiba/ps3_gelic_net.c-1428- goto out;
--
drivers/net/ethernet/toshiba/ps3_gelic_net.c=1447=void gelic_net_tx_timeout(struct net_device *netdev, unsigned int txqueue)
--
drivers/net/ethernet/toshiba/ps3_gelic_net.c-1452- atomic_inc(&card->tx_timeout_task_counter);
]
]
|
| 143/2 |
2026/05/21 11:48 |
llm |
expert |
6m
Model:
Tokens:
input: 161775
output: 66
thoughts: 62915
|
| 144/2 |
2026/05/21 11:54 |
tool |
set-results |
0m
Args: map[DenialOfService:true Exploitable:false FilesystemTrigger:false NetworkTrigger:false PeripheralTrigger:true RemoteTrigger:false Unprivileged:true UserNamespace:false VMGuestTrigger:false VMHostTrigger:false]
Results: map[DenialOfService:true Exploitable:false FilesystemTrigger:false NetworkTrigger:false PeripheralTrigger:true RemoteTrigger:false Unprivileged:true UserNamespace:false VMGuestTrigger:false VMHostTrigger:false]
|
| 145/2 |
2026/05/21 11:54 |
llm |
expert |
0m
Model:
Tokens:
input: 224820
output: 708
thoughts: 318
Thoughts: **My Analysis of the Device Shutdown Warning**
Okay, the analysis is done. It's a `WARNING` in `dev_shutdown`, and the core issue stems from the watchdog timer being still pending: `timer_pending(&dev->watchdog_timer)` is the culprit. This is happening because `unregister_netdevice_many_notify` bypasses the `netif_close_many` chain – including the critical `dev_deactivate_many` and `netdev_watchdog_down` calls – when `IFF_UP` isn't set.
The tricky part is that the timer *can* get armed during `__dev_open`. This occurs if the `ops->ndo_open` function fails or blocks. Concurrently, `netif_carrier_on` is called. It checks `netif_running(dev)`, which is true at this point because `__LINK_STATE_START` has been set. Crucially, if `__dev_open` fails, the `IFF_UP` flag is never set. Then, the device gets unregistered, and `dev_shutdown` is invoked. Since `netdev_watchdog_down` never got a chance to run, the timer remains pending, leading to the warning.
While this is a `WARNING`, and it doesn't crash the kernel in production builds, there's a potential use-after-free scenario if the timer fires *after* the device has been freed. However, `free_netdev` is delayed by RCU and `netdev_run_todo`, so the timer will most likely complete before the memory is actually freed. Nevertheless, it's still accessible through a USB raw gadget.
Given that it's a `WARNING` that *could* lead to denial-of-service, I will classify it as a DenialOfService and because it is triggered via the USB raw gadget, I will also classify it as a PeripheralTrigger.
|
| 146/1 |
2026/05/21 11:55 |
action |
format-explanation |
0m
Results: map[Explanation:The bug report indicates a `WARNING` in `dev_shutdown()` at
`net/sched/sch_generic.c:1496` because `timer_pending(&dev->watchdog_timer)`
evaluates to true when a network device is being unregistered.
### Root Cause Analysis
The warning triggers because the network device's watchdog timer is still
pending when the device is being dismantled.
When a network device is unregistered, `unregister_netdevice_many_notify()` is
called. This function first checks if the device is currently running
(`dev->flags & IFF_UP`). If it is, it calls `netif_close_many()`, which
eventually calls `dev_deactivate_many()` and `netdev_watchdog_down()` to safely
delete the watchdog timer. However, if `IFF_UP` is not set, this teardown step
is skipped.
The race condition occurs during the device initialization phase
(`__dev_open()`):
1. `__dev_open()` sets the `__LINK_STATE_START` bit before calling the driver's
`ndo_open()` callback.
2. If `ndo_open()` blocks or takes a long time, an asynchronous event (like a
USB kevent or a carrier change) can call `netif_carrier_on()`.
3. `netif_carrier_on()` checks `netif_running(dev)` (which returns true because
`__LINK_STATE_START` is set) and subsequently calls `netdev_watchdog_up()`,
arming the timer.
4. If `ndo_open()` subsequently fails, `__dev_open()` clears
`__LINK_STATE_START` and returns an error, but it **never sets `IFF_UP`**.
5. When the device is later disconnected and unregistered,
`unregister_netdevice_many_notify()` sees that `IFF_UP` is false and skips
calling `netdev_watchdog_down()`.
6. Finally, `dev_shutdown()` is called, and the
`WARN_ON(timer_pending(&dev->watchdog_timer))` triggers because the timer was
armed but never deleted.
### Security Impact
* **Exploitable:** No. While a pending timer on a freed object usually leads to
a Use-After-Free (UAF), the network device unregistration process
(`netdev_run_todo()`) involves RCU grace periods and waits for all references to
drop before calling `free_netdev()`. The timer will almost certainly fire and
finish its execution before the `struct net_device` memory is actually freed,
preventing a UAF.
* **Denial of Service:** Yes. In kernels with `panic_on_warn` enabled, this will
crash the system. In production builds without `panic_on_warn`, it will just
print a warning and continue, but it indicates a state inconsistency.
* **Peripheral Trigger:** Yes. The bug can be triggered by plugging in a
malicious USB device (or using the `dummy_hcd` / `raw-gadget` interfaces as the
syzkaller reproducer does) that simulates a network interface and manipulates
the carrier state during initialization.
* **Unprivileged:** Yes. If the system allows unprivileged users to access USB
devices or the `raw-gadget` interface, they can trigger this issue.]
|