xiaomi-sm8450/android_kernel_xiaomi_sm8450
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

RDMA/ocrdma: Depend on async link events from CNA

Browse Source 
Recently Dough Ledford reported a deadlock happening
between ocrdma-load sequence and NetworkManager service
issuing "open" on be2net interface.

The deadlock happens when any be2net hook (e.g. open/close) is called
in parallel to insmod ocrdma.ko.

A. be2net is sending administrative open/close event to ocrdma holding
   device_list_mutex. It does this from ndo_open/ndo_stop hooks of be2net.
   So sequence of locks is rtnl_lock---> device_list lock

B.  When new ocrdma roce device gets registered, infiniband stack now
    takes rtnl_lock in ib_register_device() in GID initialization routines.
    So sequence of locks in this path is device_list lock ---> rtnl_lock.

This improper locking sequence causes deadlock.

With this patch we stop using administrative open and close events
injected by be2net driver. These events were used to dispatch PORT_ACTIVE
and PORT_ERROR events to the IB-stack. This patch implements a logic
to receive async-link-events generated from CNA whenever link-state-change
is detected. Now on, these async-events will be used to dispatch
PORT_ACTIVE and PORT_ERROR events to IB-stack.

Depending on async-events from CNA removes the need to hold device-list-mutex
and thus breaks the busy-wait scenario.

Reported-by: Doug Ledford <dledford@redhat.com>
CC: Sathya Perla <sathya.perla@avagotech.com>
Signed-off-by: Padmanabh Ratnakar <padmanabh.ratnakar@avagotech.com>
Signed-off-by: Selvin Xavier <selvin.xavier@avagotech.com>
Signed-off-by: Devesh Sharma <devesh.sharma@avagotech.com>
Signed-off-by: Doug Ledford <dledford@redhat.com>
This commit is contained in:
Devesh Sharma
2015-12-24 13:14:07 -05:00
committed by Doug Ledford
parent d310a344e1
commit 3b1ea43009
6 changed files with 119 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
drivers/infiniband/hw/ocrdma/ocrdma_main.c
View File

@@ -289,6 +289,7 @@ static void ocrdma_remove_sysfiles(struct ocrdma_dev *dev)
static struct ocrdma_dev *ocrdma_add(struct be_dev_info *dev_info)
{
int status = 0, i;
u8 lstate = 0;
struct ocrdma_dev *dev;
dev = (struct ocrdma_dev *)ib_alloc_device(sizeof(struct ocrdma_dev));
@@ -318,6 +319,11 @@ static struct ocrdma_dev *ocrdma_add(struct be_dev_info *dev_info)
if (status)
goto alloc_err;
/* Query Link state and update */
status = ocrdma_mbx_get_link_speed(dev, NULL, &lstate);
if (!status)
ocrdma_update_link_state(dev, lstate);
for (i = 0; i < ARRAY_SIZE(ocrdma_attributes); i++)
if (device_create_file(&dev->ibdev.dev, ocrdma_attributes[i]))
goto sysfs_err;
@@ -372,7 +378,7 @@ static void ocrdma_remove(struct ocrdma_dev *dev)
ocrdma_remove_free(dev);
}
static int ocrdma_open(struct ocrdma_dev *dev)
static int ocrdma_dispatch_port_active(struct ocrdma_dev *dev)
{
struct ib_event port_event;
@@ -383,7 +389,7 @@ static int ocrdma_open(struct ocrdma_dev *dev)
return 0;
}
static int ocrdma_close(struct ocrdma_dev *dev)
static int ocrdma_dispatch_port_error(struct ocrdma_dev *dev)
{
struct ib_event err_event;
@@ -396,7 +402,7 @@ static int ocrdma_close(struct ocrdma_dev *dev)
static void ocrdma_shutdown(struct ocrdma_dev *dev)
{
ocrdma_close(dev);
ocrdma_dispatch_port_error(dev);
ocrdma_remove(dev);
}
@@ -407,18 +413,28 @@ static void ocrdma_shutdown(struct ocrdma_dev *dev)
static void ocrdma_event_handler(struct ocrdma_dev *dev, u32 event)
{
switch (event) {
case BE_DEV_UP:
ocrdma_open(dev);
break;
case BE_DEV_DOWN:
ocrdma_close(dev);
break;
case BE_DEV_SHUTDOWN:
ocrdma_shutdown(dev);
break;
default:
break;
}
}
void ocrdma_update_link_state(struct ocrdma_dev *dev, u8 lstate)
{
if (!(dev->flags & OCRDMA_FLAGS_LINK_STATUS_INIT)) {
dev->flags |= OCRDMA_FLAGS_LINK_STATUS_INIT;
if (!lstate)
return;
}
if (!lstate)
ocrdma_dispatch_port_error(dev);
else
ocrdma_dispatch_port_active(dev);
}
static struct ocrdma_driver ocrdma_drv = {
.name = "ocrdma_driver",
.add = ocrdma_add,