IB/ipath: Fix IB compliance problems with link state vs physical state

Subnet manager SetPortinfo messages distingush between changing the link
state (DOWN, ARM, ACTIVE) and the link physical state (POLL, SLEEP,
DISABLED).  These are somewhat independent commands and affect when link
width and speed changes take effect.  Without this patch, a link DOWN
physical state NOP command was causing the link width and speed settings
to take effect which should only happen when the link physical state is
goes down (either by a SMP or some link physical error like link errors
exceeding the threshold).

Signed-off-by: Ralph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: Roland Dreier <rolandd@cisco.com>

drivers/infiniband/hw/ipath/ipath_mad.c

@@ -555,10 +555,7 @@ static int recv_subn_set_portinfo(struct ib_smp *smp,
 		/* FALLTHROUGH */
 	case IB_PORT_DOWN:
 		if (lstate == 0)
-			if (get_linkdowndefaultstate(dd))
-				lstate = IPATH_IB_LINKDOWN_SLEEP;
-			else
-				lstate = IPATH_IB_LINKDOWN;
+			lstate = IPATH_IB_LINKDOWN_ONLY;
 		else if (lstate == 1)
 			lstate = IPATH_IB_LINKDOWN_SLEEP;
 		else if (lstate == 2)
@@ -568,6 +565,8 @@ static int recv_subn_set_portinfo(struct ib_smp *smp,
 		else
 			goto err;
 		ipath_set_linkstate(dd, lstate);
+		ipath_wait_linkstate(dd, IPATH_LINKINIT | IPATH_LINKARMED |
+				IPATH_LINKACTIVE, 1000);
 		break;
 	case IB_PORT_ARMED:
 		ipath_set_linkstate(dd, IPATH_IB_LINKARM);