mlx4_core: Allocate and map sufficient ICM memory for EQ context

The current implementation allocates a single host page for EQ context
memory, which was OK when we only allocated a few EQs.  However, since
we now allocate an EQ for each CPU core, this patch removes the
hard-coded limit (which we exceed with 4 KB pages and 128 byte EQ
context entries with 32 CPUs) and uses the same ICM table code as all
other context tables, which ends up simplifying the code quite a bit
while fixing the problem.

This problem was actually hit in practice on a dual-socket Nehalem box
with 16 real hardware threads and sufficiently odd ACPI tables that it
shows on boot

    SMP: Allowing 32 CPUs, 16 hotplug CPUs

so num_possible_cpus() ends up 32, and mlx4 ends up creating 33 MSI-X
interrupts and 33 EQs.  This mlx4 bug means that mlx4 can't even
initialize at all on this quite mainstream system.

Cc: <stable@kernel.org>
Reported-by: Eli Cohen <eli@mellanox.co.il>
Tested-by: Christoph Lameter <cl@linux-foundation.org>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
This commit is contained in:
Roland Dreier
2009-09-05 20:24:49 -07:00
parent 338a8fad27
commit fa0681d212
3 changed files with 7 additions and 51 deletions

View File

@@ -525,48 +525,6 @@ static void mlx4_unmap_clr_int(struct mlx4_dev *dev)
iounmap(priv->clr_base);
}
int mlx4_map_eq_icm(struct mlx4_dev *dev, u64 icm_virt)
{
struct mlx4_priv *priv = mlx4_priv(dev);
int ret;
/*
* We assume that mapping one page is enough for the whole EQ
* context table. This is fine with all current HCAs, because
* we only use 32 EQs and each EQ uses 64 bytes of context
* memory, or 1 KB total.
*/
priv->eq_table.icm_virt = icm_virt;
priv->eq_table.icm_page = alloc_page(GFP_HIGHUSER);
if (!priv->eq_table.icm_page)
return -ENOMEM;
priv->eq_table.icm_dma = pci_map_page(dev->pdev, priv->eq_table.icm_page, 0,
PAGE_SIZE, PCI_DMA_BIDIRECTIONAL);
if (pci_dma_mapping_error(dev->pdev, priv->eq_table.icm_dma)) {
__free_page(priv->eq_table.icm_page);
return -ENOMEM;
}
ret = mlx4_MAP_ICM_page(dev, priv->eq_table.icm_dma, icm_virt);
if (ret) {
pci_unmap_page(dev->pdev, priv->eq_table.icm_dma, PAGE_SIZE,
PCI_DMA_BIDIRECTIONAL);
__free_page(priv->eq_table.icm_page);
}
return ret;
}
void mlx4_unmap_eq_icm(struct mlx4_dev *dev)
{
struct mlx4_priv *priv = mlx4_priv(dev);
mlx4_UNMAP_ICM(dev, priv->eq_table.icm_virt, 1);
pci_unmap_page(dev->pdev, priv->eq_table.icm_dma, PAGE_SIZE,
PCI_DMA_BIDIRECTIONAL);
__free_page(priv->eq_table.icm_page);
}
int mlx4_alloc_eq_table(struct mlx4_dev *dev)
{
struct mlx4_priv *priv = mlx4_priv(dev);