xiaomi-sm8450/android_kernel_xiaomi_sm8450
1
0
Forka 0
Codice Problemi Pull Requests Actions Pacchetti Progetti Rilasci Wiki Attività

RDMA/mlx5: Improve PI handover performance

Sfoglia il codice sorgente 
In some loads, there is performance degradation when using KLM mkey
instead of MTT mkey. This is because KLM descriptor access is via
indirection that might require more HW resources and cycles.
Using KLM descriptor is not necessary when there are no gaps at the
data/metadata sg lists. As an optimization, use MTT mkey whenever it
is possible. For that matter, allocate internal MTT mkey and choose the
effective pi_mr for in transaction according to the required mapping
scheme.

The setup of the tested benchmark (using iSER ULP):
 - 2 servers with 24 cores (1 initiator and 1 target)
 - ConnectX-4/ConnectX-5 adapters
 - 24 target sessions with 1 LUN each
 - ramdisk backstore
 - PI active

Performance results running fio (24 jobs, 128 iodepth) using
write_generate=1 and read_verify=1 (w/w.o/baseline):

bs      IOPS(read)                IOPS(write)
----    ----------                ----------
512   1262.4K/1243.3K/1147.1K    1732.1K/1725.1K/1423.8K
4k    570902/571233/457874       773982/743293/642080
32k   72086/72388/71933          96164/71789/93249

Using write_generate=0 and read_verify=0 (w/w.o patch):
bs      IOPS(read)                IOPS(write)
----    ----------                ----------
512   1600.1K/1572.1K/1393.3K    1830.3K/1823.5K/1557.2K
4k    937272/921992/762934       815304/753772/646071
32k   77369/75052/72058          97435/73180/94612

Signed-off-by: Israel Rukshin <israelr@mellanox.com>
Reviewed-by: Max Gurtovoy <maxg@mellanox.com>
Suggested-by: Max Gurtovoy <maxg@mellanox.com>
Suggested-by: Idan Burstein <idanb@mellanox.com>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
This commit is contained in:
Israel Rukshin
2019-06-11 18:52:55 +03:00
committato da Jason Gunthorpe
parent 5c171cbe3a
commit de0ae958de
3 ha cambiato i file con 165 aggiunte e 25 eliminazioni
Scarica il file Patch Scarica il file Diff Expand all files Collapse all files

7
drivers/infiniband/hw/mlx5/mlx5_ib.h
Vedi File

@@ -605,7 +605,12 @@ struct mlx5_ib_mr {
int access_flags; /* Needed for rereg MR */
struct mlx5_ib_mr *parent;
struct mlx5_ib_mr *pi_mr; /* Needed for IB_MR_TYPE_INTEGRITY */
/* Needed for IB_MR_TYPE_INTEGRITY */
struct mlx5_ib_mr *pi_mr;
struct mlx5_ib_mr *klm_mr;
struct mlx5_ib_mr *mtt_mr;
u64 pi_iova;
atomic_t num_leaf_free;
wait_queue_head_t q_leaf_free;
struct mlx5_async_work cb_work;