Release Date: 26 Dec 2023
Release Version: V_01-03-59 : Limited-tested version
TC956X PCIe EMAC driver is based on "Fedora 39, kernel-6.6.1".
By default, DMA_OFFLOAD_ENABLE is enabled. Execute following commands:
#make clean
#make
If IPA offload is not needed, disable macro DMA_OFFLOAD_ENABLE in common.h. set DMA_OFFLOAD = 0 in Makefile and execute following commands:
#make clean
#make
To compile driver with load firmware header (fw.h) use the below command #make TC956X_LOAD_FW_HEADER=1
In order to compile the Driver to include the code for applying Gen3 setting, execute Make with below argument #make TC956X_PCIE_GEN3_SETTING=1
Please note, incase both fw.h and Gen3 settings are needed, then both arugments need to be specified.
Load phylink module
#modprobe phylink
Load the driver
#insmod tc956x_pcie_eth.ko pcie_link_speed=X
In the module parameter pcie_link_speed, X is the desired PCIe Gen speed. X can be 3 or 2 or 1. Passing module parameter (pcie_link_speed=X) is optional. If module parameter is not passed, by default Gen3 speed will be selected by the driver.
Please note that driver should be compiled using below command to use this feature: #make TC956X_PCIE_GEN3_SETTING=1
Remove the driver
#rmmod tc956x_pcie_eth
Use below commands to advertise with Autonegotiation ON for speeds 10Gbps, 5Gbps, 2.5Gbps, 1Gbps, 100Mbps and 10Mbps as ethtool speed command does not support.
ethtool -s advertise 0x7000 autoneg on --> changes the advertisement to 10Gbps
ethtool -s advertise 0x1000000006000 autoneg on --> changes the advertisement to 5Gbps
ethtool -s advertise 0x800000006000 autoneg on --> changes the advertisement to 2.5Gbps
ethtool -s advertise 0x6020 autoneg on --> changes the advertisement to 1Gbps
ethtool -s advertise 0x6008 autoneg on --> changes the advertisement to 100Mbps
ethtool -s advertise 0x6002 autoneg on --> changes the advertisement 10Mbps
Use the below command to insert the kernel module with specific modes for interfaces:
#insmod tc956x_pcie_eth.ko mac0_interface=x mac1_interface=y
argument info:
mac0_interface: For PORT0 interface mode setting
mac1_interface: For PORT1 interface mode setting
x = [0: USXGMII, 1: XFI (default), 2: RGMII (unsupported), 3: SGMII, 4: 2500Base-X]
y = [0: USXGMII (unsupported), 1: XFI (unsupported), 2: RGMII, 3: SGMII(default), 4: 2500Base-X]
If invalid and unsupported modes are passed as kernel module parameter, the default interface mode will be selected.
Regarding the performance, use the below command to increase the dynamic byte queue limit
$echo "900000" > /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/0000:02:03.0/0000:05:00.0/net/enp5s0f0/queues/tx-0/byte_queue_limits/limit_min
900000 is the random value chosen. It needs to adjust this value on their system and check "0000:00/0000:00:01.0/0000:01:00.0/0000:02:03.0/0000:05:00.0/" value can be obtained from the "lspci -t" command
The debug counters to check the interrupt count is available.
"#ethtool -S " needs to be executed and sample output is as below
total_interrupts: 120109 lpi_intr_n: 0 pmt_intr_n: 0 event_intr_n: 0 tx_intr_n: 120000 rx_intr_n: 51 xpcs_intr_n: 0 phy_intr_n: 46 sw_msi_n: 12
tx_intr_n = No of. Tx interrupts originating from eMAC sw_msi_n = No. of SW MSIs triggered by Systick Handler as part of optimized Tx Timer based on Systick approach. So total number of interrupts for Tx = tx_intr_n + sw_msi_n Please note that whenever Rx interruts are generated, the Host ISR will process the Tx completed descriptors too.
With V_01-00-07, when IPA API start_channel() is invoked for Rx direction, MAC_Address1_High is updated with 0xBF000000. This register setting is almost similar to promiscuous mode. So please install appropriate FRP instructions.
From V_01-00-08 onwards, Port0 ethernet interface will not be created only if there is no ethernet PHY attached to it
Enable TC956X_PHY_INTERRUPT_MODE_EMAC0 macro for supporting PORT0 Interrupt mode. Disable the macro if the phy driver supports only polling mode. Enable TC956X_PHY_INTERRUPT_MODE_EMAC1 macro for supporting PORT1 Interrupt mode. Disable the macro if the phy driver supports only polling mode.
Change below macro values for configuration of Link state L0 and L1 transaction delay. /* Link state change delay configuration for Upstream Port */ #define USP_L0s_ENTRY_DELAY (0x1FU) #define USP_L1_ENTRY_DELAY (0x3FFU)
/* Link state change delay configuration for Downstream Port-1 */ #define DSP1_L0s_ENTRY_DELAY (0x1FU) #define DSP1_L1_ENTRY_DELAY (0x3FFU)
/* Link state change delay configuration for Downstream Port-2 */ #define DSP2_L0s_ENTRY_DELAY (0x1FU) #define DSP2_L1_ENTRY_DELAY (0x3FFU)
/* Link state change delay configuration for Virtual Downstream Port */ #define VDSP_L0s_ENTRY_DELAY (0x1FU) #define VDSP_L1_ENTRY_DELAY (0x3FFU)
/* Link state change delay configuration for Internal Endpoint */ #define EP_L0s_ENTRY_DELAY (0x1FU) #define EP_L1_ENTRY_DELAY (0x3FFU)
Formula:
L0 entry delay = XXX_L0s_ENTRY_DELAY * 256 ns
L1 entry delay = XXX_L1_ENTRY_DELAY * 256 ns
XXX_L0s_ENTRY_DELAY range: 1-31
XXX_L1_ENTRY_DELAY: 1-1023
To check vlan feature status execute: ethtool -k | grep vlan
To enable/disable following vlan features execute:
(a) rx-vlan-filter:
ethtool -K <interface> rx-vlan-filter <on|off>
(b) rx-vlan-offload:
ethtool -K <interface> rxvlan <on|off>
(c) tx-vlan-offload:
ethtool -K <interface> txvlan <on|off>
Use following to configure VLAN:
(a) modprobe 8021q
(b) vconfig add <interface> <vlanid>
(c) vconfig set_flag <interface>.<vlanid> 1 0
(d) ifconfig <interface>.<vlanid> <ip> netmask 255.255.255.0 broadcast <ip mask> up
Default Configuraton:
(a) Rx vlan filter is disabled.
(b) Rx valn offload (vlan stripping) is disabled.
(c) Tx vlan offload is enabled.
Please use the below command to insert the kernel module for passing pause frames to application except pause frames from PHY:
#insmod tc956x_pcie_eth.ko mac0_filter_phy_pause=x mac1_filter_phy_pause=x
argument info:
mac0_filter_phy_pause: For PORT0
mac1_filter_phy_pause: For PORT1
x = [0: DISABLE (default), 1: ENABLE]
If invalid values are passed as kernel module parameter, the default value will be selected.
Use below commands to check WOL support and its type: #ethtool
WOL command Usage : #ethtool -s wol .
Supported WOL options and meaning:
Option | Meaning |
---|---|
p | Wake on phy activity |
g | Wake on MagicPacket(tm) |
d | Disable (wake on nothing). (Default) |
Example - To wake on phy activity and magic packet use : ethtool -s eth0 wol pg
Please use the below command to insert the kernel module to enable EEE and configure LPI Auto Entry timer:
#insmod tc956x_pcie_eth.ko mac0_eee_enable=X mac0_lpi_timer=Y mac1_eee_enable=X mac1_lpi_timer=Y
argument info:
mac0_eee_enable: For PORT0
mac1_eee_enable: For PORT1
X = [0: DISABLE (default), 1: ENABLE]
This module parameter is to Enable/Disable EEE for Port 0/1 - default is 0.
If invalid values are passed as kernel module parameter, the default value will be selected.
mac0_lpi_timer: For PORT0
mac1_lpi_timer: For PORT1
Y = [0..1048568 (us)]
This module parameter is to configure LPI Automatic Entry Timer for Port 0/1 - default is 600 (us).
If invalid values are passed as kernel module parameter, the default value will be selected.
In addition to above module parameter, use below ethtool command to configure EEE and LPI auto entry timer. #ethtool --set-eee eee tx-timer
Use below command to check the status of EEE configuration #ethtool --show-eee
Please use the below command to insert the kernel module for RX Queue size, Flow control thresholds & TX Queue size configuration.
#insmod tc956x_pcie_eth.ko mac0_rxq0_size=x mac0_rxq0_rfd=y mac0_rxq0_rfa=y
mac0_rxq1_size=x mac0_rxq1_rfd=y mac0_rxq1_rfa=y
mac0_txq0_size=x mac0_txq1_size=x
mac1_rxq0_size=x mac1_rxq0_rfd=y mac1_rxq0_rfa=y
mac1_rxq1_size=x mac1_rxq1_rfd=y mac1_rxq1_rfa=y
mac1_txq0_size=x mac1_txq1_size=x
argument info:
mac0_rxq0_size: For PORT0 RX Queue-0
mac0_rxq1_size: For PORT0 RX Queue-1
mac1_rxq0_size: For PORT1 RX Queue-0
mac1_rxq1_size: For PORT1 RX Queue-1
mac0_txq0_size: For PORT0 TX Queue-0
mac0_txq1_size: For PORT0 TX Queue-1
mac1_txq0_size: For PORT1 TX Queue-0
mac1_txq1_size: For PORT1 TX Queue-1
x = [Range Supported : 3072..44032 (bytes)], default is 18432 (bytes)
mac0_rxq0_rfd: For PORT0 Queue-0 threshold for Disable flow control
mac0_rxq1_rfd: For PORT0 Queue-1 threshold for Disable flow control
mac0_rxq0_rfa: For PORT0 Queue-0 threshold for Enable flow control
mac0_rxq1_rfa: For PORT0 Queue-1 threshold for Enable flow control
mac1_rxq0_rfd: For PORT1 Queue-0 threshold for Disable flow control
mac1_rxq1_rfd: For PORT1 Queue-1 threshold for Disable flow control
mac1_rxq0_rfa: For PORT1 Queue-0 threshold for Enable flow control
mac1_rxq1_rfa: For PORT1 Queue-1 threshold for Enable flow control
y = [Range Supported : 0..84], default is 24 (13KB)
If invalid values are passed as kernel module parameter, the default value will be selected for Queue Sizes and for Flow control 80% of Queue size will be used.
Note:
Please use the below command to insert the kernel module for counting Link partner pause frames and output to ethtool:
#insmod tc956x_pcie_eth.ko mac0_en_lp_pause_frame_cnt=x mac1_en_lp_pause_frame_cnt=x
argument info:
mac0_en_lp_pause_frame_cnt: For PORT0
mac1_en_lp_pause_frame_cnt: For PORT1
x = [0: DISABLE (default), 1: ENABLE]
If invalid values are passed as kernel module parameter, the default value will be selected. Note: It is required to enable kernel module parameter "mac0_filter_phy_pause/mac1_filter_phy_pause" along with this module parameter to count link partner pause frames.
Please use the below command to insert the kernel module for power saving at Link Down state:
#insmod tc956x_pcie_eth.ko mac_power_save_at_link_down=x
argument info:
mac_power_save_at_link_down: Common for both PORT0 & PORT1
x = [0: DISABLE (default), 1: ENABLE]
If invalid values are passed as kernel module parameter, the default value will be selected.
Debufs directory will be created for port specific in debug path of kernel i.e. "/sys/kernel/debug" in x86 Linux platfrom. Under port specific debugfs directory (tc956x_port0_debug/tc956x_port1_debug), module specific files are created to get dump of debug information related to module. Example: config_stats --> Registers related to CONFIG module mac_stats --> Registers related to MAC block mtl_stats --> Registers related to MTL block dma_stats --> Registers related to DMA block m3_stats --> Debug information related to M3 Firmware interrupt_stats --> Registers related to MSI & INT blocks other_stats --> Information related to Driver & Firmware, TAMAP, Flexible Receiver Parser, mmc counters reg_dump --> Dumps all registers of MAC, MTL, DMA and CNFG modules
Information will be printed to "dmesg" console, when files related to specific module are invoked.
debugfs file can be invoked by using "cat" command. Example: cat /sys/kernel/debug/tc956x_port0_debug/config_stats
Use the below command to insert the kernel module for SW reset during link down.
#insmod tc956x_pcie_eth.ko mac0_link_down_macrst=x mac1_link_down_macrst=y
argument info:
mac0_link_down_macrst: For PORT0
x = [0: DISABLE, 1: ENABLE (default)]
mac1_link_down_macrst: For PORT1
y = [0: DISABLE (default), 1: ENABLE]
If invalid values are passed as kernel module parameter, the default value will be selected.