A while back I had to configure a bunch of servers with Mellanox Connect-X 4 NIC’s to talk to an ethernet Switch(Dell S6000-ON to be exact) and the NIC’s that my customer had fitted not only had really old firmware but they were also in Infiniband(I think) mode, not Ethernet. So i had to do some legwork..
First job was to update the firmware.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
apt update && apt install wget unzip gcc make dkms unzip linux-headers-$(uname -r) -y mkdir /tmp/mellanox && cd /tmp/mellanox wget https://downloads.hpe.com/pub/softlib2/software1/pubsw-linux/p84948234/v182403/fw-ConnectX3Pro-rel-2_42_5700-764285-B21_Ax-CLP-8025-UEFI-14.11.49-FlexBoot-3.4.754.tgz gunzip fw-ConnectX3Pro-rel-2_42_5700-764285-B21_Ax-CLP-8025-UEFI-14.11.49-FlexBoot-3.4.754.tgz tar -xf fw-ConnectX3Pro-rel-2_42_5700-764285-B21_Ax-CLP-8025-UEFI-14.11.49-FlexBoot-3.4.754.tar mv fw-ConnectX3Pro-rel-2_42_5700-764285-B21_Ax-CLP-8025-UEFI-14.11.49-FlexBoot-3.4.754/* . rm -rf fw-ConnectX3Pro-rel-2_42_5700-764285-B21_Ax-CLP-8025-UEFI-14.11.49-FlexBoot-3.4.754 cd /tmp/mellanox wget http://www.mellanox.com/downloads/MFT/mft-4.10.0-104-x86_64-deb.tgz tar -xvf mft-4.10.0-104-x86_64-deb.tgz cd /tmp/mellanox/mft-4.10.0-104-x86_64-deb/ bash ./install.sh mst start mst status export OUT1=$(mst status | grep cr0 ) TRIMMEDOUT=(${OUT1//-/ }) CARDID=${TRIMMEDOUT[0]} echo $CARDID flint -d $CARDID dc /tmp/mellanox/orig_firmware.ini flint -y -d $CARDID -i /tmp/mellanox/fw-ConnectX3Pro-rel-2_42_5700-764285-B21_Ax-CLP-8025-UEFI-14.11.49-FlexBoot-3.4.754.bin -allow_psid_change burn |
Set both ports on the NIC to Ethernet mode
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
mst start export OUT1=$(mst status | grep cr0 ) TRIMMEDOUT=(${OUT1//-/ }) CARDID=${TRIMMEDOUT[0]} echo $CARDID mlxconfig -d $CARDID query #Turn both ports from VPI/Auto to Ethernet only: mlxconfig -y -d $CARDID set LINK_TYPE_P1=2 LINK_TYPE_P2=2 #turn off bootrom mlxconfig -y -d $CARDID set BOOT_OPTION_ROM_EN_P1=false mlxconfig -y -d $CARDID set BOOT_OPTION_ROM_EN_P2=false mlxconfig -y -d $CARDID set LEGACY_BOOT_PROTOCOL_P1=0 mlxconfig -y -d $CARDID set LEGACY_BOOT_PROTOCOL_P2=0 |