A while back I had to configure a bunch of servers with Mellanox Connect-X 4 NIC’s to talk to an ethernet Switch(Dell S6000-ON to be exact) and the NIC’s that my customer had fitted not only had really old firmware but they were also in Infiniband(I think) mode, not Ethernet. So i had to do some legwork..
First job was to update the firmware.
apt update && apt install wget unzip gcc make dkms unzip linux-headers-$(uname -r) -y
mkdir /tmp/mellanox && cd /tmp/mellanox
wget https://downloads.hpe.com/pub/softlib2/software1/pubsw-linux/p84948234/v182403/fw-ConnectX3Pro-rel-2_42_5700-764285-B21_Ax-CLP-8025-UEFI-14.11.49-FlexBoot-3.4.754.tgz
gunzip fw-ConnectX3Pro-rel-2_42_5700-764285-B21_Ax-CLP-8025-UEFI-14.11.49-FlexBoot-3.4.754.tgz
tar -xf fw-ConnectX3Pro-rel-2_42_5700-764285-B21_Ax-CLP-8025-UEFI-14.11.49-FlexBoot-3.4.754.tar
mv fw-ConnectX3Pro-rel-2_42_5700-764285-B21_Ax-CLP-8025-UEFI-14.11.49-FlexBoot-3.4.754/* .
rm -rf fw-ConnectX3Pro-rel-2_42_5700-764285-B21_Ax-CLP-8025-UEFI-14.11.49-FlexBoot-3.4.754
cd /tmp/mellanox
wget http://www.mellanox.com/downloads/MFT/mft-4.10.0-104-x86_64-deb.tgz
tar -xvf mft-4.10.0-104-x86_64-deb.tgz
cd /tmp/mellanox/mft-4.10.0-104-x86_64-deb/
bash ./install.sh
mst start
mst status
export OUT1=$(mst status | grep cr0 )
TRIMMEDOUT=(${OUT1//-/ })
CARDID=${TRIMMEDOUT[0]}
echo $CARDID
flint -d $CARDID dc /tmp/mellanox/orig_firmware.ini
flint -y -d $CARDID -i /tmp/mellanox/fw-ConnectX3Pro-rel-2_42_5700-764285-B21_Ax-CLP-8025-UEFI-14.11.49-FlexBoot-3.4.754.bin -allow_psid_change burn
Set both ports on the NIC to Ethernet mode
mst start
export OUT1=$(mst status | grep cr0 )
TRIMMEDOUT=(${OUT1//-/ })
CARDID=${TRIMMEDOUT[0]}
echo $CARDID
mlxconfig -d $CARDID query
#Turn both ports from VPI/Auto to Ethernet only:
mlxconfig -y -d $CARDID set LINK_TYPE_P1=2 LINK_TYPE_P2=2
#turn off bootrom
mlxconfig -y -d $CARDID set BOOT_OPTION_ROM_EN_P1=false
mlxconfig -y -d $CARDID set BOOT_OPTION_ROM_EN_P2=false
mlxconfig -y -d $CARDID set LEGACY_BOOT_PROTOCOL_P1=0
mlxconfig -y -d $CARDID set LEGACY_BOOT_PROTOCOL_P2=0
