Installing OFED driver and upgrading Firmware on Mellanox connectX-6 Infiniband Network Adapter.

by | Jul 16, 2020 | Howtos, Networking, News, OS, Security, Servers, Storages, Troubleshooting | 0 comments

    Step 1: Identify the Adapter model ===========================     You can use the lspci command to view the…

 

 

Step 1: Identify the Adapter model
===========================

 

 

You can use the lspci command to view the details of the Adapter.

[root@localhost ~]# lspci | grep Mellanox
a1:00.0 Infiniband controller: Mellanox Technologies MT28908 Family [ConnectX-6]

Here our Adapter is Mellanox Technologies MT28908 Family [ConnectX-6]

 

 

Step 2: Download the OFED driver for the Adapter.

===================================================================================

 

 

You can download the OFED driver from the below link and choose the OS and select the tar file for download.
https://www.mellanox.com/products/infiniband-drivers/linux/mlnx_ofed

https://www.mellanox.com/products/infiniband-drivers/linux/mlnx_ofed

You can use wget command to download the driver.

Example:  wget http://content.mellanox.com/ofed/MLNX_OFED-5.0-2.1.8.0/MLNX_OFED_LINUX-5.0-2.1.8.0-rhel8.2-x86_64.tgz

http://content.mellanox.com/ofed/MLNX_OFED-5.0-2.1.8.0/MLNX_OFED_LINUX-5.0-2.1.8.0-rhel8.2-x86_64.tgz

 

 

 

Step 3: Extract the zip file.

============================================

 

 

Untar the zip file you have just downloaded
Example: tar -xvf MLNX_OFED_LINUX-5.0-2.1.8.0-rhel8.2-x86_64.tgz

 

 

Step 4: Navigate to the extracted directory.

======================================================================

 

 

Once you have extracted the tar file you can view a directory with the same name, navigate to the directory
by cd command
Example: cd MLNX_OFED_LINUX-5.0-2.1.8.0-rhel8.2-x86_64

Once you are in the directory please list the contents on the directory using a ls command.
and from the output you can view a script file name “mlnxofedinstall”

 

 

 

Step 5: Run the script file.

==========================================

 

 

You can run the script file using “./mlnxofedinstall

Once you run the script the driver will be installed and once it is completed reload the driver or reboot the machine.

Note: The server must have Perl, python and GCC compilers for completing the driver installation.
If the installation run failed due to any unmet dependencies, please install the packages and libraries mentioned on the failed message.

 

Firmware upgrade.
————————–

 

 

Once the OFED driver is successfully installed then you can go with the firmware upgrade.

We are using the tool MSTFLINT for performing the firmware upgrade.

 

Step 1: First we need to identify the PCI bus ID of the adapter.

===========================================================================

 

 

Run “lspci | grep Mellanox” to determine the PCI bus ID

Example: [root@localhost ~]# lspci | grep Mellanox
a1:00.0 Infiniband controller: Mellanox Technologies MT28908 Family [ConnectX-6]

Here “a1:00.0” is our PCI bus ID.

 

 

Step 2: Download and unzip the Firmware Zip file.

=============================================================

 

 

 

 

You can download the firmware from the official Mellanox website.
Link: https://www.mellanox.com/support/firmware/connectx6ib

https://www.mellanox.com/support/firmware/connectx6ib

 

Please download the firmware using “wget”

Example: wget http://www.mellanox.com/downloads/firmware/fw-ConnectX6-rel-20_27_6008-MCX653105A-ECA_Ax-UEFI-14.20.25-FlexBoot-3.5.903.bin.zip

http://www.mellanox.com/downloads/firmware/fw-ConnectX6-rel-20_27_6008-MCX653105A-ECA_Ax-UEFI-14.20.25-FlexBoot-3.5.903.bin.zip

 

Once it is downloaded unzip it by using the command “unzip”

Example: unzip fw-ConnectX6-rel-20_27_6008-MCX653105A-ECA_Ax-UEFI-14.20.25-FlexBoot-3.5.903.bin.zip

Once you unzip the file you can view a .bin file which is our firmware.

Example: fw-ConnectX6-rel-20_27_6008-MCX653105A-ECA_Ax-UEFI-14.20.25-FlexBoot-3.5.903.bin

 

 

Step3: Burn the firmware to the NIC.

============================================

 

 

You need the mstflint utility to update the firmware. You can download mstflint from the OpenFabrics site at mstflint_SW for Linux.

NOTE: If OFED is installed on your machine, then mstflint is already installed.

For burning the new firmware run
the command in the same directory as the firmware file, run “mstflint –d <PCI bus ID, i.e. 05:00.0> -i <.bin file> b”

Example: mstflint -d a1:00.0 -i fw-ConnectX6-rel-20_27_6008-MCX653105A-ECA_Ax-UEFI-14.20.25-FlexBoot-3.5.903.bin b

To load new FW run mstfwreset or reboot machine and check.

Example: [root@localhost ~]# ethtool -i ib0
driver: mlx5_core[ib_ipoib]
version: 5.0-2.1.8
firmware-version: 20.27.6008 (MT_0000000222)
expansion-rom-version:
bus-info: 0000:a1:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes

Here “ib0” is our interface name, yours maybe different.

Please refer Mellanox documentation if you need any further details.

https://www.mellanox.com/products/adapter-software/firmware-tools https://www.mellanox.com/support/firmware/update-instructions

 

 

Written By Teffin Varghese

Server Administrator

Related Posts

Comments

0 Comments

0 Comments

Submit a Comment

Subscribe For Instant News, Updates, and Discounts

Pin It on Pinterest

Shares
Share This