                        ===========
                        R E A D M E
                        ===========

                Broadcom QoS Management Utility

               Copyright (c) 2018 Broadcom Ltd.
                       All rights reserved.
                          Jan 25, 2018


Introduction
~~~~~~~~~~~~
This software component is a user-mode console application for the purposes
of managing DCB parameters ETS, PFC and AppTLV's and configures the rate limit parameters
for both receive and transmit side used by Broadcom NetXtreme-E
series network controllers. The supported operations include:

- Set ETS			The Enhanced Transmission Selection (ETS) feature has a
            			configuration TLV configured with SET_ETS.
            			tsa - Transmission selection algorithm,
            			sets a comma separated list of traffic classes to the corresponding
            			selection algorithm. Valid algorithms include "ets" and "strict".
            			priority2tc - Comma separated list mapping user priorities to
            			traffic classes.
            			tcbw - Comma separated list of bandwidths for each traffic class
            			the first value being assigned to traffic class 0 and the second
            			to traffic class 1 and so on. Undefined bandwidths are presumed
            			to be 0.

- Set PFC			The PFC TLV is used to set current PFC TLV attributes. bnxtqos supports
            			"enabled" attributes only. To set the priorities with PFC enabled,
            			the set attribute takes a comma separated list of priorities to enable,
            			or the string none to disable all priorities.

- Set AppTlv			Configures a local application data field using the three
               			parameters: priority (prio), selector (sel), and protocol
               			(pid). The -d option can be provided to remove an entry
               			which matches the three parameters.

- Get QoS			Query the local TLVs which are being transmitted for a
            			given interface.

- Rate Limit			This parameter is used to add a rate limit for each TC.
               			For example, to limit tc0 to 50%, tc1 to 10%, and tc2 to 70%
               			then run below command.

               				bnxtqos -dev=p7p1 ratelimit 50,10,70

- Dump				This command is used to dump the data for user given strings.
         			Supported dump strings are "pri2cos".
         			Example:
             				bnxtqos -dev=p7p1 dump pri2cos

-buffer_pool			This command is used to configure all rx buffers. The shared option would
				assign all rx buffers to a shared buffer pool and would configure RX CoS queues
				belonging to all ports (single host) or endpoints (multi-host and multi-root) to
				use the shared buffer pool. Where as the independent option would configure an
				independent pool of equal size for each port or endpoint.

- partition_tx_ratelimit	This command is used to configure the partition rate control applies to traffic
                            	sent from a partition, which is one PF and all of its child VFs.
                            	A user can configure partition rate control through bnxtqos.


- get_partition_tx_ratelimit	Command used to query the configured partition rate control parameters.


- tx_endpoint_ratelimit		Command used to configure the transmit side rate control parameters for a given endpoint of respective port.


- get_tx_endpoint_ratelimits	Command used to query the configured transmit endpoint rate control parameters for a given port.


- rx_port_ratelimit		Command used to configure the receive side port rate control parameters.


- rx_endpoint_ratelimit		Command is used to configure the receive side rate control parameters for a given endpoint.


- get_rx_ratelimits		Command is used to query the configured receive side rate control parameters.

- get_dscp2prio                 Command is used to query the dscp to priority mapping

- get_tx_port_ratelimit         Command is used to query the transmit side of port rate limit.

- tx_port_ratelimt              Command is used to configure the transmit side of port rate limit.


Installation of bnxtqos
~~~~~~~~~~~~~~~~~~~~~~~~
bnxtqos executable is delivered for Linux only. This can be invoked from command
line directly by the user.

Dependencies
~~~~~~~~~~~~
An operational device driver for the NetXtreme-E controllers installed
in the target system is required for this application to function.

Linux kernel which supports IEEE version for DCBNL interface
needs to be installed.

Device Selection
~~~~~~~~~~~~~~~~
User needs to pass the device name of a single supported controller to be
operated on to the bnxtqos utility using the '-dev=<name>' command-line option.

Command-line Arguments
~~~~~~~~~~~~~~~~~~~~~~
To get a current list of supported command-line arguments and the usage syntax
supported by the bnxtqos utility, use the bnxtqos '-h' option.

$ bnxtqos -h

Usage: bnxtqos -dev=<interface name> <command> [-options [...]]

Broadcom NetXtreme-E QoS Configuration Utility

commands:
version          		This command is used to display program version details.

set_ets          		This command is used to configure the priority to tc and
                 		Bandwidths.This command is also used to configure only TSA
                 		and Bandwidths.

                 		Syntax:
                 			bnxtqos -dev=<interface name> set_ets tsa=<tc[0-7]:tc_type[ets/strict]>
					priority2tc=<priority[0-7]:tc> tcbw=<bandwidths> with comma seperated and
					in units of percentage (%).

                 		Example:
                 			bnxtqos -dev=p7p1 set_ets tsa=0:ets,1:ets,2:strict,3:strict,
                 			4:strict,5:strict,6:strict,7:strict priority2tc=0:0,1:0,2:0,
                 			3:0,4:0,5:1,6:0,7:0 tcbw=70,30

set_pfc          		This command is used to enable PFC on given priority. Valid values are from 0 to 7.

                 		Syntax:
                 			bnxtqos -dev=<interface name> set_pfc enabled=<0-7> the
                 			values should be with comma seperated. To disable PFC user needs
                 			to provide enabled=none.

                 		Example:
                 			bnxtqos -dev=p7p1 set_pfc enabled=5,6
                 			bnxtqos -dev=p7p1 set_pfc enabled=none

set_apptlv       		This parameter used to configure the priority of the APPTLV.

                 		Syntax:
                 			1) bnxtqos -dev=<interfac name> set_apptlv app=<priority,selector,protocol>
                 			2) bnxtqos -dev=<interfac name> set_apptlv -d app=<priority,selector,protocol>

                 		Example:
                 			1) bnxtqos -dev=p7p1 set_apptlv app=5,1,35093
                 			2) bnxtqos -dev=p7p1 set_apptlv -d app=5,1,35093

get_qos          		This command is used to get the configured priorities and bandwidth parameters.

ratelimit        		This command is used to set the rate limit for each TC in units of percentage (%).
                 		Example:
                 			bnxtqos -dev=p7p1 ratelimit 80,60,70

dump             		This command is used to dump the data for user given strings.
                 		Supported dump string is "pri2cos".
                 		Syntax:
                     			bnxtqos -dev=<device name> dump <dump string>
                 		Example:
                     			bnxtqos -dev=p7p1 dump pri2cos

buffer_pool			The command supports switching between 3 configurations: shared, independent, and ecn, descriptions below.

				Syntax:
					bnxtqos -dev=<intf> buffer_pool [rx/tx] <shared | independent | ecn> [permanent]

				Examples:

					# Size a single pool buffer and mark it as shared (between all RX CoS queues), runtime change
					./bnxtqos -dev=p1p1 buffer_pool shared

					# Evenly size independent buffer pools and assign each port / endpoint, permanent change
					./bnxtqos -dev=p1p1 buffer_pool independent permanent

				Firmware stops and resumes traffic temporarily while applying this change to hardware.

				The dev option indicates which NIC device the command applies to. The rx or tx direction option is
				non-mandatory, and will default to rx if not specified, since only rx is currently supported.
				It specifies whether the user intends to operate on rx or tx buffer pools. The shared option would
				assign all rx buffers to a shared buffer pool and would configure RX CoS queues belonging to all
				ports (single host) or endpoints (multi-host and multi-root) to use the shared buffer pool.

				The independent option would configure an independent pool of equal size for each port or endpoint.
				Up to 4 hardware buffer pools are supported. In a single host deployment, total size of the rx
				buffer available is carved up evenly between each port: total number of buffers is divided by the number
				of ports to figure out the number of buffers per port, each buffer pool is then sized to that number.
				Once all buffer pools are sized, RX CoS queues belonging to each port are configured to use an independent
				buffer pool. Similar processing is done in firmware in the multi-root or multi-host case, but instead of
				dividing resources per ports, they’re divided per endpoint, so the total is divided by the number of endpoints
				to figure out the number of buffers per endpoint, and each buffer pool is sized to that number. RX CoS queues
				belonging to each endpoint are, then, configured to use independent buffer pools.

				The default configuration is runtime, which indicates that the configuration takes effect immediately.
				The change is not stored in NVRAM.

				The permanent option is an optional parameter indicates that the configuration is written to NVRAM, but it
				does not take effect immediately. The change may be activated by resetting the firmware or rebooting the
				host or power cycling the system.


partition_tx_ratelimit		A user can configure partition transmit rate control using bnxtqos.

				syntax:
					bnxtqos -dev=<intf> partition_tx_ratelimit max=<max_val> [permanent]

				Examples:
				1) The following example configures a maximum rate of 60% of the link bandwidth for the partition
                                   anchored by PF p1p1:
					./bnxtqos -dev=p1p1 partition_tx_ratelimit max=60

				The dev option indicates the PF partition to which the configuration applies.

				The max option specifies the maximum aggregate rate at which the PF and its child VFs can transmit.
				The sum of the max rate limits for all partitions on a port may exceed the link bandwidth.

				Each rate limit is specified as a percentage of link bandwidth. The firmware automatically adjusts the rate
				limits when link speed changes. Only a maximum rate can be configured. Configuration of a minimum guaranteed
                                rate is not supported.

				By default, the command updates the running configuration. The change is not stored in NVRAM.
				The permanent option indicates that the configuration is written to NVRAM, but it does not take effect immediately.
				The change may be activated by resetting the firmware or rebooting the host or power cycling the system.

				The valid range for max rate is an integer, from 0 to 100. A value of 0 signifies no rate limit, in which case
				the firmware deletes any previous rate limit. By default, the firmware does not enforce a maximum rate.

				This command may be configured for a PF regardless of the number of PFs associated with a port, even if there is
				only one partition. This feature assumes the number of partitions is established when firmware initializes and
				cannot be changed without restarting the firmware.


get_partition_tx_ratelimit	The user can query the partition rate control configuration for a given partition using the following command.

				Syntax:
					bnxtqos -dev=<intf> get_partition_tx_ratelimit [permanent]

				Example: showing the running configuration for the partition with PF p1p1.
					./bnxtqos -dev=p1p1 get_partition_tx_ratelimit
				Output:
					max: 60%

				By default, the command queries the current running configuration. The dev parameter specifies the partition to show.
				If the permanent option is given, the configuration stored in NVRAM is shown.

tx_endpoint_ratelimit		The user can configure PCIe endpoint transmit rate control using bnxtqos. The command syntax is as follows:

				Syntax:
					bnxtqos -dev=<intf> tx_endpoint_ratelimit port_idx=<port>
                                                 ep0  max=<max_val>
   						[ep1  max=<max_val>]
   						[ep2  max=<max_val>]
   						[ep3  max=<max_val>]
   						[permanent]

				Example:
					1) On a 1-port NIC (port-idx of 0 is implied) with two endpoints, allow each endpoint to use
					   up to 50% of the transmit bandwidth:

						./bnxtqos -dev=p1p1 tx_endpoint_ratelimit ep0 max=50 ep2 max=50

					2) On a 2-port NIC with two endpoints, allow the CPU attached to EP 0 to use up to 60% of each link,
					   and allow the CPU attached to EP 2 to use up to 40% of each link:

						./bnxtqos -dev=p1p1 tx_endpoint_ratelimit port_idx=0 ep0 max=60 ep2 max=40
						./bnxtqos -dev=p1p1 tx_endpoint_ratelimit port_idx=1 ep0 max=60 ep2 max=40

					The dev option must be a valid interface on the device, but it need not be associated with the endpoint
					being configured. For example, this command, where p1p1 is on endpoint 0, is legal:

						./bnxtqos -dev=p1p1 tx_endpoint_ratelimit ep2 max=50

					An endpoint rate limit only has an effect when set below the endpoint’s effective PCIe bandwidth.

				The dev option can specify any interface on the NIC. On a multi-host system, the dev option must be an interface on
				the host where the command is issued, but does not need to be an interface on the endpoint being configured.

				The port_idx option specifies the index of an external port. The range is 0 to one less than the number of
				external ports on the NIC.

				The user may specify rate limits for one or more endpoints on the port. Of course, rate limits can only be specified
				for endpoints that exist on the system. Only a maximum rate can be configured. Configuration of a minimum guaranteed
                                rate is not supported.

				The max option specifies the maximum aggregate transmit rate of traffic from all functions on the endpoint.
				It is specified as a percentage of the bandwidth of the specified port. The command accepts integer values from 0 to 100.
				A value of 0 indicates no maximum rate limit.

				By default, this command updates the running configuration. The change is not stored in NVRAM. The permanent option
				indicates that the configuration is written to NVRAM, but it does not take effect immediately. The change may be
				activated by resetting the firmware or rebooting the host or power cycling the system.


get_tx_endpoint_ratelimits	The user can query the endpoint rate limits.

				Syntax:
					bnxtqos -dev=<intf> get_tx_endpoint_ratelimits port_idx=<port> [permanent]

				Example:
					1) The response shows the configured rate limits as a table, such as the following:

						Endpoint       Max (%)
       					    	    0           60
       					   	    2           60

				The output only lists active endpoints where a rate limit has been configured. On a system with two endpoints,
				the output only includes rates for EP0 and EP2. If no endpoint has a transmit rate limit, bnxtqos prints a message, such as:

				Example:
					2) bnxtqos -dev=p1p1 get_tx_endpoint_ratelimits port_idx=>port>
					   Error: No endpoint transmit rate limits configured

				By default, the current running configuration is shown. If the permanent option is given, the configuration stored in NVRAM is shown.


rx_port_ratelimit		The user can configure receive rate control that applies to all traffic in a receive CoS queue group.
				A queue group has one queue for each traffic class. When the NIC queues received packets according to
				the packet’s destination, there is a queue group per PCIe endpoint and this rate limit shapes traffic
				to the endpoint. When the NIC queues receives packets according to the receive panel port, there is a
				queue group per port, and this rate limit shapes all traffic received on one port. Only a maximum rate
				may be configured; configuration of a minimum guaranteed rate is not supported.

				On a single host system, the syntax is as follows. The port index is inferred from the dev parameter.

				Syntax:
					bnxtqos -dev=<intf> rx_port_ratelimit max=<max_val> [permanent]

				The max option specifies an 8-bit rate limit as a percentage of total link bandwidth, with a range of 0 to 100 percent.
				A value of 0 indicates no rate limit, and deletes a previously configured rate limit.

				When the NIC queues packets according to a packet’s destination (as it does for multi-host or multi-root), the firmware
				automatically computes an endpoint rate limit as a percentage of the endpoint’s PCIe bandwidth. The configured rate
				limit takes effect if it is less than the automatic rate limit.

				The configuration takes effect immediately. The change is not stored in NVRAM. However, if the permanent option is given,
				the configuration is written to NVRAM, but it does not take effect.

rx_endpoint_ratelimit		The user can configure receive rate control that applies to all traffic in a receive CoS queue group.
				A queue group has one queue for each traffic class. When the NIC queues received packets according to
				the packet’s destination, there is a queue group per PCIe endpoint and this rate limit shapes traffic
				to the endpoint. When the NIC queues receives packets according to the receive panel port, there is a
				queue group per port, and this rate limit shapes all traffic received on one port. Only a maximum rate
				may be configured; configuration of a minimum guaranteed rate is not supported.

				For multi-host or multi-root, the syntax is as follows. The command specifies the endpoint index explicitly,
				so that a user can specify the endpoint rate limit for all endpoints from one host.
				On 4-endpoint systems, ep is 0 to 3. On 2-endpoint systems, ep is 0 or 2.

				Syntax:
                                  	bnxtqos -dev=<intf> rx_endpoint_ratelimit ep0 max=<max_val> [ep1 max=<max_val>] [ep2 max=<max_val>]
                                                 [ep3 max=<max_val>] [permanent]

				The max option specifies an 8-bit rate limit as a percentage of total link bandwidth, with a range of 0 to 100 percent.
				A value of 0 indicates no rate limit, and deletes a previously configured rate limit.

				When the NIC queues packets according to a packet’s destination (as it does for multi-host or multi-root), the firmware
				automatically computes an endpoint rate limit as a percentage of the endpoint’s PCIe bandwidth. The configured rate
				limit takes effect if it is less than the automatic rate limit.

				The configuration takes effect immediately. The change is not stored in NVRAM. However, if the permanent option is given,
				the configuration is written to NVRAM, but it does not take effect.

get_rx_ratelimits		The user can query the receive rate limits using bnxtqos.

				Syntax:
					bnxtqos -dev=<intf> get_rx_ratelimits [permanent]

				Example:
					QueueGroup0 max: 25%
					QueueGroup1 max: 25%
      					QueueGroup2 max: 25%
					QueueGroup3 max: 25%

				By default, this command queries the current running configuration. If the permanent option is given,
				the configuration stored in NVRAM is shown.

get_dscp2prio                   This command is used to query the dscp to priority mapping

                                Syntax:
                                      bnxtqos -dev=<intf> get_dscp2prio


get_tx_port_ratelimit            This command is used to query the transmit side of port rate limit.
                                 And the tx rate limit value will be displayed in the units of Mbps.

                                 Syntax:

                                 bnxtqos -dev=<intf> get_tx_port_ratelimit

tx_port_ratelimt                 This command is used to configure the transmit side of port rate limit.
                                 User provided value should be in the units of Mbps. If user wants to cancel
                                 the transmit port rate limit, user has to provide the value Zero(0).

                                 Syntax:

                                 bnxtqos -dev=<intf> tx_port_ratelimit max=<tx rate limit value>


options:
-v[v][v]         Enable extra console output (increase verbosity).
-V               Enable maximum verbosity of console output.
-d               Removes the application TLV and this option supports only
                 with set_apptlv command.
-h               Display help for basic command-line syntax.

Commands
--------
Only one command should be specified per invocation of the bnxtqos command-line.

