To optimize the case of ping-ponging between two buffers, the DDP code
caches the last two buffers used keeping the pages wired and page pods
stored in the NIC's RAM. If a new aio_read() request uses one of the
same buffers, then the work of holding pages, etc. can be avoided.
However, the starting virtual address of an aio buffer was not saved,
only the page count, length, and initial page offset. Thus, an
aio_read() request could match a different buffer in the address
space. (Earlier during development vm_fault_hold_quick_pages() was
always called and the vm_page_t values were compared, but that was
eventually removed without being adequately replaced.) Fix by storing
the starting virtual address and comparing that (along with other
fields) to determine if a buffer can be reused.
Sponsored by: Chelsio Communications
cxgbe(4): Update the T4, T5, and T6 firmwares to 1.16.45.0.
The latest firmware has a number of link related fixes, support for a
new custom card, and the fix for a bug that affected rate limiting on
FreeBSD.
Relnotes: Yes
Sponsored by: Chelsio Communications
r318014:
cxgbe(4): Fixes related to the knob that controls link autonegotiation.
- Do not leak the adapter lock in sysctl_autoneg.
- Accept only 0 or 1 as valid settings for autonegotiation.
- A fixed speed must be requested by the driver when autonegotiation is
disabled otherwise the firmware will reject the l1cfg command. Use
the top speed supported by the port for now.
r318091:
cxgbe(4): Do not assume that if_qflush is always followed by inteface-down.
r318125:
Adjust whitespace and fix a comment. No functional change.
r318263:
cxgbe(4): netmap-only interrupts for a VI do not have an associated rxq
or ofld_rxq and should be ignored by vi_intr_iq.
Sponsored by: Chelsio Communications
r317702:
cxgbe(4): Support routines for Tx traffic scheduling.
- Create a new file, t4_sched.c, and move all of the code related to
traffic management from t4_main.c and t4_sge.c to this file.
- Track both Channel Rate Limiter (ch_rl) and Class Rate Limiter (cl_rl)
parameters in the PF driver.
- Initialize all the cl_rl limiters with somewhat arbitrary default
rates and provide routines to update them on the fly.
- Provide routines to reserve and release traffic classes.
r317847:
cxgbe(4): The Tx scheduler initialization either works or doesn't. It
doesn't need a refresh in either case.
r318307:
cxgbe(4): Avoid an out of bounds access when an attempt to unbind a tx
queue from a traffic class fails.
Sponsored by: Chelsio Communications
r317820:
cxgbe(4): Update the list of PCIe devices claimed by the driver. At
this point any board with a T6 should just work.
r317837:
cxgbe(4): Update the VF device ids too. This should have been part
of r317820.
Sponsored by: Chelsio Communications
cxgbe: Add tunables to control the number of LRO entries and the number
of rx mbufs that should be presorted before LRO. There is no change in
default behavior.
Sponsored by: Chelsio Communications
cxgbe: Add a tunable to configure the SGE time scaler, which is
available starting with T6. The values in the timer holdoff registers
are multiplied by the scaling factor before use.
dev.<nexus>.<n>.holdoff_timers shows the final values of the
timers in microseconds.
Sponsored by: Chelsio Communications
cxgbe(4): netmap does not set IFCAP_NETMAP in an ifnet's if_capabilities
any more (since r307394). Do it in the driver instead.
Sponsored by: Chelsio Communications
cxgbe(4): Allow tunables that control the number of queues to be set to
'-n' to tell the driver to create _up to_ 'n' queues if enough cores are
available. For example, setting hw.cxgbe.nrxq10g="-32" will result in
16 queues if the system has 16 cores, 32 if it has 32.
There is no change in the default number of queues of any type.
Sponsored by: Chelsio Communications
cxgbe/t4_tom: Fix CLIP entry refcounting on the passive side. Every
IPv6 connection being handled by the TOE should have a reference on its
CLIP entry.
Sponsored by: Chelsio Communications
r316940, and r317410.
r311880:
The iw_cxgb and iw_cxgbe drivers should not use a FreeBSD device_t where
a linuxkpi style device is expected. If OFED/linuxkpi actually starts
using this field then we'll have to figure out whether to create fake
devices for these drivers or have linuxkpi deal with NULL device.
This mismatch was first reported as part of D6585.
r314167:
cxgbe/iw_cxgbe: Minor changes for T6.
r316118:
cxgbe/iw_cxgbe: T6 has no limit on the amount of memory that can be
registered in one ib_reg_phys_mr.
r316571:
cxgbe/iw_cxgbe: Remove bad cast that resulted in incorrect length for
memory regions larger than 4GB.
r316573:
cxgbe/iw_cxgbe: Replace a magic constant with something more readable
(and accurate).
T4 and later have an extra bit for page shift so the maximum page size
is 8TB (shift of 12 + 31) instead of 128MB (12 + 15). This saves space
in the chip's PBL (physical buffer list) when registering very large
memory regions.
r316580:
cxgbe/iw_cxgbe: Remove another bad cast. This should have been
included in r316571.
r316936:
cxgbe/iw_cxgbe: hw supports 64K (not 32K) Protection Domains.
r316937:
cxgbe/iw_cxgbe: Report accurate page_size_cap in ib_query_device.
r316940:
cxgbe/iw_cxgbe: Report the actual values of various parameters as
configured by the firmware.
r317410:
cxgbe/iw_cxgbe: Pull in some updates to c4iw_wait_for_reply from the
iw_cxgb4 Linux driver.
Sponsored by: Chelsio Communications
r315201:
cxgbe(4): Fix an always-true assertion (reported by PVS-Studio).
sys/dev/cxgbe/t4_main.c: PVS-Studio: Expression is Always True (CWE-571) (3)
r315920:
cxgbe/iw_cxgbe: c4iw_connect should always returns a -ve errno on failure.
r315921:
cxgbe/iw_cxgbe: alloc_ep expects a gfp_t, and it's always ok to sleep during
alloc_ep.
r315922:
cxgbe/iw_cxgbe: allocations that use GFP_KERNEL (which is M_WAITOK on
FreeBSD) cannot fail.
r316008:
cxgbe/iw_cxgbe: Remove unused code.
r316062:
cxgbe/iw_cxgbe: Defer the handling of error CQEs and RDMA_TERMINATE to
the thread that deals with socket state changes. This eliminates
various bad races with the ithread.
r314814:
cxgbe/iw_cxgbe: Abort connection if there is an error during c4iw_modify_qp.
r315325:
cxgbe/iw_cxgbe: Use the socket and not the toepcb to reach for the
inpcb. t4_tom detaches the inpcb from the toepcb as soon as the
hardware is done with the connection (in final_cpl_received) but the
socket is around as long as the cm_id and the rest of iWARP state is.
This fixes an intermittent NULL dereference during abort.
r314509:
cxgbe/iw_cxgbe: Do not check the size of the memory region being
registered. T4/5/6 have no internal limit on this size. This is
probably a copy paste from the T3 iw_cxgb driver.
r314578:
cxgbe/iw_cxgbe: Implement sq/rq drain operation.
ULPs can set a qp's state to ERROR and then post a work request on the
sq and/or rq. When the reply for that work request comes back it is
guaranteed that all previous work requests posted on that queue have
been drained.
Sponsored by: Chelsio Communications
cxgbe/iw_cxgbe: fix various double-close panics with iWARP sockets.
Sockets representing the TCP endpoints for iWARP connections are
allocated by the ibcore module. Before this revision they were closed
either by the ibcore module or the iw_cxgbe hardware driver depending on
the state transitions during connection teardown. This is error prone
and there were cases where both iw_cxgbe and ibcore closed the socket
leading to double-free panics. The fix is to let ibcore close the
sockets it creates and never do it in the driver.
- Use sodisconnect instead of soclose (preceded by solinger = 0) in the
driver to tear down an RDMA connection abruptly. This does what's
intended without releasing the socket's fd reference.
- Close the socket in ibcore when the iWARP iw_cm_id is destroyed. This
works for all kinds of sockets: clients that initiate connections,
listeners, and sockets accepted off of listeners.
Sponsored by: Chelsio Communications
undo_offload_socket() is only called by t4_connect() during a connection
setup failure, but t4_connect() still owns the TOE PCB and frees ita
after undo_offload_socket() returns. Release a reference in
undo_offload_socket() resulted in a double-free which panicked when
t4_connect() performed the second free. The reference release was
added to undo_offload_socket() incorrectly in r299210.
Sponsored by: Chelsio Communications
- Check for Chelsio vendor ID in probe routines.
- Fail attach instead of faulting if pci_find_dbsf() doesn't find a
device.
PR: 216539
Sponsored by: Chelsio Communications
r311831:
cxgbe(4): The wraparound logic in start_wrq_wr() should not get involved
in work requests that end at the end of the descriptor ring, even though
the pidx wraps around to 0.
r311832:
cxgbe(4): Enable automatic cidx flush for all control queues.
r311569:
Fix comment in t4_tom. No functional change.
r311657:
cxgbe/t4_tom: Fix tid accounting. An offloaded IPv6 connection uses 2
tids, not 1, in the hardware.
r311949:
cxgbe/tom: Add VIMAGE support to the TOE driver.
Active Open:
- Save the socket's vnet at the time of the active open (t4_connect) and
switch to it when processing the reply (do_act_open_rpl or
do_act_establish).
Passive Open:
- Save the listening socket's vnet in the driver's listen_ctx and switch
to it when processing incoming SYNs for the socket.
- Reject SYNs that arrive on an ifnet that's not in the same vnet as the
listening socket.
CLIP (Compressed Local IPv6) table:
- Add only those IPv6 addresses to the CLIP that are in a vnet
associated with one of the card's ifnets.
Misc:
- Set vnet from the toepcb when processing TCP state transitions.
- The kernel sets the vnet when calling the driver's output routine
so t4_push_frames runs in proper vnet context already. One exception
is when incoming credits trigger tx within the driver's ithread. Set
the vnet explicitly in do_fw4_ack for that case.
Sponsored by: Chelsio Communications
r310151:
cxgbe(4): Changes to the default T6 firmware configuration file.
- Disable features that are not supported or not used on FreeBSD.
- Increase the RSS table slice per interface.
- Increase the share of the TCAM reserved for filtering.
r311173:
cxgbe(4): Update T4, T5 and T6 firmwares to 1.16.26.0.
Sponsored by: Chelsio Communications
r309666:
cxgbe(4): unsigned short isn't large enough to store link speed (which
is in Mbps) for 100Gbps links.
r310033:
cxgbe(4): Retire t4_bus_space_read_8 and t4_bus_space_write_8.
r310049:
cxgbe(4): Fix the tid range shown for T6 cards in misc.tids.
r310100:
cxgbe(4): Deal with compressed error vectors.
r310152:
cxgbe(4): Fix typo in an unused macro.
r310807:
cxgbe(4): Updates to link configuration.
- Update struct link_settings and associated shared code.
- Add tunables to control FEC and autonegotiation. All ports inherit
these values as their initial settings.
hw.cxgbe.fec
hw.cxgbe.autoneg
- Add per-port sysctls to control FEC and autonegotiation. These can be
modified at any time.
dev.<port>.<n>.fec
dev.<port>.<n>.autoneg
There are 4 independent knobs in T5+ chips to include or exclude PAUSE
frames from the "total frames" and "multicast frames" counters in either
direction. This change lets the driver deal with any combination of
these settings.
cxgbe(4): Fix bug in the calculation of the number of physically
contiguous regions in an mbuf chain.
If the payload of an mbuf ends at a page boundary count_mbuf_nsegs would
incorrectly consider the next mbuf's payload physically contiguous based
solely on a KVA comparison.
306821:
cxgbe(4): Add an ioctl to copy a firmware config file to the card's flash.
306823:
cxgbetool: Add a loadcfg subcommand to allow a user to upload a firmware
configuration file to the card.
305908,306062,306063,306137,306138,306206,306216,306273,306295,306301,
306465,309302:
Add support for adapters using the Terminator T6 ASIC.
305695:
cxgbe(4): Set up fl_starve_threshold2 accurately for T6.
305696:
cxgbe(4): Use correct macro for header length with T6 ASICs. This
affects the transmit of the VF driver only.
305699:
cxgbe(4): Update the pad_boundary calculation for T6, which has a
different range of boundaries.
305702:
cxgbe(4): Use smaller min/max bursts for fl descriptors with a T6.
305703:
cxgbe(4): Deal with the slightly different SGE_STAT_CFG in T6.
305713:
cxgbe(4): Add support for additional port types and link speeds.
305715:
cxgbe(4): Catch up with the rename of tlscaps -> cryptocaps. TLS is one
of the capabilities of the crypto engine in T6.
305827:
cxgbe(4): Use the interface's viid to calculate the PF/VF/VFValid fields
to use in tx work requests.
305852:
cxgbe(4): Attach to cards with the Terminator 6 ASIC. T6 cards will
come up as 't6nex' nexus devices with 'cc' ports hanging off them.
The T6 firmware and configuration files will be added as soon as they
are released. For now the driver will try to work with whatever
firmware and configuration is on the card's flash.
305906:
cxgbe/t4_tom: The SMAC entry for a VI is at a different location in the T6.
305908:
cxgbe/t4_tom: Update the active/passive open code to support T6. Data
path works as-is.
306062:
cxgbe(4): Show wcwr_stats for T6 cards.
306063:
cxgbe(4): Setup congestion response for T6 rx queues.
306137:
cxgbetool: Add T6 support to the SGE context decoder.
306138:
Fix typo.
306206:
cxgbe(4): Catch up with the different layout of WHOAMI in T6.
Note that the code moved below t4_prep_adapter() as part of this change
because now it needs a working chip_id().
306216:
cxgbe(4): Fix the output of the "tids" sysctl on T6.
306273:
cxgbe(4): Fix netmap with T6, which doesn't encapsulate SGE_EGR_UPDATE
message inside a FW_MSG. The base NIC already deals with updates in
either form.
306295:
cxgbe(4): Support SIOGIFXMEDIA so that ifconfig displays correct media
for 25Gbps and 100Gbps ports. This should have been part of r305713,
which is when the driver first started reporting extended media types.
306301:
cxgbe(4): Use the port's top speed to figure out whether it is "high
speed" or not (for the purpose of calculating the number of queues etc.)
This does the right thing for 25Gbps and 100Gbps ports.
306465:
cxgbe(4): Claim the T6 -DBG card.
309302:
cxgbe(4): Include firmware for T6 cards in the driver. Update all
firmwares to 1.16.12.0.
Sponsored by: Chelsio Communications
cxgbe/t4_tom: toepcb should be all-zero on allocation because the code
that cleans up on failure assumes that non-NULL values indicate
initialized items.
303688:
cxgbe/t4_tom: Read the chip's DDP page sizes and save them in a
per-adapter data structure. This replaces a global array with hardcoded
page sizes.
303750:
cxgbe/t4_tom: The page pod arena allocates from pod address space and
not index space. The minimum valid allocation out of this arena is the
size of a single page pod.
305166:
cxgbe/t4_tom: Add general purpose routines to deal with page pod regions
and allocations within them. Switch to these routines to manage the TOE
DDP region.
305167:
cxgbe/t4_tom: Two new routines to allocate and write page pods for a
buffer in the kernel's address space.
Sponsored by: Chelsio Communications
cxgbe(4): Initialize the adapter queues (fwq and mgmtq) instead of
returning EAGAIN if they aren't available when the user tries to program
a filter. Do this after validating the filter so that the driver
doesn't bring up the queues if it doesn't have to.
302440:
cxgbe(4): Add sysctl to display the RSS indirection table size for an
interface.
dev.cxl.<n>.rss_size
dev.vcxl.<n>.rss_size
304873:
cxgbe(4): Provide more details about the card in the sysctl MIB.
dev.t5nex.0.%desc: Chelsio T580-CR
dev.t5nex.0.hw_revision: 1
dev.t5nex.0.sn: PT13140042
dev.t5nex.0.pn: 110117150A0
dev.t5nex.0.ec: 0000000000000000
dev.t5nex.0.na: 0007432AF490
dev.t5nex.0.vpd_version: 3
dev.t5nex.0.scfg_version: 53255
dev.t5nex.0.bs_version: 1.1.0.0
dev.t5nex.0.er_version: 1.0.0.68
dev.t5nex.0.tp_version: 0.1.4.9
dev.t5nex.0.firmware_version: 1.16.2.0
305704:
cxgbe(4): Rename the debug_flags driver tunable/sysctl to dflags.
Tunables that end with _flags are special.
305985:
cxgbe(4): Fixes to wrq stats.
- Increment tx_wrs_copied in the correct place.
- Add tx_wrs_sspace to the sysctl MIB.
306787:
cxgbe(4): Fix whitespace in the pm_stats display.
307531:
cxgbe(4): Adjust whitespace to line up the column titles in cim_qcfg
with the values displayed.
Sponsored by: Chelsio Communications
- Return appropriate error code instead of ENOMEM when sosend() fails in
send_mpa_req.
- Fix for problematic race during destroy_qp.
- Abortive close in the failure of send_mpa_reject() instead of normal close.
- Remove the unnecessary doorbell flowcontrol logic.
Sponsored by: Chelsio Communications
Specifically, the FW_PORT_CMD may or may not work for a VF (the PF
driver can choose whether or not to permit access to this command),
so don't attempt to fetch port information on a VF if permission is
denied by the PF.
If a packet contains the Ethernet header (14 bytes) in the first mbuf
and the payload (IP + UDP + data) in the second mbuf, then the attempt
to fetch the l3hdr will return a NULL pointer. The first loop iteration
will drop len to zero and exit the loop without setting 'p'. However,
the desired data is at the start of the second mbuf, so the correct
behavior is to loop around and let the conditional set 'p' to m_data of
the next mbuf (and leave offset as 0).
Chelsio T4/T5 VF driver.
303522:
Various fixes to the t4/5nex character device.
- Remove null open/close methods.
- Don't set d_flags to 0 explicitly.
- Remove t5_cdevsw as the .d_name member isn't really used and doesn't
warrant a separate cdevsw just for the name.
- Use ENOTTY as the error value for an unknown ioctl request.
- Use make_dev_s() to close race with setting si_drv1.
303647:
Store the offset of the KDOORBELL and GTS registers in the softc.
VF devices use a different register layout than PF devices. Storing
the offset in a value in the softc allows code to be shared between the
PF and VF drivers.
303860:
Reserve an adapter flag IS_VF to mark VF devices vs PF devices.
303880:
Track the base absolute ID of ingress and egress queues.
Use this to map an absolute queue ID to a logical queue ID in interrupt
handlers. For the regular cxgbe/cxl drivers this should be a no-op as
the base absolute ID should be zero. VF devices have a non-zero base
absolute ID and require this change. While here, export the absolute ID
of egress queues via a sysctl.
304168:
Make SGE parameter handling more VF-friendly.
Add fields to hold the SGE control register and free list buffer sizes to
the sge_params structure. Populate these new fields in
t4_init_sge_params() for PF devices and change t4_read_chip_settings() to
pull these values out of the params structure instead of reading
registers directly. This will permit t4_read_chip_settings() to be reused
for VF devices which cannot read SGE registers directly.
While here, move the call to t4_init_sge_params() to
get_params__post_init(). The VF driver will populate the SGE parameters
structure via a different method before calling t4_read_chip_settings().
304169:
Update mailbox writes to work with VF devices.
- Use alternate register locations for the data and control registers for
VFs.
- Do a dummy read to force the writes to the mailbox data registers to
post before the write to the control register on VFs.
- Do not check the PCI-e firmware register for errors on VFs.
304170:
Add support for register dumps on VF devices.
- Add handling of VF register sets to t4_get_regs_len() and t4_get_regs().
- While here, use t4_get_regs_len() in the ioctl handler for regdump
instead of inlining it.
304479:
Add structures for VF-specific adapter parameters.
While here, mark which parameters are PF-specific and which are
VF-specific.
304485:
Reorder sysctls so that nodes shared with the VF driver are added first.
This permits a single early return for VF devices in the routines that
add sysctl nodes.
305549:
Chelsio T4/T5 VF driver.
The cxgbev/cxlv driver supports Virtual Function devices for Chelsio
T4 and T4 adapters. The VF devices share most of their code with the
existing PF4 driver (cxgbe/cxl) and as such the VF device driver
currently depends on the PF4 driver.
Similar to the cxgbe/cxl drivers, the VF driver includes a t4vf/t5vf
PCI device driver that attaches to the VF device. It then creates
child cxgbev/cxlv devices representing ports assigned to the VF.
By default, the PF driver assigns a single port to each VF.
t4vf_hw.c contains VF-specific routines from the shared code used to
fetch VF-specific parameters from the firmware.
t4_vf.c contains the VF-specific PCI device driver and includes its
own attach routine.
VF devices are required to use a different firmware request when
transmitting packets (which in turn requires a different CPL message
to encapsulate messages). This alternate firmware request does not
permit chaining multiple packets in a single message, so each packet
results in a firmware request. In addition, the different CPL message
requires more detailed information when enabling hardware checksums,
so parse_pkt() on VF devices must examine L2 and L3 headers for all
packets (not just TSO packets) for VF devices. Finally, L2 checksums
on non-UDP/non-TCP packets do not work reliably (the firmware trashes
the IPv4 fragment field), so IPv4 checksums for such packets are
calculated in software.
Most of the other changes in the non-VF-specific code are to expose
various variables and functions private to the PF driver so that they
can be used by the VF driver.
Note that a limited subset of cxgbetool functions are supported on VF
devices including register dumps, scheduler classes, and clearing of
statistics. In addition, TOE is not supported on VF devices, only for
the PF interfaces.
Sponsored by: Chelsio Communications
John Baldwin