310048
hyperv: Implement "enlightened" time counter, which is rdtsc based.
Reviewed by: kib
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8763
310101
hyperv: Allow userland to ro-mmap reference TSC page
This paves way to implement VDSO for the enlightened time counter.
Reviewed by: kib
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8768
310239
hyperv: Implement userspace gettimeofday(2) with Hyper-V reference TSC
This 6 times gettimeofday performance, as measured by
tools/tools/syscall_timing
Reviewed by: kib
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8789
309240
hyperv/vmbus: Add result polling support for xact API.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8633
309242
hyperv/vmbus: Add result polling support for message Hypercall API.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8634
309244
hyperv/vmbus: Add exec cancel support for message Hypercall API.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8635
309245
hyperv/vmbus: Use poll/cancel APIs to wait for the CHOPEN response.
Since hypervisor does not respond CHOPEN to a revoked channel.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8636
309670
hyperv/vmbus: Use pause if possible.
This makes booting on Hyper-V w/ small # of vCPUs work properly.
Reported by: Hongxiong Xian <v-hoxian microsoft com>, Hongjiang Zhang <honzhan microsoft com>
Sponsored by: Microsoft
309128
hyperv/vmbus: Commit the GPADL id only after the connection succeeds.
Minor style change.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8563
309129
hyperv/vmbus: Minor style changes.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8564
309131
hyperv/vmbus: Fix sysctl tree leakage, if channel open fails.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8565
309132
hyperv/vmbus: Don't close unopened channels.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8566
309133
hyperv/vmbus: GPADL disconnect error on a revoked channel is benign.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8567
309134
hyperv/vmbus: No stranded bufring GPADL is allowed.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8568
309135
hyperv/vmbus: Return EISCONN if the bufring GPADL can't be disconnected.
So that the callers of vmbus_chan_open_br() could handle the passed in
bufring memory properly.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8569
309136
hyperv/vmbus: Don't free the bufring if its GPADL can't be disconnected.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8570
309138
hyperv/vmbus: Always try disconnect/free bufring memory upon channel close
While I'm here, minor wording and style changes.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8598
309139
hyperv/vmbus: Propagate close error.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8599
309140
hyperv/vmbus: Add a simplified version of channel close.
So that the caller can know the channel close error and react accordingly.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8600
309224
hyperv/vmbus: Zero out GPADL if error happens.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8601
309225
hyperv/vmbus: Add supportive transaction wait function.
This function supports channel revocation properly.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8611
309030
hyperv/vmbus: Set a mark on the revoked channel.
This will be used to fix device detach DEVMETHOD for revoked primary
channel.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8522
309039
hyperv/vmbus: Merge free/active locks.
These functions are only used by management stuffs, so there are
no needs to introduce extra complexity.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8524
309080
hyperv/vmbus: Implement orphan support for transaction API
It will be used to fix the primary channel revocation support.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8525
309081
hyperv/vmbus: Fix the primary channel revoking on vmbus side.
Drivers can now use vmbus_chan_{is_revoked,set_orphan,unset_orphan}() and
vmbus_xact_ctx_orphan() to fix their attach/detach DEVMETHODs for revoked
primary channels.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8545
309083
hyperv/vmbus: Fix the multi-channel revoking on vmbus side.
- Reference count the sub-channel when channel offer message is
processed, so that immediate rescind message on the same channel
will not race sub-channel open on driver side.
- Drop the above reference when sub-channel is closed, this closely
mimics the hypervisor's reaction when primary channel is closed
on the VM side. No drivers use sub-channel after primary channel
is closed.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8546
307012
hyperv/vmbus: Allow driver to inject synchronous task into channel taskq.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8208
307013
hyperv/hn: Rework link status support.
This is the preamble for network device SR-IOV and
NDIS_STATUS_NETWORK_CHANGE handling.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8209
307262
hyperv/hn: Management parts always need suspend and resume.
Sponsored by: Microsoft
306360
hyperv/vmbus: Add dynamic device add and remove support
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8008
306387
hyperv/vmbus: Add functions to test RX/TX bufring emptiness
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8044
306389
hyperv/vmbus: Add function to drain channel interrupt task.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8045
306072
hyperv/vmbus: Allow bufrings preallocation.
The assumption that the channel is only opened upon synthetic device
attach time no longer holds, e.g. Hyper-V network device MTU changes.
We have to allow device drivers to preallocate bufrings, e.g. in
attach DEVMETHOD, to prevent bufring allocation failure once the
system memory is fragmented after running for a while.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7960
306073
hyperv/hn: Allocate bufrings in attach DEVMETHOD.
So that reinitialization, e.g. MTU change, will not fail when the system
memory is excessively fragmented.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7961
306074
hyperv/vmbus: Assert that the bufring address is page aligned.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7962
306076
hyperv/hn: Put debug messages under bootverbose
While I'm here, strip blank line.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7963
304790
hyperv/vmbus: Add function to calculate max # of elements in a bufring.
Sponsored by: Microsoft
304791
hyperv/storvsc: Increase queue depth and rework channel selection.
- Increasing queue depth gives ~100% performance improvement for
randwrite fio test in Azure.
- New channel selection, which takes LUN id and the current cpuid
into consideration, gives additional ~20% performance improvement
for ranwrite fio test in Azure.
Submitted by: Hongzhang Jiang <honzhan microsoft com>
Modified by: sephe
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7622
303603
hyperv/vmbus: Remove the artificial entry limit of SG and PRP list.
Just make sure that the total channel packet size does not exceed 1/2
data size of the TX bufring.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7359
303604
hyperv/storvsc: Set maxio to 128KB.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7360
303605
hyperv/storvsc: Stringent PRP list assertions
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7361
303764
hyperv/vmbus: Only make sure the TX bufring will not be closed.
KVP can write data, whose size is > 1/2 TX bufring size.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7414
303421
hyperv/vmbus: Avoid unnecessary mb()
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7333
303422
hyperv/vmbus: Inclusion cleanup
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7334
303470
hyperv/vmbus: Reindent function declarations.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7344
303471
hyperv/vmbus: Forward declare static functions
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7346
303472
hyperv/vmbus: Move driver glue to the beginning of the files
Just as most of other drivers do. And move sysinit function close
to its SYSINIT.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7347
303473
hyperv/vmbus: Revoke unnecessary exposure of vmbus softc
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7348
303066
hyperv/vmbus: Get rid of unnecessary definition.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7240
303067
hyperv/vmbus: Move IC register definition to Hyper-V utilities
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7241
303068
hyperv/vmbus: Channel struct field rename
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7242
303069
hyperv/vmbus: Pass channel as the first argument for channel callback
The prepares to kill device private fields in channel struct, which
are not flexible and extensible.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7243
303070
hyperv/vmbus: Deprecate the device private data in channel struct
They are neither flexible nor extensible.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7244
303071
hyperv/vmbus: Hide channel struct definition.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7245
303072
hyperv/vmbus: Save MNF trigger location instead of MNF trigger index.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7246
303127
hyperv/vmbus: Save event flag location and evet flag mask.
This avoids unnecessary access to the vmbus_softc struct on sending path.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7257
303128
hyperv/vmbus: Reorder channel fields.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7258
303129
hyperv/vmbus: Shuffle function declaration and macro definition.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7259
303131
hyperv: hv_vmbus_channel -> vmbus_channel
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7260
302882
hyperv/vmbus: Function rename
And reorder the error prone parameters list.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7191
302883
hyperv/vmbus: Field rename
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7193
302884
hyperv/vmbus: Factor out macros to do channel packet length conversion.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7194
302873
hyperv/vmbus: Set vcpuid to 0, if MSR_HV_VP_INDEX does not exist.
Mainly for compatibility. While I'm here, rename cpuid related
fields in hv_vmbus_channel.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7141
302874
hyperv/vmbus: Field rename
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7146
302816
hyperv/vmbus: Release vmbus channel lock before detach devices
Device detach method may sleep.
While I'm here, rename the function, fix indentation and function
comment.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7110
302817
hyperv/vmbus: Field renaming to reflect reality
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7111
302818
hyperv/vmbus: Fix the racy channel close.
It is not safe to iterate the sub-channel list w/o lock on the
close path, while it's even more difficult to hold the lock
and iterate the sub-channel list. We leverage the
vmbua_{get,rel}_subchan() functions to solve this dilemma.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7112
302808
hyperv/vmbus: Alloc/Free monitor param in vmbus channel alloc/free.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7101
302809
hyperv/vmbus: Move device register and channel free to the caller.
This paves the way for more cleanup/disentangle.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7102
302810
hyperv/vmbus: Move new channel scan notification to device register
And nuke now unnecessary function indirection.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7103
302811
hyperv/vmbus: Cleanup vmbus_chan_msgproc_choffer
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7104
302812
hyperv/vmbus: Nuke the channel open state.
Channel is either opened or not-opened.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7105
302813
hyperv/vmbus: Cleanup vmbus_chan_add()
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7106
302814
hyperv/vmbus: Use sub-channel index to detect primary channel
In case that VMBUS_CHAN_ISPRIMARY is needed in the early place of
channel setup.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7108
302815
hyperv/vmbus: Only add primary channels to vmbus channel list
- Make the vmbus_chan_add more straightforward.
- Partially fix the hv_vmbus_release_unattached_channels().
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7109
302698
hyperv/vmbus: Add vmbus method for GUID base device probing.
Reduce the exposure of hv_device.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7024
302699
hyperv/vmbus: All ivars are read-only; nuke unnecessary write_ivar
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7025
302700
hyperv/vmbus: Add channel ivar accessor.
This makes life easier during the transition period to nuke the hv_device.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7026
302701
hyperv/stor: Avoid the hv_device and nuke the broken get_stor_device
This paves way to nuke the hv_device, which is actually an unncessary
indirection.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7027
302702
hyperv/util: Avoid the hv_device
This paves way to nuke the hv_device, which is actually an unncessary
indirection.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7028
302703
hyperv/vmbus: Deprecate the usage of hv_device.
This paves way to nuke the hv_device, which is actually an unncessary
indirection.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7032
302704
hyperv/hn: Avoid the hv_device
This paves way to nuke the hv_device, which is actually an unncessary
indirection.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7033
302706
hyperv: Get rid of hv_device, which is unnecessary indirection.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7034
302693
hyperv/vmbus: Make channel id a field of hv_vmbus_channel.
This prepares to remove the unnecessary offer message embedding in
hv_vmbus_channel.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7014
302694
hyperv/vmbus: Make subchan index a field of hv_vmbus_channel.
This prepares to remove the unnecessary offer message embedding in
hv_vmbus_channel.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7015
302695
hyperv/vmbus: Add flags field into hv_vmbus_channel for MNF indication
This prepares to remove the unnecessary offer message embedding in
hv_vmbus_channel.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7019
302696
hyperv/vmbus: Add type/instance guid fields into hv_vmbus_channel
This prepares to remove the unnecessary offer message embedding in
hv_vmbus_channel.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7020
302697
hyperv/vmbus: Remove the embedded offer message from hv_vmbus_channel
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7021
302632
hyperv/vmbus: More verbose for GPADL_connect/chan_{rescind,offer}
Reviewed by: Dexuan Cui <decui microsoft com>, Hongjiang Zhang <honzhan microsoft com>
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6976
302633
hyperv/vmbus: Free sysctl properly upon channel close.
Prepare for sub-channel re-open.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6977
302634
hyperv/vmbus: Fix sub-channel re-open support.
For multi-channel devices, once the primary channel is closed,
a set of 'rescind' messages for sub-channels will be delivered
by Hypervisor. Sub-channel MUST be freed according to these
'rescind' messages; directly re-openning sub-channels in the
same fashion as the primary channel's re-opening does NOT work
at all.
After the primary channel is re-opened, requested # of sub-
channels will be delivered though 'channel offer' messages, and
this set of newly offered channels can be opened along side with
the primary channel.
This unbreaks the MTU setting for hn(4), which requires re-
openning all existsing channels upon MTU change.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6978
302607
hyperv/vmbus: Use post message Hypercall APIs for channel open
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6876
302608
hyperv/vmbus: Remove unnecessary check and unapplied comment
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6877
302609
hyperv/vmbus: Use post message Hypercall APIs for GPADL connect.
This also fixes memory leakge if sub-connect messages are needed.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6878
302610
hyperv/vmbus: Use post message Hypercall APIs for channel close
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6906
302611
hyperv/vmbus: Use post message Hypercall APIs for GPA disconnect
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6912
302612
hyperv: Nuke unused stuffs
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6913
302543
hyperv/vmbus: Use post message Hypercall APIs for channel request
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6831
302544
hyperv/hn: Add tunable to allow tcp_lro_queue_mbuf()
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6841
302545
hyperv/vmbus: Function renaming.
And pass vmbus_softc to vmbus_doattach()
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6842
302547
hyperv/vmbus: Explicitly assign channel message process array.
While I'm here, remove the useless message type from message process
array, which is not used and serves no purposes at all.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6858
302549
hyperv/vmbus: Add sysctl to expose vmbus version.
Requested by: Hongxiong Xian <v-hoxian microsoft com>
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6860
302554
hyperv/vmbus: Use post message Hypercall APIs for unload
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6861
302556
hyperv/vmbus: Create channel synchronously.
The device probe/attach has been move to a different thread, so the
reasons to create the channel asynchronously are no longer valid.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6862
302557
hyperv/vmbus: Save vmbus softc to channels.
So that we don't need to access the global vmbus softc.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6863
302559
hyperv/vmbus: Embed channel detach task in channel itself.
GC work queue stuffs.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6864
302606
hyperv/vmbus: Reorganize vmbus scan process.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6875
hyperv/vmbus: Implement a new set of APIs for post message Hypercall
And use this new APIs for Initial Contact post message Hypercall.
More post message Hypercalls will be converted.
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6830
Submitted by: Ju Sun <junsu microsoft com>
Reviewed by: Dexuan Cui <decui microsoft com>, sephe
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5651
Each channel contains one RX ring and one TX ring. And we
try to distribute the channels to different evenly.
Note: Currently we don't have enough information to extract
the RSS type and RSS hash value from the received packets.
This greatly improves the TX/RX performance for 8 virtual CPU
Hyper-V over 10Ge: it can max out 10Ge for TCP when multiple
RX/TX rings are enabled.
This almost doubles the TX/RX performance for locally connected
Hyper-Vs: was 6Gbps w/ 128 TCP streams, now 11Gbps w/ multiple
RX/TX rings enabled.
It is not enabled by default; it will be switched on after more
tests.
Collaborated with: Hongjiang Zhang <honzhan microsoft com>
MFC after: 2 week
Sponsored by: Microsoft OSTC
This fixes the TX/RX ring selection for TX/RX done.
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5454
Sepherosa Ziehau