r318021:
Introduce pmap_remap_vm_attr(), it allows to remap one VM memattr class to
another.
r318251:
Clarify usage rules for pmap_remap_vm_attr(). Not a functional change.
r315900:
Cleanup structures related to VFP and/or mcontext_t. - in mcontext_t,
rename newer used 'union __vfp' to equaly sized 'mc_spare'.
Space allocated by 'union __vfp' is too small and cannot hold full VFP
context.
- move structures defined in fp.h to more appropriate headers. - remove
all unused VFP structures.
r315973:
Save VFP state on fork(). Update the copy of VFP state in PCB before it is
cloned for new process.
r315974:
Preserve VFP state across signal delivery.
r303261:
Add more UEFI/e820 memory types from latest specifications.
r315059:
Split overbloated machep.c to multiple files and do basic cleanup of these
fragments.
r306631:
Use C99 designated initializers to create the armv6 cpu_functions structs.
This will help with a later cleanup of what functions we implement.
r306640:
Only define the CF_* macros on ARMv4/v5. They are unused on armv6.
r306641:
Remove the parts of cpu_functions from armv6 that are unused on that
architecture.
r306650:
Add the Cortex-A{53,57,72} ID register values. These can all run 32-bit
code so could run a 32-bit kernel.
r306656:
Use the cortex functions when booting on one of the Cortex-A ARMv8 CPUs.
This list is incomplete, however we don't have the ID values for the
missing Cortex-A32 or A35.
(by cognet)
Use strexeq instead of needlessly branch.
==
(by cognet)
Remove useless labels.
==
(by cognet)
Correct the IT instruction in atomic_fcmpset_64().
==
(by andrew)
Only define atomic_fcmpset_long in the kernel. We may include
machine/atomic.h in userspace, however atomic_fcmpset_32 is unimplemented
there.
r313254,r313341
amd64: add atomic_fcmpset
==
sparc64: add atomic_fcmpset
==
Implement atomic_fcmpset_* for arm and arm64.
==
Add atomic_fcmpset_*() inlines for powerpc
Summary:
atomic_fcmpset_*() is analogous to atomic_cmpset(), but saves off the read value
from the target memory location into the 'old' pointer in the case of failure.
==
i386: add atomic_fcmpset
==
Don't retry a lost reservation in atomic_fcmpset()
The desired behavior of atomic_fcmpset_() is to always exit on error. Instead
of retrying on lost reservation, leave the retry to the caller, and return
==
Add atomic_fcmpset_*() inlines for MIPS
atomic_fcmpset_*() is analogous to atomic_cmpset(), but saves off the
read value from the target memory location into the 'old' pointer.
==
i386: fixup fcmpset
An incorrect output specifier was used which worked with clang by accident,
but breaks with the in-tree gcc version.
While here plug a whitespace nit.
==
Implement atomic_fcmpset_*() for RISC-V.
==
Use 64bit store instruction in atomic_fcmpset_64.
Stop including sys/types.h from arm's machine/atomic.h, fix the places
where atomic.h was being included without ensuring that types.h (via
param.h) was included first, as required by atomic(9).
Remove arm's cpuconf.h, and references to it, after moving a few lines from
it into pmap-v4.h where they are used. Other than those few lines of
support for different MMU types, nothing in cpuconf.h has been used in our
code for quite a while.
The file existed to set up a variety of symbols to describe the
architecture. Over the past few years we have converted all of our source
to use the new architecture symbols standardized by ARM Inc, and predefined
by both clang and gcc.
Remove bus_dma_get_range and bus_dma_get_range_nb on armv6. We only need
this on a few earlier arm SoCs.
Restrict where we need to define fdt_fixup_table to just PowerPC and
Marvell.
Add the missing void to function signatures in much of the arm code.
r312610:
Like r310481 for i386, move the objects used to create temporary
mappings for armv6 pmap zero and copy operations to the MD PCPU region.
Change sysmap initialization to only allocate KVA pages for CPUs that
are actually present.
While here, collapse CMAP3 into CMAP2 (their use was mutually exclusive
anyway) and "recover" some space in PCPU padding that has always been
available due to 64-byte cacheline padding.
r312792:
Further cleanup of per-CPU armv6 pmap data:
- Replace pcpu_find(curcpu) with get_pcpu(), which is much more direct.
- Remove armv4 pcpu fields which I added in r286296 but never needed
to use.
- armv6 pc_qmap_addr was leftover from the old armv6 pmap
implementation. Rename it and put it to use in the new one.
Make it possible to safely use TPIDRURW from userspace.
On amd64, arm64 and i386, we have the possibility to switch between TLS
areas in userspace. The nice thing about this is that it makes it easier
to do light-weight threading, if we ever feel like doing that. On armv6,
let's go into the same direction by making it possible to safely use the
TPIDRURW register, which is intended for this purpose.
Clean up the ARMv6 code to remove md_tp entirely. Simply add a dedicated
field to the PCB to hold the value of TPIDRURW across context switches,
like we do for any other register. As userspace currently uses the
read-only TPIDRURO register, simply ensure that we keep both values in
sync where possible. The system calls for modifying the read-only
register will simply write the intended value into both registers, so
that it lazily ends up in the PCB during the next context switch.
Approved by: andrew
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D7951
fully-pessimized implementation that requires a type to be aligned to
its natural size.
On armv6+ the compiler might generate load-/store-multiple instructions
which require 4-byte alignment even though the source code is only
accessing individual uint32_t values in a way that doesn't require any
particular alignment at all. The compiler apparently feels free to
combine multiple accesses into a single instruction that requires a
more-strict alignment, and no set of compiler flags seems to disable
this behavior (at least in clang 3.8).
This fixes alignment faults on arm systems using wifi adapters. The
wifi code uses ALIGNED_POINTER(p, uint32_t) to decide whether it needs
to copy-align tcp headers. Because clang is combining several uint32_t
accesses into a single ldm instruction, we need to say that accessing a
uint32_t requires 4-byte alignment.
Approved by: re(gjb)
are no longer natural-alignment strict, there are still some restrictions.
FreeBSD network code assumes data is naturally-aligned or is running
on a platform with no restrictions; pointers are not annotated to
indicate the data pointed to may be packed or unaligned. The clang
optimizer can sometimes combine the load or store of a pair of adjacent
32-bit values into a single doubleword load/store, and that operation
requires at least 4-byte alignment. __NO_STRICT_ALIGNMENT can lead
to tcp headers being only 2-byte aligned.
Note that alignment faults remain disabled on armv6, this change reverts
only the defining of the symbol which leads to some overly-agressive code
shortcuts when building common/shared drivers and network code for arm.
Approved by: re(kib)
the exact CPU we are running on to set the cpu functions. Relax the check
to ignore the CPU revision. Even so this may still be too specific.
Reviewed by: mmel
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D6504
- Reset debug architecture and enable monitor for secondary
CPUs in init_secondary() rather than when configuring watchpoint, etc.
- Disable HW debugging capabilities when one of the CPU cores fails
to set up.
- Use dbg_capable() in a more atomic manner to avoid any mismatch
between CPUs.
Differential Revision: https://reviews.freebsd.org/D6009
to match the new state of affairs. The hardware we support has always been
able to do unaligned accesses, we've just never enabled it until now.
This brings FreeBSD into line with all the other major OSes, and should help
with the growing volume of 3rd-party software that assumes unaligned access
will just work on armv6 and armv7.
have ACLE support built in. The ACLE (ARM C Language Extensions) defines
a set of standardized symbols which indicate the architecture version and
features available. ACLE support is built in to modern compilers (both
clang and gcc), but absent from gcc prior to 4.4.
ARM (the company) provides the acle-compat.h header file to define the
right symbols for older versions of gcc. Basically, acle-compat.h does
for arm about the same thing cdefs.h does for freebsd: defines
standardized macros that work no matter which compiler you use. If ARM
hadn't provided this file we would have ended up with a big #ifdef __arm__
section in cdefs.h with our own compatibility shims.
Remove #include <machine/acle-compat.h> from the zillion other places (an
ever-growing list) that it appears. Since style(9) requires sys/types.h
or sys/param.h early in the include list, and both of those lead to
including cdefs.h, only a couple special cases still need to include
acle-compat.h directly.
Loves it: imp
where possible. In the places that doesn't work (multi-line inline asm,
and places where the old armv4 cpufuncs mechanism is used), annotate the
accesses with a comment that includes SCTLR. Now a grep -i sctlr can find
all the system control register manipulations.
No functional changes.
compilers can emit arm instructions that require 8-byte alignment. The
alignment-sensitive instructions were added in armv5, which has to be
supported by our combined v4/v5 kernels, so the value is set uncoditionally
for all arm architecture versions.
Also adjust the comment to explain in more detail why the macros have the
form and values they do.
Per advice from bde@, maintain the unsignedness of the value of _ALIGNBYTES
(but do so using his second choice of allowing sizeof() to supply the
unsignedness, rather than just hardcoding '8U', which in my mind would
require an even more verbose comment to explain why it's right). Also
explain in the comment that the resulting type of _ALIGN() is equivelent
to uinptr_t on arm (32-bit unsigned int), but it's purposely spelled as
"unsigned" to avoid problems with including other header files. Even
including machine/_types.h to allow use of __uintptr_t causes compilation
failures because of this header being included (indirectly) in asm code.
The discussion that led to this change (albeit at a glacial pace) is at
https://lists.freebsd.org/pipermail/svn-src-head/2014-November/064593.html
implementations. Early in the boot the kernel will use an approximate,
however after the timer has been probed it will switch to a more accurate
implementation.
Reviewed by: manu
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D5762
and R/W emulation aborts under pmap lock.
There were two reasons for using of atomic operations:
(1) the pmap code is based on i386 one where they are used,
(2) there was an idea that access and R/W emulation aborts should be
handled as quick as possible, without pmap locking.
However, the atomic operations in i386 pmap code are used only because
page table entries may be modified by hardware. At the beginning, we
were not sure that it's the only reason. So even if arm hardware does
not modify them, we did not risk to not use them at that time. Further,
it turns out after some testing that using of pmap lock for access and
R/W emulation aborts does not bring any extra cost and there was no
measurable difference. Thus, we have decided finally to use pmap lock
for all operations on page table entries and so, there is no reason for
atomic operations on them. This makes the code cleaner and safer.
This decision introduce a question if it's safe to use pmap lock for
access and R/W emulation aborts. Anyhow, there may happen two cases in
general:
(A) Aborts while the pmap lock is locked already - this should not
happen as pmap lock is not recursive. However, under pmap lock only
internal kernel data should be accessed and such data should be mapped
with A bit set and NM bit cleared. If double abort happens, then
a mapping of data which has caused it must be fixed.
(B) Aborts while another lock(s) is/are locked - this already can
happen. There is no difference here if it's either access or R/W
emulation abort, or if it's some other abort.
Reviewed by: kib
(PL1) and unprivileged (PL0) read/write access. As cp15 virtual to
physical address translation operations are used, interrupts must be
disabled to get consistent result when they are called.
These functions should be used only in very specific occasions like
during abort handling or kernel debugging. One of them is going to be
used in pmap_fault(). However, complete function set is added. It cost
nothing, as they are inlined.
While here, fix comment of #endif.
Reviewed by: kib
universal.
(1) New struct intr_map_data is defined as a container for arbitrary
description of an interrupt used by a device. Typically, an interrupt
number and configuration relevant to an interrupt controller is encoded
in such description. However, any additional information may be encoded
too like a set of cpus on which an interrupt should be enabled or vendor
specific data needed for setup of an interrupt in controller. The struct
intr_map_data itself is meant to be opaque for INTRNG.
(2) An intr_map_irq() function is created which takes an interrupt
controller identification and struct intr_map_data as arguments and
returns global interrupt number which identifies an interrupt.
(3) A set of functions to be used by bus drivers is created as well as
a corresponding set of methods for interrupt controller drivers. These
sets take both struct resource and struct intr_map_data as one of the
arguments. There is a goal to keep struct intr_map_data in struct
resource, however, this way a final solution is not limited to that.
(4) Other small changes are done to reflect new situation.
This is only first step aiming to create stable interface for interrupt
controller drivers. Thus, some temporary solution is taken. Interrupt
descriptions for devices are stored in INTRNG and two specific mapping
function are created to be temporary used by bus drivers. That's why
the struct intr_map_data is not opaque for INTRNG now. This temporary
solution will be replaced by final one in next step.
Differential Revision: https://reviews.freebsd.org/D5730
- Don't convert atags address passed from U-Boot. It's real physical
address (and we have 1:1 mapping).
- Size of tags is encoded in words, not in bytes
This allow us to boot FreeBSD kernel (using uImage encapsulation) directly
from U-boot using 'bootm' command or by Android fastboot loader.
For now, kernel uImage must be marked as Linux, but we can add support for
FreeBSD into U-Boot later.
controller IPI provider.
New struct intr_ipi is defined which keeps all info about an IPI:
its name, counter, send and dispatch methods. Generic intr_ipi_setup(),
intr_ipi_send() and intr_ipi_dispatch() functions are implemented.
An IPI provider must implement two functions:
(1) an intr_ipi_send_t function which is able to send an IPI,
(2) a setup function which initializes itself for an IPI and
calls intr_ipi_setup() with appropriate arguments.
Differential Revision: https://reviews.freebsd.org/D5700
the interrupt framework is also going to be used by another (MIPS)
architecture. IPI implementations may vary much across different
architectures.
An IPI implementation should still define INTR_IPI_COUNT and use
intr_ipi_setup_counters() to setup IPI counters which are inside of
intrcnt[] and intrnames[] arrays. Those are used for sysctl and ddb.
Then, intr_ipi_increment_count() should be used to increment obtained
counter.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D5459
intr_pic_init_secondary. Replace them with a direct call. On BCM2836
and ARMADA XP we need to add this function, but it can be empty.
Reviewed by: ian, imp
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D5460
slightly wrong on the others. We should just check if mp_ncpus is set to
more than one CPU as we may wish to run on a single core even when SMP is
available.
Reviewed by: ian
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D5458
Ian Lepore