The change (008fa749e0) that moved the
node release code to a separate function broke death notifications in
some cases. When it encountered a reference without a death
notification request, it would skip looking at the remaining
references, and therefore fail to send death notifications for them.
Cc: Colin Cross <[email protected]>
Cc: Android Kernel Team <[email protected]>
Cc: stable <[email protected]> # 3.10
Signed-off-by: Arve Hjønnevåg <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Jeremy Compostella <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
We dereference "heap->task" before checking if it's an ERR_PTR.
Fixes: ea313b5f88 ('gpu: ion: Also shrink memory cached in the deferred free list')
Signed-off-by: Dan Carpenter <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
We should be returning -ENOMEM here instead of zero.
Signed-off-by: Dan Carpenter <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
There is a race condition
Assume we have *one* sync_fence object, with *one* sync_pt
which belongs to *one* sync_timeline, given this condition,
sync_timeline->kref will have two counts, one for sync_timeline
(implicit) and another for sync_pt.
Assume following is the situation on CPU
Theead-1 : (Thread which calls sync_timeline_destroy())
-> (some function calls)
-> sync_timeline_destory()
-> sync_timeline_signal() (CPU is inside this
function after putting reference to sync_timeline)
At this time Thread-2 comes and does following
Thread-2 : (fclose on fence fd)
> sync_fence_release() -> because of fclose() on fence object
-> sync_fence_free()
-> sync_pt_free()
-> kref_put(&pt->parent->kref, sync_timeline_free);
-> sync_timeline_free() (CPU is inside this because
this time kref will be zero after _put)
Thread-2 will free sync_timeline object before Thread-1
has finished its work inside sync_timeline_signal.
With this change we signals all sync_pt before putting
reference to sync_timeline object.
Cc: Colin Cross <[email protected]>
Cc: Android Kernel Team <[email protected]>
Signed-off-by: Prakash Kamliya <[email protected]>
[jstultz: minor commit subject tweak]
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Add #include <linux/device.h> to fix the following warning seen
with gcc 4.7.3:
In file included from drivers/staging/android/ion/ion_heap.c:26:0:
drivers/staging/android/ion/ion_priv.h:358:21: warning: ‘struct device’ declared inside parameter list [enabled by default]
drivers/staging/android/ion/ion_priv.h:358:21: warning: its scope is only this definition or declaration, which is probably not what you want [enabled by default]
Cc: Colin Cross <[email protected]>
Cc: Android Kernel Team <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
The compat ioctl for ION_IOC_FREE currently passes allocation data
instead of the free data. Correct this.
Cc: Colin Cross <[email protected]>
Cc: Android Kernel Team <[email protected]>
Signed-off-by: Laura Abbott <[email protected]>
[jstultz: Folded in a small build fix]
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Fix a few bugs in ion_system_heap:
Initialize the list node in the info block.
Don't store size_remaining in a signed long, allocating >2GB
could overflow, resulting in a call to sg_alloc_table with
nents=0 which panics. alloc_largest_available will never
return a block larger than size_remanining, so it can never
go negative.
Limit a single allocation to half of all memory. Prevents a
large allocation from taking down the whole system.
Cc: Colin Cross <[email protected]>
Cc: Android Kernel Team <[email protected]>
Signed-off-by: Colin Cross <[email protected]>
[jstultz: Minor commit subject tweak]
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Avoid holding ashmem_mutex across code that can page fault. Page faults
grab the mmap_sem for the process, which are also held by mmap calls
prior to calling ashmem_mmap, which locks ashmem_mutex. The reversed
order of locking between the two can deadlock.
The calls that can page fault are read() and the ASHMEM_SET_NAME and
ASHMEM_GET_NAME ioctls. Move the code that accesses userspace pages
outside the ashmem_mutex.
Cc: Colin Cross <[email protected]>
Cc: Android Kernel Team <[email protected]>
Signed-off-by: Todd Poynor <[email protected]>
[jstultz: minor commit message tweaks]
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Before this change, a timeline would only be removed from the timeline
list *after* the sync driver had its release_obj() called. However, the
driver's release_obj() may free resources needed by print_obj().
Although the timeline list is locked when print_obj() is called, it is
not locked when release_obj() is called. If one CPU was in print_obj()
when another was in release_obj(), the print_obj() may make unsafe
accesses.
It is not actually necessary to hold the timeline list lock when calling
release_obj() if the call is made after the timeline is unlinked from
the list, since there is no possibility another thread could be in --
or enter -- print_obj() for that timeline.
This change moves the release_obj() call to after the timeline is
unlinked, preventing the above race from occurring.
Cc: Colin Cross <[email protected]>
Cc: Android Kernel Team <[email protected]>
Signed-off-by: Alistair Strachan <[email protected]>
[jstultz: minor commit subject tweak]
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
The ION_DUMMY option is bool, and hence this code is either
present or absent. It will never be modular, so using
module_init as an alias for __initcall is rather misleading.
Fix this up now, so that we can relocate module_init from
init.h into module.h in the future. If we don't do this, we'd
have to add module.h to obviously non-modular code, and that
would be a worse thing.
Note that direct use of __initcall is discouraged, vs. one
of the priority categorized subgroups. As __initcall gets
mapped onto device_initcall, our use of device_initcall
directly in this change means that the runtime impact is
zero -- it will remain at level 6 in initcall ordering.
Cc: Colin Cross <[email protected]>
Cc: Jesse Barker <[email protected]>
Cc: Android Kernel Team <[email protected]>
Cc: John Stultz <[email protected]>
Cc: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Paul Gortmaker <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
use ARRAY_SIZE to count number of heaps in static array
Signed-off-by: Tomas Winkler <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Need add "linux/io.h" to pass compiling under metag architecture with
allmodconfig (which use the default 'virt_to_phys'), the related error:
CC drivers/staging/android/ion/ion_dummy_driver.o
drivers/staging/android/ion/ion_dummy_driver.c: In function 'ion_dummy_init':
drivers/staging/android/ion/ion_dummy_driver.c:81: error: implicit declaration of function 'virt_to_phys'
Signed-off-by: Chen Gang <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Add support to the dummy driver for basic carveout and chunk heaps.
Since we're generating these heaps at module_init, and we want
this driver to be generic enough to be tested on any arch, we
don't have the ability to alloc bootmem, so both of these heaps
are conventionally allocated using alloc_pages(), which limits us
to 4M in size.
Should look into using CMA for heap allocation eventually, but
this provides enough to test the basic functionality of the
heaps.
Cc: Colin Cross <[email protected]>
Cc: Greg KH <[email protected]>
Cc: Jesse Barker <[email protected]>
Cc: Android Kernel Team <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Provide a basic dummy driver to register the ion device
and to install basic SYSTEM and SYSTEM_CONTIG heaps.
This allows for basic testing with ION without having
access to drivers or systems that have been enabled to use
ION.
Cc: Colin Cross <[email protected]>
Cc: Greg KH <[email protected]>
Cc: Jesse Barker <[email protected]>
Cc: Android Kernel Team <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Prior to subitting this, Colin reworked the compat_ioctl support
for the ion_test driver, moving the structure to be the same size
on both 32 and 64 bit architectures.
Two small things were left out. The compat_ioctl ptr assignment,
and the fact that despite having uniform sized types in the
structure, the structure pads out to different sizes on different
arches.
This patch resolves this issue by adding a padding entry after
the write flag, and adding the compat_ioctl ptr.
Changes in v2:
- Add a padding int rather then making write a u64
Acked-by: Colin Cross <[email protected]>
Cc: Android Kernel Team <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
RT_MUTEXES can be configured out of the kernel, causing compile
problems with ION.
To quote Colin:
"rt_mutexes were added with the deferred freeing feature. Heaps need
to return zeroed memory to userspace, but zeroing the memory on every
allocation was causing performance issues. We added a SCHED_IDLE
thread to zero memory in the background after freeing, but locking the
heap from the SCHED_IDLE thread might block a high priority allocation
thread for a long time.
The lock is only used to protect the heap's free_list and
free_list_size members, and is not held for any long or sleeping
operations. Converting to a spinlock should prevent priority
inversion without using the rt_mutex. I'd also rename it to free_lock
to so it doesn't get used as a general heap lock."
Thus this patch converts the rt_mutex usage to a spinlock and
renames the lock free_lock to be more clear as to its use.
I also had to change a bit of logic in ion_heap_freelist_drain()
to safely avoid list corruption.
Acked-by: Colin Cross <[email protected]>
Cc: Android Kernel Team <[email protected]>
Reported-by: Jim Davis <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
The kbuild test robot reported:
drivers/staging/android/ion/ion_system_heap.c:122 alloc_largest_available() error: potential null dereference 'info'. (kmalloc returns null)
Where the pointer returned from kmalloc goes unchecked for failure.
This patch checks the return for NULL, and reworks the logic, as
suggested by Colin, so we allocate the page_info structure first.
Acked-by: Colin Cross <[email protected]>
Cc: Android Kernel Team <[email protected]>
Reported-by: kbuild test robot <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
The kbuild test robot reported a build issue w/ ION on m68k:
drivers/staging/android/ion/ion.c: In function 'ion_reserve':
drivers/staging/android/ion/ion.c:1526:4: error: implicit declaration of function 'memblock_alloc_base' [-Werror=implicit-function-declaration]
drivers/staging/android/ion/ion.c:1528:11: error: 'MEMBLOCK_ALLOC_ANYWHERE' undeclared (first use in this function)
drivers/staging/android/ion/ion.c:1528:11: note: each undeclared identifier is reported only once for each function it appears in
drivers/staging/android/ion/ion.c:1537:4: error: implicit declaration of function 'memblock_reserve' [-Werror=implicit-function-declaration]
cc1: some warnings being treated as errors
This is caused by ION using memblock functionality which m68k doesn't support.
This patch adds a HAVE_MEMBLOCK dependency to the ION config.
Acked-by: Colin Cross <[email protected]>
Cc: Android Kernel Team <[email protected]>
Reported-by: kbuild test robot <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Fixes the following sparse warnings:
drivers/staging/android/ion/tegra/tegra_ion.c:23:19: warning:
symbol 'idev' was not declared. Should it be static?
drivers/staging/android/ion/tegra/tegra_ion.c:24:19: warning:
symbol 'tegra_user_mapper' was not declared. Should it be static?
drivers/staging/android/ion/tegra/tegra_ion.c:25:5: warning:
symbol 'num_heaps' was not declared. Should it be static?
drivers/staging/android/ion/tegra/tegra_ion.c:26:17: warning:
symbol 'heaps' was not declared. Should it be static?
drivers/staging/android/ion/tegra/tegra_ion.c:28:5: warning:
symbol 'tegra_ion_probe' was not declared. Should it be static?
drivers/staging/android/ion/tegra/tegra_ion.c:66:5: warning:
symbol 'tegra_ion_remove' was not declared. Should it be static?
Signed-off-by: Wei Yongjun <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
ion_test.h should not define ion_user_handle_t, and defining it
causes a warning:
In file included from drivers/staging/android/ion/ion_test.c:31:
drivers/staging/android/ion/../uapi/ion_test.h:23: error: redefinition of typedef 'ion_user_handle_t'
drivers/staging/android/ion/../uapi/ion.h:23: note: previous declaration of 'ion_user_handle_t' was here
Reported-by: Andrew Morton <[email protected]>
Cc: John Stultz <[email protected]>
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
ION doesn't export the proper symbols for it to be a module. This
causes build issues when ION is configured as a module.
Since Andorid kernels rarely use modules (I think recent policy
requires no modules?), go ahead and set the ION config to a bool
from the tristate option.
If folks decide ION as a module is important, we will have to go
through and export the various needed symbols.
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Update the ION system heap shrinker to use the new count/scan
interfaces that landed in 3.12
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Mostly just to quiet checkpatch warnings, be more verbose
in describing the ION config option.
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Just some simple cleanups to address whitespace issues and
other issues found w/ checkpatch.
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Implement ion_cma_unmap_kernel, ion will call it unconditionally.
Use correct gfp flags when calling dma_alloc_coherent so it doesn't
try to use atomic DMA memory.
Check for invalid alignment when allocating.
Reject cached allocations - the cpu address returned by
dma_alloc_coherent is always going to be an uncached mapping, so
map_kernel will not see data written by a cached userspace mapping.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Add ion_heap_pages_zero for ion heaps to use to zero pages
during initialization or allocation, when a struct ion_buffer
may not be available. Use it from the chunk heap and carveout
heaps.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
The carveout heap wasn't zeroing its buffers after use.
Create the sg_table during allocate instead of map_dma, to allow
using the sg_table during free, and call ion_heap_buffer_zero
during free. Also fixes a missing kfree when destroying the
table.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
There is no reason to use kzalloc, just call alloc_pages directly.
Change the GFP from GFP_KERNEL to include __GFP_HIGH, to allow it
to return contiguous pages from highmem. virt_to_* functions
aren't valid on highmem pages, so store the struct page * in an
sg_table in buffer->priv_virt like most other heaps, and replace
virt_to_* with page_to_*.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Now that ion_vm_fault uses vm_insert_pfn instead of vm_insert_page
cached buffers can be supported in any heap. Remove the checks
in the chunk and system heaps.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Now that ion_vm_fault doesn't need a struct page with a nonzero
refcount, there is no need allocate heap memory for cached pages using
split_page. Remove the ion_heap_alloc_pages and ion_heap_free_pages
helpers in favor of direct calls to alloc_pages and __free_pages,
and remove the special handling in the system heap.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Most ion userspace mappings are created with remap_pfn_range. Use
vm_insert_pfn instead of vm_insert_page to make faulted cached
mappings look more like uncached mappings.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Check the return value of remap_pfn_range and return an error if
it fails.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
When the shrinkers are called with GFP_HIGH free low memory first,
it is more important to have free than high memory.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
ion_heap_buffer_zero can spend a long time in unmap_kernel_range
if it has to broadcast a tlb flush to every cpu for every page.
Modify it to batch pages into a larger region to clear using a
single mapping. This may cause the mapping size to change if
the buffer size is not a multiple of the mapping size, so
switch to allocating the address space for each chunk. This
allows us to use vm_map_ram to handle the allocation and mapping
together.
The number of pages to zero using a single mapping is set to 32
to hit the fastpath in vm_map_ram.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Ion will compile and run on other platforms now, remove the
dependency on ARM.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
In testing ion system heap allocations, I ran across two issues:
1) Not k*z*allocing the sg table. This can cause trouble if
we end up trying call sg_alloc_table() with too many entries,
then sg_alloc_table() internally fails and tries to free what it
thinks is internal table structure, which causes bad pointer
traversals.
2) The second list_for_each_entry probably should be _safe,
since I was seeing strange lock warnings and oopses on occasion.
This seems to resolve it, but could use some extra checking.
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
The `buffer' variable is being used after being freed. Fix this.
Signed-off-by: Mitchel Humpherys <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Convert the ion ioctls to use _IOW instead of _IOWR where
appropriate, and factor out the copy_from_user and copy_to_user
based on the _IOC_DIR bits. For the existing incorrect ioctls,
add a function to wrap _IOC_DIR to return the corrected value.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
ion_system_contig_heap buffers have an sglist, just call
ion_heap_map_user to map it.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Use %z for size_t and %pa for dma_addr_t to avoid warnings in printks.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
phys_to_page and __phys_to_pfn don't exist on all platforms.
Use a combination of pfn_to_page, PFN_DOWN, page_to_pfn, and
virt_to_page to get the same results.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
ion_heap_map_kernel already implements mapping a scatterlist of
pages into the kernel, and all heaps are required to have struct
pages associated with them, so delete the functions that use
__arm_ioremap and use ion_heap_map_kernel instead.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Use idr_alloc instead if idr_pre_get/idr_get_new_above, and
remove idr_remove_all.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Add a /dev/ion-test device that will be created if CONFIG_ION_TEST
is set. The device accepts a dma_buf fd and allows reading and
writing to the backing memory using DMA-like apis or kernel mapping
apis. Can be used to test the dma_buf mapping ops, including
the ion implementations, from userspace.
Signed-off-by: Colin Cross <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Arve Hjønnevåg