These options are not present in older clang versions, so when we build
for a distro that has a gcc new enough to have these options and that
the distro python build config settings use them but clang doesn't
support, b00m.
This is the case with fedora 28 and rawhide, so check if clang has the
options and remove the missing ones from CFLAGS.
Cc: Adrian Hunter <[email protected]>
Cc: David Ahern <[email protected]>
Cc: Jiri Olsa <[email protected]>
Cc: Namhyung Kim <[email protected]>
Cc: Wang Nan <[email protected]>
Link: https://lkml.kernel.org/n/[email protected]
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
The gcc 8 compiler won't compile the python extension code with the
following errors (one example):
python.c:830:15: error: cast between incompatible function types from \
‘PyObject * (*)(struct pyrf_evsel *, PyObject *, PyObject *)’ \
uct _object * (*)(struct pyrf_evsel *, struct _object *, struct _object *)’} to \
‘PyObject * (*)(PyObject *, PyObject *)’ {aka ‘struct _object * (*)(struct _objeuct \
_object *)’} [-Werror=cast-function-type]
.ml_meth = (PyCFunction)pyrf_evsel__open,
The problem with the PyMethodDef::ml_meth callback is that its type is
determined based on the PyMethodDef::ml_flags value, which we set as
METH_VARARGS | METH_KEYWORDS.
That indicates that the callback is expecting an extra PyObject* arg, and is
actually PyCFunctionWithKeywords type, but the base PyMethodDef::ml_meth type
stays PyCFunction.
Previous gccs did not find this, gcc8 now does. Fixing this by silencing this
warning for python.c build.
Commiter notes:
Do not do that for CC=clang, as it breaks the build in some clang
versions, like the ones in fedora up to fedora27:
fedora:25:error: unknown warning option '-Wno-cast-function-type'; did you mean '-Wno-bad-function-cast'? [-Werror,-Wunknown-warning-option]
fedora:26:error: unknown warning option '-Wno-cast-function-type'; did you mean '-Wno-bad-function-cast'? [-Werror,-Wunknown-warning-option]
fedora:27:error: unknown warning option '-Wno-cast-function-type'; did you mean '-Wno-bad-function-cast'? [-Werror,-Wunknown-warning-option]
#
those have:
clang version 3.9.1 (tags/RELEASE_391/final)
The one in rawhide accepts that:
clang version 6.0.0 (tags/RELEASE_600/final)
Signed-off-by: Jiri Olsa <[email protected]>
Tested-by: Arnaldo Carvalho de Melo <[email protected]>
Cc: Alexander Shishkin <[email protected]>
Cc: David Ahern <[email protected]>
Cc: Josh Poimboeuf <[email protected]>
Cc: Namhyung Kim <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Sergey Senozhatsky <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
The -spec=/path/to/file can be used to change what gcc puts in the cc,
ld, etc command lines, but this is not present in clang, filter it out
at the setup.py file by changing python2's internal variable where it
keeps its initial CFLAGS value.
With this all of perf can be built in at least Fedora 25, fixing this
problem:
GEN /tmp/build/perf/python/perf.so
CC /tmp/build/perf/builtin-buildid-list.o
clang-4.0: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]
clang-4.0: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]
error: command 'clang' failed with exit status 1
Now I need to change all the containers where I have clang to build
perf with it, so that we can check that in other distros (opensuse, debian,
ubuntu, etc) this also works.
Cc: Adrian Hunter <[email protected]>
Cc: David Ahern <[email protected]>
Cc: Jiri Olsa <[email protected]>
Cc: Namhyung Kim <[email protected]>
Cc: Wang Nan <[email protected]>
Link: http://lkml.kernel.org/n/[email protected]
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
The util/python-ext-sources file contains source files required to build
the python extension relative to $(srctree)/tools/perf,
Such a file path $(FILE).c is handed over to the python extension build
system, which builds the final object in the
$(PYTHON_EXTBUILD)/tmp/$(FILE).o path.
After the build is done all files from $(PYTHON_EXTBUILD)lib/ are
carried as the result binaries.
Above system fails when we add source file relative to ../lib, which we
do for:
../lib/bitmap.c
../lib/find_bit.c
../lib/hweight.c
../lib/rbtree.c
All above objects will be built like:
$(PYTHON_EXTBUILD)/tmp/../lib/bitmap.c
$(PYTHON_EXTBUILD)/tmp/../lib/find_bit.c
$(PYTHON_EXTBUILD)/tmp/../lib/hweight.c
$(PYTHON_EXTBUILD)/tmp/../lib/rbtree.c
which accidentally happens to be final library path:
$(PYTHON_EXTBUILD)/lib/
Changing setup.py to pass full paths of source files to Extension build
class and thus keep all built objects under $(PYTHON_EXTBUILD)tmp
directory.
Reported-by: Jeff Bastian <[email protected]>
Signed-off-by: Jiri Olsa <[email protected]>
Tested-by: Josh Boyer <[email protected]>
Cc: David Ahern <[email protected]>
Cc: Namhyung Kim <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: [email protected] # v4.2+
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
Move debugfs.* to api/fs/. We have a common tools/lib/api/ place where
the Makefile lives and then we place the headers in subdirs.
For example, all the fs-related stuff goes to tools/lib/api/fs/ from
which we get libapikfs.a (acme got almost the naming he wanted :-)) and
we link it into the tools which need it - in this case perf and
tools/vm/page-types.
acme:
"Looking at the implementation, I think some tools can even link
directly to the .o files, avoiding the .a file altogether.
But that is just an optimization/finer granularity tools/lib/
cherrypicking that toolers can make use of."
Fixup documentation cleaning target while at it.
Signed-off-by: Borislav Petkov <[email protected]>
Acked-by: Ingo Molnar <[email protected]>
Cc: Adrian Hunter <[email protected]>
Cc: Andi Kleen <[email protected]>
Cc: Arjan van de Ven <[email protected]>
Cc: David Ahern <[email protected]>
Cc: Frederic Weisbecker <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Jiri Olsa <[email protected]>
Cc: Mike Galbraith <[email protected]>
Cc: Namhyung Kim <[email protected]>
Cc: Paul Mackerras <[email protected]>
Cc: Pekka Enberg <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Robert Richter <[email protected]>
Cc: Stanislav Fomichev <[email protected]>
Cc: Stephane Eranian <[email protected]>
Cc: Steven Rostedt <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
This introduces the tools/lib/lk library, that will gradually have the
routines that now are used in tools/perf/ and other tools and that can
be shared.
Start by carving out debugfs routines for general use.
Signed-off-by: Borislav Petkov <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Steven Rostedt <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
[ committer note: Add tools/lib/lk/ to perf's MANIFEST so that its tarballs continue to build ]
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
Namhyung Kim reported that the build fails with:
GEN python/perf.so
gcc: error: python_ext_build/tmp//../../libtraceevent.a: No such file or directory
error: command 'gcc' failed with exit status 1
cp: cannot stat `python_ext_build/lib/perf.so': No such file or directory
make: *** [python/perf.so] Error 1
We need to propagate the TE_PATH variable to the setup.py file.
Reported-by: Namhyung Kim <[email protected]>
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
Link: http://lkml.kernel.org/n/[email protected]
[ Fixed superfluous variable build error. ]
Signed-off-by: Ingo Molnar <[email protected]>
Adding install-python_ext target to install python extension related
files. Installation directory is governed by python distutils package
and follows the DESTDIR variable settings.
Also moving python extension build output into '$(O)python_ext_build'
directory and making it configurable via PYTHON_EXTBUILD variable.
Keeping the '$(O)python/perf.so' file, so it could be used for testing
as of until now.
Cc: Ingo Molnar <[email protected]>
Cc: Paul Mackerras <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Jiri Olsa <[email protected]>
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
Fix the following build error:
GEN python/perf.so
In file included from util/evsel.h:10,
from util/python.c:6:
util/hist.h:106:18: error: newt.h: No such file or directory
error: command 'x86_64-pc-linux-gnu-gcc' failed with exit status 1
make: *** [python/perf.so] Error 1
by passing BASIC_CFLAGS to setup.py. BASIC_CFLAGS variable contains
the -DNO_NEWT_SUPPORT switch which prevents building python c
extension with newt.
Cc: Frederic Weisbecker <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Stephane Eranian <[email protected]>
LKML-Reference: <[email protected]>
Signed-off-by: Robert Richter <[email protected]>
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
The 023695d cset added a new file, util/cgroup.c, that is referenced from
util/evsel.c, so it needs to be present in util/setup.py so that the python
shared object binding works, fixing this:
[root@emilia linux]# export PYTHONPATH=~acme/git/build/perf/python/
[root@emilia linux]# ./tools/perf/python/twatch.py
Traceback (most recent call last):
File "./tools/perf/python/twatch.py", line 16, in <module>
import perf
ImportError: /home/acme/git/build/perf/python/perf.so: undefined symbol: close_cgroup
Cc: Frederic Weisbecker <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Mike Galbraith <[email protected]>
Cc: Paul Mackerras <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Stephane Eranian <[email protected]>
Cc: Tom Zanussi <[email protected]>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
Where there are lots of errors related to python methods receiving
'char *' for things like file open mode, which break the build, also
disable strict aliasing and fixup some other warnings. Now builds on
both 32-bit and 64-bit fedora systems.
Cc: Frederic Weisbecker <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Mike Galbraith <[email protected]>
Cc: Paul Mackerras <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Stephane Eranian <[email protected]>
Cc: Tom Zanussi <[email protected]>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
First clarifying that this kind of binding is not a replacement or an
equivalent to the 'perf script' way of using python with perf.
The 'perf script' way is to process events and look at a given script
for some python function that matches the events to pass each event for
processing.
This is a python module, i.e. everything is driven from the python
script, that merely uses "import perf" or "from perf import".
perf script is focused on tracepoints, this binding is focused on profiling as
an initial target. More work is needed to make available tracepoint specific
variables as event variables accessible via this binding.
There is one example of such usage model, in
tools/perf/python/twatch.py, a tool to watch "cycles" events together
with task (fork, exit) and comm perf events.
For now, due to me not being able to grok how python distutils cope with
building C extensions outside the sources dir the install target just
builds it, I'm using it as:
[root@emilia linux]# export PYTHONPATH=~acme/git/build/perf/lib.linux-x86_64-2.6/
[root@emilia linux]# tools/perf/python/twatch.py
cpu: 4, pid: 30126, tid: 30126 { type: mmap, pid: 30126, tid: 30126, start: 0x4, length: 0x82e9ca03, offset: 0, filename: }
cpu: 6, pid: 47, tid: 47 { type: mmap, pid: 47, tid: 47, start: 0x6, length: 0xbef87c36, offset: 0, filename: }
cpu: 1, pid: 0, tid: 0 { type: mmap, pid: 0, tid: 0, start: 0x1, length: 0x775d1904, offset: 0, filename: }
cpu: 7, pid: 0, tid: 0 { type: mmap, pid: 0, tid: 0, start: 0x7, length: 0xc750aeb6, offset: 0, filename: }
cpu: 5, pid: 2255, tid: 2255 { type: mmap, pid: 2255, tid: 2255, start: 0x5, length: 0x76669635, offset: 0, filename: }
cpu: 0, pid: 0, tid: 0 { type: mmap, pid: 0, tid: 0, start: 0, length: 0x6422ef6b, offset: 0, filename: }
cpu: 2, pid: 2255, tid: 2255 { type: mmap, pid: 2255, tid: 2255, start: 0x2, length: 0xe078757a, offset: 0, filename: }
cpu: 1, pid: 5769, tid: 5769 { type: fork, pid: 30127, ppid: 5769, tid: 30127, ptid: 5769, time: 103893991270534}
cpu: 6, pid: 30127, tid: 30127 { type: comm, pid: 30127, tid: 30127, comm: ls }
cpu: 6, pid: 30127, tid: 30127 { type: exit, pid: 30127, ppid: 30127, tid: 30127, ptid: 30127, time: 103893993273024}
The first 8 mmap events in this 8 way machine are a mistery that is still being
investigated.
More of the tools/perf/util/ APIs will be exposed via this python binding as
the need arises. For now the focus is on creating events and processing them,
symbol resolution is an obvious next step, with tracepoint variables as a close
second step.
Cc: Clark Williams <[email protected]>
Cc: Frederic Weisbecker <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Mike Galbraith <[email protected]>
Cc: Paul Mackerras <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Stephane Eranian <[email protected]>
Cc: Tom Zanussi <[email protected]>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>