Profiling foswiki with NYTProf results in incomplete profile data

Question

I've an foswiki installation which is really slow (~ 60 seconds for a uncached page). I've tried to profile the installation with NYTProf, according to http://foswiki.org/Support/NYTProfDebugging with the following command:

> sudo -u www-data NYTPROF="file=/tmp/nytprof.out:addpid=1:endatexit=1" perl -wTd:NYTProf view -topic Some.Topic -username MyUsername

The script fails with an exit code 141 when I run it with profiler. If I run it without profiler (remote d:NYTProf) it exits successful and producing output.

After the profiling I've gotten a bunch of profile files in my /tmp directory:

nytprof.out.[841-1860]

But when I try to merge these files, I've get an error for the first file:

> nytprofmerge nytprof.out.*
Profile data incomplete, inflate error -5 ((null)) at end of input file, perhaps the process didn't exit cleanly or the file has been truncated  (refer to TROUBLESHOOTING in the documentation)

I can merge the files without the first file, but the results are useless and shows only 87 calls to Foswiki::Sandbox::CORE:open and that's it.

Do I have any chance got get an valid profiling result? Or is there an other tool, that I can use in this case?

Answer 1

I'm not sure why you can't get NYTProfiler to work, we've used it to figure out some performance issues in Foswiki 2.0.2, which have been partially addressed in Foswiki 2.0.3. There are a couple of issues going on, but one major cause is our conversion to UNICODE internally, and some Perl regex issues in perl versions before 5.20. https://rt.perl.org/Public/Bug/Display.html?id=66852

Foswiki 2.0.3 made the following performance updates:

• Changed some heavily called internal functions from regular expressions, to index()
• Changed EditRowPlugin to generate less html that requires processing by regular expressions in the rendering module.
• Made some other improvements to reduce excessive re-reading of topics.

If 2.0.3 doesn't significantly help, Check to see if the problem pages have large tables in them. If so, you might try disabling the EditRowPlugin and use EditTablePlugin.

Other than that, you might try our official support channel #foswiki on IRC, http://irclogs.foswiki.org/

Answer 2

The script fails with an exit code 141 when I run it with profiler.

That suggests the process received a SIGPIPE signal. The sigexit option may help.

If I run it without profiler ... it exits successful and producing output.

You're using sudo so permissions might be an issue, but that's just a guess. You'll need to dig deeper to confirm if a SIGPIPE is being received and why.

I'm not familiar with foswiki. Perhaps someone in that community could be more helpful.