SPX - A simple profiler for PHP

[![Build Status][:badge-ci:]][:link-ci:] ![Supported PHP versions: 5.4 .. 8.x][:badge-php-versions:] ![Supported platforms: GNU/Linux, macOS & FreeBSD][:badge-supported-platforms:] ![Supported architectures: x86-64 or ARM64][:badge-supported-arch:] [![License][:badge-license:]][:link-license:]

Click here for a live demo of the analysis screen

SPX, which stands for Simple Profiling eXtension, is just another profiling extension for PHP. It differentiates itself from other similar extensions as being: * totally free and confined to your infrastructure (i.e. no data leaks to a SaaS). * very simple to use: just set an environment variable (command line) or switch on a radio button (web request) to profile your script. Thus, you are free of: * manually instrumenting your code (Ctrl-C a long running command line script is even supported). * using a dedicated browser extension or command line launcher. * multi metrics capable: 22 are currently supported (various time & memory metrics, included files, objects in use, I/O...). * able to collect data without losing context. For example Xhprof (and potentially its forks) aggregates data per caller / callee pairs, which implies the loss of the full call stack and forbids timeline or Flamegraph based analysis. * shipped with its web UI which allows to: * enable / configure profiling for the current browser session * list profiled script reports * select a report for in-depth analysis, featuring these interactive visualizations: * timeline (scale to millions of function calls) * flat profile * Flamegraph

Requirements

Platforms support is currently quite limited. Feel free to open an issue if your platform is not supported. Current requirements are:

• x86-64 or ARM64
• GNU/Linux, macOS or FreeBSD
• zlib dev package (e.g. zlib1g-dev on Debian based distros)
• PHP 5.4 to 8.5

Installation

Prerequisites

• PHP development package (corresponding to your installed PHP version).
• zlib development package:
• For Debian based distros (including Ubuntu, Kubuntu...), just run: sudo apt-get install zlib1g-dev.
• For Fedora based distros (including CentOS, AlmaLinux, Rocky Linux...), just run: sudo dnf install zlib-devel.

Install the extension

git clone https://github.com/NoiseByNorthwest/php-spx.git
cd php-spx
git checkout release/latest
phpize
./configure
make
sudo make install

Then add extension=spx.so to your php.ini, or in a dedicated spx.ini file created within the include directory. You may also want to override default SPX configuration to be able to profile a web request, with this one for example for a local development environment.

ZTS PHP (multi-thread)

ZTS PHP is supported, with these extra limitations: - a little overhead (theorically unnoticeable in most cases) is added when SPX is loaded, even if it is not enabled. - Ctrl-C a CLI script will not make the possible profiling session to be properly finished. - segfaults are more likely than for NTS PHP. In this regard, avoid more than ever mixing SPX with other instrumenting extensions (debuggers, profilers...).

Also, consider ZTS PHP support as still being in beta.

Linux, PHP-FPM & I/O stats

On GNU/Linux, SPX uses procfs (i.e. by reading files under /proc directory) to get some stats for the current process or thread. This is what is done under the hood when you select at least one of these metrics: mor, io, ior or iow.

But, on most PHP-FPM setups, you will have a permission issue preventing SPX to open a file under /proc/self directory. This is due to the fact that PHP-FPM master process runs as root when child processes run as another unprivileged user.

When this is the case, the process.dumpable = yes line must be added to the FPM pool configuration so that child processes will be able to read any file under /proc/self.

Development status

This is still experimental. API might change, features might be added or dropped, or development could be frozen.

You can still safely use it in a non-production environment.

Contributions are welcome but be aware of the experimental status of this project and please follow the contribution rules described here: CONTRIBUTING.md

Basic usage

web request

Assuming a development environment with the configuration described here and your application is accessible via http://localhost.

Just open with your browser the following URL: http://localhost/?SPX_KEY=dev&SPX_UI_URI=/ to access to the web UI control panel.

N.B.: http://localhost/ must be served by a PHP script through standard web server feature like directory index or URL rewriting. The PHP script will however not be executed, SPX will intercept and disable its execution to serve its content in place.

If you see only a blank page then make sure to set zlib.output_compression = 0 in your PHP configuration file

You will then see the following form:

Then switch on "Enabled". At this point profiling is enabled for the current domain and your current browser session through a set of dedicated cookies.

Profiling can also be triggered with Curl as shown in this example:

curl --cookie "SPX_ENABLED=1; SPX_KEY=dev" http://localhost/

N.B.: You can also enable the profiling at INI configuration level via the spx.http_profiling_enabled setting, and therefore for all HTTP requests. However, keep in mind that using this setting on a high-traffic environment could quickly exhaust the storage device's capacity of the SPX's data directory.

Then refresh the web request you want to profile and refresh the control panel to see the generated report in the list below the control panel form.

Then click on the report in the list and enjoy the analysis screen.

Command line script

Instant flat profile

Just prepend your command line with SPX_ENABLED=1 to trigger profiling. You will get the flat profile printed on STDERR at the end of the execution, even if you abort it by hitting Ctrl-C, as in the following example:

$ SPX_ENABLED=1 composer update
Loading composer repositories with package information
Updating dependencies (including require-dev)
^C
*** SPX Report ***

Global stats:

  Called functions    :    27.5K
  Distinct functions  :      714

  Wall time           :    7.39s
  ZE memory           :   62.6MB

Flat profile:

 Wall time           | ZE memory           |
 Inc.     | *Exc.    | Inc.     | Exc.     | Called   | Function
----------+----------+----------+----------+----------+----------
  101.6ms |  101.6ms |   41.8MB |   41.8MB |       12 | Composer\Json\JsonFile::parseJson
   53.6ms |   53.6ms |     544B |     544B |        4 | Composer\Cache::sha256
    6.91s |   41.5ms |   41.5MB |   -7.5MB |        4 | Composer\Repository\ComposerRepository::fetchFile
    6.85s |   32.3ms |   47.5MB |    5.4MB |        5 | 1@Composer\Repository\ComposerRepository::loadProviderListings
    7.8ms |    7.8ms |       0B |       0B |        4 | Composer\Cache::write
    1.1ms |    1.1ms |     -72B |     -72B |        1 | Composer\Console\Application::Composer\Console\{closure}
  828.5us |  828.5us |     976B |     976B |       12 | Composer\Util\RemoteFilesystem::findHeaderValue
  497.6us |  491.0us |  710.2KB |  710.2KB |        1 | Composer\Cache::read
    2.4ms |  332.6us |   20.9KB | -378.8KB |       34 | 3@Symfony\Component\Finder\Iterator\FilterIterator::rewind
  298.9us |  298.9us |    2.2KB |    2.2KB |       47 | Symfony\Component\Finder\Iterator\FileTypeFilterIterator::accept

N.B.: Just add SPX_FP_LIVE=1 to enable the live refresh of the flat profile during script execution.

Generate profiling report for the web UI

You just have to specify SPX_REPORT=full to generate a report available via the web UI:

SPX_ENABLED=1 SPX_REPORT=full ./bin/console cache:clear

Handle long-living / daemon processes

If your CLI script is long-living and/or daemonized (e.g. via supervisord), profiling its whole lifespan could be meaningless. This is especially true in case of a service waiting for tasks to process.
To handle this case, SPX allows to disable the automatic start of profiling and exposes 2 userland functions, spx_profiler_start(): void & spx_profiler_stop(): ?string, in order to respectively control the start and the end of the profiled spans.

Here is how you can instrument your script:

<?php

while ($task = get_next_ready_task()) {
  spx_profiler_start();
  try {
    $task->process();
  } finally {
    spx_profiler_stop();
  }
}

And of course this script must be run at least with profiling enabled and the automatic start disabled as in the following command:

SPX_ENABLED=1 SPX_REPORT=full SPX_AUTO_START=0 my_script.php

Automatic start can also be disabled for web requests via the spx.http_profiling_auto_start INI parameter or via the control panel.

Side notes: - spx_profiler_start() and spx_profiler_stop() can safely be nested. - when profiling with the full report type, spx_profiler_stop() returns the report key so that you will be able to store it somewhere, for instance among other information related to the profiled span. With the report key you can build the analysis screen URL which ends with this pattern /?SPX_UI_URI=/report.html&key=<report key>.
- in CLI context, when automatic start is disabled, no signal handlers (i.e. on SIGINT/SIGTERM) are registered by SPX.

Add custom metadata to the current full report

When profiling with full report as output, it could be handy to add custom metadata to the current report so that you will be able to easily retrieve it or differentiate it from other similar reports.

This is especially true for the long-living process use case which otherwise would not allow to differentiate a report from other ones of the same process.

To do that SPX exposes the spx_profiler_full_report_set_custom_metadata_str(string $customMetadataStr): void function.

As you may have notificed, this function accepts a string as custom metadata, for the sake of flexibility and simplicity on SPX side. It is up to you to encode any structured data to a string, for instance using JSON format.

The metadata string is limited to 4KB, which is large enough for most use cases. If you pass a string exceeding this limit it will be discarded and a notice log will be emitted.

This string will be stored among other current report's metadata and you will retrieve it in the report list on web UI side.

spx_profiler_full_report_set_custom_metadata_str() can be called at any moment as long as the profiler is already started and not finished yet, which means: - at any moment during the script execution when automatic start is enabled (default mode). - at any moment after the call of spx_profiler_start() and before the call of spx_profiler_stop() when automatic start is disabled.

Here is an example:

<?php

while ($task = get_next_ready_task()) {
  spx_profiler_start();

  spx_profiler_full_report_set_custom_metadata_str(json_encode(
    [
      'taskId' => $task->getId(),
    ]
  ));

  try {
    $task->process();
  } finally {
    spx_profiler_stop();
  }
}

Advanced usage

Configuration

Name	Default	Changeable	Description
spx.data_dir	/tmp/spx	PHP_INI_SYSTEM	The directory where profiling reports will be stored. You may change it to point to a shared file system for example in case of multi-server architecture.
spx.http_enabled	0	PHP_INI_SYSTEM	Whether to enable web UI and HTTP request profiling.
spx.http_key		PHP_INI_SYSTEM	The secret key used for authentication (see security concern for more details). You can use the following command to generate a 16 bytes random key as an hex string: openssl rand -hex 16.
spx.http_ip_var	REMOTE_ADDR	PHP_INI_SYSTEM	The $_SERVER key holding the client IP address used for authentication (see security concern for more details). Overriding the default value is required when your application is behind a reverse proxy.
spx.http_trusted_proxies	127.0.0.1	PHP_INI_SYSTEM	The trusted proxy list as a comma separated list of IP addresses*. This setting is ignored when spx.http_ip_var's value is REMOTE_ADDR.
spx.http_ip_whitelist		PHP_INI_SYSTEM	The IP address white list used for authentication as a comma separated list of IP addresses*.
spx.http_ui_assets_dir	`/usr/local/share/misc/php-spx/ass