Useful tweaks for Nginx & PHP-FPM

If you have ever encountered errors like the ones below, the following tweaks may prove useful for you.

# /var/log/nginx/error.log
[alert] 17086#17086: *56715 open socket #713 left in connection 2775
 
# /var/log/nginx/your-site.com-error.log
[alert] 1685#1685: *1064900 768 worker_connections are not enough while connecting to upstream
[error] 17086#17086: *49510 upstream timed out (110: Connection timed out) while reading response header from upstream
[error] 26161#26161: *713828 connect() to unix:/var/run/php/php8.2-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream

This article assumes that your configurations are located in:

Nginx: /etc/nginx/nginx.conf
PHP-FPM: /etc/php/{version}/fpm/pool.d/www.conf

Please adjust according to your system. Additionally, all values are relative to your system resources, so you need to test which values are right for you.

Tweak Nginx

# /etc/nginx/nginx.conf
 
# in general is 2x of `worker_connection`
+ worker_rlimit_nofile 8192;
 
events {
+    worker_connections  4096;
}

To check the limits of Nginx, follow these steps:

$ ps aux | grep nginx
# ...rest of output
www-data 29925  0.8  0.3 662820 14136 ?        S    Jan09  10:11 nginx: worker process
# ...rest of output
 
# run `cat /proc/[PID]/limits`
# [PID] is value of 2nd column from above
 
$ cat /proc/29925/limits
 
# ...rest of output
# search for row of "Max open files"
# the value should reflect `worker_rlimit_nofile`
Max open files            8192                 8192                 files
# ...rest of output

Tweak PHP-FPM

# /etc/php/{version}/fpm/pool.d/www.conf
 
+ listen.backlog = 4096
 
# If you have dedicated resources and a high-traffic site,
# consider using 'static' to save CPU usage and maximize RAM utilization.
pm = dynamic
 
# [Total Available RAM in MB] – [Reserved RAM in MB] – [10% buffer] / 80
# 80MB is average ram usage per worker
# see below for command to get ram usage per worker
+ pm.max_children = 30
 
# max_children * 0.25
+ pm.start_servers = 8
 
# max_children * 0.25
+ pm.min_spare_servers = 8
 
# max_children * 0.75
+ pm.max_spare_servers = 22
 
# if it's too big, RAM Usage will grow over the time
# its better to keep recycled it
+ pm.max_requests = 100

To determine the average memory usage of a process, execute the following command:

ps -ylC php-fpm --sort:rss

Please note that you may need to match php-fpm with your specific process name, which could be different, such as php-fpm8.2. In case your process name varies, adapt the command accordingly:

ps -ylC php-fpm8.2 --sort:rss
S   UID   PID  PPID  C PRI  NI   RSS    SZ WCHAN  TTY          TIME CMD
S     0 28804     1  0  80   0 31344 125139 ep_pol ?       00:00:00 php-fpm8.2
R    33  5922 28804  9  80   0 53372 128049 -     ?        00:00:00 php-fpm8.2
R    33  5923 28804  8  80   0 55032 130139 -     ?        00:00:00 php-fpm8.2
R    33  5921 28804  8  80   0 56540 130098 -     ?        00:00:00 php-fpm8.2

In the output, pay attention to the RSS column, which indicates the average memory usage in kilobytes for each process. As illustrated in the example above, the average memory usage is approximately 56540 kilobytes or around 60 megabytes per process.

Or alternatively, use this command to get the summary:

$ ps --no-headers -o "rss,cmd" -C php-fpm8.1 | awk '{ sum+=$1 } END { printf ("%d%s\n", sum/NR/1024,"M") }'
63M

This command provides a single output, representing the average memory usage of the specified process (php-fpm8.1 in this case). The result, such as 63M, indicates the average memory consumption in megabytes.

Tweak the OS (Ubuntu)

# /etc/sysctl.conf
 
# max connections
+ net.core.somaxconn = 4096
 
# max unprocessed connections
+ net.core.netdev_max_backlog = 4096
+ net.ipv4.tcp_max_syn_backlog=4096

Then run sysctl -p for changes to take effect. After that, you can also restart Nginx and PHP-FPM services.

# /etc/security/limits.conf
 
## Example hard limit for maximum number of opened files
+ *    hard    nofile    4096
## Example soft limit for maximum number of opened files
+ *    soft    nofile    4096

To make these changes take effect, restart the server.

Enable PHP-FPM Status Page

Uncomment the pm.status_path line from PHP-FPM config. The default value is /status. Adjust as needed or add a prefix if you have multiple pools.

pm.status_path = /php-fpm-status

Then, edit Nginx per-site config, add this inside the server block

location ~ ^/(php-fpm-status|ping)$ {
    access_log off;
    allow 127.0.0.1; # only from localhost
    allow 1.2.3.4; # your public IP if needed
 
    # adjust the php as needed
    include fastcgi.conf;
    fastcgi_pass unix:/var/run/php/php8.2-fpm.sock;
}

Restart Nginx and PHP-FPM services for changes to take effect. Now you can access the status page via http://your-site.com/php-fpm-status. All available options are below:

/php-fpm-status
/php-fpm-status?json
/php-fpm-status?html
/php-fpm-status?xml
/php-fpm-status?full
/php-fpm-status?json&full
/php-fpm-status?html&full
/php-fpm-status?xml&full

Enable Nginx Status Page

First check if your installed Nginx has ngx_http_stub_status_module with this command:

$ nginx -V 2>&1 | grep -o with-http_stub_status_module
with-http_stub_status_module # the output should be something like this

Edit Nginx per-site config, add this inside the server block:

location /nginx-status {
    access_log off;
 	stub_status;
 	allow 127.0.0.1; # only from localhost
    allow 1.2.3.4; # your public IP if needed
}

Restart Nginx service for changes to take effect. Now you can access the status page via http://your-site.com/nginx-status.

It’s worth mentioning, always check your Nginx & PHP-FPM config after editing and before restarting the service.

UPDATE 2024-01-12

Troubleshoot PHP-FPM

These commands are useful for obtaining detailed information about the scripts being executed:

To trace a specific process, use the following command, replacing [PID] with the process ID you want to monitor. You can identify the process ID using commands such as htop or top:

strace -ffttTo /tmp/strace.out -p [PID]

To view the trace file, open /tmp/strace.out.[PID]. Dumping the output into a temporary file makes it easier to analyze compared to streaming it in real-time.

Here’s an example of the output:

09:39:13.263962 lstat("/var/www/html/bootstrap", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 <0.000034>
09:39:13.264168 stat("/var/www/html/resources/lang", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 <0.000029>
09:39:13.266313 stat("/var/www/html/bootstrap/cache/config.php", {st_mode=S_IFREG|0644, st_size=78217, ...}) = 0 <0.006338>
09:39:13.272851 access("/var/www/html/bootstrap/cache/config.php", F_OK) = 0 <0.000013>

To identify the syscalls taking the longest time, use the following command:

strace -c -f $(pidof php-fpm8.2 | sed 's/\([0-9]*\)/\-p \1/g')

Wait for a few seconds, then press Ctrl+C. The output looks like this:

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 20.05    0.002212           2       943        19 stat
 18.38    0.002028           2       941         4 lstat
 12.83    0.001415           3       550         4 read
  9.46    0.001044           8       126           write
  8.48    0.000935           1      1067           rt_sigaction
  7.09    0.000782           3       230           poll
  4.69    0.000517           2       221        20 access