Last March, Facebook announced HACK, a new open source programming language for its HipHop Virtual Machine (HHVM) touted to interoperate seamlessly with PHP. Following the announcement, I was fascinated watching everything they attempted with the new release in trying to wring as much performance as possible from PHP. This gave me an idea: Although we don’t face the exact challenges that Facebook does on a daily basis, it would be interesting to see what would happen if I tried running Drupal under HHVM, the latest iteration of Facebook’s execution engine.
Official HHVM packages are only distributed for Ubuntu and Debian, but thankfully some enterprising people have packaged them up for CentOS. So with a quick stand up of a Virtual Machine and some additional packages, we were good to go. To test, we chose a fairly complex site that we developed last year, built on the usual Panels and Display Suite with significant relationships between the content. This meant that on any given page, Drupal was likely to load and render several entities beyond just the one on the page.
I took the current PHP and database for the site and stood it up on a local VM with 4 cores and 6.5 GB of memory. We did some minor optimization of FPM to set the number of servers, along with some increases to the caches in MySQL. Other than that, it’s pretty much a vanilla installation of PHP 5.3 with XCache, Percona 5.5, and nginx. The goal was to provide as much of an “apples to apples” comparison of the two interpreters without as much regard for making everything as fast possible. We then spidered the site and sampled out 1000 URLs at random and ran JMeter to generate 30 concurrent requests against the site running with HHVM and PHP-FPM. We recorded time and load on the server.
Starting off, we ran the test against a cold start of the site. We cleared the Drupal cache and restarted nginx, PHP-FPM and MySQL. We then hit the home page with a single request to build the persistently cached items. As the requests started ramping up, the time to complete each one went up as expected.
Average response time sampled every 5 seconds, cold start
What was happening on the server mirrors that.
Overall CPU and memory usage, cold start
We checked the actual processes that were generating the CPU and memory usage, in this case only pulling out mysqld since tracking each process from PHP-FHM was challenging.
CPU usage for mysqld, cold start
Memory usage for mysqld, cold start
MySQL was using anywhere from 40 – 60%, with a few spikes to just over 100% of one core. All the other cores were entirely used by PHP-FPM. Similarly, MySQL was using, on average, about 300 MB of memory. I’m not entirely sure what was causing the areas where processing pauses and response times spike. I saw them on all scenarios, and my guess is that they were due to some sort of I/O blocking, maybe from MySQL or nginx.
As a second test, we ran the exact same URLs against the system, clearing the Drupal cache to see if a burn in iteration for XCache and MySQL would help. Overall, it reduced the median response time by a whole 400 milliseconds and increased the throughput by a whopping 2.081 pages per minute. The response times in general were a little more choppy with fewer of the peaks and valleys from the previous run, but the results were pretty consistent.
Average response time sampled every 5 seconds, warm start
Overall CPU and memory usage, warm start
After that we restarted the machine, switching out PHP-FPM for HHVM. Thankfully, it supports fastCGI, so it was a simple matter of altering the nginx configuration slightly. The setup was the exact same as for PHP-FPM; we made sure that the Drupal cache was cleared and restarted MySQL and nginx. We then hit the homepage with a single request and started up JMeter.
Average response time sampled every 5 seconds, cold start
The first thing we noticed was that it completed in just under half the time, 6:28 as opposed to 13:21. Every part of the response graph was better, and both the peaks and valleys were significantly lower. The little hiccup at the very start before it dropped, was the JIT compiler running.
Looking at the server stats, those were improved as well.
Overall CPU and memory usage, cold start
HHVM runs as a single process, so we were able to capture CPU and memory usage separately for them.
CPU usage for mysqld and hhvm, cold start
Memory usage for mysqld and hhvm, cold start
Just looking at these graphs, it’s pretty easy to tell a couple of things. Namely, both CPU and memory usage for HHVM are improved over PHP-FPM. Peak memory usage for HHVM was just about 320 MB for a total system usage of approximately 20% compared to the 25 – 26% under PHP-FPM. Likewise, total CPU usage was lower with only a couple of spikes to over 90% and the median closer to 60%, compared to consistent spike to 95% CPU and a median closer to 70 – 75%.
Similar to the PHP-FPM test, we also ran the scenario against a warm start of HHVM. Like PHP-FPM, there was very little difference, only about a 200 millisecond difference in median response time.
If we are to look at the high level, HHVM compares very well to PHP-FPM.
We saw more than double the throughput and less than half the average response time. Combined with the decrease in system resources needed, it’s a pretty compelling argument to switch to HHVM.
There are some important considerations, however. The biggest is that while the HHVM team is attempting to get as close to Zend PHP as possible, they aren’t there yet. As of the latest reports, HHVM passes 99.83% of the unit tests for Drupal, but there’s no idea how whatever idiosyncrasies exist in various contributed modules will affect it. For instance, we couldn’t get GD to work at all, despite all indications that it should. Thankfully, it’s easily replaceable with ImageMagick – for most manipulations. In fact, we didn’t run into any pages during our test that failed with HHVM, but you never know what might not work until you actually run into it. In addition, the testing was done entirely on the front end. While we went through a couple of common scenarios on the administrative side, we didn’t test that thoroughly. Some PHP modules have been ported, such as APC and memcache, and there is work by third parties to add others, such as MongoDB, Ice and Redis, but many modules haven’t and probably will never be.
It’s also a moving target. Right now the HHVM team is looking at around an 8-week release cycle. Presumably there won’t be significant regressions as they move forward, but you never know. Similarly, they are targeting Ubuntu and Debian for official packages, so if you’re running Fedora or CentOS you have to either build from source or depend on a third party repository that may not be up to date.
The performance results shown above are from running a production site on a very much ‘non-production’ virtual machine running on MacBook Pro with a 2.3 GHz Core i7 processor. There was very little tuning on any portion of the stack to ensure best performance. Load testing was performed from a separate machine over a wireless network, albeit one that was not being used for any other purpose. The load testing did not include any wait time or requests for non-PHP assets and was not intended to simulate real usage, merely to benchmark the performance of the PHP interpreters.
Drupal Developer having 9+ year experience, implementation and having strong knowledge of technical specifications, workflow development. Ability to perform effectively and efficiently in team and individually. Always enthusiastic and interseted to study new technologies
We believe Drupal is similar or superior to what TYPO3 and others can accomplish, but it's not easy to convince others about it.
Khách hàng vay mua nhà đất, căn hộ cao cấp từ 3 tỉ đồng trở lên sẽ áp dụng hệ số rủi ro gấp 3 lần quy định trước đó.
The upcoming Drupal 8 includes many changes large and small that will improve the lives of site builders, site owners, and developers. In a series we're calling, "D8FTW," we look at some of these improvements in more detail, including and especially the non-obvious ones.
