Magento performances, a long quest

We have been hosting Magento stores since version 0.9.6 was out almost, the first customer who signed with us on this technology did it way back in march 2008. Ever since, Magento didn’t stop progressing, so did we.

They made the code and performance better, we learned through hundreds (actually > 1500+) of Magento sites we host. Big, small, enormous, EE, CE, in all versions.

Magento offers a fantastic framework to develop a business online, but also a quite tricky road to performances. I wrote roughly a hundred posts around those questions, gave tens of conferences and wrote a full howto some years ago. Still, I feel some tricky points a not that well known from the community, hence this article.

1/ Cache backends

There are important points to know about cache backends.
There are two level of cache : the fast cache and the slow cache backend.

Those backends serves different purposes, but there are key points to remember:

  • The fast backend stores “unstructured” data and thus can use several type of backends like APC, File, Database, Memcached, Redis, Xcache, etc.
  • The slow backend need a structured storage. This means that APC, Xcache, Memcached and other unstructured backend cannot be used. Stays Redis, File, Database.

Common mistake : If a slow backend is pointing to a non structured storage backend (memcached, APC, etc.) the cache is just not working. On a single front server, there is no need to share the data between several servers, hence, you can safely use File for example. Using Database to store cache or session is probably not a good option in the end since the database has a lot of other tasks to handle.

Here is a table of what works (by order of preference) :

Configuration Slow backend Fast backend
Single server (Front + DB) Redis, File, Sqlite, Database Redis, Memcached, Xcache, APC
Muti front servers Redis, Sqlite, Database Redis, Memcached, Xcache, APC

In any case: 

  • If you use a file backend, try to put it on a SSD drive or, better, in a ramdisk
  • Never ever *ever* use memcached, Xcache or APC as a slow backend, the cache will just simply not work

2/ Sessions

The session backend can get very quickly filled up. This is especially true when the indexation spiders of the search engines get excited by your site content. The backend then becomes quickly polluted by “orphan sessions”. Those orphan sessions are created because the “bots” & “spiders” are not real users and they don’t engage in a session/cookie relationship. Since those cookies are not used, the corresponding sessions in the backend are just useless. It’s not just search engines, it’s all scanners. Nowadays, 61% of the web trafic is made by robots so it roughly means (it’s not exact, just for the thinking) that 61% of the sessions are probably junk.

On the other hand, it’s useful to let your real customers store their own sessions on the long term, I would advise at minimum a week. A lot of people prepare their cart in advance nowadays and there are few things more upsetting than losing them because the store owner flushed it “just in case”. So don’t flush your users sessions, let them leave at least a week.

Since you may want to keep your real user sessions for a while & since the bots are filling constantly your backend, it’s wise to put a cron jobs to clean the orphan sessions on a daily basis. I won’t share our scripts about the subject, but it’s quite easy to do, some few lines of perl or php will do magic.

To clean the proper one, just check (for example) the declared “user agent” in the session content and wipe the one looking like bots. There are more evolved way to proceed, but already this basic sanitation will be good to take.

3/ Cron jobs

There are two mechanisms in Magento to handle the recurring jobs needed to keep the site healthy.

One involve the “classical” tool of the environment, usually Linux and usually the Cron daemon. The other one, less known, is an internal table in Magento’s database, which kick a job or another at a given time. A lot of jobs already exist by default in Magento and other are added by the SI or extensions taken from the market.

In the end, when those two cron systems are kicking in several jobs at the same time, it can become a serious hit on the site performances.

Checking the System crons

You can list the linux system cron jobs by impersonating the user running the apache process and look the the declared jobs (here is an example) :

$ su www-client
$ crontab -l

It could also be that the hosting company decided to put it in the /etc/cron.* directories, even if it’s unusual since we tend to give access to those crons to the SI or company to give them autonomy. Still, you can double check if you don’t find anything with crontab -l.

Checking Magento’s crons

$ mysql -h $HOST -u $USERNAME –password=$PASSWORD -D $DBNAME -e “select * from cron_schedule;
I let you change the $HOST, $USERNAME, $PASSWORD and $DBNAME with the adapted values.

You should get some lines, hopefully not too many, with the following format:

schedule_id : job_code : status : messages : created_at : scheduled_at : executed_at : finished_at

If you have far overdue jobs, you could sport them by looking the executed_at time and when they finished, if they did. Globally, a lot of entries, especially entries in run state are not so good. It could also be good to check the status of the jobs to see how many are running or are in error.

4/ Block cache, FPC

The block cache and the Full page cache are totally different notions.

The block cache consists in generating some blocks and prerender them since they are not changing much overtime. For example the footer, the menu, the breadcrumb, and many other are in “block cache” mode. First, if you or your SI create a new custom “home made” block, remember to hook it to the cache in order to make it efficient and not a burden for your site. (You can use Nitrogento extension to do so)

So the block cache stays active, even when a user is logged in or has a content in its cart.

On the other hand, the FPC, Full Page Cache, is a cache able to fully cache a page content of your site, triggering almost no PHP interpretation to deliver the page. It’s a great performance tool, even if it’s only active as long as the user has no active session, meaning specific content unique to its page and not generic to everyone. More customisation, recommandation and all tend to deactivate more and more the FPC, but when in use, this cache is a blessing for performances and scalability. The problem is that this cache is only available through the EE version or though some extensions. In Magento, until EE 1.12, the FPC was not really usable on a daily basis, but now it seems to be better since the EE 1.12 version.

One of the pitfalls here is to wipe the cache regularly “just to be sure” or before a major operation. The site then runs on no cache, and get what we call a “called cache effect”, diminishing its performances of around 50 to 80%… You don’t want that to happen, really. Magento now handle its cache eviction quite properly when you update / upload products, so before wiping the cache on a daily basis, be sure that it’s *absolutely* needed.

5/ Log size

Logs are a pain in a dark place of most people anatomy.

To put it short, you most likely need to keep them for a while for debug and/or legal reasons, but in the end, they stack in Magento, making it awfully slow. Exporting them and compressing them outside of the database on a weekly or monthly basis is a good practice that will cheer up your database and make it perform way better in some cases.

Cut, archive, zip, trunk, but don’t let them stack and kill your performances.

6/ Proper configuration of local.xml and several local-xxx.xml files

Ok two topics here.

1/ Learn to properly format your local.xml files. Its options, its syntax, the least glitch in that one (and in enterprise.xml for the EE) can wreck havoc on your site, literally. There are tons of billions of blog posts on this topic, plus the Magento’s documentation online, worst case scenario, copy one from another site and adapt it.

2/ You may not be aware of it, but Magento configurations parser is a little bit funny.

public function loadBase()
{

        $etcDir = $this->getOptions()->getEtcDir();
        $files = glob($etcDir.DS.’*.xml’);
        $this->loadFile(current($files));

         while ($file = next($files)) {
$merge = clone $this->_prototype;
$merge->loadFile($file);
$this->extend($merge);
}

       if (in_array($etcDir.DS.’local.xml’, $files)) {
$this->_isLocalConfigLoaded = true;
}
return $this;
}

Ok, neat, but… wait… Any file ending with xml extension, it will be parsed and loaded…

name it local-old.xml in your app/etc directory or test.xml and Magento will load it. (Got some headaches once on that one)

So clean your app/etc directory and never ever leave anything like local-foo.xml in the directory. Eventually name it .old as an extension to avoid any problem.

7/ One shall never hit the 80%… Never

Magento, when evaluating if it can store a data in the cache backend, verifies if there is space left.

On the paper, it seems quite logical. The problem is rather that Magento refuses to store anything if the backend is already filled at 80% of its capabilities, no matter what space is left. That’s an awkward choice up to me, but anyway, it’s done that way. So if you reach the 80% of filling, you’ll start to see graph like this one :

memcached_fpc

You can see on this typical graph two common problems :

  • The cache is flushed everyday
  • The cache, once regenerated after 14h reach the limit of 80% of filling and then the efficiency of the hit cache ratio falls and the cache efficiency with it

In some cases of incompatibility between the memcached php library, memcached daemon and Magento version, you could also get some “can’t get filling percentage” error in your logs. This is very bad since Magento cannot stat if it can or not store a value, hence it will not, leading to an inefficient caching.

8/ Carefully check the size of the keys stored

If you do not compress keys stored, some can get quite big, like for example the Magento configuration key.

If it grows large enough to go beyond the capacity of storage of your memcached (or redis) backend, the key won’t fit and will be either truncated or discarded, leading obviously to issues. It’s recommended to compress them if they grow too large, rather than expanding the key max size beyond 1 MB since it can lead to other performances issues.

Conclusion

There are tons of other, more advanced, troubleshooting tricks, but it would be too long to explain here.

If you look for an hosting company with serious experience in Magento, we host thousands of Magento sites and since 2008, so feel free to contact us.

Philippe Humeau
Philippe Humeau
Philippe co-founded NBS System in 1999. After a focus on cybersecurity, which he never gave up, he discovered a passion for e-commerce from 2008 on. Pentester, CTO, CCO then CEO, Philippe’s multifaceted profile drove him to becoming OT Group’s Marketing and Strategy Director.