2

I'm running into memory issues with a page that lists lots (hundreds, eventually 1000+) of little news items that consist of a title, date, and sometimes brief description. These are each channel entries, and I've taken the following steps to reduce memory usage and speed up page rendering:

  • utilize the {exp:channel:entries} disable parameter (disable="categories|category_fields|member_data|pagination")
  • cache the entire page output with CE Cache ({exp:ce_cache:it id="page" seconds="3600"})
  • maintain a relatively high PHP memory limit (256MB)
  • avoid rendering any more complex fields (Playa, Matrix, etc.), which are this case part of the entries but not used on the page

I also attempted to temporarily boost PHP's memory using ini_set('memory_limit', '1024M') before the exp:channel:entries call in the template, then ini_set('memory_limit', '256M') after, but this doesn't work. (Never tried this in a template, so I'm not surprised anyway.)

Are there any common practices or alternate approaches that would be appropriate here?

Or, is this just a nonsensical goal that's forcing a no-win outcome?

2
  • Can you double-check that you have 256MB available to PHP -- e.g., create a info.php file with <?php phpinfo(); ?> and browse it? I'd think 1000+ entries of simple data should be ok.
    – Jay F
    Aug 30, 2013 at 17:55
  • I would too! And yes, it's definitely available. I used Tools → Utilities → PHP Info which I normally forget is there – but yes I have 256MB available.
    – Matt Stein
    Aug 30, 2013 at 18:03

2 Answers 2

2

One thing worth looking at is how well the {exp:channel:entries} loop you describes performs in a template with nothing else--no embeds, plugins, etc., vs in the context of the actual page on your site. It's important to be sure that your general template / page architecture isn't putting too heavy a load on EE / your server.

In other words, in addition to what you've done above, the caching you do on other elements on the page, and across your whole site, may be very relevant. In general, I've found that caching more, and for longer times, across a whole site is a good way to leave space for queries / processes that need memory and processor time.

I've dealt with a bunch of EE memory limit issues over the years, and I rely heavily on CE Cache these days. Where possible, I especially like using the static cache in the mode where it totally bypasses EE. But, beyond caching, I've had to do some combination of the following:

  • add more memory -- the obvious!!!
  • use a CRON job to generate close-to-final-format results in the database, and pull those into the template using a simple {exp:query}
  • ditch using EE channel or query tags, and do SQL queries directly from PHP (because, it's less overhead and you can really optimize your SQL)
2
  • Thanks Jay. I'm using CE Cache pretty thoroughly throughout the site in question, so the peripheral load is relatively light. I ended up writing my own query to pull the entries, which made a huge difference. The generation of your close-to-final results (to store) still seems like it could choke the server unless you're staggering the build somehow. Anyway, I appreciate your thoughts!
    – Matt Stein
    Aug 30, 2013 at 21:34
  • @Matt pre-generation can be great if you can exploit predictable load cycles on your site--e.g., you can generate things in the middle of the night when there's little traffic; and also if some entries aren't changing, you can just incrementally add / update only the changing entries.
    – Jay F
    Aug 30, 2013 at 22:07
0

Matt, a couple different approaches for you.

I assume this is a page hit frequently.

1) use pagination in your channel entries tag to reduce the number of results displayed

2) cache the page using template preferences

3) use stash to cache the channel entries fields you need. For example here is a rough framework for the approach - ( hint - you need to add some paramaters to these tags)

{exp:stash:set_list [add params] save="yes" refresh="1440"}
{exp:channel:entries [add params] disable=[everthing except pagination]}
{stash:title}{title}{/stash:title}
{stash:date}{expiration_date}{/stash:date}
...And so on
{/exp:channel:entries}
{/exp:stash:set}

Then later in the page use get_list to display results in your markup.

4) Advanced: create your own plugin to return results, the use then stash to cache it. This approach worked us on one landing page because we needed to display up to 100 channel entries plus channel_images for each ( extra queries) plus matrix (extra queries). Using the channel entries tag, the number of queries increased relative to the number of results. In the end 200+ queries and huge memory. Now, about 33 queries (doesnt increase relative to num of results), memory about 4mb and speed about 0.43 seconds.

Do you need pagination? If so, copy the query module, replace $sql with your own query and away you go.

5) use automatee to refresh your cache results

6) limit page to only one or two embeds

2
  • Sorry if I wasn't clear: the goal is to actually render a high number of entries, so #1 is outside the scope of the question. I forgot about EE's own template caching, though I prefer CE Cache – I don't know that doubling up has any benefit. I've not experimented by adding Stash to the mix – I wasn't aware there'd be a performance gain but I'll have to give that a shot.
    – Matt Stein
    Aug 30, 2013 at 21:36
  • If it was me, I'd go number 4 route. Do a quick test using query tag and compare debugger
    – Lloyd Hill
    Aug 31, 2013 at 0:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.