Magento index – The basics, benefits and tips to avoid errors

What does Magento indexing mean?

Definition 1:

 

Indexing is the process when Magento transforms product, category etc. data in order to achieve the fastest request speed.

 

Definition 2:

Data storage in MySQL in order to have the most appropriate indexes for SQL requests.

 

What would happen if there were not indexing in Magento?

All the following examples below show that Magento indexes have significant benefits in terms of ecommerce store performance. Even in the case of only one product it is worth running the index, though you should keep in mind that with one product, there is only a little increase in speed. So if you did not use Magento indexing, the you would face some serious drawbacks:

  1. Prices would be calculated in terms of download speed, on the basis of shopping cart price rules and customer groups.
  2. Calculation of inventory for configurable and bundle products would take place only after loading the product collection.
  3. „Layered navigation” would be generated in real time on the basis of hundreds of product attributes.
  4. The products within a category’s sub-categories, would be queried in a recursive way.

 

Jargon in Magento indexing

Before describing the process of performing Magento index, it is important to clarify a few basic terms.

  • Indexed data – The data set which defines the user interface display
  • Indexer – Procedure suitable for creating data arrays
  • Index event – The event or moment when the source data is modified
  • Index process – The time when the indexer is running
  • Main controller – It gives tasks to the Index process

 

Magento Index process

Below you can see the flow chart of the Magento Index process:

 

Magento index flowchart

 

Indexing can be started by way of a Magento event and manually as well. During the index process, in the user interface, the data are read from the EAV tables, thus reindexing is not recommended only in special, justified cases.

 

Describing Magento Indexing

Catalog and product indexing is switched off by default with both Magento CE and Magento EE. To switch it on go to System -> Configuration -> CATALOG -> Catalog -> Frontend -> Use Flat Catalog Category and Use Flat Catalog Category lists and select YES.

After saving, you’ll be notified in the Magento notification area that a reindex is necessary.

 

Magento index switch on

The Magento notification area can be found in the admin panel, right below the menu.

 

How to run Magento indexing?

 

1) Magento admin

Go to System -> Index Management. Click th Select All link, select Reindex Data at the Actions option, then click Submit.

 

Magento index run admin

 

2) Shell script

Start a terminal, then go to the Magento source directory and give the following command:

php shell/indexer.php –reindexall

 

Magento index run shell script

 

You also have the possibility of running only one index if you run the indexer.php with the –reindex <indexer> parameter. For example, if you want to reindex catalog URLs, then run the following command:

php shell/indexer.php — reindex catalog_url

 

Magento index run shell script 2

 

If you set the info parameter with the indexer, then you will get a list of all the indexes and the index codes.

 

Magento index run shell script

 

+1 method: from PHP code

Run the following code for the total reindex:

<?php
/* @var $indexCollection Mage_Index_Model_Resource_Process_Collection */
$indexCollection = Mage::getModel('index/process')->getCollection();
foreach ($indexCollection as $index) {
    /* @var $index Mage_Index_Model_Process */
    $index->reindexAll();
}

If you want to update only one index, then the following code  is enough:

<?php
$process = Mage::getModel('index/indexer')->getProcessByCode('catalog_product_price');
$process->reindexAll();
}
instead of catalog_product_price insert the index name that you'd like to update.

 

 

How to speed up reading from Magento index table

It is possible to add MySQL table indexes to index tables. One way to do it is the manual method, e.g. in phpMyAdmin, but after a reindex these indexes will disappear because following every reindex, Magento deletes and creates index tables. So we need a more advanced solution.

In the example below we add indexes to a product index.

Let’s create a module and “subscribe to” the catalog_product_flat_prepare_indexes event in its config.xml in the following way:

<catalog_product_flat_prepare_indexes>
    <observers>
        <aion_catalog_product_flat_prepare_indexes>
            <type>singleton</type>
            <class>aion_catalog/observer</class>
            <method>catalogProductFlatPrepareIndexes</method>
        </aion_catalog_product_flat_prepare_indexes>
    </observers>
</catalog_product_flat_prepare_indexes>

You can see in the xml section above that before the product reindex, the catalogProductFlatPrepareIndexes method of the Aion_Catalog_Model_Observer class will be called.

At this moment the table is created, we only need to add the new indexes to the table:

 

/**
 * Add indexes to product flat table
 *
 * @param Varien_Event_Observer $observer observer
 *
 * @return void
 */
public function catalogProductFlatPrepareIndexes(Varien_Event_Observer $observer)
{
    /** @var Varien_Object $indexesObject */
    $indexesObject = $observer->getIndexes();
    /** @var array $indexes */
    $indexes = $indexesObject->getIndexes();
 
    /**
     * We add indexes to these fields for faster request processes
     */
    $addFields = array(
        'upload_date', 'news_to_date', 'special_price', 'special_from_date', 'special_to_date'
    );
 
    foreach ($addFields as $field) {
        $indexes['IDX_'.strtoupper($field)] = array(
            'type' => Varien_Db_Adapter_Interface::INDEX_TYPE_INDEX,
            'fields' => array($field)
        );
    }
 
    $indexesObject->setIndexes($indexes);
}

 

We can add other attributes to the $addFields array. Please not that it is not all column types to which you can add indexes.

 

Errors that my occur during indexing

It can happen that you see some indexes with a processing status for a long time in the Index Management menu. This means that PHP could not close the index_process_*.lock files in the var/locks folder. The easiest way to solve this problem is to delete the lock files.

Another common error appears when the flat table gets damaged and this makes indexing fail. In such a case, the damaged flat table can be deleted without a problem because when reindexing, Magento checks if the table exists or not and if not, it creates it.

The columns in MySQL has a maximum number of 4096, but in reality it is much smaller. It also depends on how many characters each column can store. Thus it can happen that the table cannot be created. This is a rather hidden error because you can only see in the admin panel that the table cannot be reindexed, but you will have to search for a reason with server settings.

If you encounter such a problem, it’s worth taking a look at the attributes – whether all of them are needed in the flat tables or not. If you manage to reduce the number and/or size of the attributes (e.g. using varchar instead of text), this problem can be easily solved.

Unfortunately, the “trickiest” error is when the index process halts. This can happen when memory or running time limit is low on the web server. You can resolve this issue by increasing memory to be used by PHP and extend runtime.

Another solution is to run indexing with a command which I described at “2. shell script “ in the “How to run Magento indexing?” section.

 

Disadvantages of Magento index

I don’t want to confuse anyone. Magento index is a very useful feature of Magento. But at the same time you need to know about its disadvantages as well. Here are some:

  1. It duplicates data. Since data are written to the EAV data structure in admin, but on frontend the data are read from the index table. Therefore every product data is duplicated in the database. If you do not pay attention to reindex, it can happen that you see different things on the user interface than in the admin area.
  2. It needs abundant resources. Magento indexing uses a lot of server capacity because it reads data from a lot of locations.
  3. It is easy to reach the maximum limit of MySQL column numbers. I’ve mentioned this in the previous section.

 

Summary

I hope I could show properly the possibilities in Magento index and also provided help in solving some common problems that may arise. If you need any help, we’d be happy to assist you. Feel free to ask any questions in the comments field.

 

 

49 replies
  1. Deloris Baiera says:

    I precisely had to thank you so much again. I do not know what I might have made to happen in the absence of the entire advice documented by you on this situation. Entirely was the frightful scenario in my opinion, nevertheless noticing the skilled manner you dealt with it made me to weep with fulfillment. Now i am thankful for the service and even hope that you recognize what a powerful job you’re accomplishing educating the others via a web site. Most likely you have never come across any of us.

  2. Drug Rehab Hospital says:

    Inpatient Substance Abuse Treatment Near Me http://aaa-rehab.com Drug Rehab Centers http://aaa-rehab.com Suboxone Clinic Near Me Free
    http://aaa-rehab.com

  3. Mass hysteria says:

    I’m curious to find out what blog platform you have been using? I’m experiencing some small security problems with my latest site and I’d like to find something more secure. Do you have any suggestions?

  4. get likes in Instagram says:

    I was suggested this blog by way of my cousin. I am now not positive whether or not this post is written by him as no one else recognize such exact approximately my difficulty. You’re wonderful! Thank you!

  5. webmaster says:

    It’s really a cool and useful piece of information. I am glad that you shared this useful information with us. Please keep us up to date like this. Thanks for sharing.

  6. Jeff Cox says:

    It’s really a great and useful piece of information. I am happy that you shared this helpful info with us. Please keep us up to date like this. Thanks for sharing.

  7. car coating protection says:

    Throughout this awesome design of things you actually get an A+ for effort. Where you misplaced me ended up being in all the particulars. As as the maxim goes, details make or break the argument.. And that could not be much more correct in this article. Having said that, allow me inform you what exactly did do the job. The authoring is certainly highly convincing and this is possibly why I am making an effort to opine. I do not make it a regular habit of doing that. Next, even though I can easily notice a leaps in logic you make, I am not necessarily convinced of how you appear to unite the points which in turn make the conclusion. For now I will, no doubt subscribe to your position but hope in the near future you connect your dots much better.

  8. web hosting services says:

    Hello, i think that i saw you visited my website so i came to “return the favor”.I am trying to find things to enhance my website!I suppose its ok to use a few of your ideas!!

  9. Canada says:

    Someone essentially help to make seriously posts I would state. This is the first time I frequented your website page and thus far? I amazed with the research you made to create this particular publish extraordinary. Fantastic job!

  10. aduslot says:

    Thank you for sharing superb informations. Your website is very cool. I’m impressed by the details that you’ve on this web site. It reveals how nicely you understand this subject. Bookmarked this website page, will come back for more articles. You, my pal, ROCK! I found just the information I already searched everywhere and simply could not come across. What a great site.

  11. Download Joker123 says:

    Great write-up, I am normal visitor of one’s blog, maintain up the excellent operate, and It’s going to be a regular visitor for a lengthy time.

  12. kevin david bbb says:

    Very well written information. It will be supportive to anyone who utilizes it, including yours truly :). Keep up the good work – i will definitely read more posts.

  13. kevin and david says:

    Thank you for another informative blog. Where else could I get that kind of information written in such an ideal way? I’ve a project that I am just now working on, and I have been on the look out for such information.

  14. FafaSlot Gamming says:

    I’ve been browsing on-line greater than three hours lately, but I never found any interesting article like yours. It is pretty worth enough for me. Personally, if all website owners and bloggers made good content as you did, the web might be much more helpful than ever before.

  15. Slot Joker123 says:

    I’ve recently started a site, the information you offer on this web site has helped me tremendously. Thanks for all of your time & work.

  16. Marcelle Darga says:

    I simply want to say I am just new to blogs and seriously savored you’re blog. Almost certainly I’m going to bookmark your blog . You really come with tremendous articles. Thank you for revealing your web-site.

  17. Daftar VivoSlot says:

    Thanks a lot for sharing this with all of us you really know what you are talking about! Bookmarked. Please also visit my web site =). We could have a link exchange agreement between us!

  18. laudo avcb palhoça says:

    Whats up! I just would like to give a huge thumbs up for the good information you could have right here on this post. I will probably be coming back to your blog for extra soon.

  19. job advertising site says:

    Excellent read, I just passed this onto a friend who was doing a little research on that. And he actually bought me lunch as I found it for him smile So let me rephrase that: Thank you for lunch! “Dreams are real while they last. Can we say more of life” by Henry Havelock Ellis.

  20. Game podcasts says:

    I love your blog.. very nice colors & theme. Did you create this website yourself? Plz reply back as I’m looking to create my own blog and would like to know wheere u got this from. thanks

Trackbacks & Pingbacks

  1. hydroxychloroquine prescription

    Magento index – The basics, benefits and tips to avoid errors – aionhills.com

  2. what happens if a girl takes viagra

    Magento index – The basics, benefits and tips to avoid errors – aionhills.com

  3. where to buy tylenol

    Magento index – The basics, benefits and tips to avoid errors – aionhills.com

  4. cialis online usa

    Magento index – The basics, benefits and tips to avoid errors – aionhills.com

  5. ciprofloxacina

    Magento index – The basics, benefits and tips to avoid errors – aionhills.com

  6. sildenifil for sale

    Magento index – The basics, benefits and tips to avoid errors – aionhills.com

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published.