1

Assume there are 10 separate EE installations: each installation is on its own domain (possibly with its own hosting), its own database, etc. Assume further that they have a particular Channel that is identical across all 10 sites (e.g. a "Products" channel that has the following fields: name, description, category, price).

Is there a way to develop a separate site that can list all 10 sites' Products entries? And is there a way to allow a user of this new site to be able to search across the 10 sites (e.g. find all Products that are category "toy" order by price)?

Say MSM is not an option.

Improve this question
2

Google Custom Search Engine could handle the cross-site search. It allows you to create a search option that can be limited to one site or multiple sites. I believe there maybe a limit on the number of sites, but I recall it being higher than 10.

As for the 10+ sites consolidated to a single site? That's harder, if you want 'pure consolidation', since the entries would have same entry_ids, possibly similar or same product names, categories, etc. But if you don't want or need everything, and want to retain details on which site which data came from, you could look at RSS or XML, publish a feed from each site, that's imported into the consolidated site, using something like Data Grab or Feed Parser.

Data Grab is more robust, Feed Parser is free. Either can pull data from one EE install into another, and you can add unique fields, for storing original site, and original entry_id for that site, if need be. If images are involved, you'd obviously want to reference the full url to the image on the site it was originally created on. This may require a find and replace on import.

The benefit of pulling the data from multiple sources into a single install and stored in a database like this, is the data can be normalized and limited. If you only need name, price, description, image, and link to original, then everything can be stored into a single channel, with just the needed data. You can either then just search the new site, since it would send the user back to the original site, or you could still use Google CSE.

The data imports can be automated, once the feeds are set-up and working, you can set-up a cron job to run once a day or every hour to find new entries and pull them over. With 10 sites, you may want to create 2 or more cron jobs and stagger them, so that sites 1-5 data are pulled at the top of the hour and sites 6-10 data is pulled at the bottom. That way, the site shouldn't behave sluggishly, on processing.

Improve this answer
5
  • My reluctance to grab data from multiple sites and store it the centralized database is that I would be duplicating data, which could possibly lead to anomalies. Furthermore, making sure the data is up to date would be another challenge. Cron would work, but what if I wanted the freshest possible data? Cron every minute? Another option: I suppose the remote sites could be designed to push the data from their site to the central site during each add/edit/delete operation (I have this option since I'm the one responsible for all the sites). Seems hackish still.
    – ExpressionEngineNewbie
    Jan 27 '13 at 9:25
  • By the way, I think MSM would make life easier. However, MSM apparently only OK if each of the sites in MSM belong to the same owner. This is not necessarily the case in my situation.
    – ExpressionEngineNewbie
    Jan 27 '13 at 9:26
  • Centralizing content from independent sources just isn't that easy. There's really not a good way to make the data up-to-the minute. RSS & XML handle data updates perfectly fine, so long as they are properly configured. And you can also do on-demand updates when needed. But the data consolidation has to happen somewhere, either at the search level w/ Google CSE (all cached, updated when crawled); at the browser level, as the javascript option (10+ queries, then processed, while user waits); or at the server as my suggestion. But you'll need to make concessions, and users prefer speed.
    – nonprofit_tech
    Jan 27 '13 at 10:00
  • Tough one. I have never used Google CSE. What has been your experience with it? I'm mainly concerned with filtering (parametric search) and not so much fulltext search.
    – ExpressionEngineNewbie
    Jan 27 '13 at 10:03
  • Google CSE works fine, depending on your needs are. We've used it on a number of client's sites instead of EE search. It's not up to the minute and not great for parameter or filters. I'd say you want to to consolidate at database level. We use isotope for filters like that, which means data tags. The tags are generally applied as CSS classes, so items are removed from original query based on options. But then the initial search can be EE channel/category pulls, then drilled down on by user.
    – nonprofit_tech
    Jan 27 '13 at 12:42
0

Use JQuery / Javascript to load the various searches from the other sites and display all on one page.

Either that, or serverside, you'ld have to make a very elaborate plugin that does it, has to connect to all databases separately.

Edit: or use a server side search and retrieval technology like Apache Lucene ( http://en.wikipedia.org/wiki/Lucene ) or similar, and feed data into it.

Improve this answer
1
  • It's probably easy enough to retrieve the data for display, especially if each site somehow exposes the data in some structure data (e.g. JSON, RSS, etc.). However, sorting search results across multiple sites might be a bit harder. Every solution I have considered seems a bit hackish.
    – ExpressionEngineNewbie
    Jan 27 '13 at 9:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.