What problem are we resolving?
A Magento system is a shop that sells good to customers or to businesses. In other words, all Magento sites need Catalog data. These data can be created manually from the Magento backend or can be imported using various methods.
Below are the Catalog Import methods that we commonly experienced
- Use a csv file to feed a one off import and then create subsequent products manually
- Use a PIM system and link the Magento system with this external environment
- Import the catalog data using an SFTP source
Of course, these 3 methods come with different challenges when the catalog is big, when the PIM system does not have Magento connector available or when the system processes the import to aim at syncing the external Catalog sources with Magento in real time.
Problem 1:
The PIM system has a connector but this connector is written with the vision to be reusable for as many e-commerce framework as possible. This intention is perfectly acceptable and from the PIM system owner point of view, it is likely a best approach to limit the custom parts in this system to exchange data with other e-commerce systems.
However, this leads to a problem for the Magento owner. Magento strength lies in its flexibility and versatility to be adjust for a Catalog Structure. To reach this level of need, Magento engineering is somewhat much more complex than Shopify for instance. As a consequence, the connector for Magento and for any other will benefit from Magento experts insights.
Problem 2:
The Catalog is big and therefore the import performance cannot meet the real-time requirements that would be ideal for a Catalog synchronisation activity.
Some connectors are likely able to meet this requirement of course. However, by overlooking this specific point, we find ourselves with possible weakness when it comes to sharing BIG Catalog data with an external system.
Problem 3:
If the Magento system requires several of these methods to be implemented in the same system, the Magento shop host is under more strains each time an import process occurs.
Whilst this can be translated in hosting adjustment, it is only too common to see Magento site build with a monolithic approach where backend and frontend activities are all hosted by a single server with one size fits all.
A Catalog Synchronisation proposal
- The system should be compatible with leaving Magento shop with the existing connectors if the latter performs well
- The system should offer real-time of close to real-time activities so that no inaccurate data appears on the site. The process should prioritise the critical data (stock, availability, prices) to be synced first
- The system should incur minimal performance side-effect to Magento for most possible Catalog Structures
Description of a possible solution
- The solution may enable a Magento shop owner to replace a PIM connector that has performance issues
- The solution may create opportunities to a Magento shop owner to add new Catalog Data without having to install additional connectors or need to build in-house connector and without incurring additional hosting strains because of the import process inner-working
- The solution may enhance real-time exchange of data between several Catalog Sources and Magento