Data Feed Manager

Data Feed Manager exports your products as XML, JSON, CSV or delimited TXT feeds for Google Shopping, Meta (Facebook and Instagram), Pinterest, Microsoft/Bing Shopping, price comparison sites and ERPs. Feeds are generated in batches by cron, from the admin or on the command line, checked before they are published, and can be uploaded by SFTP, FTP or FTPS.

• Admin demo: https://demo.softawarecommerce.com/try-admin/data-feed-manager
• Sample feeds on the demo store: https://demo.softawarecommerce.com/media/feeds/google_shopping.xml and https://demo.softawarecommerce.com/media/feeds/meta_catalogue.csv
• Changelog: Release notes · Questions: FAQ

1. Requirements

2. Installation

Install with Composer. The access keys come from your account; see https://softawarecommerce.com/shop/composer-access/ for the details.

composer config repositories.softaware composer https://repo.softawarecommerce.com
composer config --auth http-basic.repo.softawarecommerce.com PUBLIC_KEY PRIVATE_KEY
composer require softaware/module-data-feed-manager
bin/magento setup:upgrade
bin/magento setup:di:compile            # production mode only
bin/magento setup:static-content:deploy # production mode only
bin/magento cache:flush

The installation imports Google's product taxonomy in English (en-GB). Make sure Magento's cron is running (bin/magento cron:install) for scheduled feeds. To update later, run composer update softaware/module-data-feed-manager and the same bin/magento commands.

3. Settings

Stores > Configuration > Softaware > Data Feed Manager (also under Softaware > Data Feed Manager > Settings). Store view, currency and schedule are set per feed; these settings apply to all feeds.

4. Creating a feed

Go to Softaware > Data Feed Manager > Feeds and open the Add Feed split button. Choose From template: Google Shopping, Meta catalogue, Pinterest catalogue, Microsoft/Bing Shopping or generic JSON, or Blank feed. The feed form has tabs on the left.

General

Content: template (XML and JSON)

{% for product in products %}
<item>
  <g:id>{{ product.sku }}</g:id>
  <g:title>{{ product.name | strip_tags | clean | truncate(150, '') }}</g:title>
  <g:price>{{ product.regular_price_incl_tax | price }}</g:price>
  {% if product.on_sale %}<g:sale_price>{{ product.final_price_incl_tax | price }}</g:sale_price>{% endif %}
</item>
{% endfor %}

• {{ … }} prints a value, escaped for XML or JSON automatically.
• {% for product in products %} … {% endfor %} marks the repeated item (exactly one, at the top level). Everything before it is the header, everything after it the footer.
• {% if %}, {% elseif %}, {% else %}, {% for image in product.images %}, {% set x = … %} and {# comments #}; loop.index, loop.first and loop.last inside loops.
• Operators: and or not == != < > <= >= in ~ + - * /, the fallback a ?? b and a ? b : c.
• Filters include strip_tags, truncate, upper, lower, capitalize, title, trim, clean, number_format, round, price, default, replace, regex_replace, prepend, append, join, split, first, last, slice, count, unique, date, html_decode, url_encode, json, cdata, raw and escape.

Variables and Filters below the editor lists every variable with a description: product attributes (store view values and option labels, product.raw.<code> for the stored value), prices (regular_price, final_price, special_price, min_price, max_price, each also _incl_tax and _excl_tax, tier_prices, on_sale), stock (qty, is_in_stock, is_salable, availability), url, images (image, images, all_images and more), categories (category, category_path, google_category_id, google_category), parent.* and item_group_id for variants, variants and children, reviews (reviews_count, rating), store.*, feed.* and now.

The language is a sandbox: it has no access to PHP, files or configuration. For safety, XML feeds may not contain XHTML, SVG, MathML, XSLT or XLink content or an xml-stylesheet instruction.

Content: columns (CSV and TXT)

Each column has a Header, a Value (the same template syntax, or static text such as new) and a Default (when empty). Drag the handle to reorder columns. Below the columns: Delimiter (comma, semicolon, tab, pipe), Enclosure (double quote, single quote, none), Encoding (UTF-8, UTF-8 with BOM for Excel, ISO-8859-1, Windows-1252), Line Ending (LF or CRLF) and Header Row.

Product Filters

Schedule

Generate: Manual only (admin or CLI), Daily at set times (for example 03:00, 15:00) or a Cron expression (for example 0 */4 * * *), in the store view's timezone. The form shows the next run.

Delivery (FTP / SFTP)

Upload After Generation uploads the file after each generation; the local file is always written too. Fields: Protocol (SFTP, FTP, FTPS with explicit TLS), Host, Port (empty = 22 for SFTP, 21 for FTP and FTPS), Username, Password (stored encrypted), Remote Directory (must exist), SFTP Host Key and Passive Mode (FTP). Test Connection checks the settings in the form.

• SFTP host key check: the server's key fingerprint (SHA256:…) is saved on the first upload, or you enter it. If the server later presents another key, the upload stops before the password is sent. Clear the field when the server was really replaced.
• After changing the host or protocol, enter the password again; the saved password is not sent to another host.
• A failed upload marks the run as a warning; the local file is still updated.
• Tested status: FTP upload was tested against a local FTP server. SFTP has not yet been tested against a real SFTP server. FTPS does not verify the server's TLS certificate (a limit of PHP's FTP extension), so prefer SFTP.

Preview

Preview First Items renders the first items (Preview Items setting) from the values in the form, including unsaved changes, and validates them. Nothing is written. It also shows how many products match the filters.

Saving

Save keeps you on the form; the split button also has Save & Close and Save & Generate Now. Invalid file locations and template syntax errors are refused on save, and the entered values are kept.

5. Generating feeds

• By cron: feeds with a schedule are generated in the cron group softaware_datafeed.
• In the admin: Generate Now (in the form or the grid's Select menu) runs batch by batch in the browser with a progress bar, then shows items, errors, file size, duration and the URL.
• On the command line: see section 9.

Each run writes to a temporary file. Items that fail their check are skipped and logged. The finished file is validated (streamed XML parse, JSON decode) and only then replaces the old file, in one step. If the file is invalid, it is not published and the previous file stays. Two runs of the same feed cannot overlap.

The grid shows each feed's file link, schedule, next run, last generated time, result, items and size, and has a keyword search. Mass actions: enable, disable and delete. Row actions: Edit, Generate Now, View Log, Duplicate and Delete.

Softaware > Data Feed Manager > Generation Log lists every run: feed, start time, started by (cron, admin or CLI), status, seconds, items, errors, file size and messages.

6. Google category mapping

Softaware > Data Feed Manager > Google Category Mapping lists your store categories. Type in the field next to a category (for example "hoodies") and pick a Google category. A product uses the mapping of its deepest category; categories without a mapping inherit the nearest mapped parent (shown in grey). Use Find category to filter the list and Save Mapping to save. In templates use {{ product.google_category_id }} or {{ product.google_category }}. The mapping applies to all feeds and store views.

Taxonomy Language: download taxonomy-with-ids.<locale>.txt from Google (for example taxonomy-with-ids.de-DE.txt) and import it at the bottom of the page or with softaware:feed:taxonomy-import. Category IDs are the same in every language, so existing mappings stay valid.

7. File locations and privacy

• Feeds can be written below pub/ (public; below pub/media/ the URL is the media URL) or below var/ (private). pub/static/, pub/errors/ and hidden folders are not allowed.
• File names may only contain letters, digits, ., - and _.
• Outside pub/media/feeds/ a feed can only create a new file: a location where another file already exists (for example pub/robots.txt) is refused.
• Public files can be read by anyone who has the URL. Put feeds with private data (cost, stock levels) below var/ and upload them by SFTP.

8. Permissions (ACL)

Under System > Permissions > User Roles > Role Resources > Softaware > Data Feed Manager:

9. Command line

bin/magento softaware:feed:list                                   # feeds with schedule, last run, items, status and URL
bin/magento softaware:feed:generate <id> [<id> ...]               # generate the given feeds
bin/magento softaware:feed:generate --all                         # generate all enabled feeds
bin/magento softaware:feed:preview <id> [--limit=3]               # print the first items
bin/magento softaware:feed:taxonomy-import [path/to/taxonomy-with-ids.de-DE.txt]

10. Troubleshooting

11. Limitations

• Prices are guest prices from the price index; tax uses the store's default tax destination.
• Attribute conditions apply to the selected products, not to variants added through the Configurable Products mode.
• The Google category mapping is global, not per store view or feed.
• CSV values are written as they are, without protection against spreadsheet formulas; do not open feeds built from untrusted text in a spreadsheet program.
• SFTP upload has not yet been tested against a real SFTP server; FTPS does not verify TLS certificates.

12. Uninstall

bin/magento module:disable Softaware_DataFeed
composer remove softaware/module-data-feed-manager
bin/magento setup:upgrade

Feed files that were already generated stay where they are; delete them yourself if channels should no longer read them.