I was recently asked to help maintain a dashboard of store performance at work. This is a pretty time-consuming and error-prone task because of our omni-channel Shopify setup.
We rely on multiple Shopify instances at work for our online and physical retail stores. That makes checking store performance more time consuming and error prone, because we need to pull data from the different stores and then combine and analyze it on the back end. We’ve tried using some services that automatically consolidate the data, but the results haven’t been great. And when we run into any discrepancies, it’s hard to figure out what went wrong because we don’t have a good view of how the data has been manipulated.
After taking a look at the Shopify API documentation, I decided that I would try to automate the data collection and consolidation as much as possible. At first, I tried to do the automation via Google Apps script, but I quickly ran into a problem. Google Apps Script has a 6-minute runtime limit; it automatically kills any scripts that run for longer than 6 minutes. The Shopify API also has a time limit; you can only do up to 2 API calls per second, and you need to do an API call per order to pull more information about it. That means that if you try to get data for more than ~720 orders, the script will run into that 6-minute Google Apps limit and die.
Finally, I turned to Perl. It took a while to reacquaint myself with the language, as I haven’t used it since leaving IBM 7 years ago, but after a week of tweaking, I was able to get it working beyond my expectations (love you, Perl). I wanted to share the script in case it’s useful for anyone else.
The script is called get_shopify_data.pl, and you can use it to pull data from any number of Shopify stores for any given time range via the command:
perl get_shopify_data.pl -s START_DATE -e END_DATE
That creates two CSV files in the same directory that the script is in: a summary file and a raw data file. The summary file includes the overall metrics for each store and the raw data file includes that data pulled for all of the individual orders. Here’s what those files look like.
Here is the code for the Perl script, get_shopify_data.pl. You need to go in and add your store’s information to get it working. Just look for “INSERT_” in the code below to find all of the places where you need to add your own info.