Perl script to pull Shopify store data

I was recently asked to help maintain a dashboard of store performance at work. This is a pretty time-consuming and error-prone task because of our omni-channel Shopify setup.

We rely on multiple Shopify instances at work for our online and physical retail stores. That makes checking store performance more time consuming and error prone, because we need to pull data from the different stores and then combine and analyze it on the back end. We’ve tried using some services that automatically consolidate the data, but the results haven’t been great. And when we run into any discrepancies, it’s hard to figure out what went wrong because we don’t have a good view of how the data has been manipulated.

After taking a look at the Shopify API documentation, I decided that I would try to automate the data collection and consolidation as much as possible. At first, I tried to do the automation via Google Apps script, one of my favorite Google Apps features, but I quickly ran into a problem. Google Apps Script has a 6-minute runtime limit; it automatically kills any scripts that run for longer than 6 minutes. The Shopify API also has a time limit; you can only do up to 2 API calls per second, and you need to do an API call per order to pull more information about it. That means that if you try to get data for more than ~720 orders, the script will run into that 6-minute Google Apps limit and die.

Next, I tried to do the data pull via Javascript in an HTML file that I could “run” in the browser. Unfortunately, that also kept failing because the browser timed out waiting for the data pull to finish (darn you Shopify API limit). 

Finally, I turned to Perl. It took a while to reacquaint myself with the language, as I haven’t used it since leaving IBM 7 years ago, but after a week of tweaking, I was able to get it working beyond my expectations (love you, Perl). I wanted to share the script in case it’s useful for anyone else.

The script is called get_shopify_data.pl, and you can use it to pull data from any number of Shopify stores for any given time range via the command:

perl get_shopify_data.pl -s START_DATE -e END_DATE

That creates two CSV files in the same directory that the script is in: a summary file and a raw data file. The summary file includes the overall metrics for each store and the raw data file includes that data pulled for all of the individual orders. Here’s what those files look like.

get_shopify_data.pl Raw Data CSV file headers

get_shopify_data.pl Raw Data CSV file headers

get_shopify_data.pl Summary CSV file

get_shopify_data.pl Summary CSV file

Here is the code for the Perl script, get_shopify_data.pl. You need to go in and add your store’s information to get it working. Just look for “INSERT_” in the code below to find all of the places where you need to add your own info.