Capture Tweets with Alteryx Designer

Alteryx Designer comes with a Twitter Search Connector that allows you to pull tweets using your Twitter Account and App. In this Tutorial, I'll walk you through creating a Twitter App and setting up an Alteryx workflow to pull tweets.

In Alteryx, we'll be creating an initial local repository of tweets.

And then modify the workflow so that new tweets are appended to the initial dataset.

Why append? The Twitter Search Macro allows you to search tweets by given search terms, with location as an optional property. The search will only retrieve tweets from the 7 days previous to the search.

When working in an environment that has an Alteryx Server, you can schedule weekly runs of the workflow to automatically stay on top of your twitter feed. You may also want to consider adding the scheduler license to your desktop, see http://insightsthroughdata.com/how-to-automate-data-refresh-operations-to-tableau-server-with-alteryx/.

Don't feel like building it yourself?

Download the workflow (note that you will have to put in your own key and secret!)

Registering a Twitter App

Getting a Twitter account and creating an app is super easy, just follow the steps below to get going!

Signing up for a Twitter Account

You need a twitter account to create a Twitter App, so therefore if you don't already have one, go to dev.twitter.com.

• Click Sign-in at the top right.

• And then Sign up now below the log-in form. 
• Now sign in using the account you just created.

Creating a Twitter App

In order for Alteryx to pull the data from Twitter you'll need to create an App with your Twitter account.

Make sure you are signed into Twitter, and then go to apps.twitter.com. 

If this is your first app the page will look something like this...if it's not you might want to just skip to the Alteryx Workflow sections!

          

Click  Create New App

Complete the New App form. 

Don't worry about entering a callback URL, we are configuring Alteryx to use your account to pull the tweets.

Read the develop agreement (and agree) and then click Create Your Twitter App at the bottom.

Getting your consumer key and secret

We're almost done. In order to set up the workflow we need the Key and Secret. If you are following along in order the App Details page is now displayed.

Toggle over to the Keys and Access Tokens tabs.

You should not share your key/secret with anybody else! Leave this page open we'll be using the key and secret in the Alteryx workflow created in the next step.

Capturing Tweets

I'm going to quickly walk you through the process of creating the initial workflow, downloading the first dataset and then modifying that workflow to append new tweets to the existing data set. 

We're using an Alteryx Database file to capture the tweets over time, there are other supported file types that you could use that have a built in append option, such as Tableau Data Exchange.

Creating a Workflow

Open Designer and create a new workflow. 

Create a folder on your local system and save the workflow in that folder. Click File > Save As > Browse.

In the Save As dialog, click the New Folder icon at the top. 

Rename the folder something like TwitterApp.

Click the folder and enter a name for the workflow, for example Twitter_Feed.

We're going to use relative paths to define where the tweets are saved so this step is important, put the workflow in its own folder!

Adding the Twitter Search tool

Find the Twitter Search tool. Just start typing Twitter Search in the search menu.

Drop the Twitter Search tool on the Canvas.

Configure the Twitter Search tool

Hopefully you left the Twitter App page open, because this step is just a matter of copy and pasting the keys from the app page in Twitter to your Alteryx Twitter Search configuration.Copy the Consumer key from the Twitter App page to the corresponding box in the configuration. Do the same for the Consumer Secret. To paste in the Alteryx Configuration box use key strokes (ctrl+v).

Enter the name of the application, be sure to type it exactly!

Toggle over to the Search tab and enter some search criteria, I'm looking for tweets that contain "alteryx", you may want to put in your criteria.

Now run your workflow to test it.

Review the results by clicking the D output in the Results window.

Make sure you are saving your workflow as you go along! 

Writing the Tweets to a local file

Because Alteryx Database files load quickly and I am collecting tweets from today forward and think that my dataset will grow quite large, I want to write out the data to an Alteryx Database file.After the Twitter Search tool, add an Output Data tool.

In the Configuration select the dropdown arrow next to Write to a File or Database and go to the folder where you saved this workflow, for me that is the TwitterApp folder.

 Enter a name for the file, such as Alteryx_Tweets, make sure that the Save as Type is Alteryx database, and click Save.Run the workflow again to create the initial dataset.

 

You CAN STOP right here, if you want, BUT the next time you run this workflow that file will be overwritten. However if you are writing to Tableau Data Exchange format using the append option you SHOULD stop right here. You may find this blog post interesting in keeping your TDEs up-to-date http://insightsthroughdata.com/how-to-automate-data-refresh-operations-to-tableau-server-with-alteryx/.The next section shows you how to create a workflow that appends only the new data in an Alteryx database. Note that this same practice will work regardless of your source!

Appending new tweets to the initial dataset

So far I've walked you through the process step-by-step but here in this section I'm going to cheat a little and just tell you which tools to use and why. Put in a Block Until Done (BUD)tool between the Twitter Search and Output Data tool.BUD causes Alteryx to wait until all the records reach the tool before sending data downstream.Next bring in the Alteryx_tweets (or whatever you called the file) dataset using an Input Data tool... yep we are reading from the same file we are writing too, it can be done!Join the Tweets dataset to Twitter Search using the Tweet ID field and connect the Left output of the join to the block until done. Now only Tweets that did not get downloaded on the last run are being captured.But wait there is more. We also need to get the Tweets we already captured so Union the Tweets dataset with the Left Join output. 

Watch Appending New Records to an Alteryx Database