Datafiniti Property Data provides updated list data for real estate and rental properties around the world. Each day, Datafiniti aggregates listings from multiple websites for businesses in its database.
This tutorial will provide you with step-by-step instructions how to access this data and import into your own data warehouse.
This tutorial assumes you are familiar with the following concepts:
If you are not familiar with these concepts, we highly recommend reading through the resources in our references section to become familiar, or requesting the help of a developer with your project.
Step 1: Get Your Account Token
The Datafiniti API requires you authenticate each time you access it. This is done by passing in an account token with each API call you make. If you have a Datafiniti account, you can access your account token from the "My Account" page on the Datafiniti website. If you don't have an account, please contact us.
Step 2: Make Your First Request
For this tutorial, we're going to assume you're interested in getting a list of all property listings in Texas.
We recommend doing an initial request for all currently-available data in Datafiniti, so that you have an initial data set to work with.
For the example in this tutorial, we would first do this preview query:
The response will look something like:
This API call only shows 10 matching records. If we needed to download all available matches, we would add "async=true" to our call:
This will initiate a download request, which will take some time to finish. The download call we just made will give us a request ID, which we can use to monitor the download:
Once we see this request has completed, we can download the available data from:
Step 3: Import the Data to Your Data Warehouse
When you open the data files, you'll see data similar to the API call we made in Step 2. This file, however, will contain all the matching data, and will most likely be much larger.
At this point, Datafiniti has delivered the data to you, but you'll still need to import it into your own data warehouse. Since Datafiniti doesn't have knowledge of the schema for your data warehouse, it will be up to you to set up this import. However, the schema of the data file you received from Datafiniti will be in a consistent, intuitive schema, which should make it easier to import.
Step 4: Requesting Updated Data
Now that you've imported the initial data file, you'll probably want to get updated data as it becomes available. Datafiniti makes this possible by allowing you to filter on a "dateUpdated" field during your API call.
Let's assume we ran our initial API call from Step 2 on September 1, 2014, and that today we want to get all the Texas listings that have been updated since that day. This is really easy with Datafiniti. You just modify the API call from Step 2 like so:
https://account_token:@api.datafiniti.co/v3/data/properties?view=properties_all_json&q=province:TX AND dateUpdated:[2014-09-01 TO *]&async=true
The dateUpdated field in the above query has been given a range to search on (from September 1, 2014 to any date greater than that).
Once you're ready to start requesting updated data like this, you'll want to setup a script or application on your side that makes regular API calls with updated dateUpdated values. This script should also automatically check for the availability of the download file, and download and import the file once it's ready.