How to push data via API to Databox

In this document, we'll outline how to sync custom data regularly into Databox from a Cloud-Based application with API access. Databox offers Java, PHP, Ruby, Node.js, GO and Python SDKs.

IN THIS ARTICLE

• General Guidelines
• What to define before syncing custom data
• How to prepare custom data
• How to sync custom data
• How to prepare a Scheduler
• How to sync data with a Scheduler
• Additional Information

General Guidelines

There are a few general guidelines to keep in mind while syncing custom data: 

1. Maximum of 10 requests per second
2. Maximum of 500kb in size
3. Sync frequency should not be less than every 5 minutes to ensure optimal performance

What to define before syncing custom data

First, we must identify the Data Source where we want to pull our data from, the SDK where we want to push our data to, and a scheduler application so that the data can automatically be synced with Databox on a regular basis. 

For our example, we have defined the following: 
Data Source: Sentry
SDK: Databox SDK
Scheduler: Heroku (free)

How to prepare custom data

Sentry is an open-source error tracking platform that provides complete app logic, deep context, and visibility across the entire stack in real time. At Databox, we use Sentry to track our backend errors and events. All events are organized in projects. 

Our goal is to get create a Metric in Databox to track Events per Project. 

Project Name	Today	Yesterday	This Week	This Month
Project 1	10	5	30	200
Project 2	1	30	70	150

Next, we'll look at Sentry's API Documentation to determine what we need in Sentry. According to this documentation, we will need the following:

• Authentication token (https://docs.sentry.io/api/auth/)
• List of Event counts for the organization (https://docs.sentry.io/api/organizations/get-organization-stats/)
• List of Projects in the organization (https://docs.sentry.io/api/organizations/get-organization-projects/)

How to sync custom data

Now that our data is prepared in Sentry, we need to sync the data with our Databox Account. Based on the Databox Developer Documentation, we will need the following in Databox:  

• Data Source Token (https://developers.databox.com/create-token/)
• Databox SDK (https://developers.databox.com/sdks)

The code used for pulling and pushing the data can be found here: http://github.com/databox/sentry2databox.

How to prepare a Scheduler

Now we need to schedule regular data syncs, so the data is automatically synced with Databox on a daily basis. Log in to your Heroku Account, or create one for free.

Learn how to copy your code to Heroku in the README section of this document. You will need to use Heroku CLI to upload your code to Heroku. If you don't already have this in place, use this document for installation steps.

Within your Heroku Account, use the following prompts to create a new SSH public key.

$ heroku login
$ heroku git:remote -a sentry2databox
$ git push heroku master

When the command completes, you should see a message verifying the deploy. 

How to sync data with a Scheduler

Go to the Overview page and click Add a New Addon. You will need to enter a credit card to add addons, but you will have enough free credits per month that syncing this data with Databox will not create any charges. 

Search for Heroku Scheduler and click Provision.

 

Click on the Scheduler and Add a new job.

 

The last step is to set up the configuration variables. Navigate to Settings > Config vars and enter your SENTRY_TOKEN, SENTRY_ORG, DATABOX_TOKEN and LAST_X_DAYS.

You can verify that everything works correctly by selecting More > Run console in Heroku and running the job.

 

If everything is set up correctly, you will see the following: 

Additional Information

• Each Rest API Custom Metric supports only one data Aggregation function. To view the same Rest API Custom Metric with different Aggregation functions selected (i.e., "SUM" on one vs "AVG" on another) duplicate Custom Metrics and create the views you desire by following these steps.