homechevron_rightDocschevron_rightNotehubchevron_rightConfiguring a Snowpipe Route

Configuring a Snowpipe Route

Notehub allows you to automatically route specified Notefiles to Snowpipe . Snowpipe is a continuous data ingestion service from Snowflake that allows you to automate the process of loading data into Snowflake tables as soon as Notes are synced with Notehub.

note

Creating a route for Snowpipe requires access to the Analyze & Scale Edition of Notehub. Learn more about Notehub pricing.

Configuring the Route

To create a Snowpipe route, open your Notehub project, head to the Settings tab and click on the Snowpipe menu item.

snowpipe menu option

On the next screen you'll be asked to provide a variety of values to configure your Snowpipe route (all fields are required).

  1. Organization: This is the name of your Snowflake organization. You can find this value by navigating to the Projects menu, creating a new SQL Worksheet, and executing the show accounts query like so:

    snowpipe organization name

  2. Account: This is the name of your Snowflake account. You can also get the account name from the previous step.

  3. Username and Password: This is the username and password for the Snowflake user you want the process to run under. If needed, you can create a new user account by navigating to the Admin > Users & Roles menu option. Please note that this must be performed by a user with the "ACCOUNTADMIN" role:

    snowpipe new user

  4. Database and Schema: This is the Snowflake database and the schema within the database into which you wish to automatically load data. In the example below, "NOTEHUB_DATA" is the database and "SENSOR_DATA" is the schema.

    snowpipe database and schema

  5. Compute Warehouse: This is the name of the cluster of compute warehouses for your organization. The default compute warehouse is "COMPUTE_WH", but yours can be found under the Admin > Warehouses menu option.

    snowpipe compute warehouse

  6. Sync Frequency in Minutes: This determines how frequently you would like Snowpipe to pull data from specified Notefiles stored in Notehub. The minimum frequency is one minute.

Setting Up Filters

More than likely you will not want to sync every Notefile from every device in your fleet via Snowpipe. Therefore, you may specify one or more Notefile names (comma-separated) and one or more fleets from which to sync data:

snowpipe filters

Save the Route

When you're all set, toggle the Enable Snowpipe service "on" and click the Save button to save your Snowpipe route.

Testing the Route

With your Snowpipe route created, you're now ready to test it out.

  1. Click on the Test connection button to verify the integration between Snowpipe and Notehub. You should see a "success" message like so:

    snowpipe test connection

  2. To test the delivery of Notefile data to Snowpipe, simply sync a new Note to Notehub that matches your filters. Then, at the cadence you previously specified, the data should automatically appear in your Snowflake database in a new "BLUES_EVENTS" table:

    snowpipe data