Control-M for Snowflake - Documentation for Helix Control-M Integrations

Control-M for Snowflake - Documentation for Helix Control-M Integrations

Snowflake is a cloud-computing platform that enables you to process, analyze, and store your data.

Control-M for Snowflake enables you to do the following:

  • Connect to any Snowflake endpoint from a single computer with secure login, which eliminates the need to provide authentication.
  • Integrate Snowflake jobs with other Control-M jobs into a single scheduling environment.
  • Execute any of the following Snowflake job operations:
    • Execute a Snowflake-supported SQL statement.
    • Copy a queried database and schema into an existing or new file in cloud storage.
    • Copy from an existing table.
    • Create a table, populated by a query, in a specified database and schema.
    • Copy data from a cloud storage location into an existing table in Snowflake.
    • Start or pause an existing Snowpipe.
    • Call an existing procedure and its arguments.
    • Monitor the status of a Snowpipe for a set period of time.
  • Monitor the status, results, and output of Snowflake jobs in the Monitoring domain.
  • Attach an SLA job to your Snowflake jobs.
  • Introduce all Control-M capabilities to Control-M for Snowflake including advanced scheduling criteria, complex dependencies, Resource Pools, Lock Resources, and variables.
  • Run 50 Snowflake jobs simultaneously per Agent.

Setting up the API Connection with Snowflake

To set up the API connection with Snowflake and enable the use of the Control-M for Snowflake plug-in, you must perform several setup tasks, as described in Setting Up a Snowflake API Connection.

Setting up Control-M for Snowflake

This procedure describes how to install the Snowflake plug-in or the Snowflake IdP plug-in, create a connection profile, and define a Snowflake job in Helix Control-M and Automation API.

Before you Begin

  • Verify that Automation API is installed, as described in Setting up the API.
  • Verify that Agent version 9.0.21.080 or later is installed.

Begin

  1. On the Agent host, set the Java environment variable by running one of the following commands through a command line:
    • Linux:
      • Bourne shell/bash: export BMC_INST_JAVA_HOME=<java_11_directory>
      • csh/tcsh: setenv BMC_INST_JAVA_HOME <java_11_directory>
    • Windows: set BMC_INST_JAVA_HOME="<java_11_directory>"
  2. Run one of the following API commands:
    • For a fresh installation of Control-M for Snowflake, which uses token-based authentication, use the provision image command:
      • Linux: ctm provision image SNF_plugin.Linux
      • Windows: ctm provision image SNF_plugin.Windows
    • For a fresh installation of Control-M for Snowflake IdP, which authenticates using an Identity Provider (IdP), use the provision image command:
      • Linux: ctm provision image SFI_plugin.Linux
      • Windows: ctm provision image SFI_plugin.Windows
    • For an upgrade, use the following command:
      ctm provision agent::update
  3. Create a Snowflake connection profile in Helix Control-M or Automation API, as follows:
  4. Define a Snowflake job in Helix Control-M or Automation API, as follows:

Note

To remove this plug-in from an Agent, follow the instructions in Removing a Plug-in. The plug-in ID for Snowflake is SNF092022, and for Snowflake IdP it is SFI122022.

David Miller
Author

David Miller

David Miller brings 15 years of experience in global economics, personal finance strategy, and market dynamics. He specializes in turning complex economic trends into actionable insights for everyday readers.