Task Factory Azure
Azure Rest Connection Manager
Connection Properties
Used with the Azure Rest Source and Azure Rest Destination.
Option | Description |
---|---|
Storage Account Name |
Storage account name provided by Azure to connect to the desired container. |
Storage Account Key |
The account key provided by Azure. |
Endpoint Suffix |
Endpoint suffix provided by Azure. Default: core.windows.net |
Proxy Configuration
Option | Description |
---|---|
Proxy Host |
Identifies the proxy address. |
Proxy Port |
Identifies the port used by the proxy. |
Proxy User Name |
Enter the user name for proxies that require user authentication. |
Proxy Password |
Enter the password for proxies that require user authentication. |
Azure ML Batch Execution Task
Task Icon | Task Description |
---|---|
The Azure ML Batch Execution Task is used to execute a batch job for an Azure ML web service. See the Azure Storage Connection Manager to learn more about setting up the connection manager. |
Function | Description |
---|---|
Storage Container | Identifies the container within Azure storage where the input and output files are stored. |
Input File | The name of the source file located in the Azure storage container. |
Output File | The name of the destination file located in the Azure storage container. Note: This does not need to exist. |
Web Service Output Name | The name of the web service output defined in the Azure ML experiment web service. |
Base Web Service Url | The request Url defined in the batch execution api documentation for your web service. To obtain this Url, users need to do the following in Azure ML Studio:
|
Execution Timeout | The amount of time in seconds before the component sends a cancel command. |
Parameters | If users have a web service with parameters, this is where to define them. |
Azure ML Storage Source
Task Icon | Task Description |
---|---|
The Azure ML Storage Source retrieves data from blob storage on Azure for an ML web service. See the Azure Storage Connection Manager to learn more about setting up the connection manager. |
Function | Description |
---|---|
Storage Container | Identifies the container within Azure storage where the data will be retrieved. |
File Name | The name of the .csv file located in the Azure storage container. |
Skip First Row | This option is selected by default and used if the first row of data contains headers. |
Azure ML Service Metadata URI | The base web service URI with the bold parameters added to the base URL (e.g.https://ussouthcentral.services.azureml.net/odata/workspaces/3b47fa 82350146aaa16a80d10ed01ba7/services/4171be6bcb584f20afa396ab8590 196c/$metadata.) To obtain the URI, users need to do the following in Azure ML Studio:
|
Refresh Metadata | Selecting the button updates the Destination Column window. By default, all columns are included in the output. |
Row Error Handling |
Additional Information: See the Task Factory Error Row Handling article for more information about this functionality. |
Azure Rest Source
Source
Begin by creating a connection manager that connects to the service's storage container.
Format
The following formats are available:
Delimited Format
Users can manually configure the delimiter, quote identifiers, the lines to skip, and lines to scan. Select Detect Formats to configure these fields automatically.
Option | Description |
---|---|
Delimiter | Identifies the character that separates columns. The default value is the comma ( , ). |
Quote | Identifies the character used to indicate a value (if present). |
Lines To Skip | Users can designate the number of lines to skip at the beginning of the delimited file. |
Auto | Automatically detects which lines to skip. |
Lines To Scan | Determines the number of lines to scan when detecting the file. |
Detect Formats | Select this button to begin scanning the chosen file. |
Detected Delimited View - After the format is detected, a new view appears that allows users to configure or change different properties of the file.
JSON Format
Option | Description |
---|---|
Root Json Path | JSON queries can return multiple levels, therefore, this field Identifies the root to be used. |
Output Columns | In this window, users can add and remove columns, define their name, token path (defined within the raw data), data type, length, precision, scale, and code page. |
Date Parse Handling | Specifies how date formatted strings are parsed when reading JSON text.
|
Date Time Zone Handling | Specifies how to treat time values converted between string and DateTime.
|
Float Parse Handling |
Specifies how floating point numbers are parsed.
|
XML Format
Option | Description |
---|---|
Root XPath Query | XML queries can return multiple levels, therefore, this field Identifies the root to be used. |
Namespaces: |
|
Columns | Users can create, remove, and configure the name, index (zero-based), data type, length, precision, and scale of the columns being extracted from the file. |
Output Columns
Users can select which columns to include/exclude in the output. Additionally, users can rename the columns by selecting into its corresponding output name.
Error Handling
Option | Description |
---|---|
Not Used | Error handling is not used and no error constraint is available. |
Ignore Failure | All errors are ignored and the package continues to execute. |
Redirect Row | All error rows are directed to an error output. |
Fail Component | (Default selection) On error, the component fails and the package execution stops. |
Preview
Select Preview Data after configuring the previous tabs to view an output sample.
Azure ML Destination
Destination Icon | Destination Description |
---|---|
The Azure ML Destination is used to send data to a blob storage on Azure storage to be consumed by an Azure ML web service. This is the input file of the Azure ML batch execution task. See the Azure Storage Connection Manager to learn more about setting up the connection manager. |
Function | Description |
---|---|
Storage Container | Identifies the container within Azure storage where the data is stored. |
File Name | The name of the file in the Azure storage container. |
Azure ML Service Metadata Url | The base web service URL with the bold parameters added to the base URL (e.g. https://ussouthcentral.services.azureml.net/odata/workspaces /3b47fa82350146aaa16a80d10ed01ba7/services/4171be6bcb584f20afa 396ab8590196c/$metadata.) To obtain the Url, users need to do the following in Azure ML Studio:
|
Output Options | This selection includes header columns in the output by default. |
Refresh Metadata | Selecting this button updates the Input and Destination columns window (input and destination columns with the same names are auto-mapped.) |
Azure Rest Destination
Target
Begin by creating a connection manager that connects to an Azure Storage container. After a connection manager is created, the source window populates with files and folders. Select the desired file to continue configuration.
Option | Description |
---|---|
Target Item Name | Defines the name and extension of the file you would like to create when the component is executed. (For example: MyNewXmlList.xml) |
Delimited Format
Option | Description |
---|---|
Delimiter | Identifies the character that separates text. |
Quote | Identifies the character (single or double quote) that indicates value. |
Include Header Row | Writes the column headers at the beginning of the document. |
Encoding | Selects which encoding is used to create the document. |
Culture | Used to identify the document's language culture. |
Json Array Format
Option | Description |
---|---|
Generate | Select to automatically generate the Row Body metadata. Users can also add custom header and footer data. |
Encoding | Select which encoding is used to create the document. |
Culture | Used to identify the document's language culture. |
XML Array Format
Option | Description |
---|---|
Generate | Select to automatically generate the Row Body metadata. Users can also add custom header and footer data. |
Encoding | Select which encoding is used to create the document. |
Culture | Used to identify the document's language culture. |