Use S3 reader, write and Logger task

Introduction

S3 stands for Amazon Simple Storage Service (Amazon S3). It is an object storage service offering industry-leading scalability, data availability, security, and performance. You can use Amazon S3 Reader to store and retrieve any amount of data at any time, from anywhere. On the other hand, the role of the S3 writer is to write-type snap that can read a binary data stream from its input view and writes it to an S3 file destination.

The logger tasks allow you to write log messages, which is useful for error or bug diagnosis purposes.

You can elect to log errors, info, warning, or note messages based on the requirements.

Overview

 

S3 reader

The S3 reader helps to specify the URL for the S3 file, from which the binary data is to be read. It must start with "s3:///". The suggested feature can be used to view the list of buckets, subdirectories, and files. Bucket names are suggested if the property is empty or "s3:///". It’s an important part of core task. The core nodes helps you to incorporate into your automation seamlessly. The S3 reader helps you to connect your application for reading the file appropriately.

For more information visit: https://sappersoftware.atlassian.net/wiki/spaces/SD/pages/700448779

S3 Writer

S3 File Writer is a write-type snap that reads a binary data stream from its input view and writes it to an S3 file destination. If you provide values for file permissions, snap sets the permissions to the file. It’s an important part of core task. The core nodes helps you to incorporate into your automation effectively. The S3 writer task helps you to connect your application for writing the snaps properly.

For more information visit: https://sappersoftware.atlassian.net/wiki/spaces/SD/pages/701202435

Logger task

Logger task is a detailed report of the automation, that helps us to understand the process of running the application in a better way. With the help of the logger task you will be able to add any kind of message. It can be information related to errors, notes, warnings, or any kind of details related to that automation. This is an important helper task for connecting the automations seamlessly.

 

How to use S3 Reader?

  1. If the connection is already created then you need to select S3 from the connection. Now, mention the bucket name as per the requirement.

  2. Fill the region field. A region filed is S3 buckets specific. When you create a new bucket you need to select the target region for it. For example, the region field can be US West (N. California).

  3. Now, add the directory path. That is adding a specific location for fetching data for performing configuration.

  4. Select how you want to read the file, and the form will appear on the screen.

 

 

 

5. If you choose using regular expression then the mention field will be open and you need to fill the input based on your requirements.

6. Now, click on save to complete the configuration. After completing the configuration you can use it for your future automation.

 

Input 

Description

Example 

Input 

Description

Example 

Name

This section associated with the name of the task.

S3 Reader

Description

In the description you will get the details for the specific task.

 

Connection

A connection is a mechanism by which a task authenticates with a third-party app or service.

For more information please visit : https://sappersoftware.atlassian.net/wiki/spaces/SD/pages/63602585

Bucket Name

A bucket name is a virtual-hosted–style URL. It is a part of the domain name in the URL.

http://bucket.s3.amazonaws.com.

Region

In the region field you will need to mention the information related to the region associated with the particular bucket name.

US West (N. California)

Directory path

It’s a location from where you can fetch the data for creating the configuration.

 

File name

Select the file which you want to upload for reading.

 

Regular expression

A regular expression is a sequence of characters that specifies a search pattern in text.

 

 

How to use S3 Writer?

  1. If the connection is already created then you need to select S3 from the connection dropdown. Now, provide a bucket name as per the requirement. A bucket name is a virtual-hosted–style URL. It is a part of the domain name in the URL.

  2. Fill the region field. A region filed is S3 buckets specific. When you create a new bucket you need to select the target region for it. For example, the region field can be US West (N. California).

  3. Now, add the directory path. It’s a location from where you can fetch the data for creating the configuration.

  4. Provide an appropriate file name. It is a unique name or title of a file and the extension. The name of the file can be associated with .CSV or .JSON based on the requirement.

  5. Add a file content. The file content can be associated with a list of your objects and metadata. The object can be assigned a MIME type that describes the format of the contents. You can also add a formula based on the requirements. The format can be CSV and JSON. For example: x-world/x-3dmf. 3dmf.

  6. Now, click on save to complete the configuration. After completing the configuration you can use it for your future automation.

Input 

Description

Example 

Input 

Description

Example 

Name

This section associated with the name of the task.

S3 Writer

Description

In the description you will get the details for the specific task.

 

Connection

A connection is a mechanism by which a task authenticates with a third-party app or service.

For more information please visit :

Bucket Name

A bucket name is a virtual-hosted–style URL. It is a part of the domain name in the URL.

http://bucket.s3.amazonaws.com.

Region

In the region field you will need to mention the information related to the region associated with the particular bucket name.

US West (N. California)

Directory Path

It’s a location from where you can fetch the data for creating the configuration.

 

File Name

Select the file which you want to upload for writing.

 

File Content

In this section you need to mention the content of the file .

 

How to user logger task?

  1. To use the logger task you need to drag and drop it in the automation. Once you select the task the form will appear on the right side of the screen. Now, you can add a log message based on your requirement.

  2. You can add multiple log messages as per the need. You can add various logs by clicking on the add symbol. For removing any logs you need to click on the minus symbol.

  3. The log message can be a note, error, information, task, or message. It can store detailed issues from the specific automation for providing information's to the end-user. Once the log is completed save it for further process.

4. Save the automation by selecting the save option. Now click on reports to download and view the logs. For more information please refer to

5. In the report, section click on the below option from the actions field. After you click on this option the log for the automation will be downloaded in a JSON format. Now you can view it from the notepad as shown in the below format.

 

6. Here you can check for the specific notes or message that has been generated in the log message. For example: INFO=========Started execution soap task=========

Input 

Description

Example 

Input 

Description

Example 

Name

Provide the name of the field.

Logger

Description

Add a suitable description for this field.

 

Log Message

Log message helps to store detailed issued from various application and system components. Therefore, logs need to be easy to read, understand, and parse.

INFO=========Started execution soap task=========

Reports

The reports tab lists all the automation’s execution summary reports with the details. The details consist of a holistic view about the flow of execution of particular automation i.e., data flow and the steps executed throughout the automation.