S3 Reader
Introduction
S3 stands for Amazon Simple Storage Service (Amazon S3). It is an object storage service offering industry-leading scalability, data availability, security, and performance. You can use Amazon S3 Reader to store and retrieve any amount of data at any time, from anywhere.
Overview
This property specifies the URL for the S3 file, from which the binary data is to be read. It must start with "s3:///". The suggested feature can be used to view the list of buckets, subdirectories, and files. Bucket names are suggested if the property is empty or "s3:///". Once a bucket is selected, it can list subdirectories and files immediately below the bucket. Names of subdirectories end with a forward-slash ("/"). The suggested feature is not supported if the properties in the S3 dynamic account are parameters.
After you complete the configuration for the S3 reader task, a fixed set of dataslot gets generated for further use in the automation.
Steps to configure
Drag and drop the S3 reader task in the automation and click on it. The below form will appear on the right side of the screen.
Provide the connection from the drop-down list. It can be done by using https://sappersoftware.atlassian.net/wiki/spaces/SD/pages/63602585 template or use the pre-defined connection.
Select the existing connection that has been created for the S3 source or create a new one from the plus button and the below form will appear on the screen.
Now select the application from the drop-down list.
In the connection type select S3.
Provide a name and description for the connection.
Mention an Access key. An access key is used to sign the requests you send to Amazon S3. Like the username/password pair you use to access your AWS Management Console, Access Key Id and Secret Access Key are used for programmatic (API) access to AWS services. You can manage your Access Keys in AWS Management Console.
Provide a Secret Access Key. It is a secret like your password. This key is used for your security, AWS doesn’t reveal your password to you if you forgot it (you’d have to set a new password). Similarly, AWS does not allow the retrieval of a secret access key after its initial creation. This applies to both root secret access keys and AWS Identity and Access Management (IAM) user secret access keys.
If the connection is already created then you need to select S3 from the connection dropdown.
Now, provide a bucket name as per the requirement. A bucket name is a virtual-hosted–style URL. It is a part of the domain name in the URL. For example http://bucket.s3.amazonaws.com.
Fill the region field. A region filed is S3 buckets specific. When you create a new bucket you need to select the target region for it. For example, the region field can be US West (N. California).
Now, add the directory path. It’s a location from where you can fetch the data for creating the configuration. A path is a string of characters used to uniquely identify a location in a directory structure. It is composed by following the directory tree hierarchy in which components, separated by a delimiting character, represent each directory. The use of the directory path is to add the file that needs to be read.
Now select how you want to read the file, and the below form will appear on the screen.
If you choose using the file name, then you need to add the file in the select file field.
If you choose using regular expression then the below field will be open and you need to fill the input based on your requirements.
Now, click on save to complete the configuration.
After completing the configuration you can use it for your future automation.
Related articles
https://sappersoftware.atlassian.net/wiki/spaces/SD/pages/88375173
https://sappersoftware.atlassian.net/wiki/spaces/SD/pages/88244181