File exchange

Connect your storage systems to exchange files with Enigma. You can send data for batch enrichment and receive processed results delivered directly to your connected storage.

How file exchange works

File exchange provides bidirectional data transfer between your systems and Enigma:

1. Send files to Enigma - Upload data files for batch enrichment, matching, or processing
2. Receive results from Enigma - Get enriched records, match results, and processed data delivered to your storage

This workflow supports large-scale data operations that go beyond individual API calls.

File path length limit

File paths within any connected source cannot exceed 110 characters. This includes the full path from the root of your bucket or SFTP directory.

Connection options

Type	Best for
Enigma SFTP	Getting started quickly with Enigma-hosted infrastructure
Your SFTP	Organizations with existing SFTP servers
Your S3	AWS-native workflows and large file volumes

Connect a storage source

1. From your account menu, select Data Sources.
2. Click Connect New Source.
3. Select your source type from the dropdown.
4. Enter a nickname to identify this connection in the Console.
5. Click Confirm to create the source.
6. Configure the source settings (see the following sections for each source type).

---

Enigma SFTP

Enigma provides a dedicated SFTP server for your organization. This option requires minimal setup since Enigma manages the server infrastructure.

Setup

1. Select Enigma SFTP Source when creating a new source.
2. Enter a nickname for the connection.
3. After creating the source, configure the following:
   • SSH public key (required) - Enigma installs this key on your dedicated server for authentication
   • PGP public key (optional) - Enigma encrypts all delivered files with your key

Connection details

After setup, the Console displays your SFTP connection details:

• Hostname - The SFTP server address
• Username - Your organization's username

Use these details with your SSH private key to connect from any SFTP client.

---

Your SFTP server

Connect Enigma to your own SFTP infrastructure when you need to use existing servers or maintain control over the SFTP environment.

Prerequisites

• An SFTP server accessible from the internet
• Credentials for Enigma to authenticate (username with password or SSH key)

Setup

1. Select Your SFTP Source when creating a new source.
2. Enter a nickname for the connection.
3. After creating the source, configure the following:
   • Host - Your SFTP server hostname
   • Username - The account Enigma uses to connect
   • Password - Account password (if using password authentication)
4. Optionally configure:
   • Read folder path - Directory where Enigma reads your input files
   • Write folder path - Directory where Enigma delivers results
   • PGP public key - Enigma encrypts all delivered files with your key

When using SSH key authentication, install the Enigma SSH public key (displayed in the Console) on your SFTP server.

---

Your S3 bucket

Connect an AWS S3 bucket for file exchange. S3 is ideal for large file volumes and integrates with AWS-native workflows.

Authentication options

Choose the authentication method that fits your security requirements.

Option 1: Access keys

1. Create an IAM user in your AWS account.
2. Attach the S3 permissions policy (see the following section).
3. Generate an access key for the IAM user.
4. In the Console, enter the AWS Access Key ID and AWS Secret Access Key.

Option 2: IAM role (recommended)

IAM roles provide temporary credentials and don't require managing long-term access keys.

1. Create an IAM role in your AWS account.
2. Attach the S3 permissions policy (see the following section).
3. Add Enigma to the role's trust policy (see the following section).
4. In the Console, enter the AWS Role ARN.

Required S3 permissions

Attach this policy to your IAM user or role. Replace YOUR-BUCKET-NAME with your actual bucket name.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::YOUR-BUCKET-NAME"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::YOUR-BUCKET-NAME/*"
    }
  ]
}

Trust policy for IAM role

If using IAM role authentication, add this statement to your role's trust policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::435097078665:role/file-manager-service-prod-lambdas",
          "arn:aws:iam::435097078665:role/file-manager-service-prod-ecs-task"
        ]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

S3 configuration options

After creating the source, you can configure:

• Bucket name - Your S3 bucket
• Region - AWS region where the bucket is located
• Read folder path - Path prefix where Enigma reads your input files
• Write folder path - Path prefix where Enigma delivers results
• PGP public key - Enigma encrypts all delivered files with your key

---

File encryption

For additional security, provide your PGP public key when configuring any source type. When a PGP key is configured, Enigma encrypts all files before delivering them to your storage. You decrypt the files using your corresponding PGP private key.

This encryption is in addition to the transport-layer security (TLS for S3, SSH for SFTP) already protecting data in transit.