# Amazon S3

Query CSV, Parquet, and JSON files stored in S3 buckets

- Category: Files & storage
- Runs in: the Phloem cloud

Lets you and Hamming query CSV, Parquet and JSON files in your S3 buckets with SQL.

## What Phloem reads
- CSV, Parquet (`.parquet`, `.pqt`) and JSON files in the buckets you pick
- Each table's columns, listed in Trace

Files are read in place when queried. Nothing is copied into Phloem first.

## What you need
An access key, or a role Phloem can assume, allowed to:
- list your buckets (`s3:ListAllMyBuckets`)
- list objects in the buckets you connect (`s3:ListBucket`)
- read those objects (`s3:GetObject`)

For **IAM Role (STS Assume Role)**, the role's trust policy must let Phloem assume it, with your external ID if you set one.

## Connect
1. In the console, open **Connectors**, choose **Add connector**, then **Amazon S3**.
2. On **AWS Credentials**, choose an **Authentication Method** (**Access Key & Secret** or **IAM Role (STS Assume Role)**), fill in its fields, and pick an **AWS Region**. **Session Token (Optional)** is only for temporary credentials.
3. Phloem tests the credentials by listing your buckets.
4. On **Select Buckets**, pick the buckets to connect.
5. On **Discovered Tables**, review the tables found. Untick any you don't want, or edit a table's **Table Name**, **File Pattern** (`raw/events/**/*.parquet`) or **Description (Optional)**.
6. On **Save Connection**, name the connection and save it.

## What Hamming can do
- Run SQL over the connected tables: query, sample rows, count rows, list columns

Nothing it does changes your buckets.

## Notes
- Discovery proposes one table per file. Set a **File Pattern** glob (`raw/events/**/*.parquet`) to query many files as one table.
- An assumed role's credentials last one hour and renew as needed.
- Credentials with a **Session Token** stop working when the token expires. Edit the connector with fresh ones.

## Connection fields

| Field | Type | Required | Notes |
|---|---|---|---|
| Authentication Method | One of | Yes | Choose how to authenticate with AWS |
| AWS Access Key ID | Text | No |  |
| AWS Secret Access Key | Secret | No |  |
| Session Token (Optional) | Secret | No | Only needed for temporary credentials |
| IAM Role ARN | Text | No |  |
| External ID (Optional) | Text | No | Required if the role's trust policy requires an external ID |
| AWS Region | One of | Yes | Select the AWS region where your S3 buckets are located |
| Path Prefix Filter (Optional) | Text | No | Only scan files under this prefix path |
| File Formats | Any of | No | Select file formats to include. You can add custom extensions. |
| Scan Subfolders | On / off | No | Recursively scan subfolders for files |