Integration
Azure Blob Storage
Query CSV, Parquet, and JSON files stored in Azure Blob containers
- Category
- Files & storage
- Runs in
- The Phloem cloud — nothing to install
Turns CSV, JSON and Parquet files in your Azure storage containers into tables you and Hamming can query, listed in Trace.
What Phloem reads
- CSV, JSON and Parquet files, including gzipped
.csv.gzand.json.gz, in the containers you pick - Files are grouped into tables by path and format, with partition folders detected and column names and types sampled
- The tables you keep, with their columns, listed in Trace for Hamming and SQL autocomplete
Files are read in place. Phloem never writes to your storage account.
What you need
- A storage account, and one of: its connection string, its account name and key, or a SAS token
- A credential that can list the account's containers and read their blobs
Connect
- In the console, open Connectors, choose Add connector, then Azure Blob Storage.
- On Azure Credentials, choose an Authentication Method:
- Connection String — paste it into Connection String (Azure Portal › Storage Account › Access Keys).
- Account Name & Key — enter Storage Account Name and Account Key.
- SAS Token — enter Storage Account Name and SAS Token, including the leading
?.
- On Testing Credentials, Phloem lists the account's containers.
- On Select Containers, pick the containers to connect.
- On Discovered Tables, review the proposed tables. Each is a File Pattern (e.g.
raw/events/**/*.parquet) matching files with the same layout. Edit a table's Table Name, File Pattern or Description, or turn it off. - On Save Connection, name the connector and save it.
Notes
- Discovery samples each folder instead of listing every file. A table it is unsure of starts turned off, so check the patterns before saving.
- Files and folders starting with
_or., andtmp,tempandlogfolders, are skipped.
#Connection fields
What the Add connector form asks for.
| Field | Type | Required |
|---|---|---|
| Authentication Method Choose how to authenticate with Azure | One of Connection String · Account Name & Key · SAS Token | Yes |
| Connection String Full connection string from Azure Portal > Storage Account > Access Keys | Secret | — |
Storage Account Namemystorageaccount | Text | — |
| Account Key | Secret | — |
Storage Account Namemystorageaccount | Text | — |
| SAS Token Shared Access Signature token (include the leading '?') | Secret | — |
| Path Prefix Filter (Optional) Only scan files under this prefix path data/exports/ | Text | — |
| File Formats Select file formats to include. You can add custom extensions. | Any of Parquet (.parquet) · CSV (.csv) · JSON (.json) · TSV (.tsv) · Avro (.avro) · ORC (.orc) · Excel (.xlsx) · Excel Legacy (.xls) | — |
| Scan Subfolders Recursively scan subfolders for files | On / off | — |
Updated