Integration
Google Cloud Storage
Query CSV, Parquet, and JSON files stored in GCS buckets
- Category
- Files & storage
- Runs in
- The Phloem cloud — nothing to install
Lets you and Hamming query CSV, Parquet and JSON files in your Cloud Storage buckets with SQL.
What Phloem reads
- CSV, Parquet (
.parquet,.pqt) and JSON files in the buckets you pick - Each table's columns, listed in Trace
Files are read in place when queried. Nothing is copied into Phloem first.
What you need
A service account JSON key whose service account can:
- list the buckets in its project (
storage.buckets.list) - list and read objects in the buckets you connect (
storage.objects.list,storage.objects.get)
Phloem lists buckets in the project named in the key.
Connect
- In the console, open Connectors, choose Add connector, then Google Cloud Storage.
- On GCP Credentials, paste your key file into Service Account JSON Key.
- Phloem tests the key by listing the project's buckets.
- On Select Buckets, pick the buckets to connect.
- On Discovered Tables, review the tables found. Untick any you don't want, or edit a table's Table Name, File Pattern (
raw/events/**/*.parquet) or Description (Optional). - On Save Connection, name the connection and save it.
What Hamming can do
- Run SQL over the connected tables: query, sample rows, count rows, list columns
Nothing it does changes your buckets.
Notes
- Discovery proposes one table per file. Set a File Pattern glob (
raw/events/**/*.parquet) to query many files as one table.
#Connection fields
What the Add connector form asks for.
| Field | Type | Required |
|---|---|---|
| Service Account JSON Key Paste the contents of your service account JSON key file {"type": "service_account", "project_id": "...", ...} | Text | Yes |
Updated