PhloemDocs
Markdown
Integration

Google Cloud Storage

Query CSV, Parquet, and JSON files stored in GCS buckets

Category
Files & storage
Runs in
The Phloem cloud — nothing to install

Lets you and Hamming query CSV, Parquet and JSON files in your Cloud Storage buckets with SQL.

What Phloem reads

  • CSV, Parquet (.parquet, .pqt) and JSON files in the buckets you pick
  • Each table's columns, listed in Trace

Files are read in place when queried. Nothing is copied into Phloem first.

What you need

A service account JSON key whose service account can:

  • list the buckets in its project (storage.buckets.list)
  • list and read objects in the buckets you connect (storage.objects.list, storage.objects.get)

Phloem lists buckets in the project named in the key.

Connect

  1. In the console, open Connectors, choose Add connector, then Google Cloud Storage.
  2. On GCP Credentials, paste your key file into Service Account JSON Key.
  3. Phloem tests the key by listing the project's buckets.
  4. On Select Buckets, pick the buckets to connect.
  5. On Discovered Tables, review the tables found. Untick any you don't want, or edit a table's Table Name, File Pattern (raw/events/**/*.parquet) or Description (Optional).
  6. On Save Connection, name the connection and save it.

What Hamming can do

  • Run SQL over the connected tables: query, sample rows, count rows, list columns

Nothing it does changes your buckets.

Notes

  • Discovery proposes one table per file. Set a File Pattern glob (raw/events/**/*.parquet) to query many files as one table.

#Connection fields

What the Add connector form asks for.

FieldTypeRequired
Service Account JSON Key
Paste the contents of your service account JSON key file
{"type": "service_account", "project_id": "...", ...}
TextYes

Updated