QBoard » Big Data » Big Data on Cloud » Big Query table to be extracted as JSON in Local machine

Big Query table to be extracted as JSON in Local machine

  • I have an idea on how to extract Table data to Cloud storage using Bq extract command but I would like rather like to know, if there are any options to extract a Big Query table as NewLine Delimited JSON to Local Machine?

    I could extract Table data to GCS via CLI and also download JSON data from WEB UI but I am looking for solution using BQ CLI to download table data as JSON in Local machine?. I am wondering is that even possible?

      December 27, 2021 1:58 PM IST
    0
  • You need to use Google Cloud Storage for your export job. Exporting data from BigQuery is explained here, check also the variants for different path syntaxes.

    Then you can download the files from GCS to your local storage.

    Gsutil tool can help you further to download the file from GCS to local machine.

    You first need to export to GCS, then to transfer to local machine.

    If you use the BQ Cli tool, then you can set output format to JSON, and you can redirect to a file. This way you can achieve some export locally, but it has certain other limits.

    this exports the first 1000 line as JSON

    bq --format=prettyjson query --n=1000 "SELECT * from publicdata:samples.shakespeare" > export.json
    
      January 8, 2022 2:43 PM IST
    0