Reltio Connect

 View Only
Back to discussions
Expand all | Collapse all

Databricks connector : same S3 bucket can be used for Stagging and target bucket

  • 1.  Databricks connector : same S3 bucket can be used for Stagging and target bucket

    Reltio Partner
    Posted 20 days ago

    need help on data bricks connector using AWS:

    Regarding data bricks connector configuration on AWS, do we need to have 2 separate AWS S3 bucket entirely, or we can use same S3 bucket credential having different location/folder structure? I am talking about stagging and target bucket while configure.



    ------------------------------
    Pradipta Chakraborty
    Cognizant Technology Solution (P) India Limited
    ------------------------------


  • 2.  RE: Databricks connector : same S3 bucket can be used for Stagging and target bucket

    Reltio Employee
    Posted 14 days ago

    There are two ways to write target tables.
    1. Unity Catalog (Recommended): Using unity catalog only bucket will be required(staging) and delta tables will be written to Unity catalog. This way no need to configure two buckets, only 1 bucket will be sufficient to stage the data. Also using unity catalog is better for Data Governance as well.

    2. Hive Metastore: Using Hive Metastore it is recommended to have two separate tables for Staging and table.



    ------------------------------
    Diptesh Savla
    ------------------------------

    Original Message


Related Content