Configure PostgreSQL as a Source

In this Article:

• Overview
• Adding PostgreSQL Source in DvSum
  • Prerequisite: Enabling General Query Log for PostgreSQL
• Step-by-Step Configuration
  • Step 1: Add a New Postgres DataSource
  • Step 2: Environment Assignment
  • Step 3: Configure Connection Settings
  • Step 4: Select Database and Schema(s)
  • Step 5: Save the Connection
  • Step 6: Run a Data Scan
• Reviewing Scan Results
• Video Tutorial

Overview:

This article outlines the process of configuring PostgreSQL as a data source in DvSum, enabling integration for data cataloging and profiling. The steps provided apply to both DvSum Data Insights (DI) and DvSum Data Quality (DQ), with only minor variations based on the specific platform.

Adding PostgreSQL source in DvSum:

Prerequisite: Enabling General Query Log for PostgreSQL

Before configuring MySQL as a source, ensure that the general query log is enabled for your PostgreSQL server. This is essential for tracking data lineage and obtaining insights into usage patterns. For more information, refer to the Enabling General Query Log for Data Sources article.

You can follow the steps mentioned below to configure and authenticate a PostgreSQL source:

Step-by-Step Configuration

Step 1: Add a New Postgres DataSource

1.  
   1. Navigate to Data Sources.
   2. Click on Add Source.
   3. Select PostgreSQL as the data source.
   4. Enter a source name and click Save.

Step 2: Environment Assignment

If Environments are enabled in the application, users can assign the Data Source to an Environment (for example, Development, UAT, or Production) and associate it with a Logical Source.

Environment assignment helps organize Data Sources across different deployment stages, while Logical Sources represent the same business dataset across multiple environments. Together, they support environment-aware asset promotion and management.

To assign an Environment and Logical Source:

1. Open the Data Source details page --> Settings --> General.
2. Select the appropriate Environment.
3. Select or create a Logical Source, if applicable.
4. Save the changes.

For detailed information about configuring and managing Environments and Logical Sources, refer to the Environment Management article.

Step 3: Configure Connection Settings

1. After saving, you will be redirected to the connection settings page.
2. Enable the On-premise Web Service checkbox if applicable.
3. Select the SAWS (Smart Adaptive Web Service) that is set up and running.
4. Enter the following details:
   • Host
   • Port
   • Database Login
   • Password

Note: By Default the SAWS type will be cloud. For more information regarding Cloud SAWS, click here

Step 4: Select Database and Schema(s)

1. Once authentication is successful, the Database section appears.
2. Select the database you wish to scan.
3. If scanning all schemas, leave the Schema checkbox unchecked.
4. To scan specific schemas:
   • Check the Schema checkbox.
   • Select the required schemas from the Available Schemas list.
   • Move them to the Selected Schemas tab.

Once the database is selected, we now have an option of limiting the scan to some specific schema(s) or we have to scan all of them. For the PostgreSQL source, we would have a Schema field as well which will contain a checkbox, If we want to scan all the schemas then we shouldn't check this checkbox and proceed with saving and scanning the source. But if we want to limit our scan to some specific schemas then check this checkbox.

Once it is checked then the list of available schemas will be displayed. Users can select single or multiple schemas from the Available Schemas list and move them to the Selected Schemas tab on the right. 

Step 5: Save the Connection

1. Scroll to the top of the page.
2. Click Done.
3. Click Save.
4. The PostgreSQL source is now configured successfully.

After that click the “Save” button. The source will get saved successfully.

Note: When adding or editing a Data Source, if incorrect details or invalid credentials are entered, it will still allows to save the Data Source. However, authentication will fail, and the Data Source will be marked as unusable with a red icon. The Data Source will not be usable until the correct credentials are provided.

The connection is saved, but it cannot be used until valid authentication details are updated.

Step 6: Run a Data Scan

1. Navigate to the Scan History page.
2. Click Scan Now.
3. The scan will run, and a job will be created.
4. Once completed, the status will change to Completed.
5. Click on the Scan Name to view the Scan Summary.

• Once the status of the job gets Completed, our new PostgreSQL source's scan will be completed successfully. 

After the scan completion, click on Scan Name and it will open the Scan Summary page of this scan.

Reviewing Scan Results

1. The Scan Summary page displays insights such as the number of tables and columns fetched.
2. Navigate to Data Dictionary to view the scanned tables.\
3. Click on the Recently Refreshed tab to see newly added tables.
4. Click on individual table names for more details.
5.  

Video Tutorial:

Watch this quick video tutorial on how to add and configure a PostgreSQL source into DvSum app.