Atlas Example

This comprehensive example demonstrates a real-world enterprise application of Atlas. We'll walk through how a payroll team can leverage Atlas and the Pontus platform to automate, standardize, and audit their payroll process.

Creating and Managing Entities and Relationships

The payroll team maintains their payroll history in a Google Sheet with the following structure:

Year	Month	Employee Name	Team	Currency	Income
2025	March	Jane Smith	Engineering	USD	7,500
2025	March	Alex Johnson	Marketing	USD	6,200
2025	February	Jane Smith	Engineering	USD	7,500
...	...	...	...	...	...

To register this data as an entity in Atlas, a team member makes a POST request to the Atlas API endpoint: POST /atlas/entity

{
    "name": "Payroll History",
    "description": "A table that contains the payroll history for the company",
    "columns": [
        {
            "name": "Year",
            "type": "number"
        },
        {
            "name": "Month",
            "type": "string"
        },
        {
            "name": "Employee Name",
            "type": "string"
        },
        {
            "name": "Team",
            "type": "string"
        },
        {
            "name": "Currency",
            "type": "string"
        },
        {
            "name": "Income",
            "type": "number"
        }
    ],
    "primary_key": ["Year", "Month", "Employee Name", "Team"],
    "google_sheets_config": {
        "spreadsheet_id": "1234567890",
        "sheet_name": "Payroll History",
        "origin": {
            "row": 0,
            "col": 0
        }
    }
}

When a new monthly payroll dataset is created for April 2025, with the following structure:

Year	Month	Employee Name	Team	Currency	Income
2025	April	Jane Smith	Engineering	EUR	6,600
2025	April	Alex Johnson	Marketing	USD	6,200
2025	April	Mary Joe	Marketing	USD	5,500
...	...	...	...	...	...

The payroll team can register this new dataset as a separate entity by sending a POST request to the Atlas API endpoint: /atlas/entities/google_sheet/register

{
    "name": "Monthly Payroll",
    "description": "A table that contains the monthly payroll data for the company",
    "columns": [
        {
            "name": "Year",
            "type": "number"
        },
        {
            "name": "Month",
            "type": "string"
        },
        {
            "name": "Employee Name",
            "type": "string"
        },
        {
            "name": "Team",
            "type": "string"
        },
        {
            "name": "Currency",
            "type": "string"
        },
        {
            "name": "Income",
            "type": "number"
        }
    ],
    "primary_key": ["Year", "Month", "Employee Name", "Team"],
    "google_sheets_config": {
        "spreadsheet_id": "2345678901",
        "sheet_name": "Monthly Payroll",
        "origin": {
            "row": 0,
            "col": 0
        }
    }
}

Once registered, Atlas will automatically update its internal representation of the "Monthly Payroll" entity with this configuration.

To establish a relationship between the monthly payroll data and the historical payroll records, the team can create a relationship definition using a POST request to the Atlas API endpoint: POST /atlas/entity/relationship

{
    "name": "CONTAINS",
    "relationship_type": "foreign_key",
    "source_entity": "Monthly Payroll",
    "target_entity": "Payroll History",
    "foreign_key_config": {
        "source_columns": ["Year", "Month", "Employee Name", "Team"],
        "target_columns": ["Year", "Month", "Employee Name", "Team"],
        "enforce_foreign_key": false
    }
}

Data Processing

The payroll team now needs to update the payroll history with the new monthly data. However, before appending the data, they must standardize the currency to USD.

To accomplish this, the team can create a PySpark transformation script:

from pyspark.sql import SparkSession
from pyspark.sql.functions import col, when, lit, current_timestamp
import os
import pontus
 
# Initialize Spark Session
spark = SparkSession.builder \
    .appName("Payroll Processing") \
    .getOrCreate()
 
# Define currency conversion rates to USD (example rates)
conversion_rates = {
    "USD": 1.0,
    "EUR": 1.1,  # 1 EUR = 1.1 USD
    "GBP": 1.3,  # 1 GBP = 1.3 USD
    "CAD": 0.75, # 1 CAD = 0.75 USD
    # Add more currencies as needed
}
 
# Load the Monthly Payroll entity data
monthly_df = pontus.atlas.entities.get("Monthly Payroll")
 
# Convert income to USD based on currency
for currency, rate in conversion_rates.items():
    monthly_df = monthly_df.withColumn(
        "Income", 
        when(col("Currency") == currency, col("Income") * lit(rate))
        .otherwise(col("Income"))
    )
 
# Update all records to indicate the currency is now USD
monthly_df = monthly_df.withColumn("Currency", lit("USD"))
 
# Write the transformed data to the Payroll History entity
pontus.atlas.entities.set("Payroll History", monthly_df, mode="overwrite")
 
spark.stop()

With the transformation script ready, the payroll team can now create a workflow to automate the currency standardization and data integration process using Atlas actions:

{
    "name": "Payroll ETL Process",
    "steps": [
        {
            "name": "convert_currency",
            "type": "atlas",
            "atlas_data": {
                "action": "transform",
                "input_data": {
                    "language": "pyspark",
                    "code_file": "payroll_currency_conversion.py"
                },
                "input_entities": ["monthly_payroll"],
                "output_entities": [    
                    {
                        "name": "payroll_history",
                        "write_mode": "append"
                    }
                ]
            },
        }
    ],
    "wires": []
}

When this workflow executes, it automatically standardizes the currency to USD and integrates the new payroll data with the historical records. The result is a comprehensive payroll history table:

Year	Month	Employee Name	Team	Currency	Income
2025	March	Jane Smith	Engineering	USD	7,500
2025	March	Alex Johnson	Marketing	USD	6,200
2025	February	Jane Smith	Engineering	USD	7,500
2025	April	Jane Smith	Engineering	USD	7,260
2025	April	Alex Johnson	Marketing	USD	6,200
2025	April	Mary Joe	Marketing	USD	5,500
...	...	...	...	...	...

Reporting

With the standardized payroll data now integrated into the Payroll History entity, the payroll team can generate detailed reports for analysis and compliance purposes.

To generate a filtered report by year, the team can create a workflow that combines Atlas's data operations with Pontus's L2 actions:

{
  "name": "Payroll Report Generator",
  "workflow_input": {
    "name": "input",
    "input_schema": {
      "type": "object",
      "properties": {
        "entity_name": {
          "type": "string"
        },
        "year_filter": {
          "type": "number"
        },
        "output_file_name": {
          "type": "string"
        },
        "google_drive_folder_id": {
          "type": "string"
        }
      },
      "required": [
        "entity_name",
        "output_file_name",
        "google_drive_folder_id"
      ]
    },
    "default_data": {
      "entity_name": "Payroll History",
      "output_file_name": "Payroll History Report",
      "google_drive_folder_id": "3456789012"
    }
  },
  "workflow_output": {
    "name": "output",
    "output_data": {
      "file_url": "{{upload_report.file_url}}"
    }
  },
  "steps": [
    {
      "name": "get_entity_data",
      "type": "atlas",
      "atlas_data": {
        "action": "get",
        "input_data": {
          "operation": {
            "filters": [
              {
                "column": "Year",
                "operator": "EQUALS",
                "value_type": "NUMBER",
                "value": "{{input.year_filter}}"
              }
            ]
          }
        },
        "input_entities": ["{{input.entity_name}}"]
      }
    },
    {
      "name": "create_report_pdf",
      "type": "l2",
      "l2_data": {
        "provider": "pontus",
        "module": "converters",
        "action": "create_pdf",
        "input_data": {
          "template": "<!DOCTYPE html><html><head><style>body{font-family:Arial,sans-serif;max-width:800px;margin:20px auto;padding:20px}.header{text-align:center;margin-bottom:30px}table{width:100%;border-collapse:collapse;margin:20px 0}th,td{border:1px solid #ddd;padding:8px;text-align:left}th{background-color:#f8f8f8}.footer{margin-top:30px;text-align:center;font-size:0.8em;color:#666}</style></head><body><div class=\"header\"><h1>Data Report</h1><p>Filtered for Year: [[.year_filter]]</p></div>[[.table]]<div class=\"footer\">Report generated via Atlas data extraction workflow</div></body></html>",
          "template_type": "html",
          "variables": [
            {
              "name": "year_filter",
              "value_type": "NUMBER",
              "value": "{{input.year_filter}}"
            },
            {
              "name": "table",
              "value_type": "TABLE",
              "value": "{{get_entity_data.table}}"
            }
          ]
        }
      }
    },
    {
      "name": "upload_report",
      "type": "l2",
      "l2_data": {
        "provider": "google",
        "module": "gdrive",
        "action": "upload_file",
        "input_data": {
          "file_data": "{{create_report_pdf.pdf_data}}",
          "file_name": "{{input.output_file_name}}.pdf",
          "file_type": "pdf",
          "folder_id": "{{input.google_drive_folder_id}}"
        }
      }
    }
  ],
  "wires": [
    {
      "source_name": "get_entity_data",
      "target_name": "create_report_pdf"
    },
    {
      "source_name": "create_report_pdf",
      "target_name": "upload_report"
    }
  ]
}

By combining Atlas's data operations with Pontus's L2 actions, the payroll team can generate filtered reports, convert them to professionally formatted PDFs, and automatically distribute them to stakeholders via Google Drive.

User Interface Integration

To create a seamless end-user experience, the payroll team can develop an application that:

Initiates the Payroll ETL Process workflow
Displays the consolidated payroll history in an interactive table
Executes the Payroll Report Generator workflow to create filtered reports
Provides direct access to generated reports

The application configuration can be defined as follows:

{
    "screens": [
        {
            "components": [
                [
                    {
                        "name": "PayrollUploadHeading",
                        "type": "text",
                        "formatting": {},
                        "text": {
                            "contents": "Payroll ETL Process",
                            "type": "heading"
                        }
                    },
                    {
                        "name": "PayrollUploadForm",
                        "type": "form",
                        "formatting": {},
                        "form": {
                            "form_items": [],
                            "job": {
                                "type": "sync",
                                "workflow_name": "Payroll ETL Process"
                            },
                            "do": {
                                "type": "redirect",
                                "redirect_data": {
                                    "url": "/screens/payroll_history"
                                }
                            }
                        }
                    }
                ]
            ],
            "url": "start_payroll_upload"
        },
        {
            "components": [
                [
                    {
                        "name": "PayrollHistoryHeading",
                        "type": "text",
                        "formatting": {},
                        "text": {
                            "contents": "Payroll History",
                            "type": "heading"
                        }
                    },
                    {
                        "name": "PayrollDataGrid",
                        "type": "datagrid",
                        "formatting": {},
                        "datagrid": {
                            "entity_name": "Payroll History"
                        }
                    },
                    {
                        "name": "ExportReportForm",
                        "type": "form",
                        "formatting": {},
                        "form": {
                            "form_items": [
                                {
                                    "label": "Year Filter",
                                    "type": "number",
                                    "required": true,
                                    "number_config": {}
                                }
                            ],
                            "job": {
                                "type": "sync",
                                "workflow_name": "Payroll Report Generator"
                                },
                            "do": {
                                "type": "redirect",
                                "redirect_data": {
                                    "url": "/screens/report?file_url={{output.file_url}}"
                                }
                            }
                        }
                    }
                ]
            ],
            "url": "payroll_history"
        },
        {
            "components": [
                [
                    {
                        "name": "ReportHeading",
                        "type": "text",
                        "formatting": {},
                        "text": {
                            "contents": "Your Payroll Report is Ready",
                            "type": "heading"
                        }
                    },
                    {
                        "name": "ViewReportButton",
                        "type": "button",
                        "formatting": {},
                        "button": {
                            "label": "View Report",
                            "event": {
                                "type": "redirect",
                                "redirect_data": {
                                    "url": "{{query_params.file_url}}"
                                }
                            }
                        }
                    }
                ]
            ],
            "url": "report"
        }
    ],
    "start_screen_url": "start_payroll_upload"
}

Transformations Architecture