Glossary🔗
A🔗
Access: What you are allowed to do/view and what resources you have access to in Matillion ETL. This is managed on an individual basis, through Roles and Permissions.
Access Control List (ACL): A list in the Manage Project dialog showing all project users and their current access levels, including admin rights and project access toggles.
Access token: A secure key issued by an authorization server that Matillion ETL uses to communicate with an API endpoint during OAuth flows. Access tokens typically have short expiry durations.
Active Directory: A server which authenticates and authorizes users and resources on a domain network, storing information about each user and device, and validates credentials and access rights. It manages groups and users, and policy administration. Active Directory is underpinned by LDAP (uses its protocol).
Amazon Redshift: Amazon's fully managed cloud data warehouse. Supported as a target data warehouse in Matillion ETL, including Redshift Spectrum and multi-node cluster configurations.
Amazon Web Services (AWS): A collection of services that relate to cloud computing, including computing, storage, and development.
AMI (Amazon Machine Image): A pre-configured machine image used to launch a Matillion ETL instance on AWS EC2. It comes with a software repository for updates.
And component: A flow component that waits for all of its parallel input branches to complete before continuing the job.
API: An Application Programming Interface allows two applications to communicate with each other, such as a mobile device pulling weather information from a server, or Uber using a map service.
API Extract component: An orchestration component that uses a configured API Extract Profile to pull data from a JSON or XML-based API and load it into a target data warehouse.
API Extract Profile: A collection of endpoint configurations that define how to extract and flatten data from a JSON or XML-based API. Configured via Project → Manage API Profiles → Manage Extract Profiles.
API Profile: A specification of metadata for accessing an API endpoint, which will invoke an external API. In Matillion ETL, an API Profile can be either an Extract Profile or a Query Profile.
API Query component: An orchestration component that uses a configured API Query Profile (RSD file) to query a JSON or XML API and load the resulting data into a table.
API Query Profile: A configuration containing one or more RSD files that define how to connect to and query a specific API endpoint, including authentication, pagination, and data structure handling.
Application protocol: A sets of rules that determine how processes on clients and servers communicate with each other, and how they run on different systems.
Audit log: An Enterprise-only feature, that provides a log of your activity within a Matillion ETL instance.
Authentication: The process of verifying a user's identity before granting access to a Matillion ETL instance. Matillion ETL supports local username and password authentication, as well as external methods such as LDAP and OpenID.
Authorization: The process of controlling what an authenticated user can do within a Matillion ETL instance, such as which projects, features, or cloud platform resources they can access. In Matillion ETL, this is managed through permissions and roles assigned via the Manage Project and User Configuration dialogs.
Automatic variables: A special subset of environment variables in Matillion ETL that are populated automatically at runtime with system-level information such as job name, project name, component name, and run timestamps.
Azure Blob Storage: Microsoft Azure's cloud storage service for unstructured data such as files, images, and logs. Matillion ETL can load data from and unload data to Azure Blob Storage using the Azure Load and Azure Unload components.
Azure Portal: Microsoft's web-based console for managing Azure resources. Setting up Matillion ETL on Azure sometimes requires the Azure Portal, for example to register an app for OpenID authentication, manage Key Vault secrets, or resize a Matillion ETL instance.
Azure Synapse: Microsoft's cloud data warehouse and analytics platform. Supported as a target data warehouse in Matillion ETL.
B🔗
Backups: Additional copies of your data/workflow in Matillion ETL.
Basic mode: Builds a query for you in Matillion ETL, without the need to write code.
Bearer token: A token-based HTTP authentication scheme where a security token is passed in the Authorization header. Supported as an API authentication method in Matillion ETL API profiles.
Bucket: In cloud storage, buckets are containers that hold your data.
BYOL (Bring Your Own License): A licensing model for Matillion ETL where customers provide their own license rather than subscribing through a cloud marketplace.
C🔗
Calculator component: A transformation component that adds new columns to the data flow by performing calculations defined as SQL expressions per row.
CDC (Change Data Capture): A Matillion ETL feature (end-of-life as of December 2024) that enabled real-time replication of database changes by detecting inserts, updates, and deletes in source systems using AWS DMS and related infrastructure.
CDW (Cloud Data Warehouse): A cloud-hosted database platform—such as Snowflake, Amazon Redshift, Google BigQuery, Azure Synapse Analytics, or Delta Lake on Databricks—that Matillion ETL uses as the target for data loading and transformation.
Cloud data lake: A cloud-hosted repository where you can store structured and unstructured data at any scale, as well as images, music, and videos.
Cloud data warehouse: Stores data on machines around the world, up to petabytes of data. Unlike on-premise warehouses, which need to be managed and upgraded, with a cloud data warehouse you pay for what you use.
Cloud storage: Object storage hosted on the internet rather than on local disks, such as Amazon S3, Azure Blob Storage, or Google Cloud Storage. Matillion ETL uses cloud storage as a staging area when loading data into a cloud data warehouse.
CloudFormation template: An AWS infrastructure-as-code template used as one method for launching a Matillion ETL instance on AWS.
CloudWatch: An AWS monitoring and observability service. Matillion ETL can publish metrics to CloudWatch using the CloudWatch Publish component.
Clusters: Using two or more storage servers in tandem to enhance performance, availability, and capacity.
Collibra: A third-party data intelligence platform that integrates with Matillion ETL to provide data lineage by running a lineage harvester against Matillion ETL job metadata.
Commit: A snapshot of your repository, in Git, for example. Each time you make a change and commit, it creates a new snapshot.
Component: The fundamental building block of a Matillion ETL job. Components are placed on the job canvas and connected to form a data pipeline. Each component performs a discrete operation such as loading, filtering, joining, or outputting data.
Component exports: Output values from a component that can be captured into variables for use in subsequent components within the same job.
Concurrent connections: A Matillion ETL environment setting (Enterprise edition only) that controls the maximum number of parallel SQL statements the instance executes against the data warehouse simultaneously.
Connection: A link to a node. Matillion ETL allows multiple connections to the same node.
Connector: A connector is a data source that is used to connect to Matillion ETL.
Console: A developer tool, where you can enter commands and see the output, and configure settings.
Copied variable: A variable in Matillion ETL whose value is scoped to a single branch of a job. Changes to a copied variable in one branch do not affect parallel branches. Formerly known as a "local" variable.
Credit: The unit of consumption used by Maia to meter Matillion ETL instance usage. Credit consumption is based on the number of vCPUs per active hour.
CRUD: The four basic operations that are used in database applications: Create, Read, Update, and Delete.
CSRF: Cross-Site Request Forgery is an attack that makes the end user perform unwanted actions on a web application. Matillion ETL instances are secured by CSRF tokens that refresh automatically.
D🔗
Data: Information, facts, images, sound, and video, which is stored on a computer and can be manipulated and analyzed.
Data lineage: The progress of data, from creation to transformations over time.
Data model: The structure and model of a database: how data is connected together, processed, and stored in a system.
Maia (DPC): Matillion's cloud platform used to manage Matillion ETL instances, editions, usage, observability, and upgrades.
Data source: Where data is loaded from to be staged and transformed in Matillion ETL, for example Facebook Query Component.
Data structure variable: A variable type in Matillion ETL that can hold a structured object, allowing dynamic population of parameters that require complex data structures.
Data Vault: A data modeling methodology used to design relational tables in an enterprise data warehouse. It organizes data into three table types—Hub, Link, and Satellite—to support flexibility and scalability.
Database drivers: A type of software that lets you 'talk' to your database. It links an application to a database management system.
Databricks: A cloud analytics platform. Matillion ETL supports Delta Lake on Databricks as a target data warehouse and supports running transformation jobs on Databricks job compute clusters.
Dataset: A collection of data.
Debug: Testing and removing errors, often using a debugger tool.
Delta Lake: An open-source storage layer that runs on Databricks, used as a target data warehouse in Matillion ETL.
Delta Live Streaming component: An orchestration component that runs a continuous Delta Live Tables pipeline within Matillion ETL asynchronously on Databricks.
Delta Live Tables: A Databricks framework for building reliable, maintainable streaming and batch data pipelines. Used with the Delta Live Streaming component.
Delta load: A data load type that transfers only changed (appended, deleted, or updated) rows from the source since the last extraction, as opposed to a full load of all rows.
Destructive: Working with a component could potentially destroy or tamper with existing data.
DMS (AWS Database Migration Service): An AWS service used by Matillion ETL's CDC feature to create replication instances, endpoints, and tasks that stream database changes from source systems.
Docker: A platform that simplifies building, running, managing and deploying applications in an isolated environment, away from your infrastructure.
Document database: Uses JSON-like documents to model data instead of rows and columns, designed to store and manage document-oriented information, referred to as semi-structured data.
E🔗
EC2 (Elastic Compute Cloud): The AWS virtual machine service on which Matillion ETL instances are hosted when deployed on AWS.
ELT (Extract, Load, Transform): The data integration pattern used by Matillion ETL, where data is extracted from source systems, loaded into a cloud data warehouse, and then transformed using the warehouse's compute resources.
End Failure component: A flow component that marks a job as failed when its execution path is reached.
End Success component: A flow component that marks a job as successful when its execution path is reached.
Endpoint: One end of an API path that allows you to perform a certain function, for example /export typically exports data from the path you are on.
Enterprise edition: The highest tier of Matillion ETL licensing, required for features such as CDC, Collibra integration, and concurrent connections configuration.
Enterprise mode: An upgrade and extension of Matillion ETL's functionality. Additional features include: Git integration, generated documentation, audit log, permissions, data lineage, and concurrent connections.
Environment: A named configuration in a Matillion ETL project that defines connection settings to a specific data warehouse, including credentials, default warehouse, default schema/database, and environment variables. A project can have multiple environments (for example, dev, staging, and production).
Environment variable: A variable defined at the environment level in Matillion ETL. Its value can differ between environments, making it useful for environment-specific configuration such as table names or data paths.
EOL (End of Life): The status of a Matillion ETL feature or version that is no longer supported or actively maintained. After EOL, the feature may be removed in a future release.
Except component: A transformation component that compares two input flows and outputs only the rows from the primary input that do not exist in the comparison input.
Export: To save a copy of the current document, instance, log, image, user details, etc. in another file format for future use. In Matillion ETL, jobs, environments, and variables can be exported from one Matillion ETL instance to another.
Expression editor: An in-product editor used to write SQL expressions for components such as Calculator and Join, with field-selection assistance to reduce manual typing.
External table: A database table whose data resides in cloud storage (S3, GCS, or Azure Blob) and is referenced rather than physically stored in the data warehouse.
Extract Nested Data component: A Snowflake-specific orchestration component used to flatten structured or semi-structured data (such as JSON) returned by other components.
F🔗
FIFO: In messaging, stands for first-in, first-out. The order of messages that are sent and received is preserved, and are delivered exactly one time.
File Iterator: A flow component that iterates over files in cloud storage, running subsequent components once for each file found.
Filter component: A transformation component that removes rows from the data flow that do not meet defined conditions, generating a SQL WHERE clause.
Fixed Flow component: A transformation component that injects a fixed, user-defined set of rows into the data flow.
Fixed Iterator: A flow component that runs subsequent components a fixed, pre-defined number of times.
Flow component: A category of Matillion ETL components that control the processing order and branching of a job, including iterators, conditionals (If), and parallel join components (And, Or).
FTP: File Transfer Protocol defines how files are sent to each other. FTP also acts like a program, sharing files between remote computers.
Full load: A data load type that retrieves the complete contents of a source table or data source on each run.
G🔗
GA (General Availability): The release status indicating that a Matillion ETL feature is fully released and supported for all customers. No special label is required in documentation.
GCP (Google Cloud Platform): Google's cloud computing platform. Matillion ETL instances can be deployed on GCP and use Google BigQuery or Snowflake on GCP as the target data warehouse.
Generate Sequence component: A transformation component that creates a column of sequential numbers with a defined start value, increment, and end value.
Generated documentation: An Enterprise feature that provides a downloadable report that details the job layout, components, SQL, and properties.
Git: A version control system that lets you track your file changes, and allows you to revert to previous versions. Multiple people can collaborate on one source.
Google BigQuery: Google's fully managed, serverless cloud data warehouse. Supported as a target data warehouse in Matillion ETL.
Graph database: A type of NoSQL database that shows the connections between data, and treats the relationships as being as important as the data itself.
Grid Iterator: A flow component that iterates over rows in a grid variable, running subsequent components once per row.
Grid variable: A variable type in Matillion ETL that holds a two-dimensional table of values (rows and columns), enabling dynamic iteration or column mapping within components.
Group: A set of Matillion ETL user accounts that share the same role-based permissions. Used for administering access to projects and features.
H🔗
HA (High Availability) cluster: A deployment configuration where multiple Matillion ETL instances (nodes) operate together behind a load balancer to provide redundancy and increased throughput. Jobs are shared across nodes using the Quartz scheduler.
Historical jobs: An Enterprise feature that allows you to view a job that was set up at the time of running.
Hub table: A Data Vault table that represents the central theme of a business relationship, identified by a natural business key and surrogate key.
I🔗
IaaS: A computing infrastructure, managed through the internet by a cloud computing service provider, such as Amazon Web Services (AWS). Cloud servers are provided to clients through an API or dashboard interface, so they have full control over the infrastructure. IaaS is similar to a traditional data center, but it is "outsourced" through a virtual data center in the cloud so the consumer does not need to physically maintain it their own data center.
IAM (Identity and Access Management): The AWS service used to manage user and role permissions. Matillion ETL requires specific IAM roles and policies to access AWS services such as S3, KMS, RDS, SQS, SNS, and DynamoDB.
If component: A flow component that evaluates an expression and directs the job to one of two branches depending on whether the expression evaluates to true or false.
Import: To bring a file from outside an application, to be read and used by that application.
In-place update: The process of upgrading an existing Matillion ETL instance to a newer version, either via the in-client update dialog or via SSH using yum commands.
Indexes: Used to find data quickly without having to search for each row. It improves database performance by reducing the number of disk accesses needed during a query. Indexes are created using a few columns: a search key that contains the Primary Keys, and the data reference column which points to where a particular key is found. Developers can also create indexes on functions or expressions.
Instance: A deployed Matillion ETL server, typically hosted on a cloud provider virtual machine (EC2 on AWS, a VM on Azure, or a VM on GCP). Multiple instances can be associated with a single Maia account.
Iterator: A category of flow components (File Iterator, Fixed Iterator, Grid Iterator, Loop Iterator, Table Iterator) that repeat the execution of subsequent components in a loop.
J🔗
JCo (SAP Java Connector): A set of SAP-provided Java libraries required to enable the SAP ODP Extract component to connect to SAP systems.
JDBC (Java Database Connectivity): A Java API for connecting to relational databases. Used by Matillion ETL internally for communicating with data sources and targets. Can be enabled with SSL for secure communication.
Job: A workflow created in Matillion ETL consisting of connected components. There are two types: orchestration jobs and transformation jobs.
Job canvas: The visual workspace in the Matillion ETL UI where components are placed, connected, and configured to form a job.
Job compute configuration: A named configuration in Matillion ETL that specifies Databricks cluster parameters (Spark version, node type, worker count) used to run a transformation job on a Databricks job compute cluster.
Job variable: A variable scoped to a specific job within a Matillion ETL project. It is resolved before environment variables when a name conflict occurs. Job variables are also used as parameters when creating shared jobs.
JSON: A open standard text file format. JSON was created to hold structured data to be used in JavaScript. It's used for different applications, and is the most popular way of sending data for web APIs.
K🔗
Key-value database: Key-value databases save data as a group of key-value pairs made up of two data items each. They're also sometimes referred to as a key-value store.
KMS (Key Management Service): An AWS service for managing encryption keys. Matillion ETL can use KMS to encrypt stored passwords in the password manager.
L🔗
LDAP (Lightweight Directory Access Protocol): A protocol for accessing and managing distributed directory information. Matillion ETL supports LDAP integration for managing user authentication via an external directory, including stateless authentication configurations.
License: BYOL versions of Matillion ETL obtained from online marketplaces have no built-in limits. A license key is required immediately before the software can be used.
Limit: Setting a range to prevent very large API calls from running and causing errors.
Lineage harvester: A Collibra software application that crawls Matillion ETL job metadata and maps technical lineage information into the Collibra data catalog.
Link table: A Data Vault table that represents a relationship between two or more Hub tables, identified by the surrogate keys of those Hubs.
Load generator component: A category of orchestration components that load delimited data from cloud storage (S3, Azure Blob, GCS) into the data warehouse.
Loop Iterator: A flow component that iterates by evaluating a condition before each pass, running subsequent components until the condition is no longer met.
LTS (Long-Term Support): A release designation for Matillion ETL versions that receive high-priority bug fixes for at least 18 months, suited for customers who prefer stability over the latest features.
M🔗
Manage CDC: A deprecated Matillion ETL feature (removed as of December 2024) that provided a UI for configuring Change Data Capture tasks on supported source databases.
Manage Credentials: A dialog in the Matillion ETL project menu for storing and managing credentials for cloud platforms such as AWS, GCP, and Azure.
Manage OAuth: A dialog in the Matillion ETL project menu for storing and managing OAuth entries used to authenticate API connections.
Manage Passwords: A dialog for securely storing and managing passwords in Matillion ETL. Passwords can optionally be encrypted using KMS.
Matillion ETL: Matillion's ELT tool that runs on customer-managed cloud virtual machines and provides orchestration and transformation capabilities for cloud data warehouses.
Metadata: Metadata is data about data. It makes it easy to find and work with certain datasets.
Migration: The process of moving Matillion ETL project content (jobs, variables, environments, shared jobs) from one instance to another, distinct from an in-place update.
Multi-cluster warehouse: A Snowflake configuration that automatically adds additional clusters of compute resources when query load increases, to reduce queuing and handle concurrent Matillion ETL jobs.
N🔗
NoSQL database: NoSQL databases are document structured, rather than tabular like many other databases. NoSQL databases are more scalable and more suited for very large amounts of data compared to relational databases. There is no limitation on the types of data that can be stored together. They are not organized in tables with rows and columns, but are grouped in collections of categories such as users and orders, for example.
Notes: A feature in Matillion ETL to annotate your jobs, using formatted text boxes.
O🔗
Object-oriented database: In an OODBMS, information is represented as objects used in object-oriented programming. An object database is a hybrid of object-based data and relational databases.
ODP (Operational Data Provisioning): An SAP framework for extracting data from SAP systems. The SAP ODP Extract component uses this interface to connect to SAP data sources via ABAP CDS views and SAPI extractors.
OpenID: An authentication standard used in Matillion ETL's stateless authentication configuration to manage users via the in-instance User Configuration dialog without requiring an external directory service.
Or component: A flow component that continues the job when any one of its inputs completes, without waiting for all inputs.
Orchestration job: A type of Matillion ETL job that coordinates the overall data pipeline, running data load components, flow control components, and calls to transformation jobs.
Output component: A category of Matillion ETL components that write data from the cloud data warehouse back to an external target system such as Oracle, SQL Server, Salesforce, or Pardot.
P🔗
PaaS: PaaS provides a framework for developers to create customizable applications. A third-party provider or enterprise manages the servers, storage and networking, while developers maintain the applications. PaaS is similar to SaaS in delivery, but is geared towards creating software.
Pagination: The process of handling multi-page API responses. API profiles in Matillion ETL support multiple pagination strategies including full path, offset-based, cursor-based, page-based, link header, and relative path.
Parameters: An element that defines a type of variable used with data. They let us send information or instructions.
PAT (Personal Access Token): A Snowflake authentication mechanism supported in Matillion ETL as an alternative to username/password authentication.
PATH: The unique URL to connect to the Matillion ETL API, e.g. https://<InstanceAddress>/rest/v1/<permission>.
Permissions: Permissions define what features you enable or disable other users to use in Matillion ETL.
Persistence database: The database used by Matillion ETL to store its internal configuration, job definitions, task history, and user data. By default, this is a local PostgreSQL database on the Matillion ETL instance, but can be an external PostgreSQL database for HA cluster deployments.
Pipeline: Pipelines are configurable objects inside Data Loader that are responsible for loading data from a single source service to a single target data warehouse.
Primary key: In a relational database, this identifies each unique record.
Print Variables component: An orchestration component that outputs the current values of selected variables to the task log for debugging purposes.
Product Improvement Metrics: A system that gathers anonymous data. If enabled, this data will be reported to Matillion who will use it to build better products.
Project: A top-level organizational unit in Matillion ETL that contains jobs, environments, variables, shared jobs, and user access settings. A project is associated with a specific data warehouse type.
Properties: Metadata about your tables.
Pub/Sub: A type of asynchronous communication used in serverless and micro services.
Q🔗
Quartz: The open-source enterprise job scheduler library used internally by Matillion ETL to manage scheduled jobs, message queues, API requests, and job distribution across HA cluster nodes.
Query profile: An RSD file within an API Query Profile that defines how to connect to a specific API endpoint, including authentication, XPath definitions for extracting fields, and pagination settings.
Query tag: A Snowflake object tag applied to SQL queries submitted by Matillion ETL, used to track cost allocation at project, environment, job, or component level.
R🔗
RDS (Relational Database Service): AWS's managed relational database service. Used in Matillion ETL with the RDS Bulk Output component and associated IAM permissions.
Read Only: A role that can be applied to a user, that allows read-only access. This role is currently only available in Matillion ETL instances created via the Billing Platform.
Recovery pointer: A timestamp string used with SAP ODP Extract's recovery load type to re-extract data from a specific point in the delta queue after an interrupted extraction.
Refresh token: A long-lived token issued alongside an OAuth access token, used to obtain new access tokens when the original expires. Stored securely on the Matillion ETL instance.
Relational database: Relational databases essentially store data in tables, or 'relations', divided in rows (records) and columns (fields). SQL (Structured Query Language) is a language used for managing the database, including searching for, inserting, updating, and deleting records.
Repository: A location where data is stored and managed, for example storing metadata in a particular location.
Request: Sends a HTTP command to a server to communicate with the network. This could be to get some data or post some data.
Resources: A collection of accessible entities within a computer system, such as files.
Response: When making a HTTP request, the information sent back from the server is the response.
REST: REpresentational State Transfer that defines the standards of communication between web computer systems.
Retry component: A flow component that re-runs a portion of the job a configurable number of times if an error occurs.
Roles: Different labels given to different users, for example, admin and owner.
RPM installation: A method of installing Matillion ETL using the Universal Installer (RPM package), typically used for on-premises or custom deployments rather than cloud marketplace images.
RSD file: A Matillion-specific XML-based script file that defines how an API endpoint is queried, including column mappings, XPath expressions, URI, pagination settings, and the HTTP method. Used within API Query Profiles.
Run Orchestration component: A flow component that calls and executes another orchestration job within the current job's execution flow.
Run Transformation component: A flow component that calls and executes a transformation job from within an orchestration job.
Runner: The runner stages data in an appropriate staging area. The runner then transforms this data on the target data warehouse.
S🔗
S3 (Simple Storage Service): Amazon's object storage service, used extensively in Matillion ETL as a staging area for data loads and as a source for S3 Load Generator components.
SaaS: SaaS delivers applications to users, managed by a third-party vendor. SaaS applications usually work through a web browser, and don't require downloads or installation. Vendors manage the technical issues, like data, servers and storage. It reduces the time needed to install and upgrade software on the client side.
Sample: Shows a preview of your data and allows you to see what will be returned when working with Matillion ETL.
SAP ODP Extract component: An orchestration component that connects directly to an SAP system via the ODP framework to extract data from SAPI extractors and ABAP CDS views into a cloud data warehouse.
Satellite table: A Data Vault table that stores contextual attributes or descriptive information about a Hub or Link table, capturing historical changes over time.
Scheduling: The ability in Matillion ETL to configure jobs to run automatically at defined times or intervals, managed through the Quartz scheduler library.
Schemas: A schema defines the structure of something. A database schema is how data is organized and constructed (i.e. relational databases are divided into tables). It also describes the relationship between its entities.
Script: A set of instructions that is carried out by another program; unlike a program, which uses a compiler.
SDK: A software development kit provides the tools a developer needs to make software on a specified platform: tools, documentation, libraries, guides, code samples, etc.
Secret key: A secret key acts like a password. It's a sequence of characters like passwords, account keys, and data connection strings.
Secrets Manager: An AWS service for storing and retrieving secrets (such as database passwords and API keys). Matillion ETL can retrieve credentials from AWS Secrets Manager at runtime.
Sequences: Sequences allow users to create an automatically iterating value that can be loaded into tables.
Server: A section of a computer's hardware or software, that enables other programs or devices to function.
Shared job: A bundled, reusable workflow that packages one or more orchestration and transformation jobs into a single custom component, which can be placed on any job canvas and run with configurable parameters.
Shared variable: A variable in Matillion ETL whose value is updated globally across all branches of a job. Any branch that updates the variable immediately affects all other branches. Formerly known as a "global" variable.
Single sign-on: Single sign-on authenticates users to log in to a service with a single ID and password. One set of credentials can be used for multiple applications, for example, when a website allows you to use your Google or Facebook account to access a webpage. Single sign-on uses a central server that all applications trust.
SNC (Secure Network Communication): An SAP authentication method that uses encryption and certificates for secure communication between Matillion ETL and an SAP system, used in the SAP ODP Extract component.
Snowflake: A cloud-native data warehousing platform supported as a primary target in Matillion ETL, offering separation of compute and storage, virtual warehouses, key-pair authentication, time travel, and query tags.
Snowflake virtual warehouse: A Snowflake cluster of compute resources responsible for executing all queries pushed down by Matillion ETL. Warehouses can be resized, suspended, and configured for multi-cluster scaling.
SNS (Simple Notification Service): An AWS messaging service. Matillion ETL can publish messages to SNS topics using the SNS Message component.
SOAP: Simple Object Access Protocol is a message protocol that can be carried over HTTP. SOAP is based on XML, a human-readable file format.
Spectrum: Amazon Spectrum is a built-in feature found in Amazon Redshift that allows a user to perform complex data analysis on objects (like JSON files) stored in an Amazon S3 bucket.
SQL: SQL stands for Structured Query Language, and is best used for structured data—data found in a fixed field, such as databases and spreadsheets. It's used for relational database management systems (RDBMS) and relational data stream management systems (RDSMS).
SQL push-down: The architectural pattern in Matillion ETL where transformation logic is converted into SQL and executed directly on the cloud data warehouse, rather than being processed by the Matillion ETL server itself.
SQS (Simple Queue Service): An AWS message queuing service. Matillion ETL can read from and write to SQS queues, including using SQS events to trigger job execution.
SSL (Secure Sockets Layer): An encryption protocol for securing data in transit. Matillion ETL supports SSL for HTTPS access to the web interface and for JDBC connections to data sources and targets.
SSO (Single Sign-On): A session authentication mechanism where a single set of credentials provides access to multiple systems. Referenced in Matillion ETL security best practices.
Stage: A data staging area is an intermediate storage area that sits between your data source and your data targets (for example your data warehouse). Data staging is a temporary state during which data is processed for extract, transform, and load operations.
Star schema: A data modeling pattern used as a presentation layer on top of a Data Vault, consisting of fact tables and dimension tables designed to support reporting and analytics queries.
Start component: The mandatory starting point of every orchestration job. It marks where job execution begins.
Stateless authentication: A Matillion ETL configuration mode for HA clusters where authentication state is managed externally (via OpenID or LDAP) rather than per-node, enabling consistent authentication across all nodes in a cluster.
String: A sequence of characters that could be a variable, or a literal constant.
Structured data: Data that's held in a table form with relationships between the rows and columns. Examples include JSON and XML.
STS (Short-Term Support): A Matillion ETL release designation for versions that receive updates and are supported only until the next version is released, suited for customers wanting the latest features.
Surrogate key: An internally generated unique identifier used in Hub and Link tables to simplify joins and replace natural business keys in relational operations.
T🔗
Table Input component: A transformation component that reads selected columns from an existing table or view in the data warehouse into the current transformation job's data flow.
Table Iterator: A flow component that iterates over the rows of a specified database table, running subsequent components once per row.
Table Output component: A transformation component that writes the input data flow into an existing table in the data warehouse, either appending to or truncating the target table.
Task: An individual execution unit within a Matillion ETL job run. Multiple tasks may run concurrently within a single job depending on the job's branching structure and component concurrency settings.
Task History: A panel in the Matillion ETL UI that shows a record of past job runs, including start time, duration, and status for each task.
Time Travel: A feature of Snowflake and Delta Lake on Databricks that allows querying a table as it existed at a previous point in time. Used in Matillion ETL's Table Input component via the Offset property.
Token: An object that's used to get access to a restricted resource. A token may be a key/password, signature, or biometric data.
Tomcat: The Apache Tomcat Java application server on which Matillion ETL runs. Restarting Tomcat is required after configuration changes and updates.
Transformation job: A type of Matillion ETL job that performs data transformation operations (filtering, joining, calculating, aggregating) directly on the cloud data warehouse by generating and executing SQL. Called from orchestration jobs via the Run Transformation component.
Truncate: To shorten something.
U🔗
Unite component: A transformation component that combines rows from two or more input flows into a single output flow, equivalent to a SQL UNION ALL operation.
User: A person who uses a computer or service.
V🔗
Variable: A named value stored in Matillion ETL that can be referenced in component parameters, scripts, and expressions at runtime. Variables have types (Text, Numeric, DateTime, Data Structure) and can be scoped as environment, job, grid, or automatic variables.
vCPU: A virtual CPU core on a cloud provider VM. The number of vCPUs on a Matillion ETL instance determines the maximum number of concurrent tasks that can run.
Version: Versioning captures your development at a point in time to designate as "live" or "production", etc., and to then continue working on your default version.
Views: Virtual tables based on the results of SQL statements, that are saved in the database as named queries. The virtual table itself can be queried.
VPC (Virtual Private Cloud): A logically isolated network within a cloud provider used to host Matillion ETL instances with controlled network access.
W🔗
Webhook payload profile: A configuration in Matillion ETL that defines the structure of webhook payloads used to trigger job runs via the Matillion ETL REST API.
Wildcard: A symbol used to represent or replace one or more characters. Wildcards are useful in searching through data.
Workday Custom Reports component: An orchestration component that uses the Workday Reporting as a Service API to extract custom report data from Workday into a cloud data warehouse.
Workday Extract component: An orchestration component that uses the Workday API to retrieve structured business data (employee, financial, HR) from Workday web services and load it into a cloud data warehouse.
Writer: View, edit, and execute all parts of Matillion ETL, but not delete projects or versions.
X🔗
XML: XML is a markup language that holds structured data. It uses tags that work like HTML. Tag names must only be letters, numbers, and underscores. XML was designed to be readable by both humans and machines, containing standard words within its tags. XML has one data type: String.