Workspace Name
Configuration: WORKSPACE_NAME¶
Description¶
Specifies the logical higher level group/domain of the data ingestion pipeline and serves as the prefix for the PIPELINE_NAME configuration, forming a composite namespace for the pipeline's metadata.
Default Value¶
"default"
Example Values¶
"dev" - This might be used to represent a development environment. "prod" - This would typically be used to represent a production environment. "marketing" - This might be used to represent a specific department within your organization. "project_a" - This could represent a specific project within your organization.
Detailed Description¶
The WORKSPACE_NAME is used to uniquely identify the workspace for the ingestion pipeline, which can be a department, a specific project, or an environment. The WORKSPACE_NAME acts as a prefix to the PIPELINE_NAME configuration. This composition creates a unique namespace for each pipeline's metadata, including its schemas, buffers, and offsets database. This namespace serves as a unique identifier for the pipeline within the data ingestion tool and assists in data management and pipeline organization.
Considerations¶
When setting the WORKSPACE_NAME, make sure it uniquely and clearly represents the data ingestion pipeline's workspace. This is important for efficient management and tracking of different data ingestion pipelines, especially when multiple pipelines are running concurrently.
Also, bear in mind that while the WORKSPACE_NAME config is typically used to represent environments like dev, test, or prod, it can also be used to represent a department or project. This flexibility can be particularly useful when testing Skippr locally.
When working with multiple environments, departments, or projects, it's best practice to keep the WORKSPACE_NAME consistent across all instances of Skippr to facilitate easier management and navigation. Furthermore, changing the WORKSPACE_NAME during a pipeline's lifecycle can lead to confusion and may cause data to be written into the wrong namespace.