August 2024 changelog

Here you'll find the August 2024 changelog for the Data Productivity Cloud. Just want to read about new features? Read our New Features blog.

---

29th August

DesignerNew features 🎉

• You can now connect your own GitLab repository to the Data Productivity Cloud. This feature lets you connect your GitLab repository when you create a project in the Data Productivity Cloud. Follow the documentation to get started:
  • Prerequisites for connecting a GitLab repository to the Data Productivity Cloud
  • Troubleshooting guide for connecting a GitLab repository to the Data Productivity Cloud
• Updated Schedules so that you can now specify a time and date to enable pipeline schedules from.
• Updated the "Connect your own GitHub" workflow so that you can now view the list of authorized accounts or organizations after a successful authorization with GitHub.

---

27th August

StreamingNew features 🎉

• Added the following destinations to streaming pipelines:
  • Amazon S3
  • Azure Blob Storage

---

20th August

DesignerNew features 🎉

• Shared pipelines are now available in the Data Productivity Cloud. Any pipeline can be saved as a shared pipeline, helping you to maintain consistency of core functionality over multiple different projects across the organization.
• Added the Run Shared Pipeline component, which allows you to call a shared pipeline from within another pipeline.
• Added improvements to environments. When you create an environment, you can now set a default access level for that environment. This access level can also be set on a per-user basis.
  • As part of this change, all users in existing environments have been set to have Admin access by default, to avoid any disruption. Project admins can change a user's environment access.

---

16th August

DesignerNew features 🎉

• Added a Document AI Predict component, which invokes the Snowflake PREDICT function and allows users to call their Document AI models from the context of a Data Productivity Cloud pipeline. This component is available in public preview.

---

14th August

CDCImprovements 🔧

• Updated the CDC agent version from 2.95.22 to 2.95.27.
• CDC will now retain snapshot metrics once a pipeline snapshot completes and streaming begins. Note: these will be lost after an agent restart.
• Improved resilience of connection between Streaming agents and the platform.
• Library updates and security fixes.

---

8th August

StreamingNew features 🎉

• Streaming pipelines are now available in the Data Productivity Cloud. Streaming pipelines are a complete end-to-end solution for near-real-time data ingestion, allowing you to capture and synchronize data from source databases and write to a target data warehouse.
  • Supported data sources are:
    • Db2 for IBM i
    • Microsoft SQL Server
    • MySQL
    • Oracle
    • PostgreSQL
  • Supported data targets are:
    • Snowflake
  • Streaming pipelines are available on our Hybrid SaaS offering, meaning you'll need to create a streaming agent in your AWS or Azure infrastructure. Read Create an agent in your infrastructure to set up your agent.

DesignerNew features 🎉

• Options to run a single component or run a pipeline starting from a specific component have been added to the pipeline canvas.
• Options to control the behavior of variables in a branching pipeline have been added.
• The Push local changes dialog now shows a list of commits to be pushed.

---

6th August

DesignerNew features 🎉

• Users can now terminate running pipelines from the Pipeline Observability Dashboard.