Build a real-time GDPR-aligned Apache Iceberg data lake.
Amazon
Build a real-time GDPR-aligned Apache Iceberg data lake | Amazon Web Services
Data lakes are a popular choice for today’s organizations to store their data around their business activities. As a best practice of a data lake design, data should be immutable once stored. But regulations such as the General Data Protection Regulation…
👍3
Implement slowly changing dimensions in a data lake using AWS Glue and Delta | AWS Big Data Blog
https://aws.amazon.com/blogs/big-data/implement-slowly-changing-dimensions-in-a-data-lake-using-aws-glue-and-delta/
https://aws.amazon.com/blogs/big-data/implement-slowly-changing-dimensions-in-a-data-lake-using-aws-glue-and-delta/
Amazon
Implement slowly changing dimensions in a data lake using AWS Glue and Delta | Amazon Web Services
In a data warehouse, a dimension is a structure that categorizes facts and measures in order to enable users to answer business questions. To illustrate an example, in a typical sales domain, customer, time or product are dimensions and sales transactions…
Welcome to Marvin - Marvin
https://www.askmarvin.ai/
https://www.askmarvin.ai/
Marvin
Marvin - Marvin
A powerful framework for building AI applications
The Truth about Prefect, Mage, and Airflow.
https://dataengineeringcentral.substack.com/p/the-truth-about-prefect-mage-and
https://dataengineeringcentral.substack.com/p/the-truth-about-prefect-mage-and
Substack
The Truth about Prefect, Mage, and Airflow.
The Battle for the Orchestration Future.
MLOps Is Overfitting: Here’s Why
https://lakefs.io/blog/mlops-is-overfitting/
https://lakefs.io/blog/mlops-is-overfitting/
Git for Data - lakeFS
How To Improve ML Pipeline Development With Reproducibility
In this article we'll explore how to improve your ML pipeline development with MLOps tools for reproducible experiments. Read on to learn more.
The future of the data engineer — Part I | by Analytics at Meta | Apr, 2023 | Medium
https://medium.com/@AnalyticsAtMeta/the-future-of-the-data-engineer-part-i-32bd125465be
https://medium.com/@AnalyticsAtMeta/the-future-of-the-data-engineer-part-i-32bd125465be
Medium
The future of the data engineer — Part I
Introduction
❤1
Introducing AI Functions: Integrating Large Language Models with Databricks SQL - The Databricks Blog
https://www.databricks.com/blog/2023/04/18/introducing-ai-functions-integrating-large-language-models-databricks-sql.html?utm_source=bambu&utm_medium=social&utm_campaign=advocacy
https://www.databricks.com/blog/2023/04/18/introducing-ai-functions-integrating-large-language-models-databricks-sql.html?utm_source=bambu&utm_medium=social&utm_campaign=advocacy
Open source self-hosted Delta Sharing server | Delta Lake
https://delta.io/blog/2023-04-24-open-source-selfhosted-delta-sharing-server/
https://delta.io/blog/2023-04-24-open-source-selfhosted-delta-sharing-server/
delta.io
Open source self-hosted Delta Sharing server
This post explains Kotosiro Delta Sharing server basic instructions
👍5
dbt Guide | GitLab
https://about.gitlab.com/handbook/business-technology/data-team/platform/dbt-guide/
https://about.gitlab.com/handbook/business-technology/data-team/platform/dbt-guide/
👍5
Microsoft introduced Fabric, which is a combination of Power BI, Azure Synapse, Data Factory and Data Explorer on top of ADLS gen2 using Delta (Parquet) as data lake format. A new component is Data Activator which seems to be a no-code rule engine.
https://azure.microsoft.com/en-us/blog/introducing-microsoft-fabric-data-analytics-for-the-era-of-ai/
https://azure.microsoft.com/en-us/blog/introducing-microsoft-fabric-data-analytics-for-the-era-of-ai/
Microsoft Azure Blog
Introducing Microsoft Fabric: The data platform for the era of AI | Microsoft Azure Blog | Microsoft Azure
Announcing Microsoft Fabric—a unified analytics platform that brings together all the data and analytics tools that organizations need. Learn more.
❤1