New Amazon S3 Tables: Storage optimized for analytics workloads | AWS News Blog
https://aws.amazon.com/blogs/aws/new-amazon-s3-tables-storage-optimized-for-analytics-workloads/
https://aws.amazon.com/blogs/aws/new-amazon-s3-tables-storage-optimized-for-analytics-workloads/
Amazon
New Amazon S3 Tables: Storage optimized for analytics workloads | Amazon Web Services
Amazon S3 Tables optimize tabular data storage (like transactions and sensor readings) in Apache Iceberg, enabling high-performance, low-cost queries using Athena, EMR, and Spark.
I would also add Azure Data Explorer (Kusto) to the list. However, ADX is not open-source.
https://startree.ai/resources/a-tale-of-three-real-time-olap-databases
https://startree.ai/resources/a-tale-of-three-real-time-olap-databases
While US markets are panicking you can try to play with DeepSeek by installing it locally or using Cursor, it is already available there.
https://dev.to/lunaticprogrammer/using-deepseek-r1-in-visual-studio-code-for-free-2279
https://dev.to/lunaticprogrammer/using-deepseek-r1-in-visual-studio-code-for-free-2279
🎉1
MCP is an open protocol that standardizes how applications provide context to LLMs. Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standardized way to connect your devices to various peripherals and accessories, MCP provides a standardized way to connect AI models to different data sources and tools.
https://github.com/punkpeye/awesome-mcp-servers
https://github.com/punkpeye/awesome-mcp-servers
GitHub
GitHub - punkpeye/awesome-mcp-servers: A collection of MCP servers.
A collection of MCP servers. Contribute to punkpeye/awesome-mcp-servers development by creating an account on GitHub.
The Future of Data Engineering: AI, LLMs, and Automation
https://www.dataengineeringpodcast.com/episodepage/the-future-of-data-engineering-ai-llms-and-automation
https://www.dataengineeringpodcast.com/episodepage/the-future-of-data-engineering-ai-llms-and-automation
Data Engineering Podcast
The Future of Data Engineering: AI, LLMs, and Automation
Summary
In this episode of the Data Engineering Podcast Gleb Mezhanskiy, CEO and co-founder of DataFold, talks about the intersection of AI and data…
In this episode of the Data Engineering Podcast Gleb Mezhanskiy, CEO and co-founder of DataFold, talks about the intersection of AI and data…
🔥1
Looks like most common technologies already integrated Iceberg. In other words, it became a new standard for open table formats.
https://youtu.be/O2l5SB-camQ?si=C2JR2xayOT6aD7jJ
https://youtu.be/O2l5SB-camQ?si=C2JR2xayOT6aD7jJ
YouTube
Tableflow: Materialize Apache Kafka® Topics as Apache Iceberg™ and Delta Lake Tables With Zero ETL
Blog post: https://cnfl.io/4hNOPFn | Tim Berglund is back at the lightboard to show you the most exciting thing he’s seen in analytics in the last 40 years (nope, that’s not an exaggeration): Tableflow, a new feature in Confluent Cloud that lets you completely…
👍2❤1
v2.0.0 of the Snowflake Terraform provider, which is the officially supported Snowflake product, reached GA. Here is a tutorial.
GitHub
terraform-provider-snowflake/ROADMAP.md at main · snowflakedb/terraform-provider-snowflake
Terraform provider for managing Snowflake accounts - snowflakedb/terraform-provider-snowflake
A great article to catch up with the recent industry developments related to Iceberg and its adoption.
Getdbt
Iceberg?? Give it a REST!
The new abstraction that changes nothing... and everything
❤2
The evolution of databases (w/ Wolfram Schulte)
https://roundup.getdbt.com/p/the-evolution-of-databases-w-wolfram
https://roundup.getdbt.com/p/the-evolution-of-databases-w-wolfram
Getdbt
The evolution of databases (w/ Wolfram Schulte)
In the first episode of our season on developer experience, the cofounder and CTO of SDF Labs, now a part of dbt Labs, discusses databases, compilers, and dev tools.
dbt announced faster engine, but I am really curious about their cost reduction rather than speed of development
https://www.getdbt.com/product/fusion
https://www.getdbt.com/product/fusion
dbt Labs
Accelerate data workflows with the dbt Fusion engine | dbt Labs
Experience lightning-fast performance and intelligent SQL validation with the dbt Fusion engine, the next-generation engine for modern analytics.
Introducing Apache Spark 4.0 | Databricks Blog
https://www.databricks.com/blog/introducing-apache-spark-40
https://www.databricks.com/blog/introducing-apache-spark-40
❤2👍2
This episode is 6 hours long, but it is like a good book that you prefer to actually last.
YouTube
DHH: Future of Programming, AI, Ruby on Rails, Productivity & Parenting | Lex Fridman Podcast #474
David Heinemeier Hansson (aka DHH) is a legendary programmer, creator of Ruby on Rails, co-owner & CTO of 37signals that created Basecamp, HEY, & ONCE, and is a NYT-best-selling author (with Jason Fried) of 4 books: REWORK, REMOTE, Getting Real, and It Doesn't…
👍6
I assume you came across these lists from Microsoft, if not here is the link. In summary, we can interpret it as AI "replicability" and "resiliency". I personally think we will be surprised in 5 years. With regards to data engineering I was wondering if DE will get replaced first or DS as mentioned in the list. We kinda do "plumbing" for data 😉
🤔1