👍4
[streaming][data]
https://jack-vanlightly.com/blog/2024/3/19/tableflow-the-stream-table-kafka-iceberg-duality
https://jack-vanlightly.com/blog/2024/3/19/tableflow-the-stream-table-kafka-iceberg-duality
Jack Vanlightly
Tableflow: the stream/table, Kafka/Iceberg duality — Jack Vanlightly
Confluent just announced Tableflow, the seamless materialization of Apache Kafka topics as Apache Iceberg tables. This announcement has to be the most impactful announcement I’ve witnessed while at Confluent. This post is about why Iceberg tables aren’t just…
[stream processing]
https://www.kai-waehner.de/blog/2024/03/20/the-past-present-and-future-of-stream-processing/
https://www.kai-waehner.de/blog/2024/03/20/the-past-present-and-future-of-stream-processing/
Kai Waehner
The Past, Present and Future of Stream Processing
Stream Processing Journey with IBM, Apama, TIBCO StreamBase, Kafka Streams, Apache Flink, Streaming Databases, GenAI and Apache Iceberg.
[databases]
https://jack-vanlightly.com/analyses/2024/4/24/understanding-apache-hudi-consistency-model-part-1
https://jack-vanlightly.com/analyses/2024/4/24/understanding-apache-hudi-consistency-model-part-1
Jack Vanlightly
Understanding Apache Hudi's Consistency Model Part 1 — Jack Vanlightly
Apache Hudi is one of the leading three table formats (Apache Iceberg and Delta Lake being the other two). Whereas Apache Iceberg internals are relatively easy to understand, I found that Apache Hudi was more complex and hard to reason about. As a distributed…
[grafana]
I found this usage of Grafana quite inspiring:
https://grafana.com/about/events/grafanacon/2024/grafana-used-to-monitor-japan-slim-moon-lander/
I found this usage of Grafana quite inspiring:
https://grafana.com/about/events/grafanacon/2024/grafana-used-to-monitor-japan-slim-moon-lander/
Grafana Labs
Grafana in space: Monitoring Japan's SLIM moon lander in real time | Grafana Labs
JAXA Associate Senior Researcher Satoshi Nakahira presents an overview of the ISAS space science missions and the SLIM lunar lander.
❤🔥1
[databases]
https://www.uber.com/en-NL/blog/auto-categorizing-data-through-ai-ml/
Data categorization–the process of classifying data based on its characteristics and essence–is a foundational pillar of any privacy or security program. The effectiveness of fine-grained data categorization is pivotal in implementing privacy and security controls, such as access policies and encryption, as well as managing the lifecycle of data assets, encompassing retention and deletion. This blog delves into Uber’s approach to achieving data categorization at scale by leveraging various AI/ML techniques.
https://www.uber.com/en-NL/blog/auto-categorizing-data-through-ai-ml/
Data categorization–the process of classifying data based on its characteristics and essence–is a foundational pillar of any privacy or security program. The effectiveness of fine-grained data categorization is pivotal in implementing privacy and security controls, such as access policies and encryption, as well as managing the lifecycle of data assets, encompassing retention and deletion. This blog delves into Uber’s approach to achieving data categorization at scale by leveraging various AI/ML techniques.
[llm][usecase][text-to-sql]
https://medium.com/pinterest-engineering/how-we-built-text-to-sql-at-pinterest-30bad30dabff
https://medium.com/pinterest-engineering/how-we-built-text-to-sql-at-pinterest-30bad30dabff
Medium
How we built Text-to-SQL at Pinterest
Adam Obeng | Data Scientist, Data Platform Science; J.C. Zhong | Tech Lead, Analytics Platform; Charlie Gu | Sr. Manager, Engineering
👍1
[news][ai][hackaton]
Great projects out of the Mistral AI hackaton which took place in Paris.
https://x.com/alexreibman/status/1796349663710511114?s=46&t=eNN3Y-GKeBSlFyyj1ozvgg
Great projects out of the Mistral AI hackaton which took place in Paris.
https://x.com/alexreibman/status/1796349663710511114?s=46&t=eNN3Y-GKeBSlFyyj1ozvgg