DE – Telegram
DE
@data_engi
522 subscribers
312 photos
81 videos
15 files
405 links
Data Engineering Technologies.
SQL, Python, Kafka, Spark, Pandas, Airflow, Clickhouse, Greenplum, Postgres, dbt, LLM agentic systems, AI, robots, drones etc.

Boost channel - https://news.1rj.ru/str/boost/data_engi
Download Telegram
About
Blog
Apps
Platform
Join
DE
522 subscribers
DE
https://seattledataguy.substack.com/p/introduction-to-prestotrino-for-data

#presto #trino
Substack
Introduction to Presto/Trino for Data Engineers
Have you ever been confused about what the story is with PrestoSQL, PrestoDB, and Trino? How do they relate, which one should you use, what is Presto used for, and who uses it? We will try to answer these questions and present and position Presto for the…
👍1🤔1
423 views
DE
😁9
425 views
DE
https://www.tinkoff.ru/about/news/14062023-tinkoff-announces-the-launch-of-russias-first-private-stem-university/
Тинькофф
Тинькофф объявляет о запуске первого в России частного STEM-университета — Тинькофф новости
Новости - новости Тинькофф на 14 июн. 2023
😐6
366 views
DE
https://github.com/apache/arrow-ballista
GitHub
GitHub - apache/arrow-ballista: Apache Arrow Ballista Distributed Query Engine
Apache Arrow Ballista Distributed Query Engine. Contribute to apache/arrow-ballista development by creating an account on GitHub.
317 views
DE
😁9
422 views
DE
https://www.cpard.xyz/posts/mlops_is_mostly_data_engineering/
Kostas Heaven on Net
MLOps is Mostly Data Engineering.
The overlap between MLOps and Data Engineering
👍2🤔1
361 views
DE
378 views
DE
https://realpython.com/python312-f-strings/
Realpython
Python 3.12 Preview: More Intuitive and Consistent F-Strings – Real Python
In this tutorial, you'll preview one of the upcoming features of Python 3.12, which introduces a new f-string syntax formalization and implementation. The new implementation lifts some restrictions and limitations that affect f-string literals in Python versions…
367 views
DE
https://blog.devgenius.io/elt-is-dead-and-etlt-will-be-the-end-of-modern-data-processing-architecture-154b87c1cce0
Medium
ELT is dead, and EtLT will be the end of modern data processing architecture
When talking about data processing, people often abbreviate it as “ETL.” However, if we look closely, data processing has undergone several…
361 views
DE
https://habr.com/ru/articles/735866/
Хабр
Wireshark — подробное руководство по началу использования
Руководство по анализу трафика сети посредством Wireshark 1. Что такое Wireshark? Логотип программы Wireshark – это широко распространённый инструмент для захвата и анализа сетевого трафика, который...
403 views
DE
https://go.dev/doc/go1.21
go.dev
Go 1.21 Release Notes - The Go Programming Language
398 views
DE
https://github.com/apache/arrow-ballista


https://www.cpard.xyz/posts/mlops_is_mostly_data_engineering/
GitHub
GitHub - apache/arrow-ballista: Apache Arrow Ballista Distributed Query Engine
Apache Arrow Ballista Distributed Query Engine. Contribute to apache/arrow-ballista development by creating an account on GitHub.
337 views
DE
Forwarded from DataEng
Airflow 2.7.0

Вчера состоялся большой релиз новой версии самого популярного Workflow менеджера в Python: Apache Airflow 2.7.0

Из новых фишечек:

— Setup and Teardown tasks
— Cluster Activity UI
— OpenLineage built-in integration

Улучшения:
— Убрали поддержку Python 3.7
— В рамках улучшения безопасности Airflow запретили функцию проверки Connections в UI
— Новый Graph View теперь используется по умолчанию, старую версию удалили
— При обновлении версии рекомендуется накатывать миграции через команду airflow db migrate

Полный список можно посмотреть тут.
Apache Airflow
Apache Airflow 2.7.0 is here
Apache Airflow 2.7.0 has been released!
👍2
457 views
DE
https://www.sortvisualizer.com/
Sortvisualizer
Sort Visualizer
A visualization of 15+ sorting algorithms, including Quick Sort, Merge Sort, Selection Sort and more!
🖕3
444 views
DE
🔥5
499 views
DE
Happy programmer's day! 👨‍💻🧑‍💻👩‍💻
Please open Telegram to view this post
VIEW IN TELEGRAM
🎉7👍1😁1👨‍💻1
642 views
DE
https://www.postgresql.org/about/news/postgresql-16-released-2715/
PostgreSQL News
PostgreSQL 16 Released!
September 14, 2023 - The PostgreSQL Global Development Group today announced the release of PostgreSQL 16, the latest version of …
❤‍🔥1👏1
353 views
DE
https://pythonspeed.com/articles/pandas-sql-chunking/
Python⇒Speed
Loading SQL data into Pandas without running out of memory
Pandas can load data from a SQL query, but the result may use too much memory. Learn how to process data in batches, and reduce memory usage even further.
🆒2
469 views
DE
https://github.com/reflex-dev/reflex
GitHub
GitHub - reflex-dev/reflex: 🕸️ Web apps in pure Python 🐍
🕸️ Web apps in pure Python 🐍. Contribute to reflex-dev/reflex development by creating an account on GitHub.
🤔3
417 views
DE
https://www.paradedb.com/
ParadeDB
ParadeDB — The Transactional Elasticsearch Alternative
ParadeDB is an open source, ACID-compliant search and analytics database. Built on Postgres for update-heavy workloads.
❤‍🔥1🤩1
453 views
DE
DE
https://www.postgresql.org/about/news/postgresql-16-released-2715/
https://news.1rj.ru/str/dataeng/544
Telegram
DataEng
Обзор новых фич в новой версии PostgreSQL 16: https://www.youtube.com/watch?v=M7m3A68CTJg
❤‍🔥1🔥1
499 views