Банки получают содержание кассовых чеков клиентов по транзакциям, совершенных по собственным картам через Операторов Фискальных Данных с согласия клиента. Данные приходят в сыром текстовом формате,...

Posted in r/bigdata by u/Big_Data_Path • 1 point and 0 comments

Иногда в СМИ можно встретить мнение, что отношение к выборам и электоральные предпочтения людей, проживающих в глубинке и в крупных городах, сильно отличаются. Возможно, источником такой информации...

Данная статья является гайдом по использованию кастомных агрегаторов в Spark SQL API. Она “выросла” из моих заметок, которые я делал себе с начала работы со Spark. Сейчас, по мере накопления опыта,...

Hello everyone, I'm very happy to announce that stucloud is finally ready for beta testing ! What's stucloud ? It's a cloud storage on budget...

Всем привет, меня зовут Илья, я работаю в компании DINS на должности инженера отдела мониторинга. В этой статье расскажу о нашей боли при работе с ElasticSearch. Мне не удалось найти решение этой...

Posted in r/bigdata by u/Best_Fold_2554 • 17 points and 0 comments

Posted in r/bigdata by u/Big_Data_Path • 1 point and 0 comments

Posted in r/bigdata by u/DolphinScheduler1 • 2 points and 0 comments

I'm writing a university assignment where I have to select a company that is currently facing some kind of strategic challenge or issue and then...

Posted in r/bigdata by u/marclamberti • 1 point and 0 comments

Posted in r/bigdata by u/Big_Data_Path • 1 point and 0 comments

Working with data in big data always involves some kind of complexities related to its size, storage, and processing. What skills are needed to deal with them?

Analytics and BI is an approach for making data-driven decisions and provides information that can help businesses

Management challenges tackle privacy, security, governance, and data/metadata management. Another area of knowledge directly connected to the big data

Machine Learning is based on the idea that analytic systems can learn to identify patterns and make decisions with minimal human involvement

Discover the differences between ACID and BASE design philosophies - from strong consistency to eventual consistency. Find out which suits your project better!

Discover the crucial Architecturally Significant Requirements (ASR) for distributed systems, including Availability, Durability, Resiliency, Reliability, and Scalability. Learn how these factors impact system design and performance.

Understand the CAP and PACELC theorems in distributed systems. Learn how to navigate tradeoffs between consistency, availability, and partition tolerance for optimal system design.

Caching... There is so much in that word - the pain of invalidation and the joy of reusing computation. In Spark, this is known as an optimization technique