Connect to federated data sources from Athena.

In this article, we discuss the tools and components that make up the Hadoop Ecosystem, including Hive, Spark, MapReduce, ZooKeeper, and Oozie.

Andreas Andreakis, Ioannis Papapanagiotou

Gain technology and business knowledge and hone your skills with learning resources created and curated by O'Reilly's experts: live online training, video, books, our platform has content from 200+ of the world's best publishers.

You’re not just another dot on a map

Building clean datasets with reliable and reproducible ingestion pipelines is completely useless if it's not possible to find them and understand their provenance. The solution to discoverability and tracking of data lineage is to incorporate a metadata repository…

Data lineage for every pipeline.

Some include code and exercises

At the beginning of the year, we set out 10 big data trends to watch in 2019. We correctly called some of what unfolded, including a renewed focus on data

In part one of this series, we covered some of the biggest news events of the year to cross Datanami’s pages. We pick up where we left off and bring you

With just over a week left on the 2019 calendar, it’s now time for predictions. We’ll run several stories featuring the 2020 predictions of industry

An opinionated list of awesome Python frameworks, libraries, software and resources. - vinta/awesome-python

A community driven list of useful Scala libraries, frameworks and software. - lauris/awesome-scala

A curated list of awesome big data frameworks, ressources and other awesomeness. - oxnr/awesome-bigdata

Modin: Scale your Pandas workflows by changing a single line of code - modin-project/modin

Dask is simply the most revolutionary tool for data processing that I have encountered. If you love Pandas and Numpy but were sometimes struggling with data that would not fit into RAM then Dask is…

Chocolatey is software management automation for Windows that wraps installers, executables, zips, and noscripts into compiled packages. Chocolatey integrates w/SCCM, Puppet, Chef, etc. Chocolatey is trusted by businesses to manage software deployments.