Prior to diving deep into complex analytics modeling topics like Sessionization, Attribution, RFM, one has to understand the motivation behind it.

It is the questions that drive business decisions first, then instruments and practices to find reliable answers to these questions.

Which questions does your business ask?

#analytics #modeling

I just tried to upgrade onto 21.10 from 21.08.

After 30 minutes this cluster is never coming back.

I hope I can create a new one and restore backup onto it.

We are in the middle of migration from Amazon Redshift DC2 nodes (2nd gen) to RA3 nodes (3rd gen) at Wheely.

What this means for us:
– Almost unlimited Disk Space (RA3 separate compute and storage)
– Speeding up Data Marts to 2hrs delay from real-time
– Blue/green deployments

I will follow up as soon as we are finished.

Attached simplified checklist plan.

Any questions welcomed.

So the process of Amazon Redshift cluster migration is almost completed.
New cluster is way more powerful. Now seeking ways to fully utilize its resources 😄

I can state that not everything has gone as expected.
The most painful parts turned out to be:

– Migrating S3 bucket with 1M+ files to a new region (took ~4-5 hours) – really challenging
– Not losing data events while switching between clusters
– VPC and network issues (connecting from BI tool)
– Hotfixing several Python UDFs suddenly not working on a new environment

In some time I will publish a detailed reflection on this process.

A nice remark from Dmitry Anoshin @rockyourdata

How one can visualize its own DWH ER (Entity-Relationship) model?

I would use these two ways (applicable to my DWH @ Wheely):

- DBeaver's feature ER diagram
- Looker's LookML Diagram

Both ways require relationships to be modeled in advance i.e. defining FOREIGN KEY / REFERENCES constraints or JOIN conditions.

Can anybody suggest more options?