Continued:

Also, pay attention to how a model is built with dbt_utils.unpivot macro:

{{ dbt_utils.unpivot(
  relation=ref('dim_chauffeurs_labels'),
  cast_to='varchar',
  exclude=['actual_dt', 'chauffeur_id'],
  field_name='label',
  value_name='value'
) }}

A list of attributes unloaded could be expanded or narrowed without actual code changes. Schema is ['actual_dt', 'chauffeur_id', 'label', 'value']

Looker action -> Braze (Customer engagement platform)

Let's keep observing ways to build reverse-ETL pipelines.

Looker enables its user to leverage Actions Hub – way of integrating Looker with 3rd party tools. Amongst most notable are:

– Airtable
– S3
– Braze
– Dropbox
– gDrive / gSheets / gAnalytics / gAds
– Hubspot
– mParticle

We calculate certain labels/traits/dimensions on our passengers/chauffeurs based on their behaviour and services usage in DWH.

Looker Action enables scheduled sync of data from Redshift to Braze user cards.

All you need to perform sync is:

– Identifier (user_id / external_id)
– Dimensions which will become traits in Braze card
– Schedule for this Action

Then in Braze we are able to:

– Target our campaigns precisely
– Deliver personal experiences and communications
– Split segments and perform A/B testing
– Identify users who are about to churn

#reverse_etl

Трансляция на YouTube, начинаем!
https://www.youtube.com/watch?v=10H45iYaCc8

[RU] Слайды с митапа:
– [dbt meetup] Почему вашей компании нужен dbt?
– [dbt meetup] Статистика запусков dbt и поиск узких мест

#meetup

The world ain't all sunshine and rainbows. It's a very mean and nasty place.

Especially when you use select *

When a base table changes, all the downstream dependencies might break down.
Please don't deploy such things to production.

⬇️⬇️⬇️

#bugs

AWS denounces its own error logs

Your post may include a non-inclusive word (master)

😭😂

Hey, long time no see 🙂

We have just started our engagement with dbtLabs at Wheely.

Guys will try to conduct audit and help us improve our dbt deployment even further.

They have already access to:
– dbtCloud (jobs)
– Github repo (code)
– Redshift (database)
– Looker (bi + monitoring)
– Slack (communicating)

And yesterday was the Kick-off call. Overall great impressions.

What is going to happen:
– Audit: Deployment + Performance
– Audit: Project Structure
– Features/Fix's

I will keep you posted.

Airbyte Clickhouse destination

Airbyte deployed Clickhouse destination which I already use to gather data from multiple sources.

By default it replicates all the data as JSON blobs (all the attributes inside one String field)

To get it flattened you either have to do it yourself or use Airbyte normalization.

1. Flattening manually with JSON functions

JSONExtract(_airbyte_data, 'id', 'UInt64') as id

➕ Could be tricky and exhausting if you have a lot of attributes.

➖ Works extremely fast

2. Airbyte normalization (= dbt underneath 😉)

➕ It will flatten all your data automatically

Technically it is auto-generated dbt project

➖ Still a little bit buggy and looks like a work in progress.

I almost managed to get it done, but I use Yandex’ Managed Clickhouse which demands you use SSL / Secure connection.

Unfortunately, Airbyte’s dbt profiles.yml is hard-configured to secure: False at the moment.

I might create a PR to fix this when I have some time.

#airbyte #clickhouse #dbt