There will be no singularity – Telegram
There will be no singularity
1.99K subscribers
248 photos
15 videos
5 files
995 links
Smartface, technologies and decay
@antonrevyako
Download Telegram
demo.gif
24.9 MB
Запилили балалайку, которая позволяет исправлять и объяснять ошибки с помощью GPT-3.

1. Вставляем код
2. Вставляем текст ошибки
. . .
5. PROFIT11!!!

An AI-powered debugger
https://github.com/shobrook/adrenaline/

Попробовать тут
https://useadrenaline.com/playground

Обсуждение на Reddit
https://www.reddit.com/r/MachineLearning/comments/106q6m9/p_i_built_adrenaline_a_debugger_that_fixes_errors/
There will be no singularity
​Сейчас все пилят тиктоки свои БД. Помните про firebolt , да? Подписчик поделился ссылкой на проект своих друзей - unum.am Говорят, уделывает FoundationDB и все остальные noSQL. Правда, cloud only, без опенсорсов. Еще наткнулся на новую графовую базу - memgraph.com…
Remember these guys?

The guys from Unum have spent 2022 polishing their Key-Value Store, integrating SPDK, and starting an Open-Source ecosystem around it! The project UKV brings a shared compatibility layer between LevelDB, RocksDB, and their persistent and in-memory Key-Value Stores, some of which are also public.

- UKV can wrap any one of those Stores into a Standalone server with an Apache Arrow Flight RPC protocol, making it available across the network.

- Similar to Redis, with RedisGraph, RedisJSON, and RediSearch, UKV add supports for Graphs, Documents, and Vectors/Features. Unlike Redis, you still get ACID transactions and Strictly Serializable Consistency Guarantees. Document collections, for example, support JSON, BSON, MessagePack, and contents, as well as JSON-Pointer, JSON-Patch, and JSON-MergePatch RFC standards. Also, with the Embedded variant, you don't have to pay for Networking.

- UKV also brings structured bindings for high-level bindings. Python Graph bindings, for example, look like NetworkX. Python Tabular interface aims to become compatible with Pandas over time. In both cases, the query will operate on the entire dataset stored on disks, which can reach 100 TB+.

In 2023 guys are promising to work on replication and sharding, so stay tuned!

Designing the fastest ACID Key-Value Store / Ashot Vardanian (Unum cloud): https://www.youtube.com/watch?v=ybWeUf_hC7o

PS: The author is among us, you can ask questions in the comments to this post
Hey, it's been a while since we had any updates on our projects...

Just to refresh your memory, all of our projects dwh.dev, holistic.dev, and parsers.dev only work with raw SQL code, without any database connection.

But, dwh.dev now needs a database connection. We need to connect to Snowflake, dump the schema into PostgreSQL, and regularly check for schema changes.

The task does not seem to be difficult. But if it doesn't, you need to make it difficult, otherwise, it will be boring :)

So, the part that transfers data from one database to another was decided to be open-sourced.

"Anton invented the open-source Fivetran!", - shouts are heard from the auditorium.

There are many data transfer tools out there, but with a catch... They either no-code approach (Fivetran and others) or something in python.

But, we love SQL here, right?

Let's do it like Snowflake: everything in a SQL-like language!

Snowflake has this construction called PIPE:

create pipe mypipe2 as copy into mytable(c1, c2) from (select $5, $4 from @mystage);

And then logs can be taken from a special table.

The desire was to make something similar, but more universal.

Today's no details, it's all in progress :)

I want to talk about something else. Connecting to a server that'll understand and do everything we want can be done in different ways. One of them is pg-protocol.
Supporting pg-protocol opens up a lot of possibilities, but I'll talk about that later :)

Suddenly, it turned out that there's no proper implementation of pg-protocol in the js-ecosystem (yes, yes, all in typenoscript again). Had to do it ourselves...
Not like it was super complicated until we tried connecting to our server from the IDE.

It'd be cool to connect from any IDE with pg-support, right?
When connecting, the IDE makes a lot of requests: set encoding, ask for schemas, types, etc.

And without proper responses to these requests, the IDE refuses to connect.
It looks like we'll have to route these requests to a special pg-instance, filtering out calls that we don't need...
Or maybe use postgres-wasm?

Waiting for your ideas in the comments.