I used Postgres since 1995 ( [PG95] mailing list has less than 400 subscribers !) for my scientific projects and operating with numbers and english strings was smooth, but in 1996 I signed up to make a searchable database of electronic archive of "Учительская газета" (a popular newspaper for teachers community). I took this project for additional income to help me raise my kids. Our office was located in Vetoshny Lane, near the Read Square, I had a lot of fun working there with smart people.
I claimed to be an expert in Postgres and didn't see any problem with loading the archive and writing several SQL queries.
First, I quickly found that cyrillic letters transformed to something unreadable. The problem was that Postgres operated with ASCII only, which isn't surprising given its roots in English world, so practically it was impossible to use any national encodings like russians koi8-r, cp-1251, cp-866, etc. That time I already participated in internationalization of perl and search engine glimpse, so I knew that basically I need to replace 'char' definiton to 'unsigned char', add call setlocale(LC_ALL,"") and use functions from locale api. The reality is getting a bit more complicated than I anticipated, so it took a month to get a working Postgres on Slackware.
The second problem I found is that postgres works very slow with text, it was not only because of locale stuff — strcoll is 10 times slower that strcmp, but since Postgres used full scan — read every row of a table and performs slow text operation. This problem was addressed by me and Teodor in 2000, when we developed OpenFTS, contrib/intarray, which was eventually transformed to built-in full-text search (FTS), which we presented at the PostgreSQL Anniversary Meeting in Toronto, 2006.
I chose to use Postgres over Oracle in 2000, when we developed the Rambler portal (the biggest portal in Russia with millions of users), since I believed that we could do everything having sources and nice support from mailing list.
I handled the pressure of Rambler investor and top management, who wanted Oracle to pass an audit by PWC. Just for reference, we started with Postgres 6.5.2 (no WAL!) gradually adding GiST framework, contrib/intarray, OpenFTS, tsearch v1. Thanks to Rambler, all this work became eventually part of Postgres!
https://obartunov.livejournal.com/211100.html
I claimed to be an expert in Postgres and didn't see any problem with loading the archive and writing several SQL queries.
First, I quickly found that cyrillic letters transformed to something unreadable. The problem was that Postgres operated with ASCII only, which isn't surprising given its roots in English world, so practically it was impossible to use any national encodings like russians koi8-r, cp-1251, cp-866, etc. That time I already participated in internationalization of perl and search engine glimpse, so I knew that basically I need to replace 'char' definiton to 'unsigned char', add call setlocale(LC_ALL,"") and use functions from locale api. The reality is getting a bit more complicated than I anticipated, so it took a month to get a working Postgres on Slackware.
The second problem I found is that postgres works very slow with text, it was not only because of locale stuff — strcoll is 10 times slower that strcmp, but since Postgres used full scan — read every row of a table and performs slow text operation. This problem was addressed by me and Teodor in 2000, when we developed OpenFTS, contrib/intarray, which was eventually transformed to built-in full-text search (FTS), which we presented at the PostgreSQL Anniversary Meeting in Toronto, 2006.
I chose to use Postgres over Oracle in 2000, when we developed the Rambler portal (the biggest portal in Russia with millions of users), since I believed that we could do everything having sources and nice support from mailing list.
I handled the pressure of Rambler investor and top management, who wanted Oracle to pass an audit by PWC. Just for reference, we started with Postgres 6.5.2 (no WAL!) gradually adding GiST framework, contrib/intarray, OpenFTS, tsearch v1. Thanks to Rambler, all this work became eventually part of Postgres!
https://obartunov.livejournal.com/211100.html
Livejournal
My Postgres Story: Internationalization
I used Postgres since 1995 ( [PG95] mailing list has less than 400 subscribers !) for my scientific projects and operating with numbers and english strings was smooth, but in 1996 I signed up to make a searchable database of electronic archive of "Uchitelskaya…
👍3
Forwarded from HN Best Comments
Re: We have used too many levels of abstractions and n...
I think there is one interesting angle to this problem.
I am someone who grew up with the technology, as the levels of abstractions were being added. I am now benefiting from all those accumulated decades of knowledge.
As the IT / development world was changing, I had enormous privilege and comfort to learn the things at the pace they were happening. Being able to assimilate changes over long decades. Be a witness to the problems and logic behind all those new solutions. Understand how we come to have JavaScript and the browser mess we are in and so many other curious features of todays digital world.
I understand pretty much all of the layers of the computing from how CPUs achieve some of the things they are doing to bus protocols, to instructions, physical memory, low level OS internals, high level OS internals, virtual memory, userspace platform communication with OS, programming language runtimes and linking, shared libraries, IPC, networking, virtualization, etc.
The issue, as with any automation, is that new players on the scene (younger devs, devops, etc.) simply have no chance to learn the same things and go trough the same path.
For them, spending a decade working with a low level programming language before you jump into high level programming language is simply not an option.
We, people who really understand the technology that the world runs on, are a slowly dying breed. We are still here as tech leads, managers, directors, business owners. But there will be a point in time when we will go on retirement and there will be only precious few people who had perseverance to really understand all those things by diving into obscure, historical manuals.
onetimeuse92304, 3 hours ago
I think there is one interesting angle to this problem.
I am someone who grew up with the technology, as the levels of abstractions were being added. I am now benefiting from all those accumulated decades of knowledge.
As the IT / development world was changing, I had enormous privilege and comfort to learn the things at the pace they were happening. Being able to assimilate changes over long decades. Be a witness to the problems and logic behind all those new solutions. Understand how we come to have JavaScript and the browser mess we are in and so many other curious features of todays digital world.
I understand pretty much all of the layers of the computing from how CPUs achieve some of the things they are doing to bus protocols, to instructions, physical memory, low level OS internals, high level OS internals, virtual memory, userspace platform communication with OS, programming language runtimes and linking, shared libraries, IPC, networking, virtualization, etc.
The issue, as with any automation, is that new players on the scene (younger devs, devops, etc.) simply have no chance to learn the same things and go trough the same path.
For them, spending a decade working with a low level programming language before you jump into high level programming language is simply not an option.
We, people who really understand the technology that the world runs on, are a slowly dying breed. We are still here as tech leads, managers, directors, business owners. But there will be a point in time when we will go on retirement and there will be only precious few people who had perseverance to really understand all those things by diving into obscure, historical manuals.
onetimeuse92304, 3 hours ago
🫡4👍1