[article]
Low-Latency, High-Throughput Garbage Collection
https://users.cecs.anu.edu.au/~steveb/pubs/papers/lxr-pldi-2022.pdf
Low-Latency, High-Throughput Garbage Collection
https://users.cecs.anu.edu.au/~steveb/pubs/papers/lxr-pldi-2022.pdf
[article]
https://www.scylladb.com/2022/04/27/shaving-40-off-googles-b-tree-implementation-with-go-generics/
https://www.scylladb.com/2022/04/27/shaving-40-off-googles-b-tree-implementation-with-go-generics/
ScyllaDB
Shaving 40% Off Google’s B-Tree Implementation with Go Generics - ScyllaDB
Go generics are a new way to improve performance in #golang. Let's see what effect generics had on Google's B-tree implementation.
[article][db] Let's build a distributed Postgres proof of concept
By the end of this post, in around 600 lines of code, we'll have a distributed "Postgres implementation" that will accept writes (
https://notes.eatonphil.com/distributed-postgres.html
By the end of this post, in around 600 lines of code, we'll have a distributed "Postgres implementation" that will accept writes (
CREATE TABLE, INSERT) on the leader and accept reads (SELECT) on any node. All nodes will contain the same data.https://notes.eatonphil.com/distributed-postgres.html
👍3
[article][programming language]
https://blog.sigplan.org/2022/05/19/language-design-in-the-real-world/
https://blog.sigplan.org/2022/05/19/language-design-in-the-real-world/
SIGPLAN Blog
Language Design in the Real World
Real programming languages are living things, changing and evolving. As with any production code, most of their designer’s time is spent on bug fixing and small improvements, rather than on the rad…
[article][db]
https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-columnar-engine
https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-columnar-engine
Google Cloud Blog
AlloyDB for PostgreSQL Columnar Engine | Google Cloud Blog
In this technical deep dive, we take a look at the columnar engine that delivers industry leading query performance for AlloyDB for PostgreSQL.
[guides]
https://github.com/acantril/learn-cantrill-io-labs/tree/master/aws-cognito-web-identity-federation
https://github.com/acantril/learn-cantrill-io-labs/tree/master/aws-cognito-web-identity-federation
GitHub
learn-cantrill-io-labs/aws-cognito-web-identity-federation at master · acantril/learn-cantrill-io-labs
Standard and Advanced Demos for learn.cantrill.io courses - acantril/learn-cantrill-io-labs
👍1
[article]
https://achievers.engineering/scaling-production-globally-service-mesh-face-lift-part-1-30ad6d393d04
https://achievers.engineering/scaling-production-globally-service-mesh-face-lift-part-1-30ad6d393d04
Medium
Scaling Production Globally — The service mesh facelift (Part-1)
Achievers has been growing rapidly over the past two years, and we have been working hard to scale microservices globally using Istio
[article][distributed]
We present a formal, machine checked TLA+ safety proof of MongoRaftReconfig, a distributed dynamic reconfiguration protocol. MongoRaftReconfig was designed for and imple- mented in MongoDB, a distributed database whose replica- tion protocol is derived from the Raft consensus algorithm. We present an inductive invariant for MongoRaftReconfig that is formalized in TLA+ and formally proved using the TLA+ proof system (TLAPS). We also present a formal TLAPS proof of two key safety properties of MongoRaftReconfig, Leader- Completeness and StateMachineSafety. To our knowledge, these are the first machine checked inductive invariant and safety proof of a dynamic reconfiguration protocol for a Raft based replication system.
https://will62794.github.io/assets/papers/cpp22-formal-verification-reconfig.pdf
We present a formal, machine checked TLA+ safety proof of MongoRaftReconfig, a distributed dynamic reconfiguration protocol. MongoRaftReconfig was designed for and imple- mented in MongoDB, a distributed database whose replica- tion protocol is derived from the Raft consensus algorithm. We present an inductive invariant for MongoRaftReconfig that is formalized in TLA+ and formally proved using the TLA+ proof system (TLAPS). We also present a formal TLAPS proof of two key safety properties of MongoRaftReconfig, Leader- Completeness and StateMachineSafety. To our knowledge, these are the first machine checked inductive invariant and safety proof of a dynamic reconfiguration protocol for a Raft based replication system.
https://will62794.github.io/assets/papers/cpp22-formal-verification-reconfig.pdf
[article]
Many organizations are shifting to a data management paradigm called the “Lakehouse,” which implements the functionality of struc- tured data warehouses on top of unstructured data lakes. This presents new challenges for query execution engines. The execu- tion engine needs to provide good performance on the raw un- curated datasets that are ubiquitous in data lakes, and excellent performance on structured data stored in popular columnar file formats like Apache Parquet. Toward these goals, we present Pho- ton, a vectorized query engine for Lakehouse environments that we developed at Databricks. Photon can outperform existing cloud data warehouses in SQL workloads, but implements a more general exe- cution framework that enables efficient processing of raw data and also enables Photon to support the Apache Spark API. We discuss the design choices we made in Photon (e.g., vectorization vs. code generation) and describe its integration with our existing SQL and Apache Spark runtimes, its task model, and its memory manager. Photon has accelerated some customer workloads by over 10× and has recently allowed Databricks to set a new audited performance record for the official 100TB TPC-DS benchmark.
https://www-cs.stanford.edu/~matei/papers/2022/sigmod_photon.pdf
Many organizations are shifting to a data management paradigm called the “Lakehouse,” which implements the functionality of struc- tured data warehouses on top of unstructured data lakes. This presents new challenges for query execution engines. The execu- tion engine needs to provide good performance on the raw un- curated datasets that are ubiquitous in data lakes, and excellent performance on structured data stored in popular columnar file formats like Apache Parquet. Toward these goals, we present Pho- ton, a vectorized query engine for Lakehouse environments that we developed at Databricks. Photon can outperform existing cloud data warehouses in SQL workloads, but implements a more general exe- cution framework that enables efficient processing of raw data and also enables Photon to support the Apache Spark API. We discuss the design choices we made in Photon (e.g., vectorization vs. code generation) and describe its integration with our existing SQL and Apache Spark runtimes, its task model, and its memory manager. Photon has accelerated some customer workloads by over 10× and has recently allowed Databricks to set a new audited performance record for the official 100TB TPC-DS benchmark.
https://www-cs.stanford.edu/~matei/papers/2022/sigmod_photon.pdf
[article]
https://github.blog/2022-06-29-improve-git-monorepo-performance-with-a-file-system-monitor/
https://github.blog/2022-06-29-improve-git-monorepo-performance-with-a-file-system-monitor/
The GitHub Blog
Improve Git monorepo performance with a file system monitor
Monorepo performance can suffer due to the sheer number of files in your working directory. Git’s new builtin file system monitor makes it easy to speed up monorepo performance.
[article][interview]
https://4dayweek.medium.com/netflix-interview-process-questions-best-practices-2022-1fdca3bb36c6
https://4dayweek.medium.com/netflix-interview-process-questions-best-practices-2022-1fdca3bb36c6
Medium
Netflix interview process, questions & best practices (2022)
Netflix is part of the FAANG (Facebook, Amazon, Apple, and Google) group of companies. Apart from these companies being part of the NASDAQ, they are also known for being even harder to get into than…
[article][sharding]
https://blog.twitter.com/engineering/en_us/topics/infrastructure/2021/sharding-simplification-and-twitters-ads-serving-platform
https://blog.twitter.com/engineering/en_us/topics/infrastructure/2021/sharding-simplification-and-twitters-ads-serving-platform
X
Sharding, simplification, and Twitter’s ads serving platform
Learn how Shardlib, a new sharding library at Twitter simplifies the management of sharded ads services and enables dynamic resharding of these services without redeploying their clients.
👍2