​​A crawler for automated functional testing of #python web applications.

Crawling a server-side-rendered web application is a low cost way to get low quality test coverage of your JavaScript-light web application.

If you have only partial test coverage of your routes, but still want to protect against silly mistakes, then this is for you.

Why should I use this? Here's an example: Flaskr, the Flask tutorial application has 166 lines of test code to achieve 100% test coverage.

Using Python Testing Crawler in a similar way to the Usage example below, we can hit 73% with very little effort. Disclaimer: Of course! It's not the same quality or utility of testing! But it is better than no tests, a complement to hand-written unit or functional tests and a useful stopgap.

https://github.com/python-testing-crawler/python-testing-crawler

Your desktop is a mess? You cannot find anything in your downloads and documents? Sorting and renaming all these files by hand is too tedious? Time to automate it once and benefit from it forever with organize!

Config example:

  - folders: ~/Downloads
    subfolders: true
    filters:
      - extension: pdf
    actions:
      - echo: "Found PDF!"
      - move: "~/Documents/pdfs"

https://github.com/tfeldmann/organize

#python

​​A next-generation curated knowledge sharing platform for data scientists and other technical professions.

The Knowledge Repo project is focused on facilitating the sharing of knowledge between data scientists and other technical roles using data formats and tools that make sense in these professions. It provides various data stores (and utilities to manage them) for "knowledge posts", with a particular focus on notebooks (R Markdown and Jupyter / IPython Notebook) to better promote reproducible research.

For more information about the motivation and inspiration behind this project, we encourage you to read our Medium Post: https://medium.com/airbnb-engineering/scaling-knowledge-at-airbnb-875d73eff091

https://github.com/airbnb/knowledge-repo

#python #ds #docops

A process for automating #docker container base image updates.

With watchtower you can update the running version of your containerized app simply by pushing a new image to the Docker Hub or your own image registry. Watchtower will pull down your new image, gracefully shut down your existing container and restart it with the same options that were used when it was deployed initially. Run the watchtower container with the following command:

 docker run -d \
    --name watchtower \
    -v /var/run/docker.sock:/var/run/docker.sock \
    containrrr/watchtower

https://github.com/containrrr/watchtower

#go #devops

Pyanalyze is a tool for programmatically detecting common mistakes in #python code, such as references to undefined variables and some categories of type mismatches. It can be extended to add additional rules and perform checks specific to particular functions.

It was opensourced just yesterday, so its still WIP.

https://github.com/quora/pyanalyze

​​Listens to changes in a PostgreSQL Database and broadcasts them over websockets.

It works like this:

1. the Phoenix server listens to PostgreSQL's replication functionality (using Postgres' logical decoding)
2. it converts the byte stream into JSON
3. it then broadcasts over websockets.

https://github.com/supabase/realtime

#elixir

​​#django gRPC framework is a toolkit for building gRPC services, inspired by djangorestframework.

https://github.com/fengsp/django-grpc-framework

#python

​​Super-fast #ts / #js compiler written in #rust

It consumes a javanoscript or typenoscript file which uses recently added features like async-await and emits javanoscript code which can be executed on old browsers. It's 20x faster than babel.

https://github.com/swc-project/swc

​​Lightweight, fault-tolerant message streams.

Liftbridge provides lightweight, fault-tolerant message streams by implementing a durable stream augmentation for the NATS messaging system. It extends NATS with a Kafka-like publish-subscribe log API that is highly available and horizontally scalable. The goal of Liftbridge is to provide a message-streaming solution with a focus on simplicity and usability. Use it as a simpler and lighter alternative to systems like Kafka and Pulsar or to add streaming semantics to an existing NATS deployment.

https://github.com/liftbridge-io/liftbridge

#go

​​The webworkers driven UI #js framework.

This is a new paradigm. If you want to enter a new era of making better Web Based User Interfaces, the following concepts will be addictive.

What if a framework & all the apps you build are running inside a separate thread (web worker)? You get extreme performance!

https://github.com/neomjs/neo

WebPipe allows you to pipe from your servers to the browser!

Just a single command to run:

docker run --rm --publish 8000:8000 hyperngn/webpipe

https://github.com/hyperngn/webpipe

#elixir

​​snakeware is a free Linux distro with a Python userspace inspired by the Commodore 64. You are booted directly into a Python interpreter, which you can use to do whatever you want with your computer.

The idea is that a #python OS would be fun to use and very easy to contribute to. Even relative beginners might be able to find ways to meaningfully contribute apps and other code to this distro.

We also are not going to be using any other huge and opaque software such as systemd, etc. The goal is to eventually have a usable set of userspace apps and utilities written entirely in Python, because Python is fun and it Just Werks.

https://github.com/joshiemoore/snakeware

​​Rich is a #python library for rendering rich text and beautiful formatting to the terminal.

The Rich API makes it easy to add colorful text (up to 16.7 million colors) with styles (bold, italic, underline etc.) to your noscript or application. Rich can also render pretty tables, progress bars, markdown, syntax highlighted source code, and tracebacks -- out of the box.

Rich works with Linux, OSX, and Windows. True color / emoji works with new Windows Terminal, classic terminal is limited to 8 colors.

https://github.com/willmcgugan/rich

​​Interactive #git with the help of fzf

Features:
- You can run git add and git reset by selecting or cursoring. You can commit interactively
- You can search the diff from the query bar and the RHS diff will be highlighted accordingly
- Search the log and corresponding diff at once. Notice that when you use | the left hand side is sent to log while the right hand side is sent to diff

https://github.com/bigH/git-fuzzy

#shell

​​How async should have been.

Your sync and async code can be identical, but still, can work differently. It is a matter of right abstractions. In this article, I will show how one can write sync code to run async programs in #python

Basically, I explain how one can use monads to abstract away technical details of your infrastructure. I also share my vision about the future of this approach and what we need to do to make this even cooler. There's a lot of work to be done!

This article also lists new features of dry-python/returns@0.14 which was released today. Try it out, it is trully awesome. With this realease I can say that we have built a nice foundation for functional programming in Python.

Github: https://github.com/dry-python/returns
Docs: https://returns.readthedocs.io

​​Penrose is a platform that enables people to create beautiful diagrams just by typing mathematical notation in plain text. The goal is to make it easy for non-experts to create and explore high-quality diagrams, providing deeper insight into challenging technical concepts. We aim to democratize the process of creating visual intuition.

https://github.com/penrose/penrose

#haskell

Here's a simple Penrose visualization in the domain of set theory:

​​kaffy: Powerfully simple admin package for phoenix applications.

Kaffy was created out of a need to have a powerfully simple, flexible, and customizable admin interface without the need to touch the current codebase. It was inspired by django’s lovely built-in admin app and rails’ powerful activeadmin gem.

https://aesmail.github.io/kaffy/

#elixir

​​A small #js library to create and animate annotations on a web page

Rough Notation uses RoughJS to create a hand-drawn look and feel. Elements can be annotated in a number of different styles. Animation duration and delay can be configured, or just turned off.

Rough Notation is about 3.2kb in size when gzipped, and the code is available on GitHub.

https://roughnotation.com/

​​Wren is a small, fast, class-based concurrent noscripting language written in #c

Think Smalltalk in a Lua-sized package with a dash of Erlang and wrapped up in a familiar, modern syntax.

- Wren is small. The VM implementation is under 4,000 semicolons. You can skim the whole thing in an afternoon. It's small, but not dense. It is readable and lovingly-commented.
- Wren is fast. A fast single-pass compiler to tight bytecode, and a compact object representation help Wren compete with other dynamic languages.
- Wren is class-based. There are lots of noscripting languages out there, but many have unusual or non-existent object models. Wren places classes front and center.
- Wren is concurrent. Lightweight fibers are core to the execution model and let you organize your program into an army of communicating coroutines.
- Wren is a noscripting language. Wren is intended for embedding in applications. It has no dependencies, a small standard library, and an easy-to-use C API. It compiles cleanly as C99, C++98 or anything later.

https://wren.io/

​​TiKV is an open-source, distributed, and transactional key-value database, originally created to complement TiDB. Unlike other traditional NoSQL systems, TiKV not only provides classical key-value APIs, but also transactional APIs with ACID compliance. Built in Rust and powered by Raft, TiKV was originally created to complement TiDB, a distributed HTAP database compatible with the MySQL protocol.

The design of TiKV ('Ti' stands for titanium) is inspired by some great distributed systems from Google, such as BigTable, Spanner, and Percolator, and some of the latest achievements in academia in recent years, such as the Raft consensus algorithm.

TiKV has the following key features:
- Geo-Replication: TiKV uses Raft and the Placement Driver to support Geo-Replication.
- Horizontal scalability: With PD and carefully designed Raft groups, TiKV excels in horizontal scalability and can easily scale to 100+ TBs of data.
- Consistent distributed transactions: Similar to Google's Spanner, TiKV supports externally-consistent distributed transactions.
- Coprocessor support: Similar to Hbase, TiKV implements a coprocessor framework to support distributed computing.
- Cooperates with TiDB: Thanks to the internal optimization, TiKV and TiDB can work together to be a compelling database solution with high horizontal scalability, externally-consistent transactions, support for RDBMS, and NoSQL design patterns.

https://github.com/tikv/tikv

#rust

​​immudb is a lightweight, high-speed immutable database for systems and applications. Written in #go. With immudb you can track changes in sensitive data in your transactional databases and then record those changes permanently in a tamperproof immudb database. This allows you to keep an indelible history of sensitive data, for example debit/credit card transactions.

Traditional DB transactions and logs are hard to scale and are mutable. So there is no way to know for sure if your data has been compromised.

As such, immudb provides unparalleled insights retroactively of changes to your sensitive data, even if your perimeter has been compromised. immudb guarantees immutability by using a Merkle tree structure internally.

immudb gives you the same cryptographic verification of the integrity of data written with SHA-256 like a classic blockchain without the cost and complexity associated with blockchains today.

- immudb is immutable. You can add records, but never change or delete records.
- Data stored in immudb is cryptographically coherent and verifiable, like blockchains, just without all the complexity and at high speed.
- Anyone can get started with immudb in minutes. Whether you're using node.js, Java, Python, Go, .Net, or any other language. It's very easy to use and you can have your immutable database running in just a few minutes.
- Finally, immudb is Open Source. You can run it on premise, or in the cloud. It's completely free. immudb is governed by the Apache 2.0 License

https://github.com/codenotary/immudb