Linux - Reddit – Telegram
Linux - Reddit
770 subscribers
4.19K photos
207 videos
39.9K links
Stay up-to-date with everything Linux!
Content directly fetched from the subreddit just for you.

Powered by : @r_channels
Download Telegram
LVM on top of hardware RAID?

I have RAID5 disk and recently added new 2 disks to the server as RAID1 configured.

LVM is configured on the RAID5 disk and I would like to extend the logical volume to include RAID1, newly added one.

Are there any data corruption or lossing risk?

What happen if one of disk in RAID5 or RAID1 is broken?

https://redd.it/cduyjf
@r_linux
Why we don't use Docker

When you don’t have clients, you don’t have to follow the herd. You cut your own path. And that is what we do. Every day.

Technology is a solution to a problem. And our problems are quite unique. As a low-latency proprietary trading firm, we need systems that strip-away complexity and for us, Docker is not it.


**Not having clients, liberates us**

While it seems like a small detail, not having clients makes a substantial difference to how we code, compared to most companies. We trade 24 hours a day with our own capital at our own risk, in a wide range of financial products on all major financial markets globally. This means, we are only accountable to ourselves.

And our code is written exclusively for our traders. The resulting auto trading application will run on our servers, monitored by our application engineers. This is quite unique and amazing position to be in: nobody influences our choices when it comes to technology! While many companies are being pushed to use various languages or frameworks due to dependencies - or trends... -, we are not.

Which leads me to the first reason why we don't use Docker: we like consistency, and if a technology is not a perfect fit for our stack, we won't use it.

The most quoted Docker benefit, is reproducibility. Yes, our developers need production-like servers to test their code. We simply buy some more and configure them the exact same way as production.

​

**A simple, fast growing environment**

Our industry moves fast. We’re talking nano seconds. Because we offer competitive, tradable prices to market participants in all market conditions, continuously, across the world, we as engineers have to move equally fast.

We’ll discuss a new feature with our traders on Monday, have a design meeting the next day and put the feature in production at the end of the same week. There is only one way to move that fast: things need to be as simple as possible - every tiny bit of complexity that doesn't pay-off is stripped away. We always favour the simple - almost naive-version of an algorithm to an overcomplicated and over-engineered one that supposedly performs better.

This ties in nicely with our view on consistency: as our entire stack is developed in-house and uses the same set of languages and libraries, our production environment is lean, homogeneous and simple enough to not being in need of containers.

You could argue that Docker is simple to use and while this is true, for our business the added complexity of a containerisation layer between your application and the OS, by far negates all of Docker's advantages.

​

**We like fast code, running on bare metal**

Unlike other industries, we like bare metal servers and we don't use virtual machines nor containers for our trading systems. All our production code is running on Linux and bare Intel CPUs.

The reason for this choice is simple: we need to be fast, and when you want to be the fastest, virtual machines are a no-go for almost all of our applications. Like other Market Makers, we use FPGAs for the most speed-sensitive part of our business, but we still need [low latency C++](https://www.youtube.com/watch?v=NH1Tta7purM) for the code operating our trading strategies.

Over and above the FPGAs, we are also trading with full software automated trading systems on many exchanges around the world and network latency on those applications is crucial. In this perpetual fight against jitter, Docker would be the antithesis to [ultra-low latency network cards](https://solarflare.com/), userspace drivers and optimized C++.

​

**Fat binaries as a container**

Even with handcrafted C++ applications, we might hit some dependency issues: dependencies between applications, between applications and libraries, with conflicting versions. The latter is often mentioned as the reason for Docker to use a single container per application.

This problem has been effectively solved a long time ago by Linux package managers and CMake, but we didn't even need to go as far: at Optiver, we like
fat binaries!

Fat binaries is an incredibly simple, powerful (and old) way to develop and deploy binaries. Recent languages such as Golang and Rust support it as a way to containerize their noscripts instead of using Docker. By "fat binaries", I refer to a binary that has been statically linked to all its dependencies. It doesn't seem great at first glance, I know.

There are a few reasons for this choice:

* *Simplicity of deployment*: the same way a Docker container runs your application with all its dependencies in an isolated environment, a fat binary doesn't have any external dependencies (except libstdc++ and libc)
* *Safety*: when you test a binary, you are certain that you are testing the exact same code as the one that will run in production. This might seem paranoid, but [you could imagine why](https://en.wikipedia.org/wiki/Knight_Capital_Group#2012_stock_trading_disruption).
* *Performance*: static linking will always be faster: at the very least you will skip a jump through the PLT, at most you can benefit from in-lining

The main downside with fat binaries is that the binary itself is bigger - still much smaller than a Docker image... - this is not a problem for us at all, as we deploy them on our servers. Our main automated trading system is thus a binary around 150MB, 14MB stripped. Again, if the size would ever be an issue we could strip them, but so far it isn’t, so we don't.

​

**Tech fetishes are real**

Some engineers out there, get so hung-up on the technology, they forget to consider the context. For us, that is working backwards. We don’t contain our options by starting with a set container tech ;-). We consider the problem first and build from there.

This way of thinking is real, it’s called having a tech fetish, and we see it surprisingly often. Especially during interviews. And as you can probably guess, it's not our cup of tea.

​

David Gross,

Engineer at [Optiver](https://www.optiver.com/eu/en?utm_source=reddit&utm_medium=blog-post&utm_campaign=optiver-eu-tech-blog-david)

https://redd.it/cdw9om
@r_linux
Weekly Questions and Hardware Thread - July 17, 2019

Welcome to r/linux! If you're new to Linux or trying to get started this thread is for you. Get help here or as always, check out r/linuxquestions or r/linux4noobs

This megathread is for all your question needs. As we don't allow questions on r/linux outside of this megathread, please consider using r/linuxquestions or r/linux4noobs for the best solution to your problem.

Ask your hardware requests here too or try r/linuxhardware!

https://redd.it/ce7uqg
@r_linux
Discussion: the future of the operating system

It is well known that the architecture that GNU/Linux is built on is robust but old and especially POSIX feels outdated. also, some new operating systems (fuchsia, redox, haiku) are slowly emerging. there are few old but interesting OSes as well ( eg: Plan9, hurd ).

Now, what new inventions / breakthroughs / gradual upgrades do you expect in OS space? ( server / desktop / embedded etc.. ).. What features / techniques from existing research is interesting in this regard? I would like to hear a bit more about OS research....

https://redd.it/ce8j2d
@r_linux