ShellCheck Wrapper Script for Bash Scripting

Hello r/bash,

I've written a Bash noscript that enhances the use of ShellCheck for linting shell noscripts. This is a utility aimed at those who need to ensure their noscripts adhere to best practices and are free of common errors.

Key Features:

Recursive or single-directory checking.
Verbose mode for detailed analysis.
Ability to specify ShellCheck exclusions.
Option to output results to a file.
Automatic ShellCheck installation if not present.
Moving error-free noscripts to a specified directory.
Summary report of ShellCheck results.
Color output for easier reading.

The noscript supports various configurations, allowing you to tailor the linting process to your needs, including the exclusion of specific checks and the organization of noscripts based on their linting results.

It's a straightforward tool designed to integrate with existing workflows, offering practical options for those looking to improve the quality of their Bash noscripts.

Feel free to try it and see if it fits your noscripting routine.

Example Usage:

./shellcheck.sh --color -s -d GitHub-Projects -m "$PWD/GitHub-Projects/completed"
./shellcheck.sh --recursive -v --output noscript-errors.txt

GitHub Script

https://redd.it/1bk5fu2
@r_bash

BashPitfall local -g var is Pretty Useless

[Credit: This post was inspired by what seemed to be an odd comment in another thread.\]

Can you guess what this code prints?

z=42
a() {
local z
z=27
b
echo "a/z=$z"
}
b() {
local -g z
z=79
}
a
echo "main/z=$z"

If you guessed a/z is 27 and main/z is 79, because z in b() points to the global z, I'm afraid you've been misled by the bash docs:

$ ./test.sh
a/z=79
main/z=42

This code, however, prints a/z=27 and main/z=79:

z=42
a() {
local z
z=27
b
echo "a/z=$z"
}
b() {
local -g z=79
}
a
echo "main/z=$z"

And this one complains z: unbound variable:

set -u
a() {
local z
z=27
b
echo "a/z=$z"
}
b() {
local -g z
z=79
}
a
echo "main/z=$z"

But this one, that combines the declaration and assignment of global z, doesn't:

set -u
a() {
local z
z=27
b
echo "a/z=$z"
}
b() {
local -g z=79
}
a
echo "main/z=$z"

See, the denoscription for declare (to which local is a rough twin) says this:

>The -g option forces variables to be created or modified at the global scope, even when declare is executed in a shell function. It is ignored in all other cases.

But you also have to take into account dynamic scoping, also documented in the man page:

>The shell uses dynamic scoping to control a variable's visibility within functions. With dynamic scoping, visible variables and their values are a result of the sequence of function calls that caused execution to reach the current function. The value of a variable that a function sees depends on its value within its caller, if any, whether that caller is the "global" scope or another shell function. This is also the value that a local variable declaration "shadows", and the value that is restored when the function returns.

Everyone who's used local to shadow a throwaway variable like i from another variable named i in its caller function knows this in their gut.

But what the docs don't say is that the only thing local -g var is good for is enabling functions to create/modify global variables with special declare attributes, like the -A (associative array) flag, and only if you assign a value at the same time:

set -u
a() {
local z
z=27
b
echo "a/z=$z"
}
b() {
local -gA z=(Hi=there Lo=blow)
}
a
declare -p h

yields, instead of z: unbound variable...

$ ./test.sh
a/z=27
declare -A z=(Lo="blow" Hi="there" )

but divorce the assignment from the declaration:

set -u
a() {
local z
z=27
b
echo "a/z=$z"
}
b() {
local -gA z
z=(Hi=there Lo=blow)
}
a
declare -p z

and you get this, because this b's z is actually a's z, which is clearly not an associative array:

$ ./test.sh
./test.sh: line 11: Hi: unbound variable

Confused? So was I, so I filed a bug report, which led to an interesting email chat with Chet Ramey and the other bash maintainers. The upshot:

1. NOTABUG, but we'll accept documentation suggestions to remove this confusion.
2. Yes, a local -g var declaration-without-assignment is Pretty Useless.

https://redd.it/1bk9iyb
@r_bash

Terminals are Colorful and Beautiful with Powerlevel10k
https://dly.to/eIIlIJUvevj

https://redd.it/1bl2zo4
@r_bash

Docker log monitoring noscript help

I have a noscript that's monitoring the new log entries using:

docker logs -f -n 0 <container>

The problem is that when the container restarts, the noscript stops. I was able to get around it by putting it in a while loop that checks for the container name in docker ps:

docker ps | while read line; do if [ ${line} != *"<container>"* ]; then sleep 30; ...

I added the sleep to give the container the chance to spin up, and it works, but I'm sure there's probably an easier way. I'm completely new to bash, fairly new to linux, but with a little programming experience. Is there a way to keep the noscript alive even after the container stops without it erroring out?



https://redd.it/1bltzyc
@r_bash

performance between xargs and arrays in bash? External programs

In general, how do the performance between xargs and arrays in bash compare? I don't write noscripts professionally but for personal noscripts, I tend to prefer posix when possible for being ubiquitous (even though this will probably never benefit me for home use) and whatever marginal performances there are.

But it seems arrays are mainly the deciding factor for switching to bash and I was wondering:

How performance compares between xargs in posix noscript to get array-like features vs. bash's native array support (obviously you can use xargs in bash too but that's irrelevant). Are there other reasons to use one over the other?

Somewhat related to above, is calling external program like xargs always slower than something that can be done natively in the shell? Why is this generally the case, doesn't it depend more on how it's implemented in the external program and in bash, such as the coding language it's implemented in and how well it's optimized?

Unless you handling with a ton of data (not usually the case for simple home noscripts unless you're dealing with logs or databases I assume), are there any other reasons to not simply write a noscript in the simplest way possible to quickly understand what's going on? E.g. Except in the case of logs, databases, or lots of files in the filesystem, I'm guessing you will not shave more than a second or two off execution time if you liberally pipe commands involving e.g. grep, sed, cut, column vs. a single long awk command but unless you're regularly dealing with awk the former seems preferable. I was initially set on learning enough awk to replace all those commands with just awk but now I'm having second thoughts.

I'm also wondering if there's a modern alternative to awk that might be less archaic in syntax/usage (e.g. maybe even a general programming language with libraries to do what awk can). Or perhaps awk is still worth learning in 2024 because it can do things modern applications/languages can't do as well?

https://redd.it/1bmbvy5
@r_bash

Is it better to use if or [ for simple checks?

I'm not new to bash, but I don't know most things behind the scenes of bash noscript. So, if I'm doing something simple like checking if a variable has a value, is it better to use an if statement, or \[\[? Or does it even matter?

Example:

my_var="some value"

[[ "$my_var" ] && {
: Do something
}

if [ "$my_var" ]; then
: Do something
fi

&#x200B;

https://redd.it/1bn3gu7
@r_bash

I need to have something to detect and unmount a Windows OS drive. I have some code that I wrote in order to detect it whether it's mounted or not. This works fine so far...

...can I get some help smoothing this code down to something that is just as reliable.

Here is the function. It may be a bit crude but so far it has worked without any hitches. Can I do the same in a better way? This is for a backup noscript that allows me to find a Windows OS drive whether mounted or not in order to exclude that partition from any backups that are initiated, and the only way I could think of to do that was to look for the "Microsoft reserved" flag that is attached on every drive where an OS is installed. I couldn't reliably look solely for "Microsoft basic data" or anything merely "ntfs" because of /media drives that are formatted as ntfs and don't have any associated OS. This is what I came up with to find the drive partition where the OS is installed. It has been good finding only one Windows OS installed, but I have yet to try it on more than one (one is enough).

[Edit:\] I can't do anything about the formatting. As soon as I enter the change the browser left justifies everything, removing all my white spaces.

unmount=true
FIND_WIN_PARTITION(){
unset FIND_WIN_OS FIND_WIN_REC WIN_OS_DRIVE ;
local FIND_WIN_OS FIND_WIN_REC WIN_OS_DRIVE ;
FIND_WIN_REC=$(sudo fdisk -l | grep "Microsoft reserved" | awk '{print $1}') ; # Find the reserved drive flag.
FIND_WIN_OS="${FIND_WIN_REC:5:3}" ; # removes ("/dev/") echoing the next 3 characters (sd?, nvm, dis...)
if eval sudo fdisk -l | grep "${FIND_WIN_OS}" | grep "Microsoft basic data" | awk '{print $1}' ; then
# Now we can find the Windows OS drive
WIN_OS_DRIVE="$(sudo fdisk -l | grep "${FIND_WIN_OS}" | grep "Microsoft basic data" | grep -v "/media" | awk '{print $1}')" &>/dev/null ;
# unmount, true or false
if [[ "${unmount}" == true ]] ; then
if df | grep "$WIN_OS_DRIVE" &>/dev/null ;then sudo umount -f "$WIN_OS_DRIVE" ;fi ;
fi ;
if [[ -n "${FIND_WIN_OS}" ]] ; then { echo "${FIND_WIN_OS}" ; return 0 ; } ; fi ;
else [[ -z ${FIND_WIN_REC} ]] && return 1 ;fi ;
}

https://redd.it/1bn484p
@r_bash

compgen -c

Hi,

I'm getting a bit confused with this command, I have a few machines that are configured in the same way, same bash version and distro, etc.

I've noticed that in one of those machines the output of `compgen -c` is sligthly different than on the rest, for example, commands that start with the letter k such as keepass* are grouped together, but on this machine it isn't.

Does anyone know what determines the output/ordering with this command? Does the number of installed programs affect it? I've checked things like the locale and it's all configured exactly the same.

Thanks.

https://redd.it/1bni2k2
@r_bash

Enhance ArchLinux with this Bash Wrapper for Reflector

Greetings, r/bash and /r/archlinux enthusiasts,

I'm sharing a Bash noscript designed to automate the updating of your Pacman mirrorlist, ensuring you're always fetching packages from the fastest, most reliable mirrors. This tool not only updates your mirrorlist but also offers the flexibility to create a systemd service for automatic updates at your preferred frequency.

Key Features:

Update your Pacman mirrorlist with mirrors best suited for your region.
Customize the number of mirrors to test, filter by country, protocol, and more.
Create a systemd service to automate updates on an hourly, daily, weekly, or monthly basis.
Includes options for dry runs, logging, and verbose output for detailed process insights.

The noscript requires root or sudo privileges and is designed with options to tailor its operation to your needs, such as setting the country or region code for mirror filtering, excluding specific mirrors, and adjusting the number of mirrors to test.

Usage: ./update_mirrorlist.sh [OPTIONS]

Options include:

-h, --help for displaying the help menu.

-c, --country <code> to set the country or region code.

--config <path> to use a specific configuration file.

And more, detailed within the noscript's help section.

Example Commands:

To set the service frequency to daily and test 100 mirrors: ./update_mirrorlist.sh -f daily -m 100

For a dry run using HTTP protocol and creating the systemd service: ./update_mirrorlist.sh -s -p http --dry-run

The noscript is thoroughly commented, making customization straightforward for those different levels of Bash knowledge. Feel free to adapt, improve, and share your enhancements.

Half of why I post these things is to get feedback on how my submission can be improved and/or just done plain better. You don't know what you don't know.

Cheers!

GitHub Script

Config File

https://redd.it/1bnoxqv
@r_bash

crap

alias crap to cat so you can crap out files

https://redd.it/1bnrd5a
@r_bash

grep \ sed \ awk random parameter from a line between special characters

I have a echo of "getend passwd <user>" with format:

randomUsername:<random>:<random>:<random>:randomFullname:<random>:<random>

How to get randomFullname parameter between fixed quantity of special characters like : ?

THX

https://redd.it/1bog8s4
@r_bash

rsync-based mv--remove dirs in source as they get synced?

I'm implementing an rsync-based version of mv because destination is not reliable (not using --checksum though, rsync by default compares file sizes and modification times which I assume is good enough for media files)?

Any way, with --remove-source-files it only removes files, leaving behind empty directories under the source directory. I handle this manually by checking if there are files in source dir after the rsync operation and if not, rm -r it.

However, the rsync operation is potentially very long and I would like it to behave like mv where as a file gets moved, it gets deleted (vs. the existing behavior described above where empty remaining directories do not get deleted).

Is there a way to implement this without invoking an rsync on every sub directory of source directory (so that when each sub directory gets synced, rsync ends so clean up of files can begin for that sub directory)? I imagine that would significantly slow down performance of the overall sync. I guess rync would need to support some kind of hook otherwise (not that I would expect it to, it's not inline with unix philosophy of just doing one thing well).

https://redd.it/1boex01
@r_bash

duplicate a telnet entry

I am trying to replicate something simple like this

telnet IP
#011000 (typed in manually)

I hit the enter key and the command executes

when I try to duplicate that

echo '#011000' | netcat -N IP

this does NOT work. I'm guessing because of the carriage return isn't being sent.

What am I missing to get this to work. I'm guessing the echo isn't sending an "enter" key at the end

I also tried

echo '#011000\r' | netcat -N IP

but that didn't work either.

What am I missing?

https://redd.it/1bokgip
@r_bash

Help with variables

Hi all. I am not a programmer though I have some basics of coding from some years ago. I tried writing a noscript for my computer (Linux) mostly for fun.
I don't know much of bash, but since the noscript was overall easy, I only read the very basics.
The idea of the noscript is to launch some games with MelonDS or mgba.
The command for melonDS is "melonDS path/of/the game".

The noscript:
- lists the games
-asks for input for the name of the game and stores the name in a variable (a)
- if the last three letters are "nds" then executes the command "melonDS path/$a" (where a is the name of the game)
- haven't written the rest yet

The problem is that I don't understand what I might be doing wrong, because the final result is indeed "melonDS /path/of/the game" but melonDS just launches without the actual game. When I run the exact same command from terminal, it works.
Sorry, if this is maybe a noob mistake, that's my real first noscript.

Update:code

Update 2: I noticed that the "d" variable part was wrong. Thanks for pointing out!
Now, I noticed that the name and path is not stored in the correct way. There are the '\' signs, so the final command comes out wrong.
picture

https://redd.it/1bpc37r
@r_bash

Validating input and adding a prefix before executing ansible playbook

I am creating a bash noscript that runs an ansible playbook. So far I have

cd /path/to/playbook
python3 inventory_updater.py

# Check if there are no errors from executing python noscript.
[ $? -eq 0 ] # Is this an appropriate way to check with an if condition and exit code ?

read -p "Enter store numbers separated by commas (e.g., 22345,28750): " store_numbers
    ansible-playbook update_specific_stores.yml -e "target_hostnames=$store_numbers"

The targethostnames variable accepts one or more store number. I need to do the following:

1)Make sure that each store number is five digits long and starts with number 2

Something like `$storenumber =~ ^20-9{4}$. Prompt user if incorrect.


while [ ! $store_number =~ ^2[0-9{4}$ ]]; do
read -p "Store number must be five digits starting with 2. Please enter a valid store number: " storenumber
done
```
The above validates for one user input but I don't know how to validate for several inputs separated by comma.

2) Add a "store" prefix before each store number. So it would be like store22345

I am expecting the value of "targethostnames=store22345,store28750" and so on. One input is minimum.

https://redd.it/1bpeuxc
@r_bash

What is the role of bash noscript in Machine Learning?

This is the requirement of a ML intern. Can anyone tell me what is the use cases of bash noscript in ML field?

Thanks in Advance
Qualifications

A bachelor’s, master's, or PhD (ongoing or complete) degree or equivalent from a top university, available to join for an in-office Summer Internship from May 2024.
Prior experience with training, building, and deploying models via Tensorflow/Pytorch (or similar) is mandatory.
Experience with CI/CD pipelines and MLOps for automating model deployments.
Skilled in using Linux, SQL, Git, and BASH noscripting.
Strong knowledge of Python and hands-on.

https://redd.it/1bpnvp0
@r_bash

why does /bin/bash -c 'ssh user@host' work?

I expected keyboard input for ssh running in a child shell to break, and that I would have to do something fancy with wiring the child process input/output to the parent shell's tty, but to my surprise this worked without any fanciness required? Would someone be able to explain what is happening at a low level that enables this to work? Thank you!

https://redd.it/1bptn6k
@r_bash

TIL: not all line continuations are the same

An unquoted slash `\` can be used to continue some command onto the next line:

$ echo abc \
> def
abc def

Both `||` and `&&` act like that:

$ echo abc &&
> echo def
abc
def
$ ! echo 123 ||
> echo 345
123
345

But there is something more to the last two: you can put multiple newlines OR comments in-between:

$ echo abc &&
>
> # some
> # comment
>
> echo def
abc
def

Or in a more practical code:

[[ $repack_early3 == n ]] ||
# The symlink is no longer needed
rm "$initrd_main/lib/modules/$kernel/kernel"

https://redd.it/1bq0enr
@r_bash

command help

I want to use a simple one-liner something like this:


whois $domainname | grep "Expir" | xargs $domainname = example.com

Something I can quickly change (or make into an useful alias).
I'm on the hunt for a new domain name, and want to avoid the long wall of text that comes from each whois search.


Is this possible without making it complicated and using noscript files etc?

https://redd.it/1bq8wii
@r_bash

Bash noscript help to launch docker container

I’m running Debian 12 and I have a Windows 11 docker container that I use with FreeRDP. I would like to have an “app” for lack of a better term that when clicked would start the windows container and start FreeRDP. I’ve written an easy little bash noscript that starts both the container and FreeRDP. I’m running into trouble checking to see if the windows container is already running. If it is, then just launch FreeRDP. If not, launch both.

I also need to have the noscript automatically stop the Windows container when I close out of FreeRDP.

Anyone have any ideas about the best way to do this?

https://redd.it/1bqbad2
@r_bash

I've implemented a few utilities to enumerate/disable/enable Linux input devices using Bash shell noscripts
https://gitlab.com/brlin/linux-input-utils

https://redd.it/1bqkrca
@r_bash