Beginner-friendly bash noscripting tutorial

There are plenty of excellent bash noscripting tutorial videos, so I thought one more is not going to hurt.

I've put together a beginner practical tutorial video, building a sample noscript and explaining the concepts along the way. https://www.youtube.com/watch?v=kFovBYgtEuI

The idea is to take you from 0 to 60 with creating your own noscripts. The video doesn't aim to explain all the concepts, but just enough of the important ones to get you started.

https://redd.it/1ds25n3
@r_bash

Share your $PS1 prompt config

Intrested how people use prompts to get most of it.

https://redd.it/1ds7dkv
@r_bash

Can someone help me reed bash even if it's a screenshot?



https://redd.it/1dsian6
@r_bash

New To Bash Scripting

I am an aspiring devOps Engineer and I have been using Linux for sometime now. I am currently in a BootCamp that just give tasks and asks students to go find solutions to it within a specific deadline.

I was tasked to write a bash noscript that does the following:

1. Creates Users and Groups of random users.

2. And also sets up home dirctories with appropriate permissions and ownership, generate random passwords for the users.

3. And also log all actions to the /var/log/user_management.log

4. And also store the generated passwords securely in /var/secure/user_passwords.txt

5. Ensure error handling for scenarios like existing users and provide clear documentation and comments within the noscript.

I am still new to this.

My question is do any one has any material or links to where I can learn quickly and do this task?

I can't find good materials or course within the short period given to me to submit the task.

https://redd.it/1dsk19t
@r_bash

VERY new to this, why is my directory '/' and not '~' when I run git bash?

As the noscript says, I am very new to this. I did a codecademy course learning the command line just yesterday, in that course, it says multiple times that in Git Bash, I would start in my '\~' (home) directory, but I actually start in the directory '/' (which is C:/Program Files/Git). I do however start in my home directory when I run Git Bash as an admin.

I'm a bit unsure as to why I start here,if it matters that I do start there, and how this effects my bash profile.

If someone could ELI5, that would be amazing.

https://redd.it/1dsvckd
@r_bash

Script Text Manipulation

I'm stumped on this one. I'm unsure how to approach taking the output from this command and put it into a list due to the formatting.

Command:
sudo so-elasticsearch-query _cat/shards | grep UN

Output:
.ds-metrics-elasticagent.filebeatinput-default-2024.06.27-000001 0 r UNASSIGNED                                 
.ds-metrics-windows.perfmon-default-2024.06.28-000002              0 r UNASSIGNED                                 
.ds-metrics-system.core-default-2024.06.27-000001                  0 r UNASSIGNED                                 
.ds-metrics-system.process-default-2024.06.27-000001               0 r UNASSIGNED                                 
.ds-metrics-system.fsstat-default-2024.06.27-000001                0 r UNASSIGNED                                 
.ds-metrics-system.memory-default-2024.06.27-000001                0 r UNASSIGNED                                 
.ds-metrics-elasticagent.filebeat-default-2024.06.27-000001       0 r UNASSIGNED                                 
.ds-metrics-system.network-default-2024.06.27-000001               0 r UNASSIGNED                                 
.ds-metrics-system.load-default-2024.06.27-000001                  0 r UNASSIGNED                                 
.ds-metrics-system.filesystem-default-2024.06.27-000001            0 r UNASSIGNED                                 
.ds-metrics-elasticagent.elasticagent-default-2024.06.27-000001  0 r UNASSIGNED                                 
.ds-metrics-system.diskio-default-2024.06.27-000001                0 r UNASSIGNED                                 
.ds-metrics-windows.service-default-2024.06.27-000001              0 r UNASSIGNED                                 
.ds-metrics-system.uptime-default-2024.06.27-000001                0 r UNASSIGNED                                 
.ds-metrics-elasticagent.metricbeat-default-2024.06.27-000001     0 r UNASSIGNED                                 
.ds-metrics-windows.perfmon-default-2024.06.27-000001              0 r UNASSIGNED                                 
.ds-metrics-system.process.summary-default-2024.06.27-000001       0 r UNASSIGNED                                 
.ds-metrics-system.cpu-default-2024.06.27-000001                   0 r UNASSIGNED                                 
.ds-metrics-elasticagent.osquerybeat-default-2024.06.27-000001    0 r UNASSIGNED                                 
.ds-metrics-system.socketsummary-default-2024.06.27-000001        0 r UNASSIGNED

As you can see, this is in an odd tabular output that makes it difficult to assign the filename to a variable (it can go to a file, too, I haven't decided yet).

Follow-up command uses the $index variable as a placeholder for the filenames. My goal is to automate this so that any of my techs can run this noscript without issue.

sudo so-elasticsearch-query $index/_settings -d '{"number_of_replicas":0}' -XPUT

How do I manipulate the output so I can use it?

https://redd.it/1dswlye
@r_bash

a Mathematics for Bash - and how that relates to AI, hopefully 🥸

TLDR: I have found a Mathematical way to build custom languages for AI applications using a mix of Bash and Python. below are examples, and here is the Mathematics: https://kamangir-public.s3.ca-central-1.amazonaws.com/giza-v1/giza.pdf this is very much a wip, \~6 years old, started on raspberry pi's. I think new Mathematical ways of making sense of AI are possible. this might be one step towards them.

---

here is an example,

vanwatch ingest help
vanwatch ingest area=vancouver,count=5,gif
@download open 2024-07-01-15-40-04-67778

https://preview.redd.it/cntempy9jz9d1.png?width=1946&format=png&auto=webp&s=c7df0769fac63ff8881391ac6610ecd480e2c7cc

https://i.redd.it/bcisse4plz9d1.gif

vanwatch lives in: https://github.com/kamangir/Vancouver-Watching and is (almost*) pip-installable: https://pypi.org/project/vancouver-watching/

* almost because this is wip, see my blog for the progress report and plans: https://arash-kamangir.medium.com/%EF%B8%8F-open-ai-experiments-121-298ff881cfa7

---

here is another example,

ukrainetimemap ingest help
ukrainetimemap ingest - - open

https://preview.redd.it/key6t9nwjz9d1.png?width=1946&format=png&auto=webp&s=8d971293312169c7a556b69881000208bf112c7c

ukraine_timemap lives in https://github.com/kamangir/blue-geo it ingests the latest data on civilian harm in Ukraine from Bellingcat: https://github.com/bellingcat/ukraine-timemap as well as other things and is pip-installable: https://pypi.org/project/blue-geo/

https://preview.redd.it/xyz1a64rjz9d1.png?width=1946&format=png&auto=webp&s=8a160cbba796a8a9e5830d99409802ad82fd1e75

https://preview.redd.it/4votxeusjz9d1.png?width=989&format=png&auto=webp&s=ef4168971bd2a68239718cffc46e9a62f4a0e414

https://preview.redd.it/zjpbjrn7nz9d1.png?width=2470&format=png&auto=webp&s=8438f6db51bb665ddc13632eb80e2ca5ad45a2b4

---

more here: https://github.com/kamangir

https://redd.it/1dt6wfz
@r_bash

Why is This If-Then Not Working as Expected?

I know that this is redundant, but it will be easier for me. Can someone tell me why the pattern match is not working correctly? I am trying to match against the EXACT pattern, but so long as there is AT LEAST the pattern in the argument, it evaluates matching.

eg... I am looking for EXACTLY 00:00, but if you put f00:00, that still qualifies as matching. How can I force the pattern to match EXACTLY as shown an NOTHING additional? I hope that makes sense.

#! /bin/bash

# ..........................
# noscript to call 'at' alarm
# ..........................

timePattern="0-90-9:0-90-9"
datePattern="0-90-9\.0-90-9\.0-90-90-90-9"
usage=0

if $# -eq 0
then usage=1
elif ! [ $1 =~ $timePattern ]
then
echo; echo "!! incorrect TIME format !!"
usage=1
elif ! [ $2 =~ $datePattern ]
then
echo; echo "!! incorrect DATE format !!"
usage=1
fi

if "$usage" = "1"
then
echo; echo "USAGE: setAlarm TIME DATE"
echo; echo "where TIME = hh:mm in 24-hour format"
echo " and DATE = dd.mm.yyyy"
echo
exit
fi

# echo DISPLAY=:0.0 vlc music/alarm.mp3 | at $1 $2

echo; echo "To show active alarms, use 'atq'"
echo "To remove active alarm, use 'atrm #', where # is shown using atq"
echo

https://redd.it/1dtax34
@r_bash

Quirk while getting pwd user info in prompt

Hey all, have had a setup in my PS1 which would display the current user and group owner of the current working directory, however when going in to get a list of my epubs found that if the pathname contains spaces it will break down. Stat sees the spaces as separation for different directories, attempted different ways of referencing the working directory to stat, have been trying different combinations of qoutes and backslashes at this bs, but hey probably some more clever people in this sub.

Just for context RANDPROMPTICON is an icon choosen randomly and dircol sets the color for the folder displaying before the current working directory.

# Fix for split name PS1="\${SEPDEND}${L_SEPERATOR}${SETDARK}  \u ${RANDPROMPTICON} \${SEPD2L}${R_SEPERATOR}${SETLIGHT}  \A ${SEPD2L}${L_SEPERATOR}\${SETDARK}󰒓 \j${SEPDEND}${R_SEPERATOR}\n\ ${SEPDEND}${L_SEPERATOR}${SETDARK}"'$(dircol)'" ${SETDARK}\w${SEPD2L}${R_SEPERATOR}\${SETLIGHT}"'$(stat -c "%U" .)'"${SEPL2D}${R_SEPERATOR}\${SETDARK}"'$(stat -c "%G" .)'"${SEPDEND}${R_SEPERATOR}${SETUNDO}\n\ ${SEPDEND}${L_SEPERATOR}${SETDARK} exec${SEPDEND}${R_SEPERATOR}${SETUNDO} "fi

https://redd.it/1du5adf
@r_bash

How to highlight my bash prompt, to colorize my terminal?

I'm trying to do customization to my terminal and I would like to use some power-fonts to do so, In order to get the desired affect I want the background of my bash prompt to be highlighted the same color as my terminal with black lettering--what would I set "PS1" to?

https://redd.it/1du572v
@r_bash

Copy previous output in tmux

i have this really neat config for `foot` terminal which lets me copy previous output

file `~/.bashrc`:

command_done() {
printf '\e]133;D\e\\'
}
PROMPT_COMMAND=${PROMPT_COMMAND:+$PROMPT_COMMAND; }command_done


file `~/.config/foot/`:

[key-bindings]
pipe-command-output=[sh -c "f=$(mktemp); cat - > $f; wl-copy < $f"] Control+Shift+g

for some reason this doesn't work in tmux

i know tmux can select with `tmux prefix` + `[`, `ctrl` + `space`, `enter`

but this requires to manually select with arrow keys.

https://redd.it/1du9wwu
@r_bash

Bug in bash? (CWD changing in weird ways)

I've found an interesting issue in bash:

[\~\] $ mkdir a a/b
[\~\] $ cd a/b
[\~\] $ rm -r ../../a
[\~\] $ env -i PS1='[\\w\] $ ' bash --norc
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
[.\] $ cd ..
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
[..\] $ cd .
[..\] $ ls
directory listing for \~, although I only did `cd ..` once.

From now on, every subsequent `cd .` or `cd ""` will apply a `cd ..` operation (instead of staying in the CWD). Similarly, a `cd ..` would go up two directories (instead of one), then three, then four, etc.
What could be some reason for this?

https://redd.it/1duhm4f
@r_bash

Is this bash noscript safe ?

This was posted online and meant to be downloaded from a web server and ran like this:

curl -fsSL https://example.com/url | bash -s --

or this

curl -fsSL https://example.com/url | bash -s -- 1.1.1.1

The purpose of the noscript is to verify that your system resolver or the resolver added as a positional parameter will be accepted by spamhaus for spam verification.

I will never run a curl http://example.com/url | bash noscript. but i did download this one and checked it out.

Is it safe ? Can somehow additonal positional paramaters be added to the downloaded noscript ?

my concerns

1. It is way to convoluted
2. the positional parameter later has an @ sign prepended. This is to define the nameserver to query with dig but what does on later bothers me . line 33
3. line 47 is the main line . somehow can the positional paramaters which are now in $dns be manipulated to do other things. Then the whole reversal with awk. then it echos a variable ?

I added line numbers to the noscript. thanks

1> #!/usr/bin/env bash

2> dnsip="$1"

3> test_IP='127.0.0.4'

4> if [[ -t 1 ]]; then

5> AZ="$(tput setaf 45)"

6> RO="$(tput setaf 160)"

7> VE="$(tput setaf 40)"

8> NO="$(tput sgr0)"

9> else

10> AZ=""

11> RO=""

12> VE=""

13> NO=""

14> fi

15>

16> # Function to validate ipv4

17> validate_ipv4() {

18> if [[ -z "$1" ]]; then

19> return 1

20> fi

21> if grep -Pq '^(([1-9]?\d|1\d\d|25[0-5]|2[0-4]\d)\.){3}([1-9]?\d|1\d\d|25[0-5]|2[0-4]\d)$' <<<"$1"; then

22> return 0

23> else

24> return 1

25> fi

26> }

27> # Check whether user has used an ip as dns server to test or use the default system name server

28> if [[ -n "$dnsip" ]]; then

29> if ! validate_ipv4 "$dnsip"; then

30> echo "${RO}Error${NO}: $dnsip is not a valid ip v4"

31> exit 1

32> fi

33> dns="@$dnsip"

34> else

35> dns=''

36> current_dns="$(grep '^nameserver' /etc/resolv.conf | head -1 | awk '{print $2}')"

37> if ! validate_ipv4 "$current_dns"; then

38> echo "${RO}Error${NO}: $current_dns is not a valid ip v4"

39> exit 1

40> fi

41> fi

42>

43> # Loop to test 10 times whether dns server is blocked/ignored by Spamhaus

44> errors=0

45> for i in {01..10}; do

46> # shellcheck disable=SC2086

47> out="$(dig +short txt $dns "$(echo "$test_IP" | awk -F '.' '{OFS="."; print $4,$3,$2,$1}' 2>/dev/null | sed 's/$/\.zen\.spamhaus\.org/')" | tr -d '"')"

48> if grep -q 'Listed' <<<"$out" &>/dev/null; then

49> echo "${VE}Test $i${NO}: $out"

50> elif grep -q 'open resolver' <<<"$out" &>/dev/null; then

51> echo "${RO}Test $i${NO}: $out"

52> ((errors += 1))

53> else

54> echo "${RO}Test $i${NO}: Request ignored by Spamhaus or DNS Resolver timed out"

55> ((errors += 1))

56> fi

57> sleep .1

58> done

59>

60> # Show result

61> echo

62> if [[ $errors -gt 0 ]]; then

63> if [[ -n $dnsip ]]; then

64> echo "Result is ${RO}bad${NO}, Spamhaus is blocking/ignoring the DNS Resolver ${AZ}$dnsip${NO}"

65> exit 100

66> else

67> echo "Result is ${RO}bad${NO}, Spamhaus is blocking/ignoring your current DNS Resolver ${AZ}$current_dns{$NO}"

68> exit 100

69> fi

70> else

71> if [[ -n $dnsip ]]; then

72> echo "Result is ${VE}good${NO}, Spamhaus works fine with the DNS Resolver ${AZ}$dnsip${NO}"

73> else

74> echo "Result is ${VE}good${NO}, Spamhaus works fine with your current DNS Resolver ${AZ}$current_dns${NO}"

75> fi

76>

77> fi

https://redd.it/1duq3ic
@r_bash

Why can't we inline command exit codes when using 'return'?

Why can't we inline command usage with the return keyword to simplify function exit codes?

Something like this:

function myrandomcommand() {
return anotherrandomfunction
}

Not to get this confused with getting the command output of the function, just the exit code from it.

I ask because there have been some occasions where all I want is the exit code from a command and have to call the command and then reference $? (It's not like that's bad, but it would be cool to have something else to get the code).

Maybe like a command substitution but dedicated to retrieving the exit code like this:

function myrandomcommand() {
return @(anotherrandomfunction)
}


Has something like this already been implemented into bash and I'm just unaware of it, or is there a specific reason that this was left out? This might be too specific of an operator to be useful for everyone, so I'd understand if it was in fact left out.

https://redd.it/1duunm1
@r_bash

How can I automate these tree commands I frequently need to type out?

I would like to run:

git add .
git commit -m "Initial "commit"
git push

I got bored of typing them out each time. Can I make an alias or something like "gc" (for git commit). The commit message is always the same "Initial commit".

https://redd.it/1duw6ac
@r_bash

Is there a way I can ctrl-z a noscript without it stopping after resume?

I'm having to do processing of data using a noscript that will take a couple weeks. I would like to be able to pause the operations temporarly so that I can run other stuff as needed and then resume, but when I do this, it will finish whatever process the noscript happened to be on and then just quit.

I would like to be able to pause and resume a noscript without it doing this. Any help would be appreciated.

https://redd.it/1duxbbg
@r_bash

Bash custom completion not working correctly

I have this noscript

#!/bin/bash
fuzzycomplete() {
# Check if initcompletion is available
if declare -F initcompletion >/dev/null 2>&1; then
initcompletion || return
else
# Manual initialization for older bash versions
COMPREPLY=()
cur="${COMPWORDS[COMPCWORD]}"
prev="${COMPWORDS[COMPCWORD-1]}"
words=("${COMPWORDS[@]}")
cword="${COMPCWORD}"
fi
local IFS=$'\n'
local suggestions=($(compgen -f -- "${cur}" | /home/vrin/rustfuzzer/target/debug/rustfuzzer "${cur}"))
if "${#suggestions[@}" -eq "1" ]; then
COMPREPLY=("${suggestions0}")
else
COMPREPLY=("${cur}")
fi
}
complete -F fuzzycomplete ls cd


and it's supposed to generate best-match completions through another rust program. When i interface with the program manually, using this command


ls | /****/****/rust_fuzzer/target/debug/rust_fuzzer hash

it does output the best possible match. like it's not perfect but still, something is being printed. but when I try the same using this noscript, ie, source the noscript and then do ls <some text> and hit tab, it just takes me 4 spaces further. Won't give an output unless it matches exactly (like if i put Car it'll give me Cargo.lock, but nothing for car). can anyone help me in figuring out where this is going wrong


https://redd.it/1duw54i
@r_bash

Portable alternative to fmt/fold with multibyte chars support?

Recently, I've found out multibyte chars support in fmt/fold is a BSD thing. Sample text in Greek:

Αυτό είναι ένα παράδειγμα κειμένου στα ελληνικά. Αυτό το κείμενο χρησιμοποιείται για την επίδειξη της μορφοποίησης των γραμμών.

FreeBSD, OpenBSD, NetBSD:

> LCALL=enUS.UTF-8 fold -s -w60 < gr
Αυτό είναι ένα παράδειγμα κειμένου στα ελληνικά. Αυτό το
κείμενο χρησιμοποιείται για την επίδειξη της μορφοποίησης
των γραμμών.
> LCALL=enUS.UTF-8 fmt < gr
Αυτό είναι ένα παράδειγμα κειμένου στα ελληνικά. Αυτό το κείμενο
χρησιμοποιείται για την επίδειξη της μορφοποίησης των γραμμών.

Ubuntu:

> LCALL=enUS.UTF-8 fold -s -w60 < gr
Αυτό είναι ένα παράδειγμα
κειμένου στα ελληνικά. Αυτό το
κείμενο χρησιμοποιείται για την
επίδειξη της μορφοποίησης των
γραμμών.
> LCALL=enUS.UTF-8 fmt < gr
Αυτό είναι ένα παράδειγμα κειμένου
στα ελληνικά. Αυτό το κείμενο
χρησιμοποιείται για την επίδειξη της
μορφοποίησης των γραμμών.

Evidently, GNU fold/fmt in Ubuntu do count bytes, not chars.


Is there some portable alternative, which is not a custom awk, perl etc noscript?

https://redd.it/1dv67wl
@r_bash

What is the best and faster tool for counting lines in a file that matches a specific pattern. The text file is quite a large one about 4GB



https://redd.it/1dva3ph
@r_bash

Add command into an existing variable (curl+torsocks usage)

I have an existing variable

PREVIEW=$(curl -Ls $URL)

if the output of the variable $PREVIEW results empty (maybe because api limit is reached), I want to add torsocks before curl and then retry

what is the correct way to launch torsocks curl -Ls $URL? I've tried to eval $PREVIEW without success.

Thanks in advance.



https://redd.it/1dvffrr
@r_bash

Help with color formatting / redirection in bash wrapper function?

**TD;LR - This one is probably more involved. I have a `wrapper` function ([pastebin](https://pastebin.com/skxQJpQ2)) that works perfectly for capturing `stdout` but seems to blow up when I attempt the same tricks with `stderr`. I'm assuming I'm doing *something* wrong but have no idea what.**

A little over a week ago, I had asked a question about [redirection](https://old.reddit.com/r/bash/comments/1dnxehx/question_about_stream_redirection_file_denoscriptors/) and got some excellent answers from you guys that really helped. Since then, I've been trying to adapt what I learned there to create a more flexible wrapper function capable of the following:

- wrapping a call to some passed application + its args (e.g. `curl`, `traceroute`, some other bash function, etc)
- capturing stderr, stdout, and return code of the passed call to local variables (with the intention of being able to export these to named variables that are passed to the wrapper function - I have done this in other functions and am not worried about this part, so that's out of scope in the examples below): Solved
- allow selectively printing stderr / stdout in real time so that certain commands like `traceroute reddit.com` (progress on stdout) / `curl --no-clobber -L -A "${userAgent}" "${url}" -O "${fileName}"` (progress on stderr) / etc can still be displayed while the command is still running: Solved - mostly based on adapting [this](https://stackoverflow.com/questions/12451278/capture-stdout-to-a-variable-but-still-display-it-in-the-console)
- Preserve colors in captured variables: Solved
- Preserve colors in realtime output: partially solved (works for stdout but not for stderr)

Using u/Ulfnic 's excellent [suggestion](https://old.reddit.com/r/bash/comments/1dnxehx/question_about_stream_redirection_file_denoscriptors/la6gcuu/) as a base, I've almost got everything I want but I'm stumped by the color output I'm getting. I've been over this a dozen times and I'm not seeing anything that sticks out... but obviously it is not behaving as desired.

I'm (currently) on Fedora 39 which has

$ bash --version | head -1
GNU bash, version 5.2.26(1)-release (x86_64-redhat-linux-gnu)

The functions I am using are defined [here](https://pastebin.com/skxQJpQ2) which I have saved as `funcs.sh` and am loading using `. funcs.sh`.

The expected usages:

A) running the wrapper function with no options and passing it a command (plus args) to be executed, it will capture stderr, stdout, and return code to separate internal variables which can be acted on later. This works perfectly and its output looks like this

https://files.catbox.moe/rk02vz.png

B) running the wrapper function with the `-O` option will print stdout in realtime so commands like `traceroute` can give progress updates without waiting for the app to finish running before output is displayed. Should still do all the same things as (A) but additionally print stdout in realtime, while preserving color. This also works perfectly and its output looks like this

https://files.catbox.moe/8a7iq0.png

C) running the wrapper function with the `-E` option will print stderr in realtime so commands like `curl` can give progress updates without waiting for the app to finish running before output is displayed. Should still do all the same things as (A) but additionally print stderr in realtime, while preserving color.

This one is broken but I don't even understand why the code isn't working as expected. Its output looks like this

https://files.catbox.moe/obryvu.png

Functionally, it has a few issues:

1. It is incorrectly capturing stderr output to the local variable `outstr`.
2. The realtime printing of stderr loses all color for some reason, even though AFAICT the handling for stdout and stderr is identical
3. The local variable `errstr` loses all color formatting, despite the incorrectly assigned `outstr` preserving it.

When I run `wrapper -E realTimeStderrTest` (e.g. the un-colorized version of the same test), it works perfectly (issue \#1 does