a Mathematics for Bash - and how that relates to AI, hopefully 🥸

TLDR: I have found a Mathematical way to build custom languages for AI applications using a mix of Bash and Python. below are examples, and here is the Mathematics: https://kamangir-public.s3.ca-central-1.amazonaws.com/giza-v1/giza.pdf this is very much a wip, \~6 years old, started on raspberry pi's. I think new Mathematical ways of making sense of AI are possible. this might be one step towards them.

---

here is an example,

vanwatch ingest help
vanwatch ingest area=vancouver,count=5,gif
@download open 2024-07-01-15-40-04-67778

https://preview.redd.it/cntempy9jz9d1.png?width=1946&format=png&auto=webp&s=c7df0769fac63ff8881391ac6610ecd480e2c7cc

https://i.redd.it/bcisse4plz9d1.gif

vanwatch lives in: https://github.com/kamangir/Vancouver-Watching and is (almost*) pip-installable: https://pypi.org/project/vancouver-watching/

* almost because this is wip, see my blog for the progress report and plans: https://arash-kamangir.medium.com/%EF%B8%8F-open-ai-experiments-121-298ff881cfa7

---

here is another example,

ukrainetimemap ingest help
ukrainetimemap ingest - - open

https://preview.redd.it/key6t9nwjz9d1.png?width=1946&format=png&auto=webp&s=8d971293312169c7a556b69881000208bf112c7c

ukraine_timemap lives in https://github.com/kamangir/blue-geo it ingests the latest data on civilian harm in Ukraine from Bellingcat: https://github.com/bellingcat/ukraine-timemap as well as other things and is pip-installable: https://pypi.org/project/blue-geo/

https://preview.redd.it/xyz1a64rjz9d1.png?width=1946&format=png&auto=webp&s=8a160cbba796a8a9e5830d99409802ad82fd1e75

https://preview.redd.it/4votxeusjz9d1.png?width=989&format=png&auto=webp&s=ef4168971bd2a68239718cffc46e9a62f4a0e414

https://preview.redd.it/zjpbjrn7nz9d1.png?width=2470&format=png&auto=webp&s=8438f6db51bb665ddc13632eb80e2ca5ad45a2b4

---

more here: https://github.com/kamangir

https://redd.it/1dt6wfz
@r_bash

Why is This If-Then Not Working as Expected?

I know that this is redundant, but it will be easier for me. Can someone tell me why the pattern match is not working correctly? I am trying to match against the EXACT pattern, but so long as there is AT LEAST the pattern in the argument, it evaluates matching.

eg... I am looking for EXACTLY 00:00, but if you put f00:00, that still qualifies as matching. How can I force the pattern to match EXACTLY as shown an NOTHING additional? I hope that makes sense.

#! /bin/bash

# ..........................
# noscript to call 'at' alarm
# ..........................

timePattern="0-90-9:0-90-9"
datePattern="0-90-9\.0-90-9\.0-90-90-90-9"
usage=0

if $# -eq 0
then usage=1
elif ! [ $1 =~ $timePattern ]
then
echo; echo "!! incorrect TIME format !!"
usage=1
elif ! [ $2 =~ $datePattern ]
then
echo; echo "!! incorrect DATE format !!"
usage=1
fi

if "$usage" = "1"
then
echo; echo "USAGE: setAlarm TIME DATE"
echo; echo "where TIME = hh:mm in 24-hour format"
echo " and DATE = dd.mm.yyyy"
echo
exit
fi

# echo DISPLAY=:0.0 vlc music/alarm.mp3 | at $1 $2

echo; echo "To show active alarms, use 'atq'"
echo "To remove active alarm, use 'atrm #', where # is shown using atq"
echo

https://redd.it/1dtax34
@r_bash

Quirk while getting pwd user info in prompt

Hey all, have had a setup in my PS1 which would display the current user and group owner of the current working directory, however when going in to get a list of my epubs found that if the pathname contains spaces it will break down. Stat sees the spaces as separation for different directories, attempted different ways of referencing the working directory to stat, have been trying different combinations of qoutes and backslashes at this bs, but hey probably some more clever people in this sub.

Just for context RANDPROMPTICON is an icon choosen randomly and dircol sets the color for the folder displaying before the current working directory.

# Fix for split name PS1="\${SEPDEND}${L_SEPERATOR}${SETDARK}  \u ${RANDPROMPTICON} \${SEPD2L}${R_SEPERATOR}${SETLIGHT}  \A ${SEPD2L}${L_SEPERATOR}\${SETDARK}󰒓 \j${SEPDEND}${R_SEPERATOR}\n\ ${SEPDEND}${L_SEPERATOR}${SETDARK}"'$(dircol)'" ${SETDARK}\w${SEPD2L}${R_SEPERATOR}\${SETLIGHT}"'$(stat -c "%U" .)'"${SEPL2D}${R_SEPERATOR}\${SETDARK}"'$(stat -c "%G" .)'"${SEPDEND}${R_SEPERATOR}${SETUNDO}\n\ ${SEPDEND}${L_SEPERATOR}${SETDARK} exec${SEPDEND}${R_SEPERATOR}${SETUNDO} "fi

https://redd.it/1du5adf
@r_bash

How to highlight my bash prompt, to colorize my terminal?

I'm trying to do customization to my terminal and I would like to use some power-fonts to do so, In order to get the desired affect I want the background of my bash prompt to be highlighted the same color as my terminal with black lettering--what would I set "PS1" to?

https://redd.it/1du572v
@r_bash

Copy previous output in tmux

i have this really neat config for `foot` terminal which lets me copy previous output

file `~/.bashrc`:

command_done() {
printf '\e]133;D\e\\'
}
PROMPT_COMMAND=${PROMPT_COMMAND:+$PROMPT_COMMAND; }command_done


file `~/.config/foot/`:

[key-bindings]
pipe-command-output=[sh -c "f=$(mktemp); cat - > $f; wl-copy < $f"] Control+Shift+g

for some reason this doesn't work in tmux

i know tmux can select with `tmux prefix` + `[`, `ctrl` + `space`, `enter`

but this requires to manually select with arrow keys.

https://redd.it/1du9wwu
@r_bash

Bug in bash? (CWD changing in weird ways)

I've found an interesting issue in bash:

[\~\] $ mkdir a a/b
[\~\] $ cd a/b
[\~\] $ rm -r ../../a
[\~\] $ env -i PS1='[\\w\] $ ' bash --norc
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
[.\] $ cd ..
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
[..\] $ cd .
[..\] $ ls
directory listing for \~, although I only did `cd ..` once.

From now on, every subsequent `cd .` or `cd ""` will apply a `cd ..` operation (instead of staying in the CWD). Similarly, a `cd ..` would go up two directories (instead of one), then three, then four, etc.
What could be some reason for this?

https://redd.it/1duhm4f
@r_bash

Is this bash noscript safe ?

This was posted online and meant to be downloaded from a web server and ran like this:

curl -fsSL https://example.com/url | bash -s --

or this

curl -fsSL https://example.com/url | bash -s -- 1.1.1.1

The purpose of the noscript is to verify that your system resolver or the resolver added as a positional parameter will be accepted by spamhaus for spam verification.

I will never run a curl http://example.com/url | bash noscript. but i did download this one and checked it out.

Is it safe ? Can somehow additonal positional paramaters be added to the downloaded noscript ?

my concerns

1. It is way to convoluted
2. the positional parameter later has an @ sign prepended. This is to define the nameserver to query with dig but what does on later bothers me . line 33
3. line 47 is the main line . somehow can the positional paramaters which are now in $dns be manipulated to do other things. Then the whole reversal with awk. then it echos a variable ?

I added line numbers to the noscript. thanks

1> #!/usr/bin/env bash

2> dnsip="$1"

3> test_IP='127.0.0.4'

4> if [[ -t 1 ]]; then

5> AZ="$(tput setaf 45)"

6> RO="$(tput setaf 160)"

7> VE="$(tput setaf 40)"

8> NO="$(tput sgr0)"

9> else

10> AZ=""

11> RO=""

12> VE=""

13> NO=""

14> fi

15>

16> # Function to validate ipv4

17> validate_ipv4() {

18> if [[ -z "$1" ]]; then

19> return 1

20> fi

21> if grep -Pq '^(([1-9]?\d|1\d\d|25[0-5]|2[0-4]\d)\.){3}([1-9]?\d|1\d\d|25[0-5]|2[0-4]\d)$' <<<"$1"; then

22> return 0

23> else

24> return 1

25> fi

26> }

27> # Check whether user has used an ip as dns server to test or use the default system name server

28> if [[ -n "$dnsip" ]]; then

29> if ! validate_ipv4 "$dnsip"; then

30> echo "${RO}Error${NO}: $dnsip is not a valid ip v4"

31> exit 1

32> fi

33> dns="@$dnsip"

34> else

35> dns=''

36> current_dns="$(grep '^nameserver' /etc/resolv.conf | head -1 | awk '{print $2}')"

37> if ! validate_ipv4 "$current_dns"; then

38> echo "${RO}Error${NO}: $current_dns is not a valid ip v4"

39> exit 1

40> fi

41> fi

42>

43> # Loop to test 10 times whether dns server is blocked/ignored by Spamhaus

44> errors=0

45> for i in {01..10}; do

46> # shellcheck disable=SC2086

47> out="$(dig +short txt $dns "$(echo "$test_IP" | awk -F '.' '{OFS="."; print $4,$3,$2,$1}' 2>/dev/null | sed 's/$/\.zen\.spamhaus\.org/')" | tr -d '"')"

48> if grep -q 'Listed' <<<"$out" &>/dev/null; then

49> echo "${VE}Test $i${NO}: $out"

50> elif grep -q 'open resolver' <<<"$out" &>/dev/null; then

51> echo "${RO}Test $i${NO}: $out"

52> ((errors += 1))

53> else

54> echo "${RO}Test $i${NO}: Request ignored by Spamhaus or DNS Resolver timed out"

55> ((errors += 1))

56> fi

57> sleep .1

58> done

59>

60> # Show result

61> echo

62> if [[ $errors -gt 0 ]]; then

63> if [[ -n $dnsip ]]; then

64> echo "Result is ${RO}bad${NO}, Spamhaus is blocking/ignoring the DNS Resolver ${AZ}$dnsip${NO}"

65> exit 100

66> else

67> echo "Result is ${RO}bad${NO}, Spamhaus is blocking/ignoring your current DNS Resolver ${AZ}$current_dns{$NO}"

68> exit 100

69> fi

70> else

71> if [[ -n $dnsip ]]; then

72> echo "Result is ${VE}good${NO}, Spamhaus works fine with the DNS Resolver ${AZ}$dnsip${NO}"

73> else

74> echo "Result is ${VE}good${NO}, Spamhaus works fine with your current DNS Resolver ${AZ}$current_dns${NO}"

75> fi

76>

77> fi

https://redd.it/1duq3ic
@r_bash

Why can't we inline command exit codes when using 'return'?

Why can't we inline command usage with the return keyword to simplify function exit codes?

Something like this:

function myrandomcommand() {
return anotherrandomfunction
}

Not to get this confused with getting the command output of the function, just the exit code from it.

I ask because there have been some occasions where all I want is the exit code from a command and have to call the command and then reference $? (It's not like that's bad, but it would be cool to have something else to get the code).

Maybe like a command substitution but dedicated to retrieving the exit code like this:

function myrandomcommand() {
return @(anotherrandomfunction)
}


Has something like this already been implemented into bash and I'm just unaware of it, or is there a specific reason that this was left out? This might be too specific of an operator to be useful for everyone, so I'd understand if it was in fact left out.

https://redd.it/1duunm1
@r_bash

How can I automate these tree commands I frequently need to type out?

I would like to run:

git add .
git commit -m "Initial "commit"
git push

I got bored of typing them out each time. Can I make an alias or something like "gc" (for git commit). The commit message is always the same "Initial commit".

https://redd.it/1duw6ac
@r_bash

Is there a way I can ctrl-z a noscript without it stopping after resume?

I'm having to do processing of data using a noscript that will take a couple weeks. I would like to be able to pause the operations temporarly so that I can run other stuff as needed and then resume, but when I do this, it will finish whatever process the noscript happened to be on and then just quit.

I would like to be able to pause and resume a noscript without it doing this. Any help would be appreciated.

https://redd.it/1duxbbg
@r_bash

Bash custom completion not working correctly

I have this noscript

#!/bin/bash
fuzzycomplete() {
# Check if initcompletion is available
if declare -F initcompletion >/dev/null 2>&1; then
initcompletion || return
else
# Manual initialization for older bash versions
COMPREPLY=()
cur="${COMPWORDS[COMPCWORD]}"
prev="${COMPWORDS[COMPCWORD-1]}"
words=("${COMPWORDS[@]}")
cword="${COMPCWORD}"
fi
local IFS=$'\n'
local suggestions=($(compgen -f -- "${cur}" | /home/vrin/rustfuzzer/target/debug/rustfuzzer "${cur}"))
if "${#suggestions[@}" -eq "1" ]; then
COMPREPLY=("${suggestions0}")
else
COMPREPLY=("${cur}")
fi
}
complete -F fuzzycomplete ls cd


and it's supposed to generate best-match completions through another rust program. When i interface with the program manually, using this command


ls | /****/****/rust_fuzzer/target/debug/rust_fuzzer hash

it does output the best possible match. like it's not perfect but still, something is being printed. but when I try the same using this noscript, ie, source the noscript and then do ls <some text> and hit tab, it just takes me 4 spaces further. Won't give an output unless it matches exactly (like if i put Car it'll give me Cargo.lock, but nothing for car). can anyone help me in figuring out where this is going wrong


https://redd.it/1duw54i
@r_bash

Portable alternative to fmt/fold with multibyte chars support?

Recently, I've found out multibyte chars support in fmt/fold is a BSD thing. Sample text in Greek:

Αυτό είναι ένα παράδειγμα κειμένου στα ελληνικά. Αυτό το κείμενο χρησιμοποιείται για την επίδειξη της μορφοποίησης των γραμμών.

FreeBSD, OpenBSD, NetBSD:

> LCALL=enUS.UTF-8 fold -s -w60 < gr
Αυτό είναι ένα παράδειγμα κειμένου στα ελληνικά. Αυτό το
κείμενο χρησιμοποιείται για την επίδειξη της μορφοποίησης
των γραμμών.
> LCALL=enUS.UTF-8 fmt < gr
Αυτό είναι ένα παράδειγμα κειμένου στα ελληνικά. Αυτό το κείμενο
χρησιμοποιείται για την επίδειξη της μορφοποίησης των γραμμών.

Ubuntu:

> LCALL=enUS.UTF-8 fold -s -w60 < gr
Αυτό είναι ένα παράδειγμα
κειμένου στα ελληνικά. Αυτό το
κείμενο χρησιμοποιείται για την
επίδειξη της μορφοποίησης των
γραμμών.
> LCALL=enUS.UTF-8 fmt < gr
Αυτό είναι ένα παράδειγμα κειμένου
στα ελληνικά. Αυτό το κείμενο
χρησιμοποιείται για την επίδειξη της
μορφοποίησης των γραμμών.

Evidently, GNU fold/fmt in Ubuntu do count bytes, not chars.


Is there some portable alternative, which is not a custom awk, perl etc noscript?

https://redd.it/1dv67wl
@r_bash

What is the best and faster tool for counting lines in a file that matches a specific pattern. The text file is quite a large one about 4GB



https://redd.it/1dva3ph
@r_bash

Add command into an existing variable (curl+torsocks usage)

I have an existing variable

PREVIEW=$(curl -Ls $URL)

if the output of the variable $PREVIEW results empty (maybe because api limit is reached), I want to add torsocks before curl and then retry

what is the correct way to launch torsocks curl -Ls $URL? I've tried to eval $PREVIEW without success.

Thanks in advance.



https://redd.it/1dvffrr
@r_bash

Help with color formatting / redirection in bash wrapper function?

**TD;LR - This one is probably more involved. I have a `wrapper` function ([pastebin](https://pastebin.com/skxQJpQ2)) that works perfectly for capturing `stdout` but seems to blow up when I attempt the same tricks with `stderr`. I'm assuming I'm doing *something* wrong but have no idea what.**

A little over a week ago, I had asked a question about [redirection](https://old.reddit.com/r/bash/comments/1dnxehx/question_about_stream_redirection_file_denoscriptors/) and got some excellent answers from you guys that really helped. Since then, I've been trying to adapt what I learned there to create a more flexible wrapper function capable of the following:

- wrapping a call to some passed application + its args (e.g. `curl`, `traceroute`, some other bash function, etc)
- capturing stderr, stdout, and return code of the passed call to local variables (with the intention of being able to export these to named variables that are passed to the wrapper function - I have done this in other functions and am not worried about this part, so that's out of scope in the examples below): Solved
- allow selectively printing stderr / stdout in real time so that certain commands like `traceroute reddit.com` (progress on stdout) / `curl --no-clobber -L -A "${userAgent}" "${url}" -O "${fileName}"` (progress on stderr) / etc can still be displayed while the command is still running: Solved - mostly based on adapting [this](https://stackoverflow.com/questions/12451278/capture-stdout-to-a-variable-but-still-display-it-in-the-console)
- Preserve colors in captured variables: Solved
- Preserve colors in realtime output: partially solved (works for stdout but not for stderr)

Using u/Ulfnic 's excellent [suggestion](https://old.reddit.com/r/bash/comments/1dnxehx/question_about_stream_redirection_file_denoscriptors/la6gcuu/) as a base, I've almost got everything I want but I'm stumped by the color output I'm getting. I've been over this a dozen times and I'm not seeing anything that sticks out... but obviously it is not behaving as desired.

I'm (currently) on Fedora 39 which has

$ bash --version | head -1
GNU bash, version 5.2.26(1)-release (x86_64-redhat-linux-gnu)

The functions I am using are defined [here](https://pastebin.com/skxQJpQ2) which I have saved as `funcs.sh` and am loading using `. funcs.sh`.

The expected usages:

A) running the wrapper function with no options and passing it a command (plus args) to be executed, it will capture stderr, stdout, and return code to separate internal variables which can be acted on later. This works perfectly and its output looks like this

https://files.catbox.moe/rk02vz.png

B) running the wrapper function with the `-O` option will print stdout in realtime so commands like `traceroute` can give progress updates without waiting for the app to finish running before output is displayed. Should still do all the same things as (A) but additionally print stdout in realtime, while preserving color. This also works perfectly and its output looks like this

https://files.catbox.moe/8a7iq0.png

C) running the wrapper function with the `-E` option will print stderr in realtime so commands like `curl` can give progress updates without waiting for the app to finish running before output is displayed. Should still do all the same things as (A) but additionally print stderr in realtime, while preserving color.

This one is broken but I don't even understand why the code isn't working as expected. Its output looks like this

https://files.catbox.moe/obryvu.png

Functionally, it has a few issues:

1. It is incorrectly capturing stderr output to the local variable `outstr`.
2. The realtime printing of stderr loses all color for some reason, even though AFAICT the handling for stdout and stderr is identical
3. The local variable `errstr` loses all color formatting, despite the incorrectly assigned `outstr` preserving it.

When I run `wrapper -E realTimeStderrTest` (e.g. the un-colorized version of the same test), it works perfectly (issue \#1 does

not happen but issues \#2 and \#3 aren't applicable in black and white mode) so I am assuming it is something related to colors that it doesn't like but I have no clue what exactly. That output is [here](https://files.catbox.moe/zqnae9.png)

https://redd.it/1dvmij1
@r_bash

why are mode bits represented in bash in both octally? AND symbolically? or alphabetically?

so i understand that file permissions in mode bits are represented alphabetically, or symbolically?

r = read

w = write

x = execute

now if i want to change this with chmod, i need to do this with octals?

x = 1

w = w

r = 4

can someone explain this to me? why are there two systems?

https://redd.it/1dw0hgq
@r_bash

Displaying stdout from continuously running program and run command if string present

Hi, I have a noscript that runs in a terminal window, and I need to see the displayed stdout from a program that it launches, which continues running. But I also need to monitor the program's stdout and run a command if a string eventually appears in the output. Once that condition is met then I don't want to see the terminal anymore so I kill the terminal, but the program keeps running until I exit its window. I would prefer to not have to write the stdout to a file for parsing. This is as close as I can get, but it doesn't show the program's output. Any tips? Thanks!

#!/bin/bash
thisPID="$(echo $$)"
docker container start Something
nohup xfreerdp /v:localhost |
grep --line-buffered 'PDUTYPEDATA' |
while read; do
wmctrl -c 'FreeRDP' -b toggle,maximizedvert,maximizedhorz;
kill $thisPID
done

https://redd.it/1dw291c
@r_bash

Text pasted to a file not parsed correctly (invisible characters)

I have a noscript which parses text on a line-by-line basis. Works as expected, except for text pasted from a particular webpage that involves both English text and foreign characters (cannot share more details than that), where it seems to parse every *other* line by the shell. However, if I `echo` *instead* of running `yt-dlp` on what's parsed, it still prints expected values of `$noscript` and `$url`. I was told there are invisible characters involved like `$'\r'` byte (0x0d) which bash does not consider as whitespaces to ignore, but how can I see if this is the case on a text editor (I use Neovim) and/or fix the cause of the issue? If I append a character manually to each line that was pasted, the noscript then parses each line as you would expect and is a workaround for the issue.


file="$2"

while IFS= read -r line || [[ -n "$line" ]]; do
noscript="${line#* }"
url="${line%% *}"

if [[ -n "$noscript" ]]; then
default_template="$noscript - %(uploader).12B (%(upload_date)s) %(id)s.%(ext)s"
else
default_template="%(noscript).140B - %(uploader).12B (%(upload_date)s) %(id)s.%(ext)s"
fi

yt-dlp --output "$default_template" "$url"
# echo "$url"

done < <( awk NF "$file" ) # Delete blank lines from $file

exit

https://redd.it/1dw8l6e
@r_bash

Is there any sense in quoting special vars like $? and $# ?

I mean, bash and other shells are aware $? and $# cant contain any spaces or patterns, so I guess they treat $? and "$?" the same? Or do they still try to perform word splitting on $? ?

https://redd.it/1dwq3no
@r_bash

Trying to send multiple flags to rsync

So I use rsync over ssh to move files over my local network. I'm not worried about security too much, but use rsync over ssh so I can do it over internet sporadically.

This is what works:

export DEN=username@den.local
export USER=/home/kitchen
rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -rptuv --delete --progress $DEN:/home/username/Music/English/A/ $USER/Music/Music/A/

I am trying to put all the flags in a variable.

The following variable doesn't work

export RSYNCFLAGS="-e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -rptuv --delete --progress"
rsync $RSYNCFLAGS $DEN:/home/username/Music/English/A/ $USER/Music/Music/A

I also tried using a variable array, but that didn't work as expected:

export RSYNCFLAGS=(-e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -rptuv --delete --progress)
rsync ${RSYNCFLAGS*} $DEN:/home/username/Music/English/A/ $USER/Music/Music/A

They both have problems with the -e at the beginning (it doesn't add it to the variable at all). When I move that later on, it still gives a problem. Can anyone help me out?

https://redd.it/1dwquoi
@r_bash