How to filter files with glob patterns?

Lets say i have these files in dir directory.

file1.jar
file2.jar
file3suffix.jar
file4.txt
...
fileX.someotherextension

I need all `*.jar` files execpt those ended with `suffix.jar. I've read this [https://www.gnu.org/software/bash/manual/html\_node/Pattern-Matching.html#Pattern-Matching](https://www.gnu.org/software/bash/manual/html_node/Pattern-Matching.html#Pattern-Matching)

... and:

* ls dir/.jar` gives me all jars
ls dir/!(_suffix.jar) gives me good jars but also files with other extensions

Preferably i would somehow merge these 2 together.

​

https://redd.it/16srr4t
@r_bash

rm function does not work

I am trying to delete some text files but I am getting this error :

rm: /path/to/files/* No such file or directory

This is the actual command :

rm -r $PATHTOFILES”*”

Used * wildcard because my usecase requires to all files deleted regardless of file extension.

I am running an airflow DAG to call a shell noscript that delete all files from a local directory, then pull files from an SFTP server and place them within the same directory. Therefore it is the same airflow user creating the directory and placing the files.

Would appreciate any advice, thank you!

https://redd.it/16thf8z
@r_bash

How to run 2x readarray ... < <(...) in parallel ?

Hi folks,

readarray -d '' a < <(my_function)
readarray -d '' b < <(my_function)

Is it possible to run the above simultaneously?

my_function runs complicated find ... -print0.

Thank you.

https://redd.it/16ti9pr
@r_bash

Hi, I'm sharing The dotfiles manager+ written in pure bash
https://github.com/yunielrc/ydf

https://redd.it/16tr45b
@r_bash

Combined output of commands into variable

I know I can do

value=$(echo "FOO"; echo "BAR")

to get the combined output of the command group and set it to the variable. AFAIU, the commands will be run in a separate shell. Now, I was wondering whether it would be possible to achieve the same thing without starting a new shell, but I couldn't find the right syntax. The closest I got was

value=$({ echo "FOO"; echo "BAR"; })

but I suppose this is not what I wanted. This will run the command group locally in a new shell. The goal was to prevent the creation a new shell in the first place.

Any ideas?

Thanks.

https://redd.it/16tuegx
@r_bash

ndarray: tools for setting up and using N-dimensional / nested arrays in bash

A recent post here inspired me to pick up an old personal project for getting bash to work with N-dimensional/nested arrays. I got it working, so I figured id share it.

The [CODE](https://github.com/jkool702/bashndarray/blob/main/ndarray.bash) is on github. There are 5 functions:

`nd_usage` gives a brief usage example
nd_create sets up the nameref framework and declares the arrays
`nd_set` writes data into the arrays at the end on the namerefs (the `A_0` and `A_1` arrays in the simple example above)
nd_get reads data out of the arrays. You can define lists/ranges on indices for any dimension and it will output all the data that falls into the n-dimensional slice of the array.
nd_clear unsets all the array and nameref variables

METHODOLOGY

It involves creating a framework of nameref arrays to handle all the dimensions except the last one (which is saved in the arrays themselves. The idea is to do something like

declare -n a_0='A_0'
declare -n a_1='A_1'
A=(a_0 a_1)
A_0=(1 2 3)
A_1=(4 5 6)

So to get the data at (1,2), you do `${A[1]}` which gives `a_1` which namerefs to A_1 then `${A_1[2]}` which gives the actual data. The use of the `a_1` and `a_0` are because bash doesnt directly support doing, say, `declare -n A[0]=A_0`...you have to nameref a dummy variable and then store that in an array.

USAGE EXAMPLE

this is the example that running `nd_usage` prints

# # # # # generate nameref framework.
# note: dont include the last dimension

source <(nd_create -a A 2 3 4)

# # # # # set array values
# pass data to be set on STDIN, and use function inputs to define basename + index ranges

source <(seq 1 $(( 2 3 4 5 )) | ndset A 0:1 0:2 0:3 0:4)

# # # # # extract various slices from the array

ndget A 0 \@ \@ \@
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25
26 27 28 29 30
31 32 33 34 35
36 37 38 39 40
41 42 43 44 45
46 47 48 49 50
51 52 53 54 55
56 57 58 59 60

ndget A \@ 0 \@ \@
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
61 62 63 64 65
66 67 68 69 70
71 72 73 74 75
76 77 78 79 80

ndget A \@ \@ 0 \@
1 2 3 4 5
21 22 23 24 25
41 42 43 44 45
61 62 63 64 65
81 82 83 84 85
101 102 103 104 105

ndget A \@ \@ \@ 0
1
6
11
16
21
26
31
36
41
46
51
56
61
66
71
76
81
86
91
96
101
106
111
116

# # # # # cleanup

ndclear A

https://redd.it/16tvadk
@r_bash

String substitution

I am new to Bash and am having a difficult time employing string substitution. In my code below, I am attempting to replace file paths that contain .csv or .xlsx extensions with the .json extension and the output for $output is always /path/to/input_files/file.csv//.csv/.json or /path/to/input_files/file.xlsx//.xlsx/.json. Could anyone help me to understand how I am using string substitution incorrectly here?

#!/bin/bash

for f in /path/to/inputfiles/*.csv /path/to/inputfiles/.xlsx
do
if [[ $f == ".csv" ]]; then
$output={$f//.csv/.json}
fi
if [ $f == *".xlsx" ]; then
$output={$f//.xlsx/.json}
fi
echo $output
python3 API.py "$f" "$output" args
done

&#x200B;

https://redd.it/16u2eoz
@r_bash

I want to add a crontab for ec2-user but it keeps adding it under root, how do I fix this?

* I am running some user data inside my EC2 instance which will always run as the root user as per AWS documentation
* I supply the EC2 instance a bunch of bash commands on launch and one of them is to invoke this file run-cron.sh

In terms of code it looks like this

cd "${ROOT_PATH}" || exit
git clone -b test/cron "${UTILS_REPO_URL}"
chown -R ec2-user:ec2-user ./utils

# https://askubuntu.com/a/889348/968824
find "${ROOT_PATH}/utils" -type f -iname "*.sh" -exec chmod +x {} \;

# shellcheck source=/dev/null
bash "${ROOT_PATH}/utils/src/ec2/run-cron.sh"

My run-cron.sh file looks like this

set -o pipefail
set -u
set -x

IFS=$'\n\t'

# https://stackoverflow.com/a/52879454/5371505
crontab <<EOF
0 0,4,8,12,16,20 * * * ec2-user /home/ec2-user/utils/src/elasticache/backup-elasticache-to-s3.sh > /tmp/backup-elasticache-to-s3.log 2>&1
0 0,4,8,12,16,20 * * * ec2-user /home/ec2-user/utils/src/rds/backup-rds-to-s3.sh > /tmp/backup-rds-to-s3.log 2>&1
EOF

My problem is that after my user data noscript runs, I logout of EC2 instance and log in again as ec2-user. When I run crontab -l it says no crontabs found for ec2-user. When I run sudo crontab -u root -l it shows the above jobs.

How do I add the crontab for ec2-user when running as root from user data noscript?

https://redd.it/16uh7jz
@r_bash

Can you guess the output of these tr(1) commands?

echo abcdefghijklmnopqrstuvwxyz

echo abcdefghijklmnopqrstuvwxyz | tr -d [:blank:\]

echo abcdefghijklmnopqrstuvwxyz | tr -d '[:blank:\]'

Now that you've tried it, and assuming you got what I did, how do you explain >!the missing letter l in the 2nd command!<?

https://redd.it/16ums5y
@r_bash

Variable with double quotes in cURL header

Hello all,

I've been trying to figure something out here for a bit. I am pulling an Etag from a request header and then attempting to use that in the "If-Match" to patch.

export ETag=$(curl -I -X GET https://{uri} -H "Authorization: Bearer ${BEARER}" 2>/dev/null | grep Etag | head -1 | cut -d":" -f2)

The tag gets stored as expected but contains double quotes: W/"12345678910"

When passing in a patch, I am struggling with how to format it.

curl --location --request PATCH https://{uri} \
-H "Authorization: Bearer ${BEARER}" \
-H "Content-Type: application/json" \
-H "If-Match: ${ETag}" \
--data '{}'

Since the tag itself contains double quotes, I am finding it difficult in how to pass it to the header for the match. Does anyone have any ideas on how I can get around this? TIA.

https://redd.it/16vcmvh
@r_bash

Provide password securely to shell noscript

Is there a built in command that can provide a password to a shell noscript? Thought I could use read but it can be sniffed. https://unix.stackexchange.com/questions/563718/sniff-password-entered-with-read-and-passed-as-a-command-line-argument

systemd-ask-password looks like it is about system passwords not for noscripts?

Any options using a command that is part of bash or available on most distros?

https://redd.it/16voh9d
@r_bash

A simple bash noscript if all you need is a simple (really, really simple) website, Zite!

Hello y'all, just thought I'd share a cool little project I've been working on called Zite. It is a simple static website generator using bash and pandoc. The main reason I created was because of the frustration of trying to use tools like Hugo and Zola for creating my personal website. While I do think they're great, I've always felt like all I needed for the website I wanted was a few lines of noscripting, so that's what I did!

The github link is https://github.com/rodrigueslazaro/zite if anyone wants to check it out. I'm no bash, or programming, expert, so I'd be glad if anyone wants to contribute to the project! I know there are some improvement that can be done, but the functionality is pretty much how I want to.

&#x200B;

https://preview.redd.it/et1rg77flhrb1.png?width=963&format=png&auto=webp&s=81a9b2bbb63a9c73b0a741f4413502483b5cae62

&#x200B;

https://redd.it/16wmw1z
@r_bash

Bash menu to execute different functions

I have a small tool to help install stuff without running all the commands.

What I'm trying to do is now that I have all the functions, I am creating a menu so that each item can be selected, and then that particular function is executed, then the menu appears again.

Here is some code to get a feel

bInstall_program_A=false
bInstall_program_B=true

function run_program_A() {
    # execute command
    echo -e "Finished"
}

function run_program_B() {
    # execute command
    echo -e "Finished"
}

if [ "$bInstall_program_A" = true ] ; then
    items+=('Program A')
fi

if [ "$bInstall_program_B" = true ] ; then
    items+=('Program B')
fi

At the top of the noscript I have booleans to turn certain things on and off (it's more complex than this, but this is an example), it depends on the distro I'm using.

Then I've got the function that performs the action

And before the menu at the bottom, I have a statement checking if the bool for the item is true or false, and if it's true, add that item to the array which will show in the selection menu.

Now for the selection menu at the bottom of the bash noscript, I have

noscript="Selection Menu"
prompt="Select an option\n\n"

while item=$(zenity --list \
--width="430" \
--height="335" \
--noscript="$noscript" \
--text="$prompt" \
--column="Installable Packages" "${items[@]}")
do
    case "$item" in
        "${items[0]}") echo "Selected $items[0], item #1";;
        "${items[1]}") echo "Selected $item, item #2";;
        "${items[2]}") echo "Selected $item, item #3";;
        *) echo "Invalid option.";;
    esac
done

The issue with this is that the list of options can change, depending on what those booleans are. So with the array I'm creating that displays all the options, I can't just hardcode the function name that needs to be executed. Because on one of my machines, Option 1 may be something different than another.

So the question is, in the array, how do I store both the menu item name and the name of the function I need to call, so that in the case statement, I can reference the option name and the function name that matches it.

https://redd.it/16wp3s9
@r_bash

UI in Next Generation Shell
https://blog.ngs-lang.org/2023/09/30/ui-in-ngs/

https://redd.it/16wthsm
@r_bash

How to delete data from 2 files and output the result

How do I tell it to remove IP in these 2 file wl.txt and bots.txt from the output file it only reads the first file also need to clean up a file find ips in ipban.txt and remove them from ipattack.txt also want to add if an IP ends with .0 to add /24 i.e: 10.10.10.0 make it 10.10.10.0/24 please add your code to the code below. Thanks

curl -sk $GLOBAL $GLOB $IDP $ATTACK $SRX |\\grep -P -o '((25[0-5\]|(2[0-4\]|1\\d|[1-9\]|)\\d)\\.?\\b){4}(/(3[0-2\]|[12\]?\\d))?\\b' |\\#awk 'NR > 0 {print $0}' > spamhaus_transformed.txtawk 'NR > 0 {print $1}' | sort -u | grep -F -v -f ipban/wl.txt | grep -F -v -f ipban/bots.txt > ipban/ipattack.txt

https://redd.it/16x15yh
@r_bash

/sbin/brltty launching randomly

Hey guys,

I’m trying to start writing some noscripts for school in my ubuntu 18.04 VM and while executing it, suddenly while doing ./noscript1.sh i get the following error:

/sbin/brltty: failed to execute /sbin/brltty

I don’t use it nor did i know of its existence up until today. Any help?

https://redd.it/16x3www
@r_bash

Bash / sed on Mac adds ^M when appending to a line

I'm writing a bash noscript to append additional tab-delimited fields to (initially) the end of the first/header line of a text file on Mac. It works, but sed adds a \^M and then the added fields to the end of each line, which I can only see when I open the file with Vim. It also adds the \^M to the end of all the other lines, even though they don't match.

How do I tell sed NOT to add the \^M?

Here's my sed command:

sed '1s/$/\\tField1\\tField2$/' TESTDATA.txt

https://redd.it/16x8cez
@r_bash

Some tricky regex and graphviz docs later, we have a decent noscript

A vimwiki graph generator using the dot language and graphviz, written in BASH.

Supports two layouts and more can be added.
Instead of a plain white elongated chart that all other such noscripts generate, this one uses the SFDP or NetworkMap layouts along with some custom coloring. Something along the lines of obsidian's graph.

link

Cheers.

https://redd.it/16xkrzf
@r_bash

Weird behavior of jobs/awk command

I'm trying to catch all the background processes belonging to a certain tmux pane and kill them in one command.

For example, if I have 4 background jobs and be using the jobs -rp the outputs would be

3 3701605 running bash -c "sleep 360"
4 3701606 running bash -c "sleep 360"
5 - 3701607 running bash -c "sleep 360"
6 + 3701610 running bash -c "sleep 360"

However when I run jobs -pr | awk '{print $3}' it would output

running
running
3701607

Or when I use jobs -pr | cut -c7- it would output

3701605 running bash -c "sleep 360"
3701606 running bash -c "sleep 360"
3701607 running bash -c "sleep 360"

which completely disregard the last line.

Does anyone have any fix ?

https://redd.it/16xm9sb
@r_bash

Calling user bash from xargs - work around

Ques: Never seen this combination of read & find .. it works but is it common?

The problem I was trying to solve calling function within the same noscript with one parameter from find and giving some additional vars. Function was exported (export -f do_stuff)

$dothis=something
$dothat=whatever
find $startdir -type d | xargs -n1 bash -c 'dostuff "$1"' - |

That will pass the directory to do_stuff but I couldn't figure out how to pass it $do_this and $do_that at the same time, gave up on that and the -exec option as well.

Found a workaround the does the job.... something I had never seen before, looks weird

while read dir ; do
dostuff $dir $dothis $dothat
done< <(find $startdir -type d -print)

&#x200B;

https://redd.it/16xs04n
@r_bash

Calling user bash from xargs - work around

Ques: Never seen this combination of read & find .. it works but is it common?

The problem I was trying to solve calling function within the same noscript with one parameter from find and giving some additional vars. Function was exported (export -f do_stuff)

$dothis=something
$dothat=whatever
find $startdir -type d | xargs -n1 bash -c 'dostuff "$1"' - |

That will pass the directory to do_stuff but I couldn't figure out how to pass it $do_this and $do_that at the same time, gave up on that and the -exec option as well.

Found a workaround the does the job.... something I had never seen before, looks weird

while read dir ; do
dostuff $dir $dothis $dothat
done< <(find $startdir -type d -print)

&#x200B;

https://redd.it/16xsdwl
@r_bash