Help with comparing two file lists, only downloading new files in second list

I wrote a noscript that functions well most of the time but occasionally there is some weird behavior that I am having difficulty reproducing.

https://pastebin.com/wb4ytP5e

The problem that occurs is sometimes (usually when running the cronjob) it downloads the entire list of files instead of just the new files. Can anyone see any reasons in my code why this would happen?

My thinking is taking me towards line 75, where it checks to see if history.tmp exists. I don't think line 76 is producing unexpected behavior, I think for some reason it's not finding history.tmp and therefore in the else statement it's taking the entire history.log as the file list. Why wouldn't it find history.tmp? And why only during the cronjob? It runs fine when I run it manually. Could there be some sort of permissions problem? Like the cronjob doesn't have permission to run line 42, where history.tmp is created in the first place?

https://redd.it/15wa2ap
@r_bash

If ctrl + r and see a command with the arguments I like, is there anyway to extract just the arguments?

Say I have in my history

vi 1
vi 2
vi 3

and I have cat in my prompt right now. If I press ctrl + r, is there a way to transfer the 2 into my prompt so that it reads cat 2 with minimal amount of maneuvering? Can my original cat command survive in the prompt (ctrl + r likes to delete everything currently in my prompt)?

Is there a manpage I can read about reverse-search utility? All of the guides online pretty much say ctrl + r is all I need to know but I'm sure there's more to it.

https://redd.it/15wil10
@r_bash

list mounted storage devices and set them to variables.

First off I'm a complete noob at writing noscripts so I very well might be doing something stupid or over complicating things. I'm working on a noscript that will back up a select number of directories and saves them to another local directory. I'm trying to add a feature to select a mounted storage device to copy said backup to but I'm kind of stuck trying figure out how to achieve this.

This is the part of the code that works

USR=$(whoami)
day=$(date +%F+%H%M)
localdest=/home/$USR/Backups
dependencies="pv espeak tar"
temp=/tmp/temp_backups
SRC=/home/$USR/Pictures
SRC1=/home/$USR/Documents
SRC2=/home/$USR/.config
SRC3=/home/$USR/Desktop
backup=/home/$USR/RiceBackups

# Setup
if ls /home/$USR | grep RiceBackups || mkdir $backup ; then
echo "Setup complete, starting backup..."
else
echo "Setup failed."
fi
# Creates Backup
mkdir $temp &&
tar -cf - $SRC | pv -s $(du -sb $SRC | awk '{print $1}') | gzip > $temp/backup_Pics$day.tar.gz
tar -cf - $SRC1 | pv -s $(du -sb $SRC1 | awk '{print $1}') | gzip > $temp/backup_Docs$day.tar.gz
tar -cf - $SRC2 | pv -s $(du -sb $SRC2 | awk '{print $1}') | gzip > $temp/backup_Dot$day.tar.gz
tar -cf - $SRC3 | pv -s $(du -sb $SRC3 | awk '{print $1}') | gzip > $temp/backup_Desk$day.tar.gz &&
mkdir $backup/Rice$day
mv $temp/*.tar.gz $backup/Rice$day && echo "Backup complete. Saved to $backup" || echo "Backup failed."
rm -r $temp
# Clean out old backups default is 7+ days old
read -p "Clean out old rice? 7+ days old (yes/no) " yn

case $yn in
yes ) echo one sec, getting a mop;;
no ) echo exiting...;
exit;;
* ) echo invalid response;
exit 1;;
esac

echo cleaning up... one moment...
find $backup -mindepth 1 -mtime +7 -delete && echo done. || echo cleanup failed. soz.

#Copies backup to external drive
read -p "Would you like to backup to an external drive? (yes/no ) " yn

case $yn in
yes ) echo Getting list of available drives...;;
no ) echo exiting...;
exit;;
* ) echo invalid responce;
exit 1;;
esac

I copied this part for a selection menu online and am adapting it to work but this is where I'm lost.

#Select storage device menu
HEIGHT=15
WIDTH=40
CHOICE_HEIGHT=4
BACKTITLE="CopyRice V2"
TITLE="Select storage device"
MENU="Choose one of the following options:"

OPTIONS=(1 "Option 1"
2 "Option 2"
3 "Option 3")

CHOICE=$(dialog --clear \
--backnoscript "$BACKTITLE" \
--noscript "$TITLE" \
--menu "$MENU" \
$HEIGHT $WIDTH $CHOICE_HEIGHT \
"${OPTIONS[@]}" \
2>&1 >/dev/tty)

clear
case $CHOICE in
1)
echo "You chose Option 1"
;;
2)
echo "You chose Option 2"
;;
3)
echo "You chose Option 3"
;;
esac

How would I go about detecting all mounted drives and then setting them as variables to be used in this section? Again I very well might be over complicating a lot here haha

https://redd.it/15wmaxf
@r_bash

Is there a way in a noscript to enter commands with semi colon like in vim? (I dont know what this is called sorry)

​

in vim you can enter a semi colon and then this appears at the end of the screen.

I tried looking this up but I had no idea how to word it.

https://redd.it/15worgw
@r_bash

Why is Bash needed to interpret the Homebrew install noscript?

I use zsh on macOS. The Homebrew installation noscript is:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Because I’m using zsh and not bash, I assumed that I should tweak the path to /bin/zsh (instead of /bin/bash) but this returns an error message saying that bash is required to interpret the noscript. Why is that? Thanks for the help!

https://redd.it/15wtydk
@r_bash

Shell/tab completion, who usually supplies it?

Is it more common for an application to supply a drop in file for tab completion, or a separate project to supply it?

Specifically, I started getting more in depth with tmux, and noticed it had no tab completion. However I found/installed a separate project offering one: https://github.com/imomaliev/tmux-bash-completion

I've never really thought much about this. I guess there was always tab completion available for everything I've needed over the years.

I filed an issue on the tmux github page pointing out the no tab completion and provided a link to the ready made one, but he wasn't interested. Suggested taking it up with my shell or distro packager. This doesn't seem like a bash thing at all to me, and Arch wouldn't include it with tmux, don't even need to ask based on their packaging policy.

Tmux has a 2890 line man page with a metric shit ton of commands, options, etc. Seems to me tab completion made using it much easier to get a handle on and learn...

Anyone have thoughts or experience with this?

https://redd.it/15wx1zf
@r_bash

Can anyone suggest me fun weekend projects to try. I want to explore bash.



https://redd.it/15xg9cc
@r_bash

Troubleshooting basic commands

Hi!
I’m a bit of a newb so bare with me. I have been tasked with a mini project because of learning path - but I’m slighting stumped. I just need to run a one liner with basic Unix commands - to extract information (file name and size). This information is given by an internal tool and I just need to append it to a file afterwards for all the files in that folder. But how can I test my code without having access to the tool? Or I’m overthinking this and should just follow the format.
Thanks

https://redd.it/15xi8eh
@r_bash

Problem with renaming files

I'm looking for files of a particular name on a system, and if those files are found, I want to rename them. My goal is to upload the noscript into JAMF and have it run on all our Macbooks at work.

The "find" part of the noscript is working correctly (I'm just testing locally on my own system for now), but the part of the noscript that does the rename is returning odd results and I don't know why.

This is my first attempt at writing a bash noscript. I've found examples of different noscripts online and wound up with the following:

​

#!/bin/bash
filenames=$(find / -type f ( -name "test-1.2.3.rtf" -o -name "test-1.2.4.rtf" ))
for old_name in "${filenames[@]}"; do
if [ -e "$old_name" ]; then echo “Found file $old_name”
extension="${old_name##*.}"
new_name="RENAMED_${old_name}"
mv "$old_name" "$new_name" echo "Renamed '$old_name' to '$new_name'"
else
echo "File '$old_name' not found."
fi
done

#

​

When I run just the "filenames" line, it correctly finds the test files I created on my system. But then when running through the "if" loop to do the rename, here are the messages I'm getting from bash that I don't understand (I've replaced the actual user name returned with "\[USERNAME\]"):

>File '/System/Volumes/Data/Users/\[USERNAME\]/Downloads/test-1.2.3.rtf/System/Volumes/Data/Users/\[USERNAME\]/Downloads/test-1.2.4.rtf/Users/\[USERNAME\]/Downloads/test-1.2.3.rtf/Users/\[USERNAME\]/Downloads/test-1.2.4.rtf/dev/fd/6/System/Volumes/Data/Users/\[USERNAME\]/Downloads/test-1.2.3.rtf/dev/fd/6/System/Volumes/Data/Users/\[USERNAME\]/Downloads/test-1.2.4.rtf/dev/fd/6/Users/\[USERNAME\]/Downloads/test-1.2.3.rtf/dev/fd/6/Users/\[USERNAME\]/Downloads/test-1.2.4.rtf' not found.“Found file ??mv: rename / to RENAMED\_/: Is a directoryRenamed '/' to 'RENAMED\_/'File '-type' not found.File 'f' not found.File '(' not found.File '-name' not found.File 'test-1.2.3.rtf' not found.File '-o' not found.File '-name' not found.File 'test-1.2.4.rtf' not found.File ')' not found.

​

https://redd.it/15xjjdp
@r_bash

"syntax error: operand expected" in mathematical operation

I have a noscript with a mathematical operation:

myfunction(){
VAR=$((bc -l <<<"$VAR_1 * $VAR_2 * $VAR_3"))
return VAR
}

I'm getting this error, but I don't know why:

`bc -l <<<3508 * 60 * 0.393701: syntax error: operand expected (error token is "<3508 * 60 * 0.393701")`

I think it can be due to the double parenthesis, but I need it to be a floating number.

https://redd.it/15xigen
@r_bash

really long flag

Why do some commands have a really long seemingly random flag? In 2048's case it resets your score but I can't tell what the ssh one does.

ex. ssh and 2048

https://redd.it/15xtn6e
@r_bash

Need to read file and remove content from results.

I have a file that generate all the bad IP for my firewall from several site I have a line to delete any of my IPs but would loved to tell it to remove any ips in a file instead of adding them to my .sh file here is the command below can anyone tell me what to change to tell it to omit whitelistips.txt

curl -sk $IPBAN $FW $MAIL $BLOCKIP $DEB $DES |\

grep -oE '[0-9]{1,3}+[.][0-9]{1,3}+[.][0-9]{1,3}+[.][0-9]{,3}+(/[0-9]{2})?' |\

awk 'NR > 0 {print $1}' | sort -u | grep -v XXX.182.158.* | grep -v 10.10.20.* | grep -v XXX.153.56.212 | grep -v XX.230.162.184 | grep -v XXX.192.189.32 | grep -v XXX.192.189.33 | grep -v >

so I need to replace grep -v XXX.153.56.212 | grep -v XX.230.162.184 | grep -v XXX.192.189.32 | grep -v XXX.192.189.33 | grep -v > with my file name

If you can modify my code because I am not a developer It would be much appreciated.

https://redd.it/15y3q95
@r_bash

{: command not found, shellcheck shows no errors though.

Using shellcheck and no errors, but keeping getting `{: command not found`,

Just using aws cli to create cloudfront invalidation and store the output in variable `invalidation_ouput` Then take that `invalidation_output` and extract the `invalidation_id` and `invalidation_status`.

And bash noscripts asynchronous? Does it wait till the aws cloudfront command to get an output?

My errors are specifically showing on `invalidation_id` line and `invalidation_status` line. The aws cloudfront doesn't seem to be succeeding.

`test.sh`

# 1. Create invalidation
invalidation_output=$(aws cloudfront create-invalidation --region "${REGION}" --distribution-id "${CLOUDFRONT_ID}" --paths '/*')

invalidation_id=$("${invalidation_output}" | jq -r '.Invalidation.Id')
invalidation_status=$("${invalidation_output}" | jq -r '.Invalidation.Status')

https://redd.it/15y8gdm
@r_bash

Assigning tar parameters using variables... help?

Bash noscripting newbie here, I'm trying to write a noscript that generates tar archives of some model data that I've been generating. I have set years for the start and end of each archive. Here's what I'm working with so far:

#!/bin/bash -l

shopt -s extglob

### DEFINE START AND END YEARS HERE
srtYr=$(2451)
endYr=$(2500)

cd /usr/scratch/data/SF_SU_55Ma_init/ocn/
tar -czvf /usr/archive/SF_SU_55Ma_init/ocn/SF_SU_55Ma_init.$srtYr-$endYr.ocn.tar.gz SF_SU_55Ma_init.ocn.h.{$srtYr..$endYr}-??.nc
cd /usr/scratch/data/SF_SU_55Ma_init/atm/
tar -czvf /usr/archive/SF_SU_55Ma_init/atm/SF_SU_55Ma_init.$srtYr-$endYr.ocn.tar.gz SF_SU_55Ma_init.atm.h.{$srtYr..$endYr}-??.nc
cd /usr/scratch/data/SF_SU_55Ma_init/ice/
tar -czvf /usr/archive/SF_SU_55Ma_init/ice/SF_SU_55Ma_init.$srtYr-$endYr.ocn.tar.gz SF_SU_55Ma_init.ice.h.{$srtYr..$endYr}-??.nc

So, I am trying to set the start and end years in both the name of the tar files, and in the brace expansion that defines the range of files to use. This doesn't work, though, as I get these errors:

bash: 2451: command not found
bash: 2500: command not found
tar: SF_SU_55Ma_init.ocn.h.{..}-??.nc: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
tar: SF_SU_55Ma_init.atm.h.{..}-??.nc: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
tar: SF_SU_55Ma_init.ice.h.{..}-??.nc: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors

It looks like I'm not able to call variables within the brace expansion. Also, I'm not sure what those initial bash errors might be. Does anyone know what I'm doing wrong, or what I could do for a workaround? Thanks in advance!

https://redd.it/15yb8v2
@r_bash

Can't pass installation command to function in POSIX noscript

I wrote this noscript to install rust alongside other tools:

#!/bin/sh

set -e

install_dependency() {
id_dependency="$1"
id_denoscription="$2"
id_installation_command="$3"

if ! command -v "$id_dependency"; then
echo "Installing '$id_denoscription' ('$id_dependency' binary)..."
$id_installation_command
echo "Installed '$id_dependency' successfully.
Please make sure '$id_dependency' binary is in your \$PATH."
exit 1
fi
}

install_dependency rustc Rust 'curl --proto "=https" --tlsv1.2 -sSf https://sh.rustup.rs | sh'

The problem appears when I run it:

Installing 'Rust' ('rustc' binary)...
curl: option --proto: is badly used here
curl: try 'curl --help' or 'curl --manual' for more information

I don't understand why: I expect everything work fine, as I didn't quote `$id_installation_command`. While I can run `curl --proto "=https" --tlsv1.2 -sSf https://sh.rustup.rs | sh` directly (and it works) I need this function to install several tools and not to duplicate code several times.

https://redd.it/15yoxle
@r_bash

POSIX-compliant noscript to generate DocOpt-compatible help messages with colors

I've written a noscript to generate help messages with colors. Here is how it looks:

&#x200B;

https://preview.redd.it/7ghkhdhglsjb1.png?width=1920&format=png&auto=webp&s=1e97f83ef40249060034a208cf8320f732e1dc29

Firstly, I tell my program explanation (Colorful help message generator. here as the first argument), then after double dashes I list all options should be put in my help message in this format: -<short>/--<long>='Some denoscription'. All options and their denoscriptions are automatically aligned to form a table.

You can preview a generated message, or the whole help() function generated via this noscript. Also, it's possible to copy the output via xclip to clipboard when --copy/-c is passed.

In this example I've obtained this Bash code generated:

help() {
echo -e "\e30mColorful help message generator.

\e[36mOptions:
\e[31m--help|-h \e[35mPrint help
\e[31m--version|-v \e[35mPrint version
\e[31m--text-color|-t \e[35mSpecify text color
\e[31m--header-color|-h \e[35mSpecify header color
\e[31m--option-color|-o \e[35mSpecify option color
\e[31m--option-denoscription-color|-O \e[35mSpecify option denoscription color
\e[31m--preview|-p \e[35mWhether to preview generated message
\e[31m--copy|-c \e[35mWhether to copy generated function to clipboard"
}


&#x200B;

[https://redd.it/15ytyu5
@r_bash

Backslash doesn't seem to remain literal inside double quotes when doesn't follow any special character in dash

Dash [man page](https://manpages.ubuntu.com/manpages/focal/en/man1/sh.1.html) states that:

> The backslash inside double quotes is historically weird, and serves to quote only the following characters:
> $ ` " \ <newline>
> Otherwise it remains literal.

I've tried to execute `echo "1\2"` and expected to see `1\2` while got `1`. In Bash it works as expected. Is it me misunderstanding man page or what? When I see `it remains literal` phrase, I interpret it automatically as `it is being displayed as it is`.

https://redd.it/15z0lm8
@r_bash

Send output of tail to another tail

I am looking to condense the following into one line

I am plumbing logs that are of the format keyword.randomstring.joblog, looking for the last line to have the keyword rc in it.

I currently use

ls -tr $LogDirectory | grep keyword | tail -1 then manually copy that to tail -1 $LogDirectory/FilenameHere | grep rc

Can this be condensed to one line?

Thanks!

https://redd.it/15zdfnf
@r_bash

How Ctrl-V prevents Terminal Driver Sending SIGINT to foreground group?

Hello,

Sorry if this question is a little mercurial and pedantic...or obvious. Trying to fully understand some behaviour rigorously.

If I am running in interactive bash session and I run something like trap report_SIGINT SIGINT to a function that just logs the reception of the SIGINT, then I Ctrl -V Ctrl - C to implement the character literally as in the bash manual I get the Ctrl - C as the manual would suggest literally printed. All good so far according to the manual.

However, this causes a problem I can't quite account for:

Since the terminal driver acts on things before readline library receives stuff from it, why doesn't the driver receive the Ctrl - C from the keyboard and then send the SIGINT to the foreground process group (which bash is in when there is no active foreground childprocess) meaning that bash receives a SIGINT? You can see that bash doesn't receive this from the trap...nothing is outputted to my logging file after Ctrl - V then Ctrl - C. I thought perhaps it turns off the intr value in terminal driver from Ctrl - C but running stty -a TERMINAL on the terminal (from a second terminal) after Ctrl - V shows no change.

Thanks for any and all help!

https://redd.it/15zcpdq
@r_bash

slack noscript

hello not sure if this is the right place for this

i have 0 noscripting experience and my boss asked me to create a noscript that will produce all of our users who are on slack and show them what build version they are on

example user 1 slack build intel
example user 2 slack build silicon

https://redd.it/15zi0is
@r_bash

Can someone tell me why my code works fine outputting to a terminal but gets "stuck" outputting to a pipe?

Usually I try and parse down the code to a easy-to-understand example, but Im really not sure what is causing this behavior. So....the code is here. It is a prototype for a new way of distributing inputs to several forked workers that is much faster than the standard "pipe --> while read; do..." loop. On my system, distributing 1 million lines (printf '%%s\n' {1..1000000}) to 28 different workers only takes ~0.3 seconds longer than the printf call itself. But that isnt the point of this post, so back on topic...

To see this behavior im trying to fix, run

# source function
source <(curl https://github.com/jkool702/forkrun/blob/main/mySplit.bash)

# this works as expected
printf '%s\n' {1..10000} | mySplit

# this gets stuck
printf '%s\n' {1..100000} | mySplit | wc -l

Ive tried a lot of different ways to fix this without success. best I can tell, mySplit runs fully and cleans up. There are no forked processes left running nor any file denoscriptors left open. mySplit passes all the data it is supposed to pass to the pipe, runs every line of code it is supposed to, and calls its exit trap. And then...it just sits there and does nothing instead of closing and sending an EOF down the pipe.

If whatever you are piping its output to expects an EOF it will wait for it forever. As such, something like wc -l will never give any output. Piping to cat instead will print everything to the screen but then just sit there not closing and giving you back the terminal until you press <crtl> + c.

Any ideas what would cause this?

Thanks in advance.

https://redd.it/15zlyld
@r_bash