Compare the differences between dirs and sync

I have two directories A (local) and B (Remote/SFTP). I would like to somehow sync them based to A.

I cant really rsync because A dir has plaintext files and B has encrypted files(GPG).

Example:

Local dir contains: A.txt, B.txt, D.txt, F.txt, G.txt

Remote dir contains: A.txt.gpg, B.txt.gpg, C.txt.gpg, D.txt.gpg, E.txt.gpg, F.txt.gpg

The result I need is remote dir to be like A.txt.gpg, B.txt.gpg, D.txt.gpg, F.txt.gpg, G.txt.gpg

Hence, the file G.txt will be transfer over and files C & F will be deleted from remote dir

P.S I have the functions ready to basebane fies, encrypt them, rsync them.

​

​

My question is how to compare the differences and run my functions accordingly?

What is coming to my mind is to create a array for each dir and compare but Im sure there is smarter and cleaner way than that. Any thoughts?

https://redd.it/13phrj7
@r_bash

save - Capture and reuse output of command (Bash function)
https://gist.github.com/thingsiplay/612d2482b8747cf20e6bf4756d43eed4

https://redd.it/13pl4xs
@r_bash

A bash noscript for converting security camera recordings in .media or similar format to .mp4/.mkv

Received a video recording from a friend for checking a burglary event. Looking at the sdcard contents, there seems to be many folders with names like 1645645354 and contents with `.media` files. Turns out the file names are unix epoch and the `.media` files can be converted using `ffmpeg`. Wrote small noscript and tested on few thousand files. It might be useful for someone else:

https://github.com/IbrahimTanyalcin/camToMP4

MIT License, logo and comments by chatGPT

https://redd.it/13pm9x6
@r_bash

solaris 10 cron failed command substitution.

0,20,40 * * * bash -c -l /usr/local/monitors/lp_mon2

\#!/bin/bash



\#grab oldest file from queue

stuff=\`ls -1 /var/spool/lp/requests/localhost/ | /usr/bin/head -1\`

\#pull printer name from lp requests (first line in file)

puff=\`cat $stuff | /usr/bin/head -1 | sed -e 's/-.*//' -e 's/$/ is having issues printing from /'\`



if [ "$(ls -A /var/spool/lp/requests/localhost/)" ]

then

echo $puff | /usr/ucb/mail foo@bar.com
else

echo lp queue is empty | /usr/ucb/mail foo@bar.com

fi

Also tried:

stuff="(ls -1 /var/spool/lp/requests/localhost/ | /usr/bin/head -1)"


puff="$(cat $stuff | /usr/bin/head -1 | sed -e 's/-.*//' -e 's/$/ is having issues printing

Suggestions??

https://redd.it/13pq6d3
@r_bash

how to prevent this variable form becoming equal to when directory is empty

list=$(for f in "$dir"/\ ; do printf '%s\\n' "$(basename $f)" ; done )

works as expected when the directory is not empty but when it's empty, it becomes equal to * and I want it to become an empty string if the directory is empty

https://redd.it/13q28rv
@r_bash

the case for bash
https://www.neversaw.us/2021/04/02/the-case-for-bash/

https://redd.it/13ql14n
@r_bash

Simple file path opening code

I’m as new as you can get to using bash and was trying to make a little code to make some of my work tasks easier. I’ve just been saving a .txt file as .bat and double clicking when I need to use it.

I figured out how to use explorer to open the file path I want, but now I want it to be a little smarter. Say I have the file path:

T:\Files\Reports

I figured how to open it using “explorer” but now I want to add the month after and have it change to the current month accordingly. For example, this month would be:

explorer T:\Files\Reports\04

I have found multiple things online on how to get current date and how to concatenate strings, but “explorer” doesn’t seem to accept a variable as a useable path. I’m not sure about many of the rules of bash code and all it’s intricacies, so I was hoping someone here could help me. Thanks!!

https://redd.it/13qoinl
@r_bash

Script not working like we had envisioned it working

I've been working with ChatGPT to create a noscript that iterates through my entire music library, looking for filenames that are like "<##> - <Artist> - <Title>", and renaming them as "<##> <Title>". I still need to add a couple conventions, like "<##>. <Artist> - <Title>" and such, but the problem is that it's currently not doing what it should, leaving filenames as "<##> - <Title>" or just leaving the filename completely unchanged. It's also supposed to add a leading zero to the track number in the filename if it's 's a single digit. It's not doing that, either. We've been working at this for a couple hours trying to figure out why it's not doing the thing. Here's the noscript. Edit: Forgot to "ask a question", so, Can you help me? Edit 2: Oh, by the way, I know the music_library path is set to '/path/to/music/library' I do have it set properly on my system. Edit 3: left out a bit of what it's supposed to do.

https://redd.it/13qq0wr
@r_bash

Two-Way File <-> FileList Check

Hello, this bash noscript will loop through lines of files from FileList.txt and check if a directory contains such file... Now I want to do it the other way, loop through a directory and check if filename exists in the FileList.txt . How do I do it? Files may contain special characters...

#!/usr/bin/env bash
#Check Files in list from file exist or doesn't exist in directory.

if $# -eq 0 || $# -eq 1
then
echo "You must pass the path to file with the list and the path to directory to check files. \nsh check-file-exist-from-list.sh path/to/file path/to/directory"
exit 1
fi

while read -r file; do
if -e "$2/$file" ; then
echo "$file exists in $2"
else
echo "$file doesn't exist in $2"
fi
done < "$1"

https://redd.it/13qwd06
@r_bash

I have a weird Syntax error near unexpected token ")"

I can't remember what I was trying to do, I think I modified something in JAVA_HOME and since then I broke everything.

Brew doesn't work anymore, the custom color scheme I had for the terminal is gone and I always get this error

&#x200B;

https://preview.redd.it/grw8emfl8u1b1.png?width=1884&format=png&auto=webp&v=enabled&s=16b1fae9a35417301bcba8c9f2301b4cdc9672bd

It says there is an unexpected token ) in line 1, but there's none there

&#x200B;

https://preview.redd.it/128o2qaq8u1b1.png?width=1972&format=png&auto=webp&v=enabled&s=7da6270a482d9077cae6ec3aee0875405ba46b1e

As you can see here. I am a complete noob with bash and I had a friend set up brew and other neat tools for me, I tried to find a solution myself for days, but I can't for the life of me understand what is going on.. can someone help me?

https://redd.it/13qwmpo
@r_bash

Calculate space savings from deduplicating with hardlinks (using find, sort/uniq, and awk)
https://gist.github.com/Sidneys1/d726c2a59f2dcb1be9dfce2dfae0a042

https://redd.it/13r0hji
@r_bash

trying to number a variable. 1. causing an error 2. I don't know if this is the best way to do it

evalWithNum(){
i=1
listWithNum=$(while read line
do
printf '%d. %s' "$i" ""$line""
i=$((i+1))
done < "$list"
)
unset i
}

"$list" is

a

b

c

&#x200B;

the error is

cannot open a
b
c: No such file

https://redd.it/13r29fw
@r_bash

while loop skips steps?

I have a text file that has variable data a space then a path to a file.

DATA ./path/to/file

This is repeated several dozen times. I am trying to convert mkv files to mp4.


I then have the following code:


#!/bin/bash

if [ -z $1 ]
then
echo "Usage:"
echo
echo '$ ./mkv2mp4.sh "Title.of.TV.Show" file.list "1080p.h264"'
echo
echo "Where file.list is a space seperated file format as:"
echo
echo "S01E01 ./path/to/episode1.mkv"
exit
else
filelist=$2
extras=$3
shownoscript=$1
declare -a array
fi

while read -r mkvfile
do
array=( $mkvfile )
echo "Read from: ${array1}" >> mkv2mp4.log
echo "Write to: $shownoscript.${array0}.$extras.mp4" >> mkv2mp4.log
echo "Read from: ${array1}"
echo "Write to: $shownoscript.${array0}.$extras.mp4"
echo "Start: $( date )" >> mkv2mp4.log
#sleep 1s
ffmpeg -i "${array1}" -c:v copy -c:a aac "$shownoscript.${array0}.$extras.mp4"
echo "Done..."
sleep 1s
echo "Finish: $( date )" >> mkv2mp4.log
done < $filelist


If I run it, it gets to the end of the first video, then it skips almost the entire list and proceeds with one of the last videos in the list. If I comment out the ffmpeg line, it doesn't skip anything.


Any help would be appreciated.

https://redd.it/13rng0x
@r_bash

Detecting Chinese characters using grep

I'm writing a noscript that automatically translates filenames and renames them in English. The languages I deal with on a daily basis are Arabic, Russian, and Chinese. Arabic and Russian are easy enough:

orig_name="$1"
echo "$orig_name" | grep -q "[ابتثجحخدذرزسشصضطظعغفقكلمنهويأءؤ]" && detected_lang=ar
echo "$orig_name" | grep -qi "[йцукенгшщзхъфывапролджэячсмитьбю]" && detected_lang=ru

I can see that this is a very brute-force method and better methods surely exist, but it works, and I know of no other. However, Chinese is a problem: I can't list tens of thousands of characters inside grep unless I want the noscript to be massive. How do I do this?

https://redd.it/13rqfjd
@r_bash

Need a firebase tools passthrough noscript

Unabasheldy asking....
Can someone help me figure out resources how to...,to write a passthrough noscript..not a bash noscripter..but I need one...if you have a fiverr let me know.


So simple thing...I want the user of my noscript (mostly me) ....to run a certain command before running the command with the arguments passed to it...in this case the command is "firebase deploy"...and it usually is something like "firebase deploy --only firestore:rules"

Problem is that sometimes my configuration is in the wrong environment.


There is another command I can use to get the current project...
"echo $(firebase use)"


So I want to run my noscript ..lacking a name...lets just call it "kissanddeploy" like this "kissanddeploy --only firestore:rules"

I want it to run..."echo $(firebase use)"
which should output "dev-foobar" or "prd-foobar"...if the name includes "prd-" in the response...I want to ask the user...


"Are you sure you want to deploy to production?"


...and a Y/N answer before itʻll execute the command passed into it from the CLI...if "N" is answered then it exits. Otherwise it executes whatever was after...like "firebase deploy --only firestore:rules"


Long story short...I accidently deployed to production when I meant to deploy to dev...and it had the potential to break something big...and I do not want that happening ever again..

Anyone wrote a passthrough noscript like this I could take a look at?


Thank you.

https://redd.it/13rw7ya
@r_bash

Environment set-up problems with bash (ros2iron)

Hello! I am very new to this, and this seems like a extremely trivial problem, but I have no idea how to fix it. I am following the steps to build ros2\iron from source on Ubuntu, everything worked great until the final step, where I have to set up the environment. The tutorial gives this command:

. \~/ros2_iron/install/local_setup.bash

I checked, the file is there, everything looks good. But, when i run that command instead of taking me to that directory to get the local_setup.bash file, it takes me here:

"/home/parallels/ros2_iron/install/rosbag2_transport/share/rosbag2_transport/local_setup.bash" so its looking for it in in the wrong place. I am new to bash and ros in general, so I really don’t know why when I explicitly type in the path where it must find the file, it looks for the file a couple of folders deeper. I have tried removing the contents from my .bashrc and .profile files, as I thought maybe there was some path variable that it was following, but that did not help. I am at a loss. Again i know this is likely a very trivial error, but I have no clue how to solve it, so any input would be greatly appreciated!! :)

Also I am running Ubuntu 22.04 ARM64 on a Mac through Parallels.

https://redd.it/13s8615
@r_bash

Speed up inter process communication

Inspired by this post by u/jkool702, I am interested in ways to speed up inter process communication in which a process reads from an anonymous FIFO to receive messages at a rate of around 10-40Hz from other processes. The context is here:

https://github.com/lynxthecat/cake-autorate/blob/16f6c467b2f3faeff935c6d0ccfa23a53eeeaba3/cake-autorate.sh#L1877

Right now each read is byte-by-byte, and I gather one way to overcome that would be to use fixed width writes and read using -N.

Any other ideas?

u/jkool702 you indicated that you ended up writing to files and using 'cat'. Would that work in the above context in which I need to read lines at around 10-40Hz?

https://redd.it/13sgo9y
@r_bash

best way to read settings from a file?

One of the noscripts I am making, I am going to put the settings in a seperate file that the user can modify. But what is the best way to get the settings into the main noscript: Grep? Awk and Line numbers?

https://redd.it/13su4kp
@r_bash

exec "$0" "$@" causing getopt invalid option

I have noscripts that update themselves to the latest version from github. After they update I want them to run the updated noscript, but `exec "$0" "$@"` results in the noscript having getopt errors.

If I run the noscript with "`noscript.sh``-snfr`" it runs as it should.

If I run the noscript with "`noscript.sh``-s -n -f -r`" it causes a getopt error.

In bash 4.3.43 the error is:

getopt: unrecognized option '- -n -f -r'

In bash 4.4.23 the error is:

getopt: invalid option -- ' '
getopt: invalid option -- '-'
getopt: invalid option -- ' '
getopt: invalid option -- '-'
getopt: invalid option -- ' '
getopt: invalid option -- '-'

Here's the part of the noscript causing the issue:

#!/usr/bin/env bash

usage(){
cat <<EOF
Usage: $(basename "$0") [options]
Options:
-s, --showedits Show edits made to <model>_host db and db.new file(s)
-n, --noupdate Prevent DSM updating the compatible drive databases
-m, --memory Disable memory compatibility checking
-f, --force Force DSM to not check drive compatibility
--restore Undo all changes made by the noscript
-a, --autoupdate Auto update noscript (useful when noscript is scheduled)
-h, --help Show this help message
-v, --version Show the noscript version
EOF
exit 0
}

# Save options used
#args="$*"
args=("$*")

# Check for flags with getopt
if options="$(getopt -o abcdefghijklmnopqrstuvwxyz0123456789 -a -l \
restore,showedits,noupdate,nodbupdate,memory,force,help,version \
-- "$@")"; then
eval set -- "$options"
while true; do
case "${1,,}" in
--restore) # Restore changes from backups
restore=yes
break
;;
-s|--showedits) # Show edits done to host db file
showedits=yes
;;
-n|--nodbupdate|--noupdate) # Disable disk compatibility db updates
nodbupdate=yes
;;
-m|--memory) # Disable "support_memory_compatibility"
ram=yes
;;
-f|--force) # Disable "support_disk_compatibility"
force=yes
;;
-h|--help) # Show usage options
usage
;;
-v|--version) # Show noscript version
noscriptversion
;;
--)
shift
break
;;
*) # Show usage options
echo -e "Invalid option '$1'\n"
usage "$1"
;;
esac
shift
done
else
echo
usage
fi

# Show options used
#echo "Using options: $args"
echo "Using options: ${args[@]}"

# Copy new noscript over current noscript
echo "Reload noscript? [y/n]"
read -r reply
if [[ ${reply,,} == "y" ]]; then
echo -e "------------------------------------------------------------\n"

#echo "debug: exec" "$0" "$@" # debug
#echo "debug: exec" "$0" "$args" # debug
echo "debug: exec" "$0" "${args[@]}" # debug

#exec "$0" "$@"
#exec "$0" "$args"
exec "$0" "${args[@]}"
fi

https://redd.it/13t2jcc
@r_bash

find, filenames with leading "-", but cannot use "--"

I have a wrapper noscript around find (and a few other) command. The noscript itself is using Bash's getopts and double dash -- to stop parsing options works as intended. However, there is a problem when giving the arguments over to find command. If a file is a relative path and starts directly with a dash such as -New File, then find command will fail. All other tools and the noscript are handling this correctly. My problem is, I can't use -- with find, because options need to appear after the filenames.

So my question, what should I do? The idea is, if filenames start with a dash, then I can safely add ./ in front of them. For anyone who wants to have a look at the code (over 500 lines of code): https://github.com/thingsiplay/findpick/blob/main/fp and here is how I run find at the moment:

files="$(find "${symlinks}" \
-O3 \
"${@}" "${stdin@}" \
-readable \
-nowarn \
-maxdepth "${optmaxdepth}" \
${xdev} \
${opttype} \
${executabletype} \
-name "${allpattern}" \
"${filtermode}" "${filterpattern}" \
-regextype posix-extended \
"${extendedmode}" "${extendedpattern}" \
-print \
2>/dev/null)"

About the unquoted options, I know that is usually not very safe to do. But these options are controlled and cannot be anything else than correct or empty (in theory). My focus is on "${@}" "${stdin[@]}" \ .

If adding ./ is my only option (the only one I can think of at the moment), how would I do that efficiently for both, positional arguments list and stdin array?

https://redd.it/13t9dmd
@r_bash

Opinions about Bash customization project

Hello all,

I would love to hear some of your opinions about my OpenSource project that allow you to enhance your bash experience. It's called HomeSetup and can be installed or tried in a docker container.

https://github.com/yorevs/homesetup

Appreciated any of your help

https://redd.it/13tcs1f
@r_bash