Is there a linux/bash command that will recursively fix invalid filenames in my 3.5TB harddrive to make them more universally compatible with most software & filesystems?

So, here's the deal.

I've been making folder names that include characters like the colon : and parenthesis ( and dashes - etc. for years without thinking too much of it until now.

This was all on linux, and I didn't think much of it because the computer allowed me to name files however I wanted.

For example, I'll have a folder called

 BOOKS

or have

(Unabridged)

in the filename etc.

...and sometimes some weird glitch would have added quotes ' or " into the filename, too - this wasn't me, but I reckon it happened when it was downloaded or something.

But when I try to rsync, many will fail due to weird characters.

It's not just rsync. it's a lot of scenarios.

It's been a huge headache.

Even when I restored some of my folders from my cloud backup on Crashplan, the software itself renamed my folders by removing colons and adding slashes / to likely make them more universally compatible etc.

My question is, is there a simple and safe command that will recursively remove or fix these filename issues to make them universally compatible?

I wish I hadn't used colons, but I certainly don't want to break any files or screw up the renaming so royally that I have tens of thousands of misnamed files/folders either.

Thanks for any help.

https://redd.it/14qix2c
@r_bash

Unable to call executable from .gitconfig even though its in PATH, but can call it with full path

I'm struggling to understand why this is occurring, but it seems that .gitconfig doesn't have permission to call some executables by name, even if they are in PATH.

For context, I am setting up diff-so-fancy, and have cloned the git repo, changed the permissions of the executable and added the directory to PATH in my dotfiles. I then added the following line in the [core\] section of my gitconfig as instructed:

pager = diff-so-fancy | less --tabs=4 -RFX

however, that results in the following error

>diff-so-fancy | less --tabs=4 -RFX: 1: diff-so-fancy: Permission denied

​

So I decided to make sure PATH contained the directory at the time of calling git diff and added the following line

pager = echo $PATH && diff-so-fancy | less --tabs=4 -RFX

which did have the appropriate path in it, prior to printing the permission-denied error

​

Just having the executable directory in PATH and calling it by name in gitconfig does work on my Mac laptop for work.

​

It also works when I just call diff-so-fancy by its name in the terminal, so it is clear to me that it is in the path and callable.

​

I then decided to check what would happen if I had the full path to the executable within the gitconfig alias, so I added

pager = ~/path/to/diff-so-fancy | less --tabs=4 -RFX

which did have the expected output.

​

It doesn't, however, work if I make a symlink to the executable and put it in /usr/bin/

​

Its also worth noting that .gitconfig can call other executables, such as less just fine, without needing to specify the full path. The only time it doesn't work is when gitconfig is trying to call the specific executables.

​

Also, yes, I know, I could just leave the full path in my gitconfig and have the expected output, but I just wanted to figure out why it wouldn't work, when it does on my Mac laptop.

​

System Information:

Shell: Bash 5.1.16

Git 2.34.1

OS: Windows 10 22H2 19045.3086

WSL version: 1.2.5.0

WSL Kernel version: 5.15.90.1

https://redd.it/14qpk0y
@r_bash

Invisible window I can't click throuh

When i open gitbash console a huge portion of my screen becomes unclickable, like if i have an invisible window over my desktop. Can't click any shortcuts or file explorer, any idea what it could be? when i close the msys2 process in the task manager it goes away, but every time the console is open it's there. Im running windows 11 btw, any help is appreciated, thanks.

https://redd.it/14w4l6n
@r_bash

Resources on learning bash noscripting

I already know how to traverse the file system, I also know the purpose of certain files in the root directory

https://redd.it/14q9k5t
@r_bash

Remove all words after another word from multiple lines lines

Hi, I have a text file containing

libargon2.so
libargon2.so.1
libasan.so
libasan.so.8
libasan.so.8.0.0
libasm-0.189.so
libasm.a
libasm.so
libasm.so.1
libasprintf.so
libasprintf.so.0
libasprintf.so.0.0.0

I want to convert it like this

libargon2.so
libargon2.so
libasan.so
libasan.so
libasan.so
libasm-0.189.so
libasm.a
libasm.so
libasm.so
libasprintf.so
libasprintf.so
libasprintf.so

i.e. I want to remove all the text after ".so".

How can I do?

Thanks in advance.

https://redd.it/14pix7v
@r_bash

Why is this happening? (Win 11)

https://redd.it/14pif5w
@r_bash

Are there any tools to analyse/modify colours directly from a bash noscript?

I am on Arch Linux and I am using pywal to generate a colour palette from my wallpaper, which I then use throughout my system. In particular, i have a bash noscript which grabs these colours and uses them for polybar. The problem is that sometimes these colours do not have enough contrast, and the bar is hard to read. Is there any tool that would allow me to check the readability of my colours, and modify them accordingly, directly from my noscript? If not, how should I be approaching this issue?

https://redd.it/14pfnqk
@r_bash

Help in determining if I should use if/then or case/esac or what?

I am trying to write a BASH Script for use at work and I am in need of assistance. The noscript I m writing will need operator input and run in sections depending on the users input. What I would like the noscript to do is ask the user how many times it needs to run the subroutine or at the end of each subroutine ask if they would like to continue with next subroutine.

We process files at work sometimes 1 sometimes 3 or 4 or 5. I would like the noscript to ask the user how many files the user is interested in processing and have the noscript run the subroutines for that many.

I don’t know its I should use if/then or case/ecase or whatever else to structure the noscript with.

Following is a step by step of what im intending to do:

Step 1- Ask user how many file values it wants to process

Step 2- Start processing the first file values saving variable values to print out at the end of the noscript

Step 3- Ask the user if they would like to continue

Step 3- Continue to the next file saving its variables to print out at the end also

Step 4- After the last file is processed print the values of the variables from each section

If I set it up for say 5 sections this will cover a number up to what we can do during the day. After it comes to the end of each section if User Input Yes/No to continue with another until the final is reached that would be good. The at the end when the user is asked do you want to continue processing and the answer no it prints the variables retrieved from the different sections.

https://redd.it/14o87vc
@r_bash

Help optimizing my box drawing function

drawbox() {
tput clear
# gettermsize
horiz="$((COLUMNS - 1))"
# vert="$((LINES - 2 ))"
vert="$LINES"
itera=1

# boxchar0="╭" boxchar1="╮" boxchar2="╰" boxchar3="╯" boxchar4="│"
# boxchar5="─" boxchar6="┑" boxchar7="┎" boxchar8="├" boxchar9="┤"

# put corner left & right top
# and then top line
cuphome && cupxeol && printf '╭'
perl -E "print '─' x $horiz"
cupright 1 && printf '╮'

# input vertical lines on both sides
for ((itera=1; itera<vert; itera++)); do
tput cup "$itera" 0 && printf "│"
tput cup "$itera" "$horiz" && printf "│"
done

# put left & right bottom corner
# and then top line
cup "$vert" 0 && cupxeol && printf '╰'
perl -E "print '─' x $horiz"
cupright 1 && printf '╯'

# put left & right bottom middle delim
# and then mid-line
cup "$((vert - 2))" 0 && cupxeol && printf '├'
perl -E "print '─' x $horiz"
cupright 1 && printf '┤'
}

So I have this noscript that I'm writing, it's pretty large at this point and it's still in it's early stages called [catnip](https://raw.githubusercontent.com/sweetbbak/catnip/main/cn). It is a little command console that lets you pick a directory and shuffle through the images in that directory. I'm on my 2nd or 3rd re-write right now and I'm having trouble optimizing some of these core functions, the worst one is this box drawing function. Its like Box() in C from Ncurses except that it has a little command line box at the bottom.

The main issues that I'm running into are speed, trouble writing unicode characters and portability. I'm using Perl here because it can easily handle unicode characters and its far far faster than using tput.

I'm curious if you all would have some advice here? I'd like to use a bash builtin instead of Perl and I'd really like to speed this up. It's not so bad with perl and only redrawing when the box gets destroyed but I'm still not happy with it.

# cup in the above function is defined as:
# Cursor to absolute position
cup() { printf '\e[%s;%sH' "$1" "$2" ; }
# Erase from cursor to end of line
cupxeol() { printf '\e[K'; }
# Cursor 0, 0
cuphome() { printf '\eH' ; }

Im using this in place of Tput but I'm open to using that instead. In my mind I felt going top left box char, across the top, top right box char... then each side and then the bottom horizontal bar and left + right box characters, then the middle line with a special right and left character on each edge.

The repo has three videos in it that show it in action if you're curious. One weird side-effect of another function is that the characters are colored blue, but only after the "gradient" function is called that takes a string and divides it up to make a color gradient. I'm sure the entire thing is pretty rough but any help would be appreciated.

[https://redd.it/14nnh95
@r_bash

Quoting value in string w/ multiple matches

I am looking to add some quotes to a string, and am having some issues with doing it with sed:

>$ value='abc=123'
>
>$ echo $value
>
>abc=123
>
>$ echo $value | sed "s/\\(.*\\)=\\(.*\\)/\\1='\\2'/g"
>
>abc='123' <-- This is what I want, with the value quoted
>
>$ value='JAVA_OPTS=-Xms1G -Xmx3G -Dcom.redhat.fips=false'
>
>$ echo $value
>
>JAVA_OPTS=-Xms1G -Xmx3G -Dcom.redhat.fips=false
>
>$ echo $value | sed "s/\\(.*\\)=\\(.*\\)/\\1='\\2'/g"
>
>JAVA_OPTS=-Xms1G -Xmx3G -Dcom.redhat.fips='false' <-- it's picking up the = inside the value

This is for use in a container, so I would rather not add something bulky like perl or python that could do this easier than sed. awk and other basic GNU Utilities are available

https://redd.it/14n4n9g
@r_bash

A little bash noscript for new Ubuntu users to help them install Flatpak with necessary software center & desktop integrations
https://github.com/hakerdefo/flatjak

https://redd.it/14ldurr
@r_bash

xdotool help - adding in --window breaks things

Can anyone help with this one, using xdotool to do a shift click .

i have to specify the window with --window which seems to make things not work

This works

xdotool keydown shift click 1 keyup shift

Adding --window in this does not.

xdotool keydown --window $windowid shift click --window $windowid 1 keyup --window $windowid shift

the keydown doesnt work for shift as soon as i add in --window

Making it into single lines also has the same issue

xdotool keydown --window $windowid shift  
xdotool click --window $windowid 1
xdotool keyup --window $windowid shift

Any suggestions?


Google only finds similar issues or fixes that dont include --window (which is required since its a background window)

https://redd.it/14ltoul
@r_bash

how to get PID to not appear from a while loop?

So I have a while loop in my PS1 prompt but everytime I open a new terminal a PID shows up at the top like [1\] 28014

I tried 2>/dev/null, it did not work

https://redd.it/14lhf7b
@r_bash

Capturing history of bash on exit

I have been using something the TRAP command in my .bashrc file for over 10 years to get a history of all commands run in the shell.

I have this line in my .bashrc:

trap $HOME/bin/save_history.sh EXIT

$ cat bin/save_history.sh

\#!/bin/bash

history > \~/.hist/`date +%d%b%y_%H%M_``echo $RANDOM`

This will capture the history into a file with the date, time and a random number so multiple files will not over write each other. A typical file name looks like

28Jun23_1047_25998

I started doing this so I had a log of whatever I did with git.

I am using git-bash on a win11 system. I have used this in my .bashrc on both linux systems and recently it has stopped working. The file is created, but it is empty. If I run that command from a command line in a live shell, it works. I get a file with 500 lines, the last 500 commands run in that shell.

Is there some reason that history output doesn't go to the file when run from a noscript?

&#x200B;

https://redd.it/14lgdq3
@r_bash

Script to make a new python virtual env and install packages into it with pip

I wrote a quick noscript to make a new python virtual env and install packages into it with pip, but only if $2 $3 $4 are provided. The noscript accepts a -h flag to display a usage message (also if run without arguments). Here is the noscript

I tried to make it compatible with the Bourne shell and used the #!/bin/sh. On Arch and derivatives, sh is symlinked to bash, so it doesn't really affect my use case, but might help others.

## A few problems I want to ask about:

It works fine, but I couldn't:

find a way to `cd` into the new env directory and source the `env/bin/activate` from the noscript before exiting the noscript because of the noscript running in its own subshell, and I couldn't find any smart fixes for that. Although sourcing the python env from inside the noscript works as far as getting the env working and installing packages with pip inside the new env.
Another problem I faced was that using echo "$PWD"/"$1" would return "$PWD"/"$1"/"$1" like /home/wolandark/env/env which didn't even exist, and that confused me a bit.

Any feedback || tips are highly appreciated and thank you for your time.
Cheers!

## Edit:
After the tips I recieved under the post, I updated the noscript. Here it is.

https://redd.it/14kx6l8
@r_bash

How do I copy, merge and append multiple files taking them as user inputs?

So firstly what is the difference between these and how do I do them separately?

https://redd.it/14l6jmf
@r_bash

How should I pass "i" in the record object array in the for loop?

for i in {0..$IDs}
do
CSs=$(echo "$jsonData" | jq -r '.result.records.ChangeSetNamec')
echo "$CSs"
done

==============Below is the JSON===========

{
"status": 0,
"result": {
"records":
{
"attributes": {
"type": "agf__ADM_Work__c",
"url": "/services/data/v58.0/sobjects/agf__ADM_Work__c/a1F26000002EhXREA0"
},
"Id": "a1F26000002EhXREA0",
"Change_Set_Name__c": "W-001730"
},
{
"attributes": {
"type": "agf__ADM_Work__c",
"url": "/services/data/v58.0/sobjects/agf__ADM_Work__c/a1F26000002EwRyEAK"
},
"Id": "a1F26000002EwRyEAK",
"Change_Set_Name__c": "W-002024"
},
{
"attributes": {
"type": "agf__ADM_Work__c",
"url": "/services/data/v58.0/sobjects/agf__ADM_Work__c/a1F26000002ETgGEAW"
},
"Id": "a1F26000002ETgGEAW",
"Change_Set_Name__c": "W-001444"
}
,
"totalSize": 7,
"done": true
},
"warnings":
"The \"force data soql query\" command has been deprecated. Use \"data query\" instead.",
"The \"-u\" flag has been deprecated. Use \"--target-org | -o\" instead."

}



So I want to take the change_set of the respective ID and later then use the ID again in some SOQL query.

If i keep the statement as is i.e. "CSs=$(echo "$jsonData" | jq -r '.result.records[\].Change_Set_Name__c')" it returns all the IDs in one go, I want to iterate them one by one to perform some tasks on each ID and their changesets.

I tried using $i but seems like the '' are restricting it., it takes it as a past of the logic. Or is there any other way to iterate through the json

https://redd.it/14kehp3
@r_bash

Why choose these to test if the shell is broken

#!/bin/sh
#
# FFmpeg configure noscript
#
# Copyright (c) 2000-2002 Fabrice Bellard
# Copyright (c) 2005-2008 Diego Biurrun
# Copyright (c) 2005-2008 Mans Rullgard
#

# Prevent locale nonsense from breaking basic text processing.
LCALL=C
export LCALL

# make sure we are running under a compatible shell
# try to make this part work with most shells

tryexec(){
echo "Trying shell $1"
type "$1" > /dev/null 2>&1 && exec "$@"
}

unset foo
(: ${foo%%bar}) 2> /dev/null
E1="$?"

(: ${foo?}) 2> /dev/null
E2="$?"

if test "$E1" != 0 || test "$E2" = 0; then
echo "Broken shell detected. Trying alternatives."
export FFCONFEXEC
if test "0$FFCONFEXEC" -lt 1; then
FFCONFEXEC=1
tryexec bash "$0" "$@"
fi
if test "0$FFCONFEXEC" -lt 2; then
FFCONFEXEC=2
tryexec ksh "$0" "$@"
fi
if test "0$FFCONFEXEC" -lt 3; then
FFCONFEXEC=3
tryexec /usr/xpg4/bin/sh "$0" "$@"
fi
echo "No compatible shell noscript interpreter found."
echo "This configure noscript requires a POSIX-compatible shell"
echo "such as bash or ksh."
echo "THIS IS NOT A BUG IN FFMPEG, DO NOT REPORT IT AS SUCH."
echo "Instead, install a working POSIX-compatible shell."
echo "Disabling this configure test will create a broken FFmpeg."
if test "$BASHVERSION" = '2.04.0(1)-release'; then
echo "This bash version ($BASHVERSION) is broken on your platform."
echo "Upgrade to a later version if available."
fi
exit 1
fi


This is a code snippet from FFmpeg's comfigure scirpt.

It uses these to test whether the current shell is broken.

unset foo
(: ${foo%%bar}) 2> /dev/null
E1="$?"

(: ${foo?}) 2> /dev/null
E2="$?"

I understand the behaviour of these code. My question is why do they choose these? What is special about them?

https://redd.it/14k9rpz
@r_bash

Splitting file by unique/not-unique

I have a couple of files that look something like this:

chr1 94160888 94161133 GAGTGTCTCTCAGAGTGA 91
chr1 94160888 94161133 GTCACTGTGTCTCAGTGT 84
chr1 94160888 94161133 CACAGTCACACTCTCTCA 56
chr1 94160888 94161133 CACTCACTGAGTGTGTGA 51
chr1 94160888 94161133 GACTGAGACACTCACACT 22
chr1 94121054 94121299 CAGTCAGACAGTCAGACT 91
chr1 94121054 94121299 GACAGTGTCTCTCACTCT 76
chr1 94121054 94121299 GACAGTGTCTCTCACTCT 62


And I need to identify them by unique strings in column 4. I was hoping to split these into two separate files. However, the closest thing I've found to at least find unique values is sort -k4,4 -u, which would be fine, but it still prints the first instance of a duplicated string, which I do not want. Is there a way to print only the lines where a column 4 string appears once, and is there a way to print lines where one appears more than once?

&#x200B;

EDIT: for file simplicity

https://redd.it/14jooud
@r_bash