basic text editor help probably

I have 10000 plus lines , that must be made into a single line

Replacing \n with "" does not seem to work. Any idea how to do it?

https://redd.it/x38hpl
@r_bash

Help with file denoscriptors and redirections.

So, I have this issue: I'm running a command that prints to both stdout and stderr.

The stdout is huge, and must be stored into a file. But I also want the errors to be printed there, for later parsing.

This would be simple, right:


$ ( echo out; >&2 echo err; echo out ) 2>&1 > /tmp/file


However, since this is running in a pipeline, I also want it to print out the errors. I cannot use tee, as it would completely drown out the errors and make the pipeline log unreadable. So in essence I want to have the stdout directed into a file, and I want the stderr to be redirected to both the file AND the terminal that calls it.


This was the best I could come up with:

$ ( echo out; >&2 echo err; echo out ) 2> >(tee -a /tmp/file) > /tmp/file


But unfortunately this doesn't even work since it either prints the err at the end of the file, or it omits it from the file completely because of race conditions (I think¹).

I know this is kind of weird, but I wonder if there's a good way of doing it.

Edit:
I now understand that either the > creates the file, does its thing and then tee appends to it, or tee creates it first, and then > overrides it. I could always replace > with >> to have all the outputs, but since this is still pretty messy, I'd like to know if there's something better than:

$ > /tmp/file
$ ( echo out; >&2 echo err; echo out ) 2> >(tee -a /tmp/file) >> /tmp/file

Edit 2:

Best I could do:

$ ( echo out; >&2 echo err; echo out ) 2>&1 1>>/tmp/file | tee -a /tmp/file

https://redd.it/x3atli
@r_bash

Need a maintainer(MacOS) for a little bash noscript

I wrote this noscript for the kitty terminal. It works well on linux but fails to work on MacOS. This is obvious because it was never tested on a Mac but I hoped it'd work because Mac uses bash.

I've had two issues reported:
https://github.com/zim0369/pretty-kitty/issues/4
https://github.com/zim0369/pretty-kitty/issues/3

BTW, the noscript must sound a little exaggerated since it is probably a one time fix.

https://redd.it/x2xxv8
@r_bash

How do you get the return code for FTP commands in a heredoc?

I’m using sshpass to check if a file has been successfully delivered to an FTP outbox, but no matter what I do, the return status is always 0.

Here is my current approach:

sshpass -e sftp user@host << EOF
cd outbox
ls notafile # non-zero status
exit
EOF
echo $? #0
# Alternatively
echo ${PIPESTATUS0} # 0

How do I get the actual non-zero status code?

https://redd.it/x2uluo
@r_bash

Why powershell has more members than bash?

IDK is it appropriate to ask this question here. I'm wondering why powershell subreddit has more members than bash subreddit? :D. Shouldn't bash be more popular than powershell due to popularity of Linux servers?

View Poll

https://redd.it/x3fyoh
@r_bash

Two pieces of advice

I have been answering shell noscripting questions on Stack Overflow, on and off since 2013. As a result of doing so, there are two things that I have learned, which I wanted to pass on to anyone here who might be interested.

a} Learn to use the three utilities Ed, tr, and cut.

In my observation, the only two shell programs that anyone on SO uses are Awk and sed. I consider Ed the single most versatile noscripting utility that I have ever discovered. If everyone who asked questions there knew how to use Ed alone, I honestly think it would reduce the number of noscripting questions the site gets by 90%. Although I do use Ed interactively as well, my main use of it is in noscripts, via embedded here documents.

Although knowledge of Ed's use has been almost completely forgotten, this book about it exists. I would encourage everyone here who is willing, to read it. I also offer my own SO Answers tab which contains examples of how to use Ed in noscripts, although I am still learning myself.

b} Learn to search vertically as well as horizontally.

Most questions which I answer on SO, are about how to extract substrings from a much larger stream of information, and a lot of the time said information is all on a single line.

I have discovered that complex regular expressions are usually only necessary for sorting through a large single line from left to right. If I use the tr utility to insert carriage returns before and after the substring I want, I can isolate the substring on its' own line, and it will then generally be much easier to use cut to isolate it further. I find writing complex regexes very difficult, but identifying nearby anchors in a data stream for inserting carriage returns is usually much easier.

I really hope these two suggestions help someone. I don't know how to pass them on to anyone on SO really, but given how valuable they have been to me, I wanted to make sure that I communicated them to someone.

https://redd.it/xf08td
@r_bash

Issue with cron running a sh noscript / app

Hi all,

&#x200B;

I am trying to make a bash noscript (called mynoscript.bash) that will check if an app (called TheApp.sh) is running and if it's not, launches it.

Here is my code :

ps aux | grep TheApp | grep -v grep > /dev/null 2>&1

if $? -eq 1
then

bash /path/to/TheApp.sh > /dev/null 2>&1 &!

fi

&#x200B;

So my noscript is running well when i run it myself (bash mynoscript.bash) but doesn't work from cron.

&#x200B;

Cron line for my noscript :

/bin/bash /path/to/mynoscript.bash

I have some log files that states that my noscript is played by crontab BUT it doesn't do the "bash" line at all.

&#x200B;

I also tried to only run the app from cron and that doesn't work either and i don't understand why :

/bin/bash /path/to/TheApp.sh

The app doesn't launches at all, i tried with /bin/sh but it's the same and the shebang of the app is /bin/bash. But if launch it myself it works just fine (bash /path/to/TheApp.sh)

&#x200B;

I am using my own account and not the root account but i also added these lines in the root's crontab but it's still the same.

I checked the /var/log/syslog for cron error but there is no error showing up at all, just some lines stating that the cron job is done i guess ?

Sep 15 17:35:01 computer CRON24307: (user) CMD (/path/to/TheApp.sh)

&#x200B;

Am i missing something ? I am running on Ubuntu 20.04.5 LTS.

https://redd.it/xf01ko
@r_bash

Calling validation when user inputs wrong options in Getops

I have noscript and i need to check if all the validations are correct and working :

   # initialize variables
    un=0
    file=””
    
      while getopts "d:uph" option; do
              case $option in
               d)
                   file=$OPTARG
                   ;;
               u)
                   un=1
                   ;;
                   
               p)
                   p=1
                   ;;
                    
               h)
                    help
                    exit ;;
    
               *)  echo "Error: Invalid option"
                    exit;;
            esac
      done
    
    #Validations
    
    # This is my first logic validation user must provide eithe -u or -p option
     
    if [ "$p" ] || [ "$un" ];then
             echo "must provide -u or -p" 1>&2 ;
             Help
             exit 3 ;
            fi
    #Second one, i want check make sure they not passing both options
    
    if (( p && u ));then
    echo "must not pass  two options";
    Help
    exit 3 

#third one is make sure pass in option if the user doesnt pass any option

https://redd.it/x2iyml
@r_bash

Deleting all files relating to an application

I've been trying to rid my Mac of a horribly persistent Microsoft Autoupdate popup, and, in what was definitely not a smart move, I deleted all my MS apps (Excel, Teams etc) from the Applications folder by hand instead of using an app cleaner.

The popup persisted, so I dug around and deleted some stuff in ~/Library/Application Support (as directed by a different thread). Five minutes later, the Autoupdate window was back. So my question is (for future reference, too): is there a bash command which would allow me to find all the files associated with an application? Surely this is what app cleaners do -- find some log of these files that came in installation and updates? Since I've already deleted the apps, I can hardly go back and do it by app cleaner, so I might as well try and learn something from this.

Alternatively, if someone has had the same Autoupdate problem and solved it, do please let me know what worked :)

&#x200B;

Edit: I just found a post someone else made here with the same question, but the only answer is what I knew all along... should have used app cleaner in the first place. It's too late now though, so I hope someone can help!

https://redd.it/xf88uv
@r_bash

Need help in extracting specific words from specific line from a text file and echo it to the user when they enter their name.

# Hi everyone basically I got stuck on this part of an assignment, I tried searching everywhere but can't seem to get an answer. I am using ubuntu terminal.

# Basically I have this student.txt

__________________________________________________________________________________________________________

Name:country:phonebrand:email

James:Singapore:Apple:outlook

Jonh:USA:samsung:gmail

Steve:England:nokia:gmail

_______________________________________________________________________________________________________

# I am suppose create a executable binbash file to allow user to enter their name, and automatically it will display the phone brand, email, country details. So any help would be appreciated.

______________________________________________________________________________________________________

Name:

________________________________________________________________________________________________

Phone brand(auto display) :

Email (auto display) :

Country (auto display) :

_____________________________________________________________________________________________________

https://redd.it/x28qb4
@r_bash

ZERO knowledge in noscripting, need help in this noscript.

Okay please don't flame me. This is probably chicken stuff from y'all.
Totally zero knowledge in noscripting and inherited an IT system.
One of daily tasks is to do backup of our redmine db to our storage server. Both local.
Last I.T. left me with a noscript to do this.
The noscript backups ALL-time redmine DB and I just need to backup this year's DB. (2022)

Hope somebody could let me know what I need to change.


#!/usr/bin/env bash
BACKUP_DATE=`date +"%Y%m%d"`
pushd /root
rm -f redmine_db_${BACKUP_DATE}.sql
rm -f redmine_db_${BACKUP_DATE}.tar.gz
rm -f redmine_file_${BACKUP_DATE}.tar.gz
mysqldump -uroot -pcomplipassword --all-databases > redmine_db_${BACKUP_DATE}.sql
tar -cvzf redmine_db_${BACKUP_DATE}.tar.gz redmine_db_${BACKUP_DATE}.sql
rm -f redmine_db_${BACKUP_DATE}.sql
scp -P 32323 redmine_db_${BACKUP_DATE}.tar.gz ubuntu@192.168.10.231:/mnt/gitbackup/
rm -f redmine_db_${BACKUP_DATE}.tar.gz
tar -cvzf redmine_file_${BACKUP_DATE}.tar.gz /opt/redmine/files
scp -P 32323 redmine_file_${BACKUP_DATE}.tar.gz ubuntu@192.168.10.231:/mnt/gitbackup/
rm -f redmine_file_${BACKUP_DATE}.tar.gz
popd

https://redd.it/x245gc
@r_bash

How to cd into an argument's directory?

I want my noscript to cd into the directory of $1. For example: if the user passes /home/user/pdf/book1.pdf as an argument, I want the noscript to cd into /home/user/pdf. Any idea?

https://redd.it/x1s7jg
@r_bash

Download Any Research Article

By simply copying the doi of any article (i.e., [https://doi.org/](https://doi.org/)...) and passing it as a second argument, the noscript scrapes [sci-hub.se](https://sci-hub.se) and opens the article in your native pdf viewer. If you have xclip or xsel installed, you can simply copy the doi and only run the noscript without passing any arguments.

You need curl as dependency.

Optional dependencies are dmenu, xclip or xsel

&#x200B;

#!/bin/bash

# To run:
# 1. Copy https://doi.org/blahblah
# 2. Then pass it as the second argument: ./paperdownloadscihub https://doi.org/blahblah
# * Or you can just run the noscript if the link is copied in your clipboard

dmenuprompt() {
answer=$(echo -e "Yes\nNo" | dmenu -i -p "Keep paper?")
[[ -z $answer ]] && rm -rf $downloaded_paper && notify-send "$downloaded_paper removed" && exit 0;
case $answer in
#"Yes") mv $downloaded_paper "$papersdir" && notify-send "$downloaded_paper saved in $papersdir" ;;
"Yes") notify-send "$downloaded_paper saved in $papersdir" && exit 0 ;;
"No") rm -rf $downloaded_paper && notify-send "$downloaded_paper removed" ;;
*) exit 0 ;;
esac
}
papersdir="$HOME/Downloads"
[[ -d $papersdir ]] || papersdir="$HOME/Downloads"
useragent="Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0"
[[ ! -z "$@" ]] && doi="$@" || \
doi=$(xclip -selection clipboard -o)
[[ -z $doi ]] && doi=$(xsel)
[[ -z $doi ]] && doi=$(wl-paste)
[[ -z $doi ]] && doi=$(pbpaste)
linkk=$(echo "https://sci-hub.se/"$doi | sed "s/ //g")
tmppaper=$(curl -s $linkk | grep "<button onclick" | awk 'BEGIN {FS="\""} {print $2}' | sed "s/location.href='//g;s/'//g;s/?download=true//g")
[[ -z $tmppaper ]] && notify-send "Paper not found" && exit 1;
two_dashes=$(echo $tmppaper | grep -o "^\/\/");
[[ -z $two_dashes ]] && paper=$tmppaper || paper=$(echo $tmppaper | sed "s/^\/\///")
paper_with_scihub=$(echo "https://$paper")
[[ $paper_with_scihub == "https://sci-hub.se" ]] && notify-send "Paper not found" && exit 1;
cd $papersdir
curl -LsO "$paper_with_scihub" \
&& downloaded_paper=$(ls -tr *.pdf | tail -n 1)
typee=$(file -b $downloaded_paper 2>/dev/null | cut -d' ' -f1)
if [[ typee == "HTML" || -z $downloaded_paper ]];
then
paper_with_scihub=$(echo "https://sci-hub.se$paper")
curl -LsO "$paper_with_scihub" \
&& downloaded_paper=$(ls -tr *.pdf | tail -n 1)
fi
xdg-open $downloaded_paper 2>/dev/null || open $downloaded_paper 2>/dev/null && [[ -z "$(which dmenu 2>/dev/null)" ]] && exit 1 || dmenuprompt

https://redd.it/xfebph
@r_bash

Authenticate using external noscript

I have a bunch of small noscripts which are interacting with a rest api. An example of such is the below list.sh noscript.

#!/usr/bin/env bash

$HTTP "$INSTANCE_HOST/instances" "Authorization: Bearer $ACCESS_TOKEN"

Note that HTTP="http --verify=no --check-status"

All of these small noscripts have the need for a valid access token ($ACCESS_TOKEN) in common.

What I'd like to have is another noscript which I can source and have the ACCESS_TOKEN variable updated as needed. I've tried to do that using the following noscript. Named auth.sh.

#!/usr/bin/env bash

exp=$(echo "$ACCESS_TOKEN" | jq -R 'split(".")? | .[1] | @base64d | fromjson | .exp')

NOW=$(date +%s)

if [[ -z "$exp" ]] || (( $exp < $NOW )); then

echo "Update token..."

# shellcheck disable=SC2155

export ACCESS_TOKEN=$($HTTP --auth "$USER_EMAIL:$PASSWORD" post "$INSTANCE_HOST/tokens" | jq -r '.access_token')

else

echo "Token still valid!"

fi

But the above noscript always update the access token.

Using the above I'd like to have my initial list.sh noscript defined as below

#!/usr/bin/env bash

source ./auth.sh

$HTTP "$INSTANCE_HOST/instances" "Authorization: Bearer $ACCESS_TOKEN"

Is this possible with bash? As mentioned before the auth.sh noscript always update the access token.

https://redd.it/xfhcr9
@r_bash

Match an exactly string 2 or more times

I've this text and i need to match all functions that returns int and has int as parameter

; unistd.h
void _exit(int);
int access(string,int);
uint alarm(uint);
int chdir(string);
int chown(string,int,int);
int close(int);

I've tried `grep -E '^int|\(int|,int\)'` but that's is an or, so i've tried `grep -E '\bint\b{2,}'` but, nothing

https://redd.it/xfjcd3
@r_bash

I want to assign a command to a variable and use it in if statement

sum_errors=$(grep -P '^(?!\[SKIPPED\]).*?[Ee]rror' $LOG_FILE)


if [[ $(grep -P '^(?!\[SKIPPED\]).*?[Ee]rror' $LOG_FILE) ]]; then
echo -e "Error details:\n\n$sum_errors\n" >> $LOG_FILE 2>&1
else
echo -e "No errors found." >> $LOG_FILE 2>&1
fi


I want to use "sum\_errors" instead of the whole grep command as condition to if statement, this is required in order to make the noscript working good.

I tried to do this:

if [[ $(${sum_errors}) ]]; then
echo -e "Error details:\n\n$sum_errors\n" >> $LOG_FILE 2>&1
else
echo -e "No errors found" >> $LOG_FILE 2>&1
fi

and it works if the grep command doesn't output nothing, but if it finds the string pattern it fails and returns this:

./test1.sh: line 31: Modular: command not found

Can you help me understand what am i doing wrong?

https://redd.it/xfq5mr
@r_bash

Help with awk in noscript

Hi,

I am trying to adapt a rofi noscript that changes the pulseaudio sink.The noscript work well, but I want to replace the sink/device names from "alsa_output.pci-0000_06_00.6.analog-stereo" to "Laptop" for example.

Can it be done with awk?The idea is: if sink name matches "alsa_output.pci-0000_06_00.6.analog-stereo" then print "Laptop", but for the next command keep "alsa_output.pci-0000_06_00.6.analog-stereo"

Here is the noscript I am using:

#!/usr/bin/bash# choose pulseaudio sink via rofi or dmenu # changes default sink and moves all streams to that sinksink=$(ponymix -t sink list|awk '/^sink/ {s=$1" "$2;getline;gsub(/^ +/,"",$0);print s" "$0}'|rofi -dmenu -p 'pulseaudio sink:' -location 6 -width 100|grep -Po '[0-9]+(?=:)') &&

ponymix set-default -d $sink && for input in $(ponymix list -t sink-input|grep -Po '[0-9]+(?=:)');do
echo "$input -> $sink"

ponymix -t sink-input -d $input move $sink done

Thank you in advance! :)

https://redd.it/xfrb3z
@r_bash

Display input line by line waiting on keypress

I would like to do something like

ls | while read n; do echo $n; read; done

For some reason the second read command doesn’t seem to be executing, is it blocked by the first read command or something?

Could anyone suggest a way to do this?

Thanks very much

https://redd.it/xfls1y
@r_bash

Tutorial: Rapid Script Development with Bash, JC, and JQ (no grep/sed/awk)
https://blog.kellybrazil.com/2022/08/29/tutorial-rapid-noscript-development-with-bash-jc-and-jq/

https://redd.it/x0rvit
@r_bash

awk is adding a space to print output

cat variables.tf | grep -i "createkmskey" -A 3 | awk -F'=' '/default/ {print "\""$2"\""}'

I'm reading a value from a file, and some outputs will have a value of true or false bash interprets this as a boolean. So I am wrapping the output with ". Doing so though awk adds a white space to the output so false will output " false" instead of "false".

Sometimes the values in the Default field will be strings, numbers, etc.

How do I strip that white space without having to resort to the likes of sed, and only use awk?

https://redd.it/x0ygeb
@r_bash

why is crontab running some commands different? (running noscript manually works as expected)

I have a noscript which is runs correctly manually and I know crontab is picking it up because the logs get updated but I have the following issues:

in crontab "lastUpdate", "now" and "diff" are always empty in the log (when I run it manually they have integer values) and because of this it never reaches the "then" statement (also I assume kill and python commands will work in crontab)

log looks like (but i expect values for 2,3,4):

1 start

2

3

4

&#x200B;

_____________________________________________________________

crontab -e: * * * * * /home/jmair/test/noscript.sh

_____________________________________________________________

\#!/bin/bash

echo "start" >> /home/jmair/log.txt

lastUpdate=$(stat -c %Y products.json)

now=$(date +%s)

let diff=${now}-${lastUpdate}

limit=10

echo $(lastUpdate) >> /home/jmair/log.txt

echo $(now) >> /home/jmair/log.txt

echo $(diff) >> /home/jmair/log.txt

echo $(diff>limit) >> /home/jmair/log.txt

if ((diff>limit))

then

echo "restarting main.py" >> /home/jmair/log.txt

kill $(pgrep -f 'main.py')

sleep 1

python3.9 /home/jmair/test/main.py

fi

https://redd.it/xfzhgm
@r_bash