build latest version of curl from source and install to correct directory

Hey, fellow bash enthusiasts. I got into building curl from source to take advantage of more recently added features. As one does, I noscripted it so I can do this programmatically. I'd love some critique on any aspects of the noscript. I have a few specific points too:

- Is it reckless of me to purge the host's installed curl and then try to build from source? Do I need to trap failures so I can re-install curl if it was originally present? (I.e., clean up after a failure?)

- For downloading the source, instead of installing/purging curl, should I just fall back to wget (and other things) if curl isn't initially available for downloading the curl source?

- Is it unreliable to use "which curl" to determine where on the system curl likes to live?

- Do I need more sophisticated error checking around the make/make install steps?

Thanks for taking a look and sharing your feedback. Much appreciated!

_

#!/usr/bin/env bash

# debugging switches
# set -o errexit # abort on nonzero exit status; same as set -e
# set -o nounset # abort on unbound variable; same as set -u
# set -o pipefail # don't hide errors within pipes
# set -o xtrace # show commands being executed; same as set -x
# set -o verbose # verbose mode; same as set -v

source ./functions.sh

die-if-not-root

apt-get -y update && apt-get -y upgrade

TEMP_DIR=$(mktemp --directory)
cd-or-die "${TEMP_DIR}"

if program-not-available curl; then
apt-get -y install curl
fi

die-if-program-not-available curl "An existing \`curl\` isn't available, so the noscript can't determine where to install a new \`curl\`."

# find where curl lives, so we can install new curl using the same directory path prefix
CUR_CURL_PATH=$(which curl)
PREFIX_FOR_MAKE=$(echo "${CUR_CURL_PATH}" | sed "s,/[^/]*$,," | sed "s,/[^/]*$,,")

# get latest stable version of curl
LATEST_VER=$(curl --silent https://curl.se/changes.html | grep --max-count 1 --only-matching --perl-regexp "(?<=<h2> Fixed in ).*?(?= -.*)")
ARCHIVE_BASENAME="curl-${LATEST_VER}"
curl --location --show-error --silent --url "https://curl.se/download/${ARCHIVE_BASENAME}.tar.gz" --output "${TEMP_DIR}/${ARCHIVE_BASENAME}.tar.gz"

# remove existing curl, so we can install a new one
apt-get -y purge curl

if ! tar -xzvf "${ARCHIVE_BASENAME}.tar.gz"; then
die "Couldn't extract source code via \`tar\`."
fi

# install dependencies for build
apt-get -y install libbrotli1 libbrotli-dev
apt-get -y install libssh2-1 libssh2-1-dev
apt-get -y install libzstd1 libzstd-dev

cd-or-die "${ARCHIVE_BASENAME}"

if ! ./configure --enable-websockets --prefix="${PREFIX_FOR_MAKE}" --with-brotli --with-libssh2 --with-nghttp2 --with-ngtcp2 --with-openssl --with-zstd; then
die "\`configure\` failed."
fi

if ! make; then
die "\`make\` failed."
fi

if ! make install; then
die "\`make install\` failed."
fi

die-if-program-not-available curl "Building \`curl\` seemed to work, but \`curl\` can't be located."



I utilize some helper functions of mine in another noscript. Here they are, if someone wants to see them:


#!/usr/bin/env bash

# usage: die "$MESSAGE"
die() {
printf >&2 "\n*\n* Error: %s\n* This error is unrecoverable. Check above for additional error messages.\n*\n\n" "${1:-Unspecified error.}"
exit 1
}

# usage: am-root
# returns 0 if root, 1 if not root
am-root() {
if ((${EUID:-$(id -u)} != 0)); then
return 1
else
return 0
fi
}

# usage: die-if-not-root
die-if-not-root() {
if ! am-root; then
die "Please run this noscript as root (e.g., using ‘sudo’)."
exit 1
fi
}

# usage: die-if-root
die-if-root() {
if am-root; then
die "Please run

this noscript as a non-root user (e.g., not as root, not using ‘sudo’)."
exit 1
fi
}

# usage: die-if-program-not-available $PROGRAM_NAME "$MESSAGE"
die-if-program-not-available() {
program-not-available "${1}" && die "${2}"
}

# usage: program-not-available $PROGRAM_NAME
# returns 0 if program isn't available, 1 if program is available
program-not-available() {
program-available "${1}" && return 1
return 0
}

# usage: program-available $PROGRAM_NAME
# returns 0 if program is available, 1 if program isn't available
program-available() {
command -v "${1}" >/dev/null 2>&1 && return 0
return 1
}

# usage: die-if-file-not-present $FILENAME_WITH_PATH "$MESSAGE"
die-if-file-not-present() {
[ ! -f "${1}" ] && die "${2}"
}

# usage: cd-or-die $DIRECTORY "$MESSAGE"
cd-or-die() {
cd "${1}" || die "Couldn't cd to ${1}."
}

https://redd.it/12b944i
@r_bash

Problems with sudo and single quote.

I executed the following command in interactive bash, but output from echo contains string '-e'.

```shell

sudo -u root sh -c 'for name in raw mangle nat filter; do echo -e "#######$name#######\\n"; iptables -t $name -nvL; done'

```

https://redd.it/12bfhw6
@r_bash

Progress bar showing the progress of a data migration between two different RDBMS?

One of the most amazing features I've ever seen in Linux terminals is that progress bar showing the going of a certain package installation, for example.

I'd like to make the same but to show the progress of a data migration between two different databases. I have a table (let's call it "followup") that records the process of the migration through a status, that is, if a table in the destiny database has been filled with all the data from the source, its status in the followup is put as "ok". It's also known the total amount of tables that is to be migrated. Thus, the relation between "the amount of tables migrated" and "the total amount to be migrated" is known.

An important point here: I have a main shell noscript that executes the SQL migration file inside it. Therefore, when the latter is executed, my shell noscript gets stuck in that specific line. One thing I've been struggling with is that when the SQL migration file is being executed, my shell noscript can't be executed in parallel to track the progress of the migration.

The following code just shows how I'm executing the SQL noscript inside my shell noscript. The nohup command is used as a way of avoiding the migration itself to stop because of network issues or stuff like that.

nohup sqlplus -s / as sysdba <<< @./noscript.sql > /tmp/log.out 2>&1

How could I make a progress bar showing the progress of this migration through a shell noscript (we're talking about bash in this case)? I showed the whole scenario to clear my intentions, but I think that knowing the relation explained before and how to execute this SQL noscript in parallel with a progress bar record might be enough to solve the problem.

Is it even reachable? I appreciate any help, thoughts, ideas, etc.

https://redd.it/12bgyeb
@r_bash

timeout vs gtimeout? (MacOS)

I am cleaning up a Makefile, and see this line:

TIMEOUT := $(shell command -v timeout || command -v gtimeout)

AFAIK gtimeout is needed for MacOS.

Is this really needed for MacOS?

I can't test it, I only use Linux.

https://redd.it/12bf462
@r_bash

How can i have this ignore files that are already gzipped?

I am working on a noscript and I want to ignore already gzipped files so that it is not trying to zip them again. What would be the best way to do so?

for LOG in $(find "${ARCHIVE}/${COMPONENT_TYPE}" -type f -mtime +"${AGE_TO_ZIP}"); do
gzip "${LOG}" -S .$(date -I).gz
done


Edit: Thanks for the help. Solution was to add ! -name \\\*.gz after my age variable

https://redd.it/12bjgq0
@r_bash

GPT quality check

I'm not very good at bash noscripting so I asked chatgpt solve a problem for me. I want to ask if any string in one array also exists in the other array. It gave me a result and it works, my question here is if it is a good way of doing it?

this noscript won't be running on huge datasets so this is for my curiosity.

#!/bin/bash

# define the first array
array1=("apple" "banana" "orange" "pear" "peach")

# define the second array
array2=("banana" "grape" "peach" "watermelon")

# loop through each element in array1
for item in "${array1@}" do
# check if the item is in array2
if [ " ${array2[*} " == " $item " ]]
then
echo "$item is in array2"
else
echo "$item is not in array2"
fi
done

https://redd.it/12biwvc
@r_bash

To provide an exit 1 status and also allow the noscript to continue running

Hello all,

I want to check if it is possible to add an exit 1 statement if $comparison_delta1 and/or $comparison_delta2 is not empty and then also allow the noscript to finish? The idea is that, I want to deploy my noscript to Jenkins, mark my Jenkins build failed when comparison_delta* exist and the output of it. I can go through my failed Jenkins jobs to find out the servers that aren't supposed to be pinging on my subnets.

&#x200B;

Here is part of my noscript. This part is to compare the ping results of the subnets to the master/gold copy (known IP addresses). Once the comparison function is completed, the noscript will be finishing with cleaning up old ping/comparison files. Thank you all in advance.

&#x200B;

\# Comparing ping result of 192.28.x.x subnet against gold copy

cat $output_file1|awk -F. {'print $1'}|while read host; do egrep -w $host $mastercopy1 >/dev/null||echo $host "${BOLD}${RED}is NOT my server!${RESET}" >> $comparison_delta1

done

echo "See result for 192.28.x.x subnet"

if [ -s "$comparison_delta1" \]

then

cat $comparison_delta1

else

echo -e "${BOLD}${GREEN}Subnet only known IP pinging!${RESET}"

fi

\# Comparing ping result of 192.27.x.x subnet against gold copy

cat $output_file2|awk -F. {'print $1'}|while read host; do egrep -w $host $mastercopy2 >/dev/null||echo -e $host "${BOLD}${RED}is NOT my server!${RESET}" >> $comparison_delta2

done

echo "See result for 192.27.x.x subnet"

if [ -s "$comparison_delta2" \]

then

cat $comparison_delta2

else

echo -e "${BOLD}${GREEN}Subnet only have known IP pinging!${RESET}"

fi

\##Cleaning up old logs

find /tmp/ \\( -name 'PingResults*' -o -name 'Non-Known_IP*' \\) -mmin +1 -exec rm -f {} \\;

https://redd.it/12bpna0
@r_bash

Here document detail - poorly written docs?

According to the Bash docs:

> This type of redirection instructs the shell to read input from the current source until a line containing only word (with no trailing blanks) is seen.

Since it explicitly called out no trailing blanks, I read that as allowing leading blanks but that does not seem to work. I'm just curious if other people agree with my reading and think the docs would be clearer it they read with no leading or trailing blanks or would this be clear if only I understood X (in which case, what is that X).

https://redd.it/12bog6k
@r_bash

Get all arguments between named arguments

I want to run a function such as

foo -e 1 2 3 -f 4 

so it recognizes 1 2 3 as arguments to be appended to -e case in the switch-like statement i have at present:

foo() {
  exclude=()
  while test $# -gt 0; do
    case $1 in
      -e) exclude+=$2; shift 2;;
       *) break;;
    esac
  done
  
  testlist=(1 2 3 4)
  for e in ${exclude[@]}; do
    testlist=(${testlist[@]/$e/})
  done

  echo ${testlist[@]}
}

The -f is just a dummy to serve as example for elements to stop being appended to -e case.

Any ideas on how to do this?

https://redd.it/12brmia
@r_bash

Is there a recommended alternative to getopts?

Hi everyone, I’m a beginner in Bash noscripting and I need some advice on how to parse options.

I know there are two common ways to do it:

Writing a custom "while loop" in Bash, but this can get complicated if you want to handle short-form flags that can be grouped together (so that it detects that “-a -b -c” is the same as “-abc”)
Using getopts, but this doesn’t support long-form options (like “–help”)

I’m looking for a solution that can handle both short-form grouping and long-form, like most noscripting languages and the Fish shell have (argparse). Is there a recommended alternative to getopts that can do this?

Thanks!

https://redd.it/12bqmyi
@r_bash

Are there any quality of life improvements that you add to bash?

I have installed bash-autocompletion and that's pretty awesome! I've also tried to use autojump and it's finnicky and I couldn't get it to work for bash, despite it claiming first class support for bash?

https://redd.it/12bptfj
@r_bash

shell appended symbol ~ %

How to change that ?

https://redd.it/12ciptb
@r_bash

Question regarding Logrotate

Hi all, I am trying to understand logrotate on my linux servers. I have read all the docs on it and how we use individual config files to control other app logs. My question is does the logrotate.conf set a precedence over the files in logrotate.d directory? I would also like to know can you do away with the files in the logrotate.d directory and just put all the info in the logrotate.conf file? Thanks any other helpful info would be appreciated as well.

https://redd.it/12cp7q1
@r_bash

zsh-style menu completion in bash?

Is there any way to get zsh-style tab menu completion in bash? Specifically, where the list of completions appears, you can tab through them, and once you've selected it the menu disappears? https://asciinema.org/a/2q4M7y9cu476xrlK6TLfCZlJg

https://redd.it/12cqxxl
@r_bash

Automating unzipping files in multiple sub folders

Hello Everyone,

I'm planning on how to write this bash noscript.

I currently have a parent directory called 'logs' containing sub-folders that populate once daily with the current dates. In these sub-folders are .csv.gzip files that I need to unzip. I want to automate the process by creating a cron job to run this bash noscript. I'm new to bash noscripting, so please bare with me.

Can anyone point me to resources that best explain how to accomplish this?

I think iterating using a for loop into each sub-folder and then having an if statement to check if it contains the .csv.gzip file extension and if the value is true to unzip the files. Then, move to the next sub-folder.

All the sub-folders have this format YYYY-MM-DD.

Any help would be greatly appreciated.

https://redd.it/12cxwba
@r_bash

YAD noscript help.....A progress bar without error window

I'm having issues implementing an error window that stops the progress and shows what failed. currently, the window will continue regardless of the error.

code is :



\#!/bin/bash -f


\#log not working
log="/home/pi/yad/logs/Rut_Check_error.log > $(date +'%FT%T%Z')"
(
\# =================================================================
echo "5"
echo "# Varifying Data Is correct." ; sleep 3
\## Calling exp. noscript
./Print_Qr_exp.sh && tail
\# =================================================================
echo "25"
echo "# Collecting Rut Infomation." ; sleep 2
grep "Mac," Rut-Info.log | tr ',' '\\n' | tail -n1 > mac.log
\# =================================================================
echo "55"
echo "# Getting the printer ready." ; sleep 3
\# Script runs Python env and prints
./activ.sh
\# =================================================================
echo "85"
echo "# Storing data logs" ; sleep 4
\## mv rut-info.txt (rename it to date) ; mv inst.log (rename with date) into new folder
\# =================================================================
echo "100"
echo "# Successful" ; sleep 1
) |
yad --progress --center --borders=70 \\
\--width=850 --height=650 \\
\--noscript="#### Progress Status ####" \\
\--text="Processing Data In Session." \\
\--text-align=center \\
\--percentage=0 \\
\--auto-kill \\
\#--auto-close \\

(( $? != 0)) && yad --error --text="Error in yad command." --width="400" --height="400" --noscript="Error Data"\\
exit 1

https://redd.it/12czvv6
@r_bash

smarter way to test bash noscripts and list all external commands

I was wondering if there were smart ways to find all uses of external commands in a bash noscript ?

i.e. is grep using built in bash or external grep etc (( can't think of a perfect command where its in both places at this time of night :P )).


The only solution in my head is strace or something but was wondering if there were a better way?

https://redd.it/12d2yhm
@r_bash

Simple dmenu config editor noscript
https://redd.it/12d3i7c
@r_bash

Is there a way to automate selecting gdm3 during Xubuntu install?



https://redd.it/12d7u7h
@r_bash

quote help

I am trying to build information for my bash prompt. The following command works in the terminal.

git worktree list 2>/dev/null | grep -v "^${HOME} " | sed -e 's/^.\[//' -e 's/\]$//'

When I add this to .bashrc, I get an error stating the following:

grep: ": No such file or directory

This is the line in the .bashrc file

PS1="${PS1}$(git worktree list 2>/dev/null | grep -v \"^${HOME} \" | sed -e 's/^.[//' -e 's/\]$//')"

https://redd.it/12dgc05
@r_bash