line buffering vs block buffering

Hi, after trying appending to a file with awk some weird occurrence happened

awk -i inplace '{print $0} ENDFILE{print "endoffile"}' somefile

the next command in terminal finish immediately and throws an error with exit status 1:

cat -A
cat: -: input file is output file

Now the `grep` (which has `--line-buffered` as a possible flag) does fine

grep -

So, my suspicion was `awk -i inplace` has done something wrong, and the [inplace extension manual](https://www.gnu.org/software/gawk/manual/htmlnode/Extension-Sample-Inplace.html) does suggest so

>redirect gawk's standard output to /dev/null

Slightly different from suggested, but this works

awk -i inplace '{print $0} ENDFILE{print "endoffile"}' somefile &>/dev/null

also `sed --in-place` has no problem at all

sed -i '$r /dev/stdin' somefile <<< "endoffile"

So what is the cause of this, and is the manual slightly wrong? It doesn't seems awk -i inplace is like sed -i emulation, like suggested. Also, is &>/dev/null mandatory to follow inplace extension?



https://redd.it/1ikudxt
@r_bash

Learning bash, trying to get it to do something stupid

I'm writing a noscript to handle my code projects, and something stupid I want to add is an ffmpeg command to play every mp3 in a folder after it opens my project in the IDE. Me & GPT (good idea for a romance novel, you're welcome) got this far:

for i in *.mp3; do

ffplay -nodisp -autoexit "/home/scottishcomedian/Music/bash_bullshit/$i"

done

And when I run it, it just hits me with the blank console. What am I doing wrong, oh wise elders?

https://redd.it/1invmde
@r_bash

illegal number problem

Hey, I struggle with some noscript.

var="nef892na9s1p9asn2aJs71nIsm"

for counter in {1..40}
do
var=$(echo $var | base64)
if $counter -eq 35
then
echo $var | WC -c
fi
done

It always give me: illegal number: {1..40}
Can someone help?

https://redd.it/1iogfzm
@r_bash

How to parse a nested JSON file in an old unix version

Hi, I'm trying to split a JSON file by transaction_id wherein 1 transaction_id=1 record. However, I feel like my shell noscript is failing due to the fact that it cannot read the JSON file and it won't proceed to processing on what I want it to do. You may see the snippet of my code below.


# Extract all transaction_id values using a regular expression

echo "$content" | sed -n 's/.*"transaction_id":\s*"\([^"]*\)".*/\1/p' | while read transaction_id; do

# Debugging: Show the current transaction_id being processed and log it

echo "Processing transaction_id: $transaction_id" | tee -a "$BATCH_LOG"



# Get the last character of the transaction_id

last_char="${transaction_id: -1}"



# Debugging: Show the last character of the transaction_id and log it

echo "Last character of '$transaction_id': $last_char" | tee -a "$BATCH_LOG"



# Check the last character and categorize

if [[ "$last_char" =~ [0-4] ]]; then

echo "$transaction_id" >> "${file%.json}_01.json"

# Debugging: Log which file the transaction_id is being saved to

echo "Saved to: ${file%.json}_01.json" | tee -a "$BATCH_LOG"

elif [[ "$last_char" =~ [5-9] ]]; then

echo "$transaction_id" >> "${file%.json}_02.json"

# Debugging: Log which file the transaction_id is being saved to

echo "Saved to: ${file%.json}_02.json" | tee -a "$BATCH_LOG"

elif [[ "$last_char" =~ [a-l] ]]; then

echo "$transaction_id" >> "${file%.json}_03.json"

# Debugging: Log which file the transaction_id is being saved to

echo "Saved to: ${file%.json}_03.json" | tee -a "$BATCH_LOG"

elif [[ "$last_char" =~ [m-z] ]]; then

echo "$transaction_id" >> "${file%.json}_04.json"

# Debugging: Log which file the transaction_id is being saved to

echo "Saved to: ${file%.json}_04.json" | tee -a "$BATCH_LOG"

else

# Debugging: Log unexpected last characters

echo "Unexpected last character '$last_char' for transaction_id: $transaction_id" | tee -a "$BATCH_LOG"

fi

done


I hope someone can help I've been losing my mind over this.

https://redd.it/1ip6rpi
@r_bash

Transposing args in noscript, including quotes

I'm trying to create a noscript to interact with my docker containers without having to shell in and run commands manually. It's a very simple noscript:

#!/bin/bash

ALLARGS="$@"
docker compose exec api ash -c "cd ../ && alembic ${ALLARGS}"

I tried a few things (${ALL_ARGS//\"/\\\"}, sed, others), but finally noticed that "$@" simply doesn't contain the double quotes. Is there a way to transpose the args as is?

https://redd.it/1iow04v
@r_bash

Check if number of arguments is one after all the flag

I have a noscript who can take more than one flag.

./noscript -a list is the same than ./noscript list all but list can have other parameter than all so what i want is ./noscript -a list somethingHere give a error.


So what i have test is if $3 is empty when -a is given.

But if the user type ./noscript -a -s list this give a error because $3 is no longer empty but the exeption behavior is to work.

if aflag = 1 and (after 'list' is empty)
do something
else
error

So my idea is this on pseudo code. But i don't know how to check dynamicly if the $n+1 after list ( $n) is empty

https://redd.it/1ip7zyv
@r_bash

Unexpected curl command behaviour ?

The following command reads the exchange rate information for the EUR/USD currency pair from HTML page and prints it.

page=$(curl -s https://www.widgets.investing.com/live-currency-cross-rates?theme=darkTheme&pairs=1); echo "$page" | pup 'div.pid-1-bid text{}'

But why doesn't the following command work, instead it prints the entire page?

curl -s https://www.widgets.investing.com/live-currency-cross-rates?theme=darkTheme&pairs=1 | pup 'div.pid-1-bid text{}'

https://redd.it/1imsp0s
@r_bash

WHAT IS BASH DOING?

**UPDATE**

So it looks like FFPMEG is interacting with the shell in some way... so adding this to the FFPMEG line seems to have resolved the issue.

</dev/null >/dev/null 2>&1



I am doing something dumb... I guess? But I can't figure out what in the heck, when using the EVAL statement, previous variables are stripping off a character for every other loop? Sound confusing? I am confused...

I am using FFMPEG and writing a quick little bash wrapper to automatically detect silences and split apart an audio file.

Let me see if I can show what is going on... This is WITHOUT the eval command...

IFS='\r\n'
while read -r line1; do
IFS= read -r line2
echo "Start: $line1"
echo "End: $line2"
echo "Prev: $PREV"
START="${PREV}"
END="${line1}"

echo "/usr/bin/ffmpeg -hidebanner -loglevel error -i ./${INPUT} -ss ${PREV} -to ${line1} output${COUNT}.wav"
COMMAND='/usr/bin/ffmpeg -hidebanner -loglevel error -i '
COMMAND+="./${INPUT} -ss ${START} -to ${END} output${COUNT}.wav"

echo "${COMMAND}"
# eval ${COMMAND}

COUNT=$(( COUNT + 1 ))
PREV=$line2
echo ''

done <<< $SILENCES

This outputs exactly what I would expect...

Start: 6.04
End: 6.30
Prev: 0
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 0 -to 6.04 output0.wav
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 0 -to 6.04 output0.wav
Start: 21.72
End: 21.98
Prev: 6.30
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 6.30 -to 21.72 output1.wav
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 6.30 -to 21.72 output1.wav
Start: 24.18
End: 24.53
Prev: 21.98
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 21.98 -to 24.18 output2.wav
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 21.98 -to 24.18 output2.wav
Start: 43.34
End: 43.58
Prev: 24.53
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 24.53 -to 43.34 output3.wav
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 24.53 -to 43.34 output3.wav

SO then I uncomment the eval command. That is the only change. I have tried with and without " ", using and not using { } to see if I am interpretting the string differently.

`eval ${COMMAND}`

SOOOO.... Here is the output

Start: 6.04
End: 6.30
Prev: 0
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 0 -to 6.04 output0.wav
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 0 -to 6.04 output0.wav

Start: 1.72
End: 21.98
Prev: 6.30
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 6.30 -to 1.72 output1.wav
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 6.30 -to 1.72 output1.wav
-to value smaller than -ss; aborting.

Start: 24.18
End: 24.53
Prev: 21.98
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 21.98 -to 24.18 output2.wav
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 21.98 -to 24.18 output2.wav

Start: 3.34
End: 43.58
Prev: 24.53
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 24.53 -to 3.34 output3.wav
/usr/bin/ffmpeg -hidebanner -loglevel error -i ./audio.wav -ss 24.53 -to 3.34 output3.wav
-to value smaller than -ss; aborting.

SO Every other iteration... the ${PREV} variable has the first digit/character stripped. So for the second iteration:

21.72 -> 1.72

BUT this ONLY happens when I have the EVAL command AFTER the echo commands. So somehow the eval command is affecting that variable, but I can't see how. Thanks!

https://redd.it/1im9sik
@r_bash

How can I combine these two awk commands?

git ls-remote -t `https://github.com/PowerShell/PowerShell.git` | awk -e '$2 ~ /v[0-9].[0-9].[0-9]$/ {sub(/refs\/tags\/v/,"");print $2}' | awk 'END {print}'


This command gives me the latest version of Powershell. I just want the result from the last line. How can I add the END rule to the first awk command?

https://redd.it/1ik223z
@r_bash

Does anyone know of an interactive BASH command builder that is built with BASH noscripts?

I'd love to have an interactive shell environment that helps students new to BASH navigates its opacity and easily survey the breadth of options. I would also like it for myself, since one can always learn more.

This is especially useful when BASH is required for a subject like Practical Control Theory with Python but is not the subject of the class. Think of it as a nice gateway drug for BASH.

And while I'm asking for ponies, I think it would be best to make this run natively inside terminal emulators. So writing it as a BASH noscript would be great.

https://redd.it/1iq37l3
@r_bash

Could anyone show me how parallel works?

Does anyone have good examples of how 'parallel' can work with bash functions or noscripts? I have several for processing filetypes that I'd like to make happen more quickly

https://redd.it/1iqaw6o
@r_bash

tree returning invalid filename
https://redd.it/1iqjnz1
@r_bash

Help with login noscript

I have created two login noscripts, one of which is working wonderfully. However, the other only works under certain conditions and I need some help making it more circumstance independent. Here's what I mean:

Both noscripts are for starting Google Chrome PWAs and then docking them to my system tray with kdocker. The first one is for Google Messages and the second is for Gmail.

Here is the first noscript:

#!/bin/bash

# Start Messages
/opt/google/chrome/google-chrome --profile-directory=Default --app-id=hpfldicfbfomlpcikngkocigghgafkph &

# Set ID variable
messages=$(xdotool search --sync --name "Messages - Google Messages for web")

# Pin to tray
kdocker -w $messages -i /home/ego/.local/share/icons/hicolor/128x128/apps/chrome-hpfldicfbfomlpcikngkocigghgafkph-Default.png &

# Quit
exit

And here is the second:

#!/bin/bash

# Start Gmail
/opt/google/chrome/google-chrome --profile-directory=Default --app-id=fmgjjmmmlfnkbppncabfkddbjimcfncm &

# Set ID variable
gmail=$(xdotool search --sync --name "Gmail - Inbox - myemail@gmail.com - Gmail")

# Pin to tray
kdocker -w $gmail -i /home/ego/.local/share/icons/hicolor/128x128/apps/chrome-fmgjjmmmlfnkbppncabfkddbjimcfncm-Default.png &

# Quit
exit

The problem with the Gmail noscript is that this string: Gmail - Inbox - myemail@gmail.com - Gmail changes based on how many emails I have in my inbox. For example, if I have three emails, it will read: Gmail - Inbox (3) - myemail@gmail.com - Gmail. This causes xdotool to not find it and subsequently causes kdocker to fail to pin it in the system tray unless I specifically have zero unread messages in my inbox, which is obviously not ideal. Can anybody help me figure out a better way to target the windows in both of my noscripts so that they are able to find the correct window in more varying conditions?

https://redd.it/1iq7bko
@r_bash

printf

There are 3 places you can get info on how to use printf in bash. One is by consulting the bash man page (or help), because bash's builtin printf command is used by default. But you probably also have an installed printf command. For example, at /usr/bin/printf. So you can check man 1 printf. There's also the printf library, which you can read about in man 3 printf. Even though bash has printf builtin, it depends on the printf library, and so some of the stuff in the two man pages applies to the builtin command as well.

Using all of that, I came up with this printf command that I put in my PS1:

printf "\\u2501%.0s" $(seq "$(tput cols)")

The argument to the format string (the seq) gets the current width of the terminal window, as an integer, and then spits out that many arguments, in the form of number strings. The format string produces a Unicode character and then one of the string arguments converted to zero-width. A zero-width string is literally just "". So the printf is printing the Unicode character and then nothing. But because there are, say, 100 string arguments, it'll repeat this over and over again, that many times.

The reason I came up with this is because, for a while, I was having trouble seeing where one command ran and ended when I was scrolling through my terminal window history. This printf creates a nice visual barrier that's easy to catch even when you're scrolling in the window.

Anyway, I thought it was pretty clever so I wanted to share with you guys.

https://redd.it/1iq782d
@r_bash

Here're 22 best free video converter software and online apps that you can use to convert video formats without any cost.
https://www.videoproc.com/media-converter/best-video-converter.htm

https://redd.it/1iqowa9
@r_bash

Where do you store reusable code snippets?

Hey folks! Curios where do you store your code snippets? If you work in a team how do you manage it?

https://redd.it/1iqwadh
@r_bash

Bash noscript explain

This is a noscript in Openwrt. I know what this noscript does at higher level but can I get explanation of every line.

case $PATH in
(*[!:]:) PATH="$PATH:" ;;
esac

for ELEMENT in $(echo $PATH | tr ":" "\n"); do
PATH=$ELEMENT command -v "$@"
done

https://redd.it/1iqrfzb
@r_bash

Can someone explain the following: mkdir ${1:-aa}

Trying to understand the following:

mkdir ${1:-aa) and it seems to work by changing 1 to another number it works as well.

also

mkdir ${a:-a} creates a directory 1

but

mkdir ${b:-b} creates b

Any help would be great as learning.

https://redd.it/1iso5zj
@r_bash

Efficient way to find outliers?

***Sorry if this is the wrong place, I use bash for most of my quick filtering, and use Julia for plotting and the more complex tasks.***

I'm trying to clean up my data to remove obvious erroneous data. As of right now, I'm implementing the following:

awk -F "\"*,\"*" 'NR>1 && $4 >= 2.5 {print $4, $6, $1}' *


And my output would look something like this, often with 100's to 1000's of lines that I look through for both a value and decimal year that I think match with my outlier. lol:

2.6157 WRHS 2004.4162
3.2888 WRHS 2004.4189
2.9593 WRHS 2004.4216
2.5311 WRHS 2004.4682
2.5541 WRHS 2004.5421
2.9214 WRHS 2004.5667
2.8221 WRHS 2004.5695
2.5055 WRHS 2004.5941
2.6548 WRHS 2004.6735
2.8185 WRHS 2004.6817
2.5293 WRHS 2004.6899
2.9378 WRHS 2004.794
2.8769 WRHS 2004.8022
2.7513 WRHS 2004.9008
2.5375 WRHS 2004.9144
2.8129 WRHS 2004.9802


Where I just make sure I'm in the correct directory depending on which component I'm looking through. I adjust the values to some value that I think represents an outlier value, along with the GPS station name and the decimal year that value corresponds to.

[Timeseries Plot](https://imgur.com/a/3p4LIqa)

Right now, I'm trying to find the three outlying peaks in the vertical component. I need to update the noscript to reflect that the lines shown are a 365-day windowed average.

I do have individual timeseries plots too, but, looking through all 423 plots is inefficient and I don't always pick out the correct one.

I guess I'm a little stuck with figuring out a solid tactic to find these outliers. I tried plotting all the station names in various arrangements, but for obvious reasons that didn't work.

Actually, now that I write this out, I could just create separate plots for the average of each station and that would quickly show me which ones are plotting as outliers -- as long as I plot the station name in the noscript...

okay, I'm going to do that. Writing this out helped. If anyone has any other idea though of how I could efficiently do this in bash, I'm always looking for efficient ways to look through my data.

:)

https://redd.it/1isop2r
@r_bash

Protect exclamation point when using double quotes and sed

Hi!

The following line

sed "/$PATTERN1/,/$PATTERN2/{/$PATTERN1/n;/$PATTERN2/!d;}" $FILE

deletes everything between the two patterns but not the lines containg them. I want to abstract this to a function. However, even when issuing the command interactively, the above line always result in this error: bash: !d}: event not foundz. This makes sense because ! is history expansion. If I use the line with single quotes, there's n problem but I cannot expand the value of shell variables, which is what I want. I also tried escaping the exclamation sign, i.e. \!, but I excpetedly get unknown command:'`.

Is there a way of protecting the exclamation point inside the sed command line when using double-quotes so it doesn't try to do history expansion?

Thanks!

https://redd.it/1itm6nk
@r_bash

🎉 FuzPad 2.0 is now released 🎉 FuzPad is a minimalistic note management solution. Powered by fzf
https://github.com/JianZcar/FuzPad

https://redd.it/1ito7d2
@r_bash