Parse XML from a larger non-XML string in BASH

Hello people, I'm using a Calibre utility to fetch e-book metadata (via a BASH noscript):

$ fetch-ebook-metadata --isbn="9780791085455" --opf

The utility works by trying several resources, it returns errors as well as the metadata in XML format,

see output:

Failed to get cached URL from google for URL: https://www.amazon.com/Peyote-Mescaline-Drugs-Straight-Facts/dp/0791085457 with error: HTTP Error 429: Too Many Requests
Failed to get cached URL from google for URL: https://www.amazon.com/Nanny-Lana-Ferguson-ebook/dp/B0B6B41KFP with error: HTTP Error 429: Too Many Requests
Failed to get cached URL from google for URL: https://www.amazon.com/Basic-Geriatric-Nursing-Patricia-Williams/dp/0323554555 with error: HTTP Error 429: Too Many Requests
Response from wayback machine: {'archivedsnapshots': {},
'url': 'https://www.amazon.com/Basic-Geriatric-Nursing-Patricia-Williams/dp/0323554555'}
Failed to get cached URL from google for URL: https://www.amazon.com/-/es/M-Foster-Olive/dp/0791085457 with error: HTTP Error 429: Too Many Requests
Response from wayback machine: {'archivedsnapshots': {},
'url': 'https://www.amazon.com/-/es/M-Foster-Olive/dp/0791085457'}
Failed to get cached URL from google for URL: https://www.amazon.com/Responding-Emergency-American-Red-Cross/dp/1584805544 with error: HTTP Error 429: Too Many Requests
<?xml version='1.0' encoding='utf-8'?>
<package xmlns="http://www.idpf.org/2007/opf" unique-identifier="uuidid" version="2.0">
<metadata xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:opf="http://www.idpf.org/2007/opf">
<dc:language>eng</dc:language>
<dc:subject>Juvenile Nonfiction</dc:subject>
<dc:subject>Health &amp; Daily Living</dc:subject>
<dc:subject>Substance Abuse</dc:subject>
</metadata>
<guide/>
</package>

I only need this part (obviously):

<?xml version='1.0' encoding='utf-8'?>
<package xmlns="http://www.idpf.org/2007/opf" unique-identifier="uuidid" version="2.0">
<metadata xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:opf="http://www.idpf.org/2007/opf">
<dc:language>eng</dc:language>
<dc:subject>Juvenile Nonfiction</dc:subject>
<dc:subject>Health &amp; Daily Living</dc:subject>
<dc:subject>Substance Abuse</dc:subject>
</metadata>
<guide/>
</package>

Running the BASH noscript via Cygwin, but I can move to a Linux box.

I Googled it, but the Stack Overflow threads I got don't mention this scenario, at least not that I saw. My Shell capabilities aren't that strong.

https://redd.it/12v1lo6
@r_bash

sleep in while clause

Hello,

i found a noscript where there is sleep () inside done clause, i am not sure... but from my point of view its pure mess?

&#x200B;

while [[ something \]\]; do

.....

sleep X;

done << (sleep Y);

&#x200B;

does it make any sense to use 2 times sleep and especially inside done clause...

&#x200B;

&#x200B;

thanks!

https://redd.it/12v4cu7
@r_bash

Cool bash CLI for optimizing your development productivity on Git and other things

This a bash CLI I have been developing in my spare time. It provides developers with the ability for optimising the execution of commonly performed tasks, commands, directory navigations, and environment setups/noscript executions.

https://github.com/hwixley/WIX-CLI

Please check it out, and leave a star if you found it useful! (it helps me out alot) :)

https://redd.it/12vascj
@r_bash

Why does this work??

I am not a programmer - I like to tinker.

I needed to batch process all file name in a folder all files having the naming convention as follows

3. 3. 2023-03-14 p.1 teaching.mp3

3. 3. 2023-03-14 p.2 discussion.mp3

etc

I needed to remove all characters upto the 2023 and keep the remaining string from 2023 to the end.

&#x200B;

Here's the noscript(which I run in the folder it needs to operate):

1 For file in * ; do
2 if [[ -f “$file” ]]; then
3 upto=“2023”
4 result=$(echo “$file” | sed “s/^.*$upto/$upto/“) // this works
5 result=$(echo “$file” | sed “s/^.*$upto//“) // this does not work
6 echo $result
7 fi
8 Done

When I use the noscript with line 5 it does not work

But it works with line 4

Any idea why??

https://redd.it/12vigo6
@r_bash

I wrote a small utility to merge my dashcam recordings automatically using Bash, RFC!
https://github.com/brlin-tw/dashcam-recording-automerge

https://redd.it/12w2n1y
@r_bash

Bash noscripting indentation: Tabs or spaces? - Poll
https://poll-maker.com/poll4789276x49bA4EBF-148

https://redd.it/12wgkl3
@r_bash

How to interact with the history of the shell that the noscript is being executed?

I've found several ways but none seem to work, the closest I got is one where I got output from the commands run by the own noscript but not from the shell. I've also tried #!/bin/bash -i, set -o, looking into \~/.bash_history... that last one seemed to work, but it didn't. I want to have the same output that I'd have when I type "history" on the terminal.

https://redd.it/12wq567
@r_bash

E: Could not lock file (13: Permission denied)

I'm trying to install Node JS on linux and I keep getting errors saying what I've described in the noscript, I've treied to fix it but I can't change the permission.

I'm on Pop_OS, if that is related to the very problem I'm getting.

Cheers.

https://redd.it/12xuekv
@r_bash

How to one-line with & ?

Normally in a .sh file, I will write:

sleep 1000 &
echo 'Say this now'

But when I do it in bash -c

bash -c "sleep 1000 & ; echo 'say this now'"

It gives me an error:

 error near unexpected token `;'

How do I write it properly with bash -c?

https://redd.it/12xuacc
@r_bash

Garbage characters appearing in terminal output when using 'watch'

So, when I run the following bash command, my terminal output file gets filled with a lot of random garbage between the data I am looking for.

watch -t -n 1 'sensors k10temp-pci-00c3 -j | jq '..Tccd1.temp3input'' >> output.txt

> \[?1049h \[22;0;0t \[1;51r (B \[m \[4l \[?7h \[H \[2J46 \[51;193H \[H44 \[51;

What's causing this? Are console control characters being injected into my output file? How can I stop this?

It only happens when I run the command with watch. Running just:

sensors k10temp-pci-00c3 -j | jq '.[].Tccd1.temp3input'' >> output.txt

yields me

>45.25
>
>47.75
>
>44.75
>
>44.5
>
>48.75

as expected.

https://redd.it/12xyje3
@r_bash

Sed and Regex question

Hi everyone! I have a brief question regarding sed and regex that's been bothering me for the better part of a day. I'm not the most experienced with Bash and this little problem has been taking much longer than planned.

I have a set of text files containing strings with a decimal and 1-3 digits at the end (ex. RMNT000021.55) I need to remove all the decimals and everything after the decimal (to get RMNT000021).

So far, I've tried to use:

sed -E 's/\.+:digit:+/ /g'
# Wanted something to grab the period and 1+ digits after it
# but the end result is a space where the period was
RMNT000021 55

# And the command below I tried just does more or less the same.
sed "s/\..//"

I've checked on forums but haven't had much luck. Would anyone be able to help me?

https://redd.it/12xuvr9
@r_bash

Syntax error near unexpected token in "while IFS= read" loop

I have a noscript that hundreds of people have used without any issue but yesterday one user has reported they are getting the following error:

syntax error near unexpected token <' synohdddb.sh: line 612: done < <(printf "%s\0" "${hdlist@}" | sort -uz)

The part of the noscript giving the error is:

while IFS= read -r -d '' x; do
hdds+=("$x")
done < <(printf "%s\0" "${hdlist@}" | sort -uz)

What could cause a syntax error in that while loop for 1 person but not for hundreds of other people?

This person does only have 1 HDD but I've tested with just 1 HDD and I could not reproduce the error.

Here's the noscript minus the irrelevant parts. The error in this short noscript occurs on line 53 https://gist.github.com/007revad/e7ca1c185f593b2d93cccf5bd0ccd0c2

In case anyone wants to see the full noscript it is here:: https://github.com/007revad/Synology\_HDD\_db

https://redd.it/12y1wb1
@r_bash

Hey, Guys just need your small help!



I have been researching Linux for DevOps for quite some time and by researching I have stacked piles of sample bash noscripts. So, I want you all to go through my GitHub repo and if you found my repo for the sample noscript good, please Fork it and give my repo some stars. Thanks in advance.

My Github Repo: - https://github.com/yashwant7chaudhari/Sample-Bash-Scripts

https://redd.it/12y7ppm
@r_bash

Fixing SQL errors when data includes "|" in the string and when noscript uses "|" as the delimeter

Hi! I am wokring on a bash noscript that extracts data from android device, i have 11 noscripts in total making up one tool. Everything so far works without any issues, except for one noscript, which extracts data from .db files into a readable txt format.

No matter what I do, i cannot get the extraction to work properly when reading data from the chroe browser. It reads the data fine and outputs to a txt file correctly, up to the point where there is a "|" character in the noscript of a page (for example in the gmail page noscript). Also I've tried it with the \\t delimeter but as far as i can remember the issue persists there, it seems like the pipe character is still responible for the issues


I tried enclosing it in double quotes, but it still did not work. here's the snippet of code responsible for that functionality/error.


# Extract browsing history to a text file
sqlite3 "$CHROMEDATAPATH/History" "SELECT noscript, url, lastvisittime FROM urls" \
| while IFS='|' read -r noscript url lastvisittime; do
# Convert unix timestamp to human-readable date
date=$(date -d "@$((lastvisittime/1000000 - 11644473600))" "+%Y-%m-%d %H:%M:%S" 2>/dev/null)
echo "Title: \"$noscript\"" >> "$extr/browsinghistory.txt"
echo "URL: \"$url\"" >> "$extr/browsinghistory.txt"
echo "Last Visit Time: $date" >> "$extr/browsinghistory.txt"
echo "" >> "$extr/browsinghistory.txt"
done

and here's the error i get in terminal (execution of the extraction stops there, but the noscript continues even tho set -e flag is set). I am pretty sure the issue is with the special characters in the noscript fields. In the database it is a Long varchar.

&#x200B;

./bash/db_extractor.sh: line 108: http://gmail.com/|13325597979782411: syntax error in expression (error token is "://gmail.com/|13325597979782411")

Does someone have any idea how to handle this? Thank you in advance, have a nice day!

https://redd.it/12y9nvu
@r_bash

RegEx

Guys pls recommend some RegEx books in context of bash noscripting.

https://redd.it/12yclt8
@r_bash

Check this link now



I have been researching Linux for DevOps for quite some time and by researching I have stacked piles of sample bash noscripts. So, I want you all to go through my GitHub repo and if you found my repo for the sample noscript good, please Fork it and give my repo some stars. Thanks in advance.

My Github Repo: - https://github.com/yashwant7chaudhari/Sample-Bash-Scripts

https://redd.it/12yd6l0
@r_bash

Langchain AI Bash Tool (Plain English converted to Linux one-liner)

I created an end-to-end llm app with a simple UI for the Langchain Bash tooling. I've been tinkering with an LLM app that generates Bash one-liners using OpenAI's Large Language Model plugged in. The app enabled users to submit plain English requests, receive practical Bash one-liners, and keep a simple command history.

Of course, there's often more than one way to tackle a problem in Bash, and OpenAI's LLM, while powerful, needs help producing focused and logical results. The key to this has been fine-tuning the prompt logic, leading to improved output quality.

I'm eager to know your thoughts and am open to constructive feedback and suggestions.


The World Famous Bash One-liner Tool! (oneliner.tech)

https://redd.it/12yj3n5
@r_bash

Use real time terminal logs in noscript

Hi! I have a game server on pi 4 that is open in a terminal and runs indefinitely in the background thanks to the command screen. It will generate some logs in the terminal and one of them is the info of which player has logged in.

I want to create a noscript that reads this info and sends an email with the name of the player. Is this possible? What would be a sample code for this?

https://redd.it/12ymwud
@r_bash

beepbeep: A nice QOL addition to my workflow

[https://github.com/SpicyLemon/SpicyLemon/blob/master/bash\_fun/generic/beepbeep.sh](https://github.com/SpicyLemon/SpicyLemon/blob/master/bash_fun/generic/beepbeep.sh)

This started out a simple a command that preserved the last exit code and beeped twice. Since then, I made a few enhancements to it.

Features:

* Exits with the same exit code as the previous command.
* Number of beeps can be provided as an argument.
* If no arguments are provided, it will beep twice if the previous exit code was \`0\`, or, three times if it was not \`0\`.
* If a non-numeric argument is provided and the \`say\` command is available, it will execute \`say\` with the text provided as args.

I like to use it at the end of commands that I expect to take longer than a few seconds so that I can shift my attention somewhere else, but still know when the command finishes.

For example:

$ make test; beepbeep

I actually have it aliased to \`alert\`: \`$ alias alert='beepbeep'\`, which is what I think of it as (and is easier to type). I didn't want the function named \`alert\` though due to high probability of conflicting with other commands.

You can download that file (at the top) and source it to add the \`beepbeep\` function to your environment. The file can also be directly executed if you prefer not to add it as a function to your environment.

Here's just the function definition from that file:

beepbeep () {
# Presever last exit code
local ec=$?
local count
if [[ "$#" -gt 1 || "$1" =~ [^[:digit:]] ]]; then
if command -v say > /dev/null 2>&1; then
say "$@"
return $ec
fi
printf 'beepbeep: Unknown argument(s): %s\n' "$*" >&2
else
count="$1"
fi

if [[ -z "$count" ]]; then
if [[ "$ec" -eq '0' ]]; then
count=2
else
count=3
fi
fi

printf '\a'
count=$(( count - 1 ))
while [[ "$count" -gt '0' ]]; do
sleep .3
printf '\a'
count="$(( count - 1 ))"
done
return $ec
}

It's just been so useful for me that I thought I'd share.

https://redd.it/12yurcx
@r_bash

Help

I need help writing a bash noscript to delete every folder that is older than 6 months, and delete 3 folder a week and save 2 for month 4 to month 6. The folders have the following naming convention; “files20230425”, “files20230424” z

https://redd.it/12yw7yo
@r_bash

5 Inbuilt Bash Variables That Every Developer Should Know
https://levelup.gitconnected.com/5-inbuilt-bash-variables-that-every-developer-should-know-a2d60721a472?sk=6debc2c4c3196f2fcd954690d7fb989f

https://redd.it/12z9snv
@r_bash