bash noscript `sed` help

Hello, I am a college student working on a summer project, but I feel like I have been stuck for too long on this one thing.

**TLDR:** I am working on a bash noscript and am having issues with \`sed\` not putting markdown for an indented bullet point in front of the line for any ports it finds.

So I am trying to work on a bash noscript and I have been stuck on part using \`sed\` for two weeks, so I come to you all for help. So I am trying to search through an nmap scan that I have happening earlier in the noscript, and add the markdown for an indented bullet point to the port lines. If I understand correctly I should be able to use regex as the searching pattern in \`sed\`, but I have been able to get every other thing I need working except for this one.

I will put a bunch of lines I have tried at the bottom so maybe you can see my thinking/attempts, but I have 2 different theories as to why what I am trying isn't working. Oh, and with the fun 3rd theory of me missing something simple and obvious.

1: I believe \`sed\` looks at \`\*\` as whatever character is right before it? So maybe because I am using that as my bullet point markdown it's thinking its a space? But things still don't seem to work when I replace it with a \`-\` instead?

2: I am missing something about what's needed to add regex into sed. Nothing too fancy here, I think I have tried the right (~~various~~) arguments. On its own I am pretty sure that my regex is right as I can verify that on its own.

Here are a number of the commands that I have tried so far

\`sed -e '/\[0-9\]+\\/\[A-Za-z\]\[A-Za-z\]\[A-Za-z\]\[\[:space:\]\]+open/gm/$\\t \* \\/'\`

\`sed 's/\[0-9\]+\\/\[A-Za-z\]\[A-Za-z\]\[A-Za-z\]\[\[:space:\]\]/\\t \* &/'\`

\`sed -e .....; /\^\[0-9\]\\{1,5\\}\\/\[a-z\]{3}$/s/\^/\\t \* /;\`

\`awk '/\[a-z\]\[a-z\]\[a-z\] open|\[a-z\]\[a-z\]\[a-z\] open/ {print " \* " $0}' /home/$ownerAccount/Desktop/$projectName/AaFinalDoc.txt >> /home/$ownerAccount/Desktop/$projectName/BbFinalDoc.md\`

This project is larger than anything I have tried before and because its fun I just keep adding to it after I finish the previous goal. I have historically been really bad in my programming classes but this feels fun so I don't want to give up!

I appreciate any help that any of you can give me, thank you!

EDIT: warrior0x7 pointed out I dont actually show my start and end goals, so here is an example that hopefully might help.

Nmap scan report for
PORT STATE SERVICE VERSION
8008/tcp open http?
8009/tcp open ssl/ajp13?
8443/tcp open ssl/https-alt?
9000/tcp open ssl/cslistener?
10001/tcp open ssl/scp-config?
MAC Address: 1C:53: (Google)
Aggressive OS guesses: Android 6.0 - 7.1.2 (Linux 3.18 - 4.4.1)
TRACEROUTE
1 66.90 ms 192.168.

Nmap scan report for 192.168.
PORT STATE SERVICE VERSION
8008/tcp open http?
8009/tcp open ssl/ajp13?
8443/tcp open ssl/https-alt?
9000/tcp open ssl/cslistener?
9080/tcp open glrpc?
10001/tcp open ssl/scp-config?
MAC Address: 1C:53: (Google)
Aggressive OS guesses: Android 6.0 - 7.1.2 (Linux 3.18 - 4.4.1)
TRACEROUTE
1 44.48 ms 192.168

Nmap scan report for 192.168.
PORT STATE SERVICE VERSION
135/tcp open msrpc Microsoft Windows RPC
139/tcp open netbios-ssn Microsoft Windows netbios-ssn
445/tcp open microsoft-ds?
5357/tcp open http Microsoft HTTPAPI httpd 2.0 (SSDP/UPnP)
MAC Address: D8:BB: (Micro-Star Intl)
Device type:
Aggressive OS guesses: Microsoft Windows 11 21H2 (97%)
TRACEROUTE
1 2.13 ms 192.168.

But the only thing I am looking at to alter (with this line that I am having issues with) is the ports. I already have adding markdown working for what I want to do to every other line. So that end result looks like this.

Nmap scan report for
PORT STATE SERVICE VERSION
* 8008/tcp open http?
* 8009/tcp open ssl/ajp13?
*

8443/tcp open ssl/https-alt?
* 9000/tcp open ssl/cslistener?
* 10001/tcp open ssl/scp-config?
MAC Address: 1C:53: (Google)
Aggressive OS guesses: Android 6.0 - 7.1.2 (Linux 3.18 - 4.4.1)
TRACEROUTE
1 66.90 ms 192.168.

Nmap scan report for 192.168.
PORT STATE SERVICE VERSION
* 8008/tcp open http?
* 8009/tcp open ssl/ajp13?
* 8443/tcp open ssl/https-alt?
* 9000/tcp open ssl/cslistener?
* 9080/tcp open glrpc?
* 10001/tcp open ssl/scp-config?
MAC Address: 1C:53: (Google)
Aggressive OS guesses: Android 6.0 - 7.1.2 (Linux 3.18 - 4.4.1)
TRACEROUTE
1 44.48 ms 192.168

Nmap scan report for 192.168.
PORT STATE SERVICE VERSION
* 135/tcp open msrpc Microsoft Windows RPC
* 139/tcp open netbios-ssn Microsoft Windows netbios-ssn
* 445/tcp open microsoft-ds?
* 5357/tcp open http Microsoft HTTPAPI httpd 2.0 (SSDP/UPnP)
MAC Address: D8:BB: (Micro-Star Intl)
Device type:
Aggressive OS guesses: Microsoft Windows 11 21H2 (97%)
TRACEROUTE
1 2.13 ms 192.168.

Hopefully that helps to clarify things.

https://redd.it/1dec1ti
@r_bash

Ignore error and continue with other files

Hi all, I can't seem to use the right search words to find what I'm looking for so I am braving r/bash with my query.

I have \~70 fastq.gz files in a directory that I need to unzip. Easy peesy, right?:

gzip -d *.gz


Turns out, some of the files are corrupted and this results in an error. The command simply stops and none of the other files get unzipped. How can I skip bad files and unzip good files?

https://redd.it/1delnyw
@r_bash

Easy question

At first I'm new to noscripting and this is for sure an easy question for you but:
I have a variable with file names separated by a space, like "FileA.txt FileB.txt FileC.txt".
I simply want to move those files to another directory. How can I get those file names readable for Linux?

https://redd.it/1deya1g
@r_bash

What does ${0%/} mean exactly

I've got a noscript that creates backups of my various machines on my network. I have the .sh file located in a directory on my nas and then my machines just access the nas to run the noscript.

I was having trouble figuring out how to set the working directory to the directory that the noscript is located in so I can store all the backups in the same directory. I did some research and discovered the line:

cd "${0%/}"

This line works and does exactly what I need but, I'd like to know what it means. I know cd and I know what the quotes mean but everything within the quotes is like a foreign language to me but I'd like to understand.

Thanks in advance

https://redd.it/1dfdjg4
@r_bash

k10s noscript feedback and next steps

I wrote a noscript to create a little CLI I dubbed k10s. I made this as a solution to more quickly open up various regional clusters next to one another in a window. I'd appreciate feedback on where to improve what I have done, as well as suggestions for any features and next steps to keep learning.

#! /usr/bin/env bash

k10s_dir=$HOME/.config/k10s
groups_file=$HOME/.config/k10s/groups

process_contexts() {
local index=0
local random=$RANDOM
local session="session-$random"
local split_times=$(($#-1))
tmux new-session -d -s "$session" \; switch-client -t "$session"

while [[ "$split_times" -gt 0 ]] ; do
tmux split-window -h -t "$session"
((split_times--))
done
tmux send-keys -t "$session:0.0" "tmux select-layout even-horizontal" C-m
for context in $@; do
tmux send-keys -t "$session:0.$index" "k9s --context $context" C-m
((index++))
done
}

save_group() {
mkdir -p "$k10s_dir"
touch "$groups_file"
local group=$(echo $@ | awk -F [=,' '] '{print $1}')
local contexts=$(echo $@ | awk -F [=,' '] '{for (i=2; i<=NF; i++) printf $i (i<NF ? OFS : ORS)}')
update_group "$group"
echo "$group"="$contexts" >> "$groups_file"
}

update_group() {
while read line; do
local group=$(echo "$line" | awk -F [=,' '] '{print $1}')
if [[ "$1" = "$group" ]]; then
sed -i "/$line/d" "$groups_file"
fi
done < "$groups_file"
}

start_group() {
while read line; do
local group=$(echo "$line" | awk -F = '{print $1}')
if [[ "$group" = "$1" ]]; then
local contexts=$(echo "$line" | awk -F = '{for (i=2; i<=NF; i++) printf $i (i<NF ? OFS : ORS)}')
process_contexts ${contexts[@]}
fi
done < "$groups_file"
}

usage() {
figlet -f slant "k10s"
cat <<EOT
k10s is a CLI that enables starting multiple k9s instances at once.

Usage: k10s [flags]

Flags:
-c, --context List of contexts to start up (e.g. k10s -c <CONTEXT_NAME> <CONTEXT_NAME> ...)
-s, --save List of contexts to save/overwrite as a group name (e.g. k10s -s <GROUP_NAME>=<CONTEXT_NAME> <CONTEXT_NAME> ...)
-g, --group Group name of contexts to start up (e.g. k10s -g <GROUP_NAME>)
-h, --help Help for k10s

EOT
exit 0
}

main() {
if [ "$#" -eq 0 ]; then
usage
fi

while [[ "$#" -gt 0 ]]; do
case "$1" in
-c | --context )
shift
contexts=()
while [[ "$1" != "" && "$1" != -* ]]; do
contexts+=("$1")
shift
done
process_contexts ${contexts[@]}
;;
-s | --save )
shift
contexts=()
while [[ "$1" != "" && "$1" != -* ]]; do
contexts+=("$1")
shift
done
save_group ${contexts[@]}
;;
-g | --group )
shift
start_group "$1"
;;
-h | --help )
shift
usage
;;
* )
shift
usage
;;
esac
shift
done
}

main $@

https://redd.it/1dffex9
@r_bash

download a website for offline use with sequential URLs

Hey everyone,I'm looking for a way to download an entire website for offline use, and I need to do it with sequential URLs. For example, I want to download all the pages from

www.example.com/product.php?pid=1

to

www.example.com/product.php?pid=100000

Does anyone know of a tool or method that can accomplish this? I'd appreciate any advice or suggestions you have.Thanks!

https://redd.it/1dfhhqj
@r_bash

open command & xdg-open command & Ubuntu?

Hi, I found the command open, then xdg-open ... both do the same, I use Lubuntu, so when I should use one and when should use another?

and how do I use the flag -a for open any web (https://ddg.com for example) using another browser that is NOT my default browser (=Falkon) like Chromium browser? the command is ....

Thank you and Regards!

https://redd.it/1dfm340
@r_bash

Official page

Hey guys, someone be so kind to write me in a reply the official bash page to find the noscripting manual and one or another link you can think of, containing tips on noscripting.

https://redd.it/1dg4awa
@r_bash

POSIX 2024 published: $'...' strings, set -o pipefail, find -0, xargs -0, sed -E, readlink, realpath, and more becoming new standards

# [1003.1-2024 - IEEE/Open Group Standard for Information Technology--Portable Operating System Interface (POSIX™) Base Specifications, Issue 8](https://ieeexplore.ieee.org/document/10555529)

Non-paywalled specification should eventually replace the current documentation [here at opengroup.org](https://pubs.opengroup.org/onlinepubs/9699919799/).

[HN thread](https://news.ycombinator.com/item?id=40679809)

Highlights on HN from [a-french-anon](https://news.ycombinator.com/threads?id=a-french-anon):

> * readlink/realpath (https://austingroupbugs.net/view.php?id=1457)
>
> * find -print0, xargs -0 and read -d (https://austingroupbugs.net/view.php?id=243)
>
> * find -iname (https://austingroupbugs.net/view.php?id=1031)
>
> * sed -E (https://austingroupbugs.net/view.php?id=528)
>
> * set -o pipefail (https://austingroupbugs.net/view.php?id=789)

Perhaps not strictly bash-related, but a rising tide lifts all boats.

https://redd.it/1dgbdp4
@r_bash

Templating in Bash, but not $foo

In a bash noscript I have a string containing a lot of dollar signs: 'asdf $ ... $'.

I want to insert a variable into that string. But if I use "..." instead of single quotes, then I need to escape all dollar signs (which I would like to avoid).

Is there a way to keep the dollar signs and insert a variable into a string?

Is there a simple templating solution like {{myvar}}?

https://redd.it/1dgdkpc
@r_bash

Word Splitting definition from man page confusing

This is from the man page of bash (5.2):

>If IFS is unset, or its value is exactly <space><tab><newline>, the default,
then sequences of <space>, <tab>, and <newline> at the beginning and end
of the results of the previous expansions are ignored, and any sequence of
IFS characters not at the beginning or end serves to delimit words.

According to that, I would expect this following behaviour:

$ A=" one two "
$ echo before-$A-after
before-one two-after

However, the actual output is:

before- one two -after

As you can see, the IFS whitespace at the beginning and end of the result of the previous expansion was NOT ignored, precisely the opposite of what the man page proclaims.

Is there something I misunderstood?

https://redd.it/1dgi39w
@r_bash

How can one reliably output text, if it contains text from variable expansions?

I want a command to easily print out text, that may include text from a variable expansion.
The bash command echo fails for FOO=-n and BAR=bar:

$ echo "$FOO" "$BAR"
bar$

There is printf, but there you always need to pass a format string, which to me seems to burdensome. One might try a function definition:

$ myecho () { printf %s "$@" ; }
$ echo $FOO $BAR
-nbar$ # space between arguments is missing.

There must be some ready to use solution, right?

https://redd.it/1dgiz6d
@r_bash

Why does ">> " result in ambiguous redirect

In a folder i have 3 files : file1 file2 file3

Doing "date >> ./" Causes error "ambiguous redirect.

https://redd.it/1dh4fpd
@r_bash

Script for doing the command line labor in making and adding to a github repository

It's very simple as far as software is concerned, mostly just designed it for myself just because i think making commits to github is kinda annoying:

https://github.com/LEDparty/lazy-commit

Tested and works like it should.

https://redd.it/1dhd2zb
@r_bash

MarCLIdown: A decent way to read markdown files directly in your linux terminal.

MarCLIdown (WIP) is a minimalist Bash program that prints a Markdown file with a beautiful, human-readable monochrome output, featuring interactive images, hyperlinks, emails, and references, as well as noscripts, emphasis, strikethrough, highlighted text, and Unicode characters for easy recognition of elements such as lists, checkboxes, collapsible sections, notes, etc.

https://preview.redd.it/5fz9y399y07d1.png?width=1365&format=png&auto=webp&s=d86457db474f9f75c760ede62ab877fe6fe284d9

I don't have much knowledge of programming, so there's still a lot to improve in the code, as well as finishing the planned features.

What do you think? And what extra formatting could this support? Any good github flavor feature I forgot about?

https://redd.it/1dhl7ij
@r_bash

Help with noscript to format markdown tables

Hi, I recently made this post asking for help on how to make hyperlinks in a markdown file appear beautifully in the terminal, as I was making a program similar to glow, I've made a lot of progress (which you can see here) since then, but I just couldn't handle formatting the markdown tables, does anyone know how this could be done using only native linux tools?

https://redd.it/1dhlerj
@r_bash

Manage your noscripts and snippets, share them and run programming languages as noscripts
https://github.com/thereisnotime/xxToolbelt

https://redd.it/1dhk5ps
@r_bash

Have you ever written a full on application in Bash? What was it?

I'm a very old hat programmer. C++ was newfangled stuff and nobody had ever spoken the word "Javanoscript" when I first learned how to code Hello World. Bsh/Bash was the first language I learned, and we called it "terminal programming" back then and not noscripting.


To this day its my go to if I need to write a linux-portable application that doesn't engage with the hardware enough to require C. I recently "finished" a program for controlling an entire network of remote Varnish server clusters, written in just under 2000 lines. It uses a pull-store-flag-edit-push-versioncontrol schema with 4 levels of granularity in managing .vcl files, and has remote tools built in for generating and pulling logs, modifying inline C include files, and controlling all the cache parameters. It even has a fancy toggling system that lets a non-VCL nerd enable and disable all the special modules, and its own Help menu.


I wrote this beast because I'm the only resident Varnish guru in our devteam, and I needed something simple that other administrators can use to control and maintain the system if I got hit by a bus. At its current line count, and with 28 menus I'm about 80% sure its the biggest Bash program I've written in my life. That got me wondering what kinds of things other people have written as their Magnum Opus.

https://redd.it/1dhsf7u
@r_bash

Site that returns protocol that can be used from the command line

Is there a site similar to ifconfig.me that you can curl so that it returns the protocol it was hit with? I.e. curl http://example.com should return http somewhere in the response and curl https://example.com should return https.

https://redd.it/1di8umt
@r_bash

Write to file keeps service restarting

I am trying to write a simple multi-line value to a file.

I've noticed as I watch the processes, and the logs, whenever I add that particular code and re-start my service, it loops over and over again, instead of running once and then waiting for the timer to re-activate it.

cat ${dir}/${file}.json << EOF
{
    "id": "${item_id}",
}
EOF

I then execute

systemctl --user start my_service_name.service

And then loop begins. As soon as I remove that particular set of code and re-execute, then the noscript only runs once, and then waits 15 minutes for the .timer to call it again.

I've tried both cat and tee hoping one or the other would work.

https://redd.it/1did9c7
@r_bash