r_bash – Telegram
How do I make remove all the extra spaces while using jq to read raw html into a JSON string

- My use case is to upload SES templates to AWS for sending emails
- It requires you to have a JSON templates uploaded to SES in a [https://docs.aws.amazon.com/ses/latest/dg/send-personalized-email-api.html!](certain format)

I wrote a bash noscript in jq that does the following thanks to the help from some of the redditors here

```
declare -A confirm_email=( [TemplateName]="confirm-email" [SubjectPart]="Please confirm your email" )
declare -A forgot_password=( [TemplateName]="forgot-password" [SubjectPart]="Reset Password" )
declare -a templates=(confirm_email forgot_password)

for ((i=0; i<"${#templates[*]}"; i++)); do
curr="${templates[$i]}"
TemplateName="${curr}[TemplateName]"
SubjectPart="${curr}[SubjectPart]"
jq -n \
--arg TemplateName "${!TemplateName}" \
--arg SubjectPart "${!SubjectPart}" \
--rawfile html "${manually_created_directory}/${!TemplateName}.html" \
--rawfile text "${manually_created_directory}/${!TemplateName}.txt" \
'
{
"Template": {
"TemplateName": $TemplateName,
"SubjectPart": $SubjectPart,
"HtmlPart": $html,
"TextPart": $text
}

}' > "${automatically_generated_directory}/${!TemplateName}.json"

aws ses update-template --cli-input-json "file://${automatically_generated_directory}/${!TemplateName}.json"
done

```
My generated json file that I can uploaded to AWS SES looks somewhat like this

```
{
"Template": {
"TemplateName": "confirm-email",
"SubjectPart": "Please confirm your ch email",
"HtmlPart": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\" xmlns=\"http://www.w3.org/1999/xhtml\">\n <head>\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\" />\n <meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" />\n <noscript>Please confirm your email on ch</noscript>\n </head>\n <body\n style=\"\n -webkit-text-size-adjust: none;\n box-sizing: border-box;\n color: #74787e;\n font-family: Arial, 'Helvetica Neue', Helvetica, sans-serif;\n height: 100%;\n line-height: 1.4;\n margin: 0;\n width: 100% !important;\n \"\n bgcolor=\"#f2f4f6\"\n >\n <style type=\"text/css\">\n body {\n width: 100% !important;\n height: 100%;\n margin: 0;\n line-height: 1.4;\n background-color: #f2f4f6;\n color: #74787e;\n -webkit-text-size-adjust: none;\n }\n @media only screen and (max-width: 600px) {\n .email-body_inner {\n width: 100% !important;\n }\n .email-footer {\n width: 100% !important;\n }\n }\n @media only screen and (max-width: 500px) {\n .button {\n width: 100% !important;\n }\n }\n </style>\n <span\n class=\"preheader\"\n style=\"\n box-sizing: border-box;\n display: none !important;\n font-family: Arial, sans-serif;\n font-size: 1px;\n line-height: 1px;\n max-height: 0;\n max-width: 0;\n mso-hide: all;\n opacity: 0;\n overflow: hidden;\n visibility: hidden;\n \"\n >Use this link to reset your password. The link is only valid for 24\n hours.</span\n >\n <table\n class=\"email-wrapper\"\n width=\"100%\"\n cellpadding=\"0\"\n cellspacing=\"0\"\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n margin: 0;\n padding: 0;\n width: 100%;\n \"\n bgcolor=\"#f2f4f6\"\n >\n <tr>\n <td\n align=\"center\"\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n
word-break: break-word;\n \"\n >\n <table\n class=\"email-content\"\n width=\"100%\"\n cellpadding=\"0\"\n cellspacing=\"0\"\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n margin: 0;\n padding: 0;\n width: 100%;\n \"\n >\n <tr>\n <td\n class=\"email-masthead\"\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n padding: 25px 0;\n word-break: break-word;\n \"\n align=\"center\"\n >\n <a\n href=\"https://example.com\"\n class=\"email-masthead_name\"\n style=\"\n box-sizing: border-box;\n color: #bbbfc3;\n font-family: Arial, sans-serif;\n font-size: 16px;\n font-weight: bold;\n text-decoration: none;\n \"\n >ch</a\n >\n </td>\n </tr>\n <tr>\n <td\n class=\"email-body\"\n width=\"100%\"\n cellpadding=\"0\"\n cellspacing=\"0\"\n style=\"\n -premailer-cellpadding: 0;\n -premailer-cellspacing: 0;\n border-bottom-color: #edeff2;\n border-bottom-style: solid;\n border-bottom-width: 1px;\n border-top-color: #edeff2;\n border-top-style: solid;\n border-top-width: 1px;\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n margin: 0;\n padding: 0;\n width: 100%;\n word-break: break-word;\n \"\n bgcolor=\"#ffffff\"\n >\n <table\n class=\"email-body_inner\"\n align=\"center\"\n width=\"570\"\n cellpadding=\"0\"\n cellspacing=\"0\"\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n margin: 0 auto;\n padding: 0;\n width: 570px;\n \"\n bgcolor=\"#ffffff\"\n >\n <tr>\n <td\n class=\"content-cell\"\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n padding: 35px;\n word-break: break-word;\n \"\n >\n <h1\n style=\"\n box-sizing: border-box;\n color: #2f3133;\n font-family: Arial, sans-serif;\n font-size: 19px;\n font-weight: bold;\n margin-top: 0;\n \"\n align=\"left\"\n >\n Hi {{name}},\n </h1>\n <p\n style=\"\n box-sizing: border-box;\n color: #74787e;\n font-family: Arial, sans-serif;\n font-size: 16px;\n line-height: 1.5em;\n margin-top: 0;\n \"\n align=\"left\"\n >\n Use the button below to confirm your email on ch.\n <strong\n
style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n \"\n >This link is only valid for the next 24\n hours.</strong\n >\n </p>\n <table\n class=\"body-action\"\n align=\"center\"\n width=\"100%\"\n cellpadding=\"0\"\n cellspacing=\"0\"\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n margin: 30px auto;\n padding: 0;\n text-align: center;\n width: 100%;\n \"\n >\n <tr>\n <td\n align=\"center\"\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n word-break: break-word;\n \"\n >\n <table\n width=\"100%\"\n border=\"0\"\n cellspacing=\"0\"\n cellpadding=\"0\"\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n \"\n >\n <tr>\n <td\n align=\"center\"\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n word-break: break-word;\n \"\n >\n <table\n border=\"0\"\n cellspacing=\"0\"\n cellpadding=\"0\"\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n \"\n >\n <tr>\n <td\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n word-break: break-word;\n \"\n >\n <a\n href=\"{{action_url}}\"\n class=\"button button--blue\"\n target=\"_blank\"\n style=\"\n -webkit-text-size-adjust: none;\n background: #007dcc;\n border-color: #007dcc;\n border-radius: 3px;\n border-style: solid;\n border-width: 10px 18px;\n box-shadow: 0 2px 3px\n rgba(0, 0, 0, 0.16);\n box-sizing: border-box;\n color: #ffffff;\n
display: inline-block;\n font-family: Arial, sans-serif;\n font-size: 16px;\n text-decoration: none;\n \"\n >Confirm Email</a\n >\n </td>\n </tr>\n </table>\n </td>\n </tr>\n </table>\n </td>\n </tr>\n </table>\n <p\n style=\"\n box-sizing: border-box;\n color: #74787e;\n font-family: Arial, sans-serif;\n font-size: 16px;\n line-height: 1.5em;\n margin-top: 0;\n \"\n align=\"left\"\n >\n Thanks, <br />The ch Team\n </p>\n <table\n class=\"body-sub\"\n style=\"\n border-top-color: #edeff2;\n border-top-style: solid;\n border-top-width: 1px;\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n margin-top: 25px;\n padding-top: 25px;\n \"\n >\n <tr>\n <td\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n word-break: break-word;\n \"\n >\n <p\n class=\"sub\"\n style=\"\n box-sizing: border-box;\n color: #74787e;\n font-family: Arial, sans-serif;\n font-size: 12px;\n line-height: 1.5em;\n margin-top: 0;\n \"\n align=\"left\"\n >\n If you’re having trouble with the button above,\n copy and paste the URL below into your web\n browser.\n </p>\n <p\n class=\"sub\"\n style=\"\n box-sizing: border-box;\n color: #74787e;\n font-family: Arial, sans-serif;\n font-size: 16px;\n line-height: 1.5em;\n margin-top: 0;\n \"\n align=\"left\"\n >\n {{action_url}}\n </p>\n </td>\n </tr>\n </table>\n </td>\n </tr>\n </table>\n </td>\n </tr>\n <tr>\n <td\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n word-break: break-word;\n \"\n >\n <table\n class=\"email-footer\"\n align=\"center\"\n
width=\"570\"\n cellpadding=\"0\"\n cellspacing=\"0\"\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n margin: 0 auto;\n padding: 0;\n text-align: center;\n width: 570px;\n \"\n >\n <tr>\n <td\n class=\"content-cell\"\n align=\"center\"\n style=\"\n box-sizing: border-box;\n font-family: Arial, sans-serif;\n padding: 35px;\n word-break: break-word;\n \"\n >\n <p\n class=\"sub align-center\"\n style=\"\n box-sizing: border-box;\n color: #bbbfc3;\n font-family: Arial, sans-serif;\n font-size: 12px;\n line-height: 1.5em;\n margin-top: 0;\n \"\n align=\"center\"\n >\n © 2023 ch. All rights reserved.\n </p>\n </td>\n </tr>\n </table>\n </td>\n </tr>\n </table>\n </td>\n </tr>\n </table>\n </body>\n</html>\n",
"TextPart": "Use this link to confirm your email. The link is only valid for 24 hours.\n\nch ( https://example.com )\n\n************\nHi {{name}},\n************\n\nUse the button below to confirm your email on ch. This link is only valid for the next 24 hours.\n\nConfirm your email ( {{ action_url }} )\n\nThanks,\nThe ch Team\n\nIf you’re having trouble with the button above, copy and paste the URL below into your web browser.\n\n{{action_url}}\n\n© 2023 ch. All rights reserved.\n"
}
}

```

Has a tonne of new lines, extra spaces and what not. How do I tell jq to remove these while reading the rawfile?

https://redd.it/15n1mkl
@r_bash
One of the insignificant portions of my shell noscript, what do you think?
https://redd.it/15n22f9
@r_bash
Unwanted expansion - no idea why this is happening in grep with Perl regex

Trying negative lookbehind with grep.

This seems reasonable, show 5 numbers not preceded by 'w':
echo -e "w12121\n888lj\n123456" | grep -P "(?<!w)[[:digit:]]{5}"

I get this error:
grep: syntax error in subpattern name (missing terminator)

and this appears in my history:

echo -e "w12121\n888lj\n123456" | grep -P "(?<wc -l current.csv )[[:digit:]]{5}"

So the <!w was expanded to wc -l current.csv.

Where is this coming from? How can I avoid it? (tried single and double quote around the regex, same result.

&#x200B;

https://redd.it/15n1jnz
@r_bash
This media is not supported in your browser
VIEW IN TELEGRAM
Got a strange bash behaviour recently, any idea how can I fix this?
https://redd.it/15nmpqd
@r_bash
How do I add a variable as an alias?

I'm not sure if that was a decent enough denoscription, but basically, I'm trying to effectively remove the --volume= part of mpv --volume =x. I want to input mpv x and for it to set mpv's volume to x, and I want to add this to my .zshrc, as opposed to a noscript, because I know I can make a noscript and just do cd [directory] && ./[noscript].sh && cd or something like that.

&#x200B;

I'm hoping what I said made sense. I've tried several ideas such as alias mpv=$1 --volume=$1 and even prefaced it with volume=$1 just in case.

&#x200B;

I showed this to a friend in my software class, and she wrote: alias mpv='volume(){ mpv ~/Music/[playlist]/* --shuffle --no-audio-display --af="volume='$1'" "$2"; unset -f volume; }; volume' which i don't understand, nor does it work.

&#x200B;

In case you're wondering/haven't noticed, the full command I want to use is: mpv ~/Music/[playlist]/* --volume=x --shuffle --no-audio-display

&#x200B;

If I'm lucky enough to have someone help me with this, I could also change: alias weather='curl -s `wttr.in/`[location] | head -n -2' To alias weather x='curl -s `wttr.in/`x | head -n -2'.

https://redd.it/15o8igq
@r_bash
I wrote a simple JSON -> JSON schema converter in jq

Here is a code:

def to_noscript: . | gsub("(?<x>[A-Z])"; " \(.x)") | ascii_downcase;
def to_denoscription: . | to_noscript + "\n" + $url;

def wrap: {
type: "object",
properties: with_entries(
if .value | type != "object" then
{
key,
value: {
noscript: .key | to_noscript,
denoscription: .key | to_denoscription,
type: (.value | type),
default: .value
}
}
else
.value = {
noscript: .key | to_noscript,
denoscription: .key | to_denoscription
} + (.value | wrap)
end)
};

{
"$schema": "http://json-schema.org/draft-07/schema",
noscript: "config",
denoscription: "A config"
} + (. | wrap)


The idea is to pass some sample config file for some tool and get a JSON schema generated for it. I wrote this noscript as I didn't like how [online generator](https://jsonformatter.org/yaml-to-jsonschema) generated JSON schemas for YAML: it used too many `$ref`s when they were redundant, and made all properties required by default. So I decided just to convert YAML -> JSON and then apply my noscript to get JSON schemas.


For the following input: `{ "a": 1, "b": { "c": 2, "d": { "e": 3 } } }` it generates this schema (note that `--arg url x` is passed to jq):

Input:

'{ "a": 1, "b": { "c": 2, "d": { "e": 3 } } }'

Output:

{
"$schema": "http://json-schema.org/draft-07/schema",
"noscript": "config",
"denoscription": "A config",
"type": "object",
"properties": {
"a": {
"noscript": "a",
"denoscription": "a\nx",
"type": "number",
"default": 1
},
"b": {
"noscript": "b",
"denoscription": "b\nx",
"type": "object",
"properties": {
"c": {
"noscript": "c",
"denoscription": "c\nx",
"type": "number",
"default": 2
},
"d": {
"noscript": "d",
"denoscription": "d\nx",
"type": "object",
"properties": {
"e": {
"noscript": "e",
"denoscription": "e\nx",
"type": "number",
"default": 3
}
}
}
}
}
}
}

https://redd.it/15oh21e
@r_bash
Running a bash noscript function as an alias

pzserver () {
if "$1" == "-backup" ; then
# run backup function
elif "$1 == "-start" ; then
# run start function
elif "$1" == "-quit" ; then
# run quit function
else
# no options set
fi
}

^(Obligatory: I'm new at bash noscripting so forgive me if I'm overlooking a simpler solution. Also, any tips and hints in general are much appreciated!)

I have the function above defined in my .bashrc and would like to use this function from a terminal. For example, I'd like to run pzserver -backup from a terminal to start the backup process. However, I'm seeing the following error when running that command: -bash: pzserver: command not found

Of course I've done some searching and found that I'll need to use bash -c to be able to do this, however I'm not sure how to get an alias to do this for me.

alias pzserver = "bash -c source pzserver.sh; pzserver $1" doesn't work, it returns -bash: alias: pzserver: not found

Any hints would be massively appreciated! Thanks!

https://redd.it/15opx5k
@r_bash
grep 'echo -' vs grep 'echo -.'

what am i missing here, the results for the second version all appear in red font? what's the difference the dot is making? i always thought .* refered to filenames?

https://redd.it/15ose0y
@r_bash
How do I output the new lines onto my terminal.

If I do echo "dog\ncow\ncat" > file1.txt and then cat file1.txt, it will output dog\ncow\ncat.

But if we do echo -e "dog\ncow\ncat" > file1.txtand then cat it, then it does output the newlines correctly.

If I forget the -e option in the initial echo step, then is there a way to still have the newlines output onto my terminal?

https://redd.it/15pfkp1
@r_bash
a little wallpaper noscript written by a noob

This is my noscript here: https://github.com/jpatzy/whaven

I was looking for a project idea to help me learn and I came across this wallpaper noscript by /u/wick3dr0se that grabs random wallpapers from unsplash, so I decided to make own noscript to get random wallpapers from Wallhaven.cc. I originally started writing this in bash (I thought) but found out that most of it wasn't bash, and so with the help of shellcheck I tried to make it POSIX compliant instead. I am not 100% that it is since shell noscripting is very new to me.

If anybody is willing to take a look, I'd appreciate any feedback, tips, or criticisms. It only works with utilities that I have personally used which are, sway, feh, and gnome (gsettings).

https://redd.it/15ph8o1
@r_bash
Isolated environment for iteratively developing dev machine set up noscript?

I want to write a noscript that will set up my machine (MacOS). Nothing too crazy - install brew, brew install a bunch of things I use, prompt me for certain logins so it can auth for me, etc.

Hypothetically, if I test a bash function to install something, then make a change, I'll need to uninstall that thing to re-test.

Is there a better way to do this? Some sort of isolated environment that I can tear down with each run?

Thanks

https://redd.it/15pkanw
@r_bash
awk - how to wrap printf for readability?

writing a really long printf, how to separate it into multiple lines? ie this doesnt work:

{printf(" stuff stuff stuff"

printf "stuff stuff");}

https://redd.it/15puett
@r_bash
text in e-mail subject 'corrupts' e-mail body

I have a slightly modified version of this noscript which monitors a zfs volume. It e-mails me after it runs with this command

echo -e "$emailMessage \n\n\n <pre>/sbin/zpool list \n\n\n /sbin/zpool status</pre>" | mailx -v -s "$emailSubject
Content-Type: text/html" my.address@gmail.com

Its output looks like this:

https://i.imgur.com/JWgOAUg.png

One part of the noscript does this:

if ${problems} -eq 0 ; then
capacity=$(/sbin/zpool list -H -o capacity)
for line in ${capacity//%/}
do
if $line -ge $maxCapacity ; then
emailSubject="$emailSubject - Capacity Exceeded"
problems=1
fi
done
fi

If this condition is true, the e-mail looks like this:

https://i.imgur.com/IhESPce.png

through some trial and error, I've found I can change this to:

emailSubject="$emailSubject - Capac"

and it will still format correctly.

However, adding an i (or any other letter) after "Capac" causes the e-mail formatting error.

I seem to be running into some kind of length limit, but I don't understand why. Is it because of the way the echo command is formatted?

For what its worth, the output of

"$emailMessage \n\n\n <pre>/sbin/zpool list \n\n\n /sbin/zpool status</pre>"

is 2303 characters long.


Another oddity is that if I were to do say:

echo -e "$emailMessage \n\n\n <pre>/sbin/zpool list \n\n\n /sbin/zpool status</pre>" | mailx -v -s "$emailSubject Content-Type: text/html" my.address@gmail.com

i.e. I don't put a new line between $emailSubject and Content-Type

instead of

echo -e "$emailMessage \n\n\n <pre>/sbin/zpool list \n\n\n /sbin/zpool status</pre>" | mailx -v -s "$emailSubject
Content-Type: text/html" my.address@gmail.com

I also get the same formatting issue

https://redd.it/15q07kx
@r_bash
Why does /*.py correctly interpret my wildcard but not \*.py?

This command works correctly to output all files ending with .py `ls | grep /*.py`

but this doesn't `ls | grep \*.py`. This doesn't output anything

I can't figure out why as isn't the backslash required to escape the *? In all of the internet resources, the backslash escapes the next character but I'm not sure why the forward slash version is working for me.

https://redd.it/15q2okr
@r_bash