Scraping weather info

Hi, I’m trying to scrape some weather information using sed and grep from: https://api.met.no/weatherapi/locationforecast/2.0/classic\?lat\=60.795\&lon\=10.691

What I’m trying to do is:
1. extract the temperature forecast for the current hour, the next hour and the hour after that.
2. extract the next precipitation forecast.
3. the date and time when the meteorological data has been scraped.

Example of how I want the output to look (I want them “stacked”/under each other, but couldn’t format that):
3
0
-4
Cloudy
2022-11-24 10:03

What I know is that I probably should grep something, but not sure what and how

curl -s https://api.met.no/weatherapi/locationforecast/2.0/classic\?lat\=60.795\&lon\=10.691 | grep

I have found these commands that I think could be useful, but I don’t know how to use them in this case.

grep -Eo "0-9{2}:0-9{2}:0-9{2}"
date +"%Y-%m-%d"
date +"%Y-%m-%d %H:%M"

https://redd.it/z3f73f
@r_bash

Is there something like w3school but for bash? or any other shell?

I'm looking for an online resource for learning shell noscripting. I dont really find youtube lessons helpful so written material works better for me. Books could also work tbh but no video plz :D

https://redd.it/z68zbh
@r_bash

I'm currently taking a Google Cloud Course that includes Qwiklabs. I've been tasked with translating one of the labs into a linux bash noscript. I've done that but i'd like someone who knows what they're doing to take a look at it before i turn it in. I'd really appreciate it. Thanks in anticipation.

I'm currently a big noob trying to wrap my head around tech. please be nice

https://redd.it/z6ala6
@r_bash

What does this noscript do? I need to explain what this noscript does for a question on my classwork but my teacher isn't satisfied with my answer. Can anyone provide some insight into what exactly is happening with this noscript?
https://redd.it/z6c7x0
@r_bash

Getopts to allow only 1 option

How can I make getopts to allow only one option?

while getopts ":ab" option
do
case $option in
a)
echo "test1"
;;
b)
echo "test2"
;;
esac
done

This is an example loop, running the noscript

./noscript.sh -a -b

will return both a and b cases. How can I make it to show an error message instead?

https://redd.it/z6hq3c
@r_bash

printf with colour using variable

I thought i was being clever using printf with colour variables to save me typing out the colour i'd like, but apparently shellcheck doesn't like that

for instance...

red="\e1;31m%s\e[0m\n"
green="\e[1;32m%s\e[0m\n"
yellow="\e[1;33m%s\e[0m\n"

and then

printf "$red" "string of sorts in red colour"

Since it complains, is there another way, or just suppress and ignore it (i've been ignoring it up until now since i discovered shellcheck, and never run into any problems)? Thought i'd ask just in case it does fluff something up i'm unaware of.

[https://redd.it/z6vhvv
@r_bash

eval vs source

I recently discovered that you can get functionality similar to eval ... with source <(...).

There is some difference in functionality - mostly that eval can be used within some code statement whereas source <(...) requires complete statements. e.g. in someFunction $(eval $someCode) the eval ... cant be replaced with source <(...). The flip side of this is that source <(...) will evaluate everything between the paranthesies, whereas eval will stop evaluating at pipes/newlines/semicolons.

Any other considerations regarding these two that are worth knowing?

Side note: I'm not advocating using these for "general noscripting", but there are some situations where they are useful. Examples of what Id call "reasonable usage" for both are:

eval ...

for nn in $(eval echo {$start..$end..$skip}); do
    <...>
done

Yes, I know you can do

nn=$start
while [ $nn -le $end]; do
    <...>
    nn=$((( $nn + $skip )))
done

but its just an example

source <(...)

# passing an array to a noscript
declare -p arrayName | noscript.sh

# in noscript.sh
source <(cat)

https://redd.it/z6wm7f
@r_bash

New to Bash

Hey everyone...

I'm super new to bash noscripting, I'm working through a class currently and I am stuck. I need help, I'm trying to isolate a particular word and print out the results.

Would I be using awk then pipe and wc? For example, this particular scenario involves searching for individuals who are creating a loss of profit. I need to isolate the people responsible and print it to a new evidence file. I think I just don't have enough practice to understand it yet, any help would be appreciated! If you are curious and want to see the exact code DM me for it.

https://redd.it/z6zq0g
@r_bash

This is my first bash noscript! What do you think? You plug in your droplet ip, your domain and your gitlab info and in 5 minutes your web app is live at https://yourdomain, and all future commits to main are automatically deployed. Included templates for new django, flask and fastApi projects!
https://github.com/johnsyncs/ezinnit

https://redd.it/z74xq9
@r_bash

hfetch, a simple and minimal system information tool written in bash. with fortune
https://redd.it/z776ux
@r_bash

associative array order

I've noticed, while playing around with associative arrays, that when I append keys/values, the appended pairs stay in the order I added them in stdout, unlike the key/value pairs I declared the array with. If that's the case, can I take advantage of that in order to control the way associative arrays are displayed?

https://redd.it/z7bcmb
@r_bash

Script to detect change in URL redirection:

I've been hesitant to submit this in fear that I have re-invented the wheel here, but please tell me if I have.

#!/usr/bin/env bash

# Script to be placed in /etc/cron.d/cron.hourly

###############################################################################################################
# Ignore below comment, it's just for shellcheck. (Must appear before any instructions.) #
# shellcheck disable=SC1090 # Can't follow non-constant source #
# shellcheck disable=SC2034 # Yea, I have some unused variables #
###############################################################################################################

requiredPackages=(
alsa-utils
coreutils
curl
wc
wget
zenity
)

noscriptBasename=$(basename "${0}") # Could also user ${BASH_SOURCE[0]} here.
kibibyte="1024"
mebibyte="$(( kibibyte * kibibyte ))"
###############################################################################################################
########################################### Configuration Variables: ##########################################
###############################################################################################################

### Inputs: Wikipedia example ######################################################################################
user="user"
workingDir="/home/${user}/${noscriptBasename}"
audioOutputDevice="plughw:CARD=NVidia,DEV=3" # Get list of available devices with: aplay -L | grep "CARD"
maxLogSize="$((1 * mebibyte))" #bytes
notificationSound="${workingDir}/notify.wav"
urlToWatch="https://en.wikipedia.org/wiki/Redirect_page" # Will redirect to https://en.wikipedia.org/wiki/URL_redirection
notificatonTitle="Redirect change"
notificationMsg="Page now points somewhere new!" # String will be followed by redirect URL and location noscript is run from.
subredditsInput="${workingDir}/subreddits" # If present, is a sourced bash noscript that should define a ${subreddits[@]} array.

### Outputs: ##################################################################################################
logFile="${workingDir}/${noscriptBasename}.log"
lastRedirectFile="${workingDir}/lastRedirect"
subredditList="${workingDir}/subreddits.list"
website="${workingDir}/${noscriptBasename}.html"
sharedMemory="/dev/shm/${noscriptBasename}.shm"
namedPipe="/dev/shm/${noscriptBasename}.pipe"

###############################################################################################################
################## No need to modify anything below this line unless changing functionality ###################
###############################################################################################################
version="4" #Version 4 outputs to subreddits.list instead of subreddits.
version="5" #Version 5 reads subreddit array from subreddits file.

# Defines checkError(), which simply converts an exit code to a string denoscription.
# https://old.reddit.com/r/linuxquestions/comments/6nuoaq/how_can_i_look_up_exit_status_codes_reliably/
if [ -f "/home/${user}/.adms/ADMS-OS/noscripts/CommonScripts/error" ]; then
source "/home/${user}/.adms/ADMS-OS/noscripts/CommonScripts/error"
fi

###############################################################################################################
################################# Ideas for extending functionality: ##########################################
###############################################################################################################
# TODO: Should I install required packages if missing ???
###############################################################################################################
# Do I really want to make a distro indepandent check here?
# A lot is already done in "/home/${user}/noscripts/CommonScripts/packages"
# But no, I really don't....
# for package in

"${requiredPackages[@]}"; do
# :
# done
###############################################################################################################
# TODO: Should we use a named pipe for communication with subprocess instead ???
###############################################################################################################
# Would have to re-do a lot of logic for this route.
# if [ ! -p "${namedPipe}" ]; then
# mkfifo -m "a=rw" "${namedPipe}" #Man page literally says "not a=rw", but what does that mean?
# fi
###############################################################################################################
# TODO: Use array of URL's for tracking multiple websites.
###############################################################################################################
# Don't actually need this at all right now, but maybe one day...
###############################################################################################################

#Does not try to handle race-conditions, but I don't think it should not be a problem.
declare -A globalState;

function setGlobalState()
{
local -r varName="${1}"
local -r varValue="${2}"

if [ ${#} -eq 2 ]; then
globalState["${varName}"]="${varValue}"
fi

printf "# Using associative array for easy addition of new variables\n" | sudo tee "${sharedMemory}" > /dev/null
declare -p globalState | sudo tee -a "${sharedMemory}" > /dev/null
}

function getGlobalState()
{
local -r varName="${1}"
local -r varType="${2}"
local success=true
if [ -f "${sharedMemory}" ]; then
source "${sharedMemory}"
if [ ${#} -ge 1 ]; then
if [[ "${globalState[${varName}]}" != "" ]]; then
printf "%s" "${globalState[${varName}]}"
else
success=false
fi
fi
else
success=false
fi

if ! ${success}; then
if [[ ${varType} == "bool" ]]; then
printf "false";
elif [[ ${varType} == "int" ]]; then
printf "0";
elif [[ ${varType} == "string" ]]; then
printf "";
fi
return 1
fi
return 0
}


function cleanupSharedMemory()
{
if [ -f "${sharedMemory}" ]; then
sudo rm -vf "${sharedMemory}"
fi
}

setGlobalState "ring" "false"

dateFmt="+%Y.%m.%d_%I:%M%P"
#dateFmt="+%Y-%m-%d_%H:%M:%S.%3N"
function getDateTimeStamp()
{
date "${dateFmt}"
}

function getLogSize()
{
wc -c "${logFile}" | cut -f 1 -d ' '
}

function getLogLength()
{
wc -l "${logFile}" | cut -f 1 -d ' '
}

function log()
{
local -r extFmt="${1}"
printf "%s | ${extFmt}\n" "$(getDateTimeStamp)" "${@:2}" | tee -a "${logFile}"
}

function truncateLog()
{
local -r percentToKeep="${1}"

local -r logSize="$(getLogSize)"
local -r numLinesStart="$(getLogLength)"
# shellcheck disable=SC2155 # Masking return values by declaring and assigning together.
local numLines="$(echo "scale=0; ${numLinesStart} * ${percentToKeep}" | bc)"
numLines="${numLines%.*}" #Round down to nearest int.

# shellcheck disable=SC2005 # It's not a useless echo! It's not! I love echo...
echo "$(tail "-${numLines}" "${logFile}" 2> /dev/null)" > "${logFile}"

log "Trimmed output size: %b -> %b" "${logSize}" "$(getLogSize)"
log "Trimmed output size: %b -> %b" "${numLinesStart}" "$(getLogLength)"
}

printf -v dividerVar "<%.0s>%.0s" {1..80}
function divider()
{
printf "%b\n" "${dividerVar}">> "${logFile}"
}



function ringer()
{
local -r startVolume=$(amixer get Master | grep -o "[0-9]*%" | head -1) #Record current volume level
# shellcheck disable=SC2034 # The variable ${uid} is used when testing as cron job.
local -r uid=$(id -u "${user}")
local ring=true #Should always be true fist call.
if [[ "$(getGlobalState "ring")" != "${ring}" ]]; then
printf "Ringer was called with incorrect ring state! Check logical flow!\n"
fi

while ${ring}; do
amixer set Master 20% > /dev/null #I use headphones, and I don't want to blast out my eardrums

# Ok, weird one. Audio will not play over

same device user is using, so we need to specify a different one.
# So, if user is using laptop speakers, we can play though properly equipped external HDMI montior.
# Also, the audio is muted for the first second of play, so we will play the sound twice, but hear it once.
sudo -H -i -u "${user}" "aplay" -D "${audioOutputDevice}" "${notificationSound}" "${notificationSound}"

# This version works if run by user directly (i.e. Not as cron job)
# aplay -D "${audioOutputDevice}" "${notificationSound}" "${notificationSound}" > /dev/null

amixer set Master "${startVolume}" > /dev/null #Reset volume to what it was before
sleep 1
ring=$(getGlobalState "ring" "bool")
done
setGlobalState "ring" "false"
}

function popup()
{
local -r website="${1}"
local -r width=400
local -r height=200
local -r noscript="${notificatonTitle}"
local -r message="${notificatonMsg}\n${1}\nThis dialoge was created by: $(realpath "${BASH_SOURCE[0]}")"

zenity \
--warning \
--text="${message}" \
--noscript="${noscript}" \
--width="${width}" \
--height="${height}"
}

function checkReDirect()
{
local -r lastRedirect="$( < "${lastRedirectFile}" )"
local currentRedirect=""

currentRedirect="$(curl -ILs -o /dev/null -w "%{url_effective}" "${urlToWatch}")"
curlCode="${?}"
if [ "${curlCode}" -ne 0 ]; then
if [[ "${ERROR_SOURCED}" == "true" ]]; then
log "$(checkError "curl" "${curlCode}")"
else
log "Error! curl failed with ${curlCode}"
fi
return
elif [[ "${currentRedirect}" == "${lastRedirect}" ]]; then
log "Executing: %b ( No news... )" "$(realpath "${BASH_SOURCE[0]}")"
return
else # This isn't needed since other cases do early return ...
log "Executing: %b ( NEWS!! )" "$(realpath "${BASH_SOURCE[0]}")"
listSubreddits
wget "${currentRedirect}" -O "${website}" # Grab page for analysis

redirectEnding=${lastRedirect%/} # Remove trailing slash if present
redirectEnding=${redirectEnding##*/} # Remove everything up to last slash (Shoud be left with last section of URL)

noscriptSuggestion="$(grep -o "<noscript>.*</noscript>" "${website}")" #TODO: Find better way to parse html
noscriptSuggestion="${noscriptSuggestion#*>}" # Strip off <noscript>
noscriptSuggestion="${noscriptSuggestion%<*}" # Strip off </noscript>
log "Title Suggestion: ${redirectEnding} - ${noscriptSuggestion}"

log "Opening %s\n" "${currentRedirect}"
printf "%s" "${currentRedirect}" > "${lastRedirectFile}"


setGlobalState "ring" "true"
ringer & # Non-blocking so it will keep ringing until killed.
ringerPID=${!} # Keep this as global variable.

#Attempt to open URL in default web-broswer
if command -v gnome-open > /dev/null; then
gnome-open "${currentRedirect}" &
[ -f "${subredditList}" ] && [ -s "${subredditList}" ] && gnome-open "${subredditList}" &
elif command -v xdg-open > /dev/null; then
xdg-open "${currentRedirect}" &
[ -f "${subredditList}" ] && [ -s "${subredditList}" ] && xdg-open "${subredditList}" &
elif command -v gio > /dev/null; then
gio open "${currentRedirect}" &
[ -f "${subredditList}" ] && [ -s "${subredditList}" ] && gio open "${subredditList}" &
fi

popup "${currentRedirect}" # Blocking command. Once popup is closed, we will kill the ringer.
setGlobalState "ring" "false" # Seems /dev/shm is the way to communicate with background processes.
printf "Popup closed. Waiting for ringer to end. [pid=%s]\n" "${ringerPID}"
wait "${ringerPID}"
printf "Ringer ended.\n"
fi
}

function listSubreddits()
{
# Maybe one of these subreddit will care about the URL change
if [ -f "${subredditsInput}" ]; then
# Expected format is simply to define an array named ${subreddits[@]}
# You can form that array however you want. Keep it simple, or go nuts.
source "${subredditsInput}"
fi

for subreddit in "${subreddits[@]%\/}"; do # Normalize sub names

by removing trailing slash if present.
subreddit="${subreddit##*\/}" # Normalize sub names by removing any subreddit prefix.
printf "https://old.reddit.com/r/%s\n" "${subreddit}"
done > "${subredditList}"
}


###############################################################################################################
############ Script Start #####################################################################################
###############################################################################################################

divider

# To list hourly noscripts that run.
# sudo run-parts --report --test /etc/cron.hourly
# If started as root, then re-start as "${user}": (https://askubuntu.com/a/1105580)
if [ "$(id -u)" -eq 0 ]; then
log "Refusing to run as root!"
#exit; #Don't run as root.
exec sudo -H -u "${user}" bash -c "${0}" "${@}" #Force run as user.
echo "This is never reached."
fi

# ${DISPLAY} is typically unset when root sudo's to user.
if [ -z "${DISPLAY+unset}" ]; then
export DISPLAY=":0.0"
fi

log "Running as \${USER}=${USER} id=$(id -u). Script version = ${version}"

function exitCron()
{
printf "Exiting due to interrupt\n" >> "${logFile}"
if [ -n "${ringerPID}" ] && ps -p "${ringerPID}" > /dev/null; then
kill -15 "${ringerPID}"
fi

# unset traps.
trap - SIGHUP
trap - SIGINT
trap - SIGQUIT
trap - SIGTERM

cleanupSharedMemory
exit 1
}

#Don't think this is nessesary, but just make sure we exit each process when told to.
trap exitCron SIGHUP SIGINT SIGQUIT SIGTERM

if [ ! -d "${workingDir}" ]; then
mkdir -p "${workingDir}"
fi

# For loop will excute once every 600 seconds (10 minutes) for an hour
# shellcheck disable=SC2034 # Not using variable ${i}
for i in {0..3599..600}; do
checkReDirect
#break; #Uncomment to run once for testing.
sleep 600;
done

logSize="$(getLogSize)"
#Trim log length by about 10% if we have gone over ${maxLogSize}.
if (( "${logSize}" > "${maxLogSize}" )); then
truncateLog "0.90"
fi

cleanupSharedMemory

###############################################################################################################
# References:
#
# Check if variable is empty or unset:
# https://stackoverflow.com/questions/3601515/how-to-check-if-a-variable-is-set-in-bash
# https://www.cyberciti.biz/faq/unix-linux-bash-noscript-check-if-variable-is-empty/
#
# Send variable to background process:
# https://stackoverflow.com/questions/13207292/bash-background-process-modify-global-variable
#
# Limit log size keeping last n lines:
# https://unix.stackexchange.com/questions/310860/how-do-you-keep-only-the-last-n-lines-of-a-log-file
#
###############################################################################################################

https://redd.it/z7endd
@r_bash

Press ctrl z and type bg from a bash noscript automatically

So i have a timer i run from telegram connected to my pc. Do anyone know if there is a possibility to automatically stop the noscript and ”automatically” put it under ”jobs” without ”&” when started.
Currently i have to manually press ctrl z and then type bg

https://redd.it/z7avv4
@r_bash

pass to xargs

Hello I'm trying to process the output of \.

I run: mynoscript * to process all files in the current directory.

The output should be the full path of *.

This is what I have (not working):

echo ${@:1} | tr '\n' '\0' | xargs -0 -I "%" realpath "%"

can someone help?

https://redd.it/z7r0aj
@r_bash

How to make tests in .bashrc so they execute as fast as possible?

I am using this snippet from an AskUbuntu post in my own .bashrc, but would like to improve it so it works also in a tmux session:

if "$color_prompt" = yes ; then
# when system is accessed via SSH, hostname with light grey background
if [ $(pstree -s $$) = *sshd* ]; then sshbg="[\03348;5;7m\"; fi
# when used as root, change username to orange and '#' to red for prompt
if $(id -u) -eq 0 ; then usercol="[\03338;5;3m\"; hashcol="[\03338;5;1m\"; else usercol="[\03338;5;2m\"; fi
# bash PS1 prompt
PS1="${usercol}\u[$(tput sgr0)\]@[$(tput sgr0)\][\03338;5;4m\${sshbg}\h[$(tput sgr0)\]:[$(tput sgr0)\][\03338;5;6m\\w[$(tput sgr0)\]${hashcol}\\$ [$(tput sgr0)\]"
unset sshbg rootcol hashcol
fi

I can test similar to line 3 above if pstree -s $$ returns tmux: server, and if it does, I can test [[ $(pstree -s $(pgrep "tmux: client")) = *sshd* ]] to get the desired result.

So far, I have:

if "$color_prompt" = yes ; then
# when system is accessed via SSH, hostname with light grey background
if [ $(pstree -s $$) = *sshd* ]; then
sshbg="[\03348;5;7m\"
elif [ $(pstree -s $$) = *"tmux: server"* ]; then
if [ $(pstree -s $(pgrep "tmux: client")) = *sshd* ]; then
sshbg="[\03348;5;7m\"
fi
fi
# when used as root, change username to orange and '#' to red for prompt
...

Is there any better, especially faster way? Since it generates PS1, I would like it to be as fast as possible to not be annoying.

https://redd.it/z8033h
@r_bash

read command removes empty lines

#!/usr/bin/env bash
ls ??-??.json | while IFS= read -r line; do ./noscripts/add-missing-strings.js "$line"; done

What I have already tried:-played with ifs-read -r-started questioning if the problem is really here and not in the .js

All files - https://github.com/RDKRACZ/for-reddit/tree/main/noscripts

https://redd.it/z8955d
@r_bash

Is there any way to have a "longterm history" in addition to the normal history?

I like to keep a record of all the commands I have run on my VPS, because it's a good reference for how I installed or configured something many months ago. So I have my

HISTFILESIZE=10000, and HISTSIZE=10000

along with

export PROMPTCOMMAND="history -a; history -c; history -r;$PROMPTCOMMAND"

to keep the history file up to date after each command. But my command item numbers are in the 5000's now, so it's a little inconvenient to search through such a long history when I'm looking for something I just used a few days ago. Is there any way to to write to a file like .historylong in addition to my normal history? That way I could keep my normal history shorter and only reference the "indefinite history" when needed. And would there be any way to have .historylong displayed with a command line number that I could use in the infrequent times that I want to run something from long ago like:

!4573

https://redd.it/z8vubd
@r_bash

Why does sed delete the file's content when an output redirection is performed on the file, and not when using the append >> redirection?

For example, assume file.txt has the following content:

This is an example.

When I invoked the following command sed '/s/example/EXAMPLE/' file.txt > file.txt, the file's content was deleted.

But invoking this command sed '/s/example/EXAMPLE/' file.txt >> file.txt, produces the following output:

This is an example.
This is an EXAMPLE.

Isn't the output of sed produced on the stdout, and then the redirection operator redirects that output and rewrites the file?

https://redd.it/z8x9c8
@r_bash