Count totals and correlate to $1

Hi all, I'm stumped by a problem and would love if I could get some help. I have a txt with lines and lines of data like this:

xxx.xxx.xx.xxx ftp ssh
yyy.yyy.yy.yyy ssh
zzz.zzz.zz.zzz smtp ftp

I need to count and correlate each service to the IP address, so the output would be similar to:

ftp count: 2
xxx.xxx.xx.xxx
zzz.zzz.zz.zzz

ssh count: 2
xxx.xxx.xx.xxx
yyy.yyy.yy.yyy

smtp count: 1
zzz.zzz.zz.zzz

I've been trying tons of stuff with awk but I'm getting nowhere and am afraid I'm deep down a rabbit hole. I think I need someone else's perspective on this one.

Anything you could give me to point me in the right direction would be awesome! Thanks!

https://redd.it/x6l7t5
@r_bash

Grepping for whitespace and double quotes as part of an if-then statement

I am attempting to create a noscript that will eventually run as a cron to check a conf file, and if it does not have a critical setting within it, to execute another noscript that regenerates the conf file.

The critical setting is: SecRuleEngine "On"

I can get the grep on my bash prompt just fine:

user@host ~# grep --color -h 'SecRuleEngine "On"' /path/to/conf-file.conf
SecRuleEngine "On"

And what I was wanting to do was to have a simple 1 for yes, 0 for no, like:

user@host ~# grep -c -h 'SecRuleEngine "On"' /path/to/conf-file.conf
1

Which I was hoping to feed into an if then statement in the noscript to test it, like:

if grep -c -h 'SecRuleEngine "On"' /path/to/conf-file.conf = "1" ; then
echo "all good"
fi

But on the line that starts with "if" I get:

: too many arguments

What I was hoping to do was get the noscript to see the grep as intended, then reverse it by putting a `!` after the `if` and then issuing the command to run the configuration noscript. What corrections do I need to make to my syntax?

[https://redd.it/xdmqhu
@r_bash

How do you export a mongo db into a JSON?

mongoexport --uri="mongodb://localhost:27017/" --username admin --password root --authenticationDatabase "admin" --db products --collection shoes --out dump.json

What's wrong with this command?

2022-09-02T18:00:24.492-0400 could not connect to server: connection() error occurred during connection handshake: auth error: sasl conversation error: unable to authenticate using mechanism "SCRAM-SHA-1": (AuthenticationFailed) Authentication failed.

There shouldn't be any authentication. Looking inside Studio 3T, I see Authentication Mode: None.

https://redd.it/x6zz1k
@r_bash

Parsing a csv with Unix time and an integer on each line.

Hello,

My csv looks something like this:

timestamp,Value

1662353808788,5

1662354108786,12

.

.

.

Hundreds of lines, ordered by time.

My goal is for the noscript to return the biggest value recorded in the last 24hours and ideally if given a parameter should be also able to return the biggest value for a specific day. I imagined something like placing a 0 as a parameter for today, 1 for the day before, 2 for the day before that, but not set on that.

Any help appreciated, even if you could just tell me what I will need to accomplish it, some algorithm, pseudo code, commands I should look at. Never worked with unix time before.

Thank you

https://redd.it/x6cacz
@r_bash

printf vs echo - Showing last command

Hey all!

I've been writing a pretty basic noscript to take care of some file backup tasks for me. I'm trying to be good by commenting and producing log output. I'm having an issue getting the last command run to display in a function I've declared.

Here are the function declarations:

# Declare conditional logging
function success()
{
echo -e ${date} "$? ran successfully!" >> ${dockerDir}/shared/archive.log
}

function fail()
{
echo -e ${date} "\xF0\x9F\x92\x80" "$? failed..." "\xF0\x9F\x92\x80" >> ${dockerDir}/shared/archive.log
}

Pretty straightforward, right?

Here is one example of calling the functions with nested if statements:

#Tar 'em up
tar --create --bzip2 --file ${dockerDir}/shared/${archiveTar} ${dockerDir}${bookstack}*bookstackFiles.tar ${dockerDir}${bookstack_db}*bookstack_db.sql
if [ $? -eq 0 ]; then
success
else
fail
if [[ -e ${dockerDir}${bookstack}*bookstackFiles.tar || ${dockerDir}${bookstack_db}*bookstack_db.sql ]]
then
rm ${dockerDir}${bookstack}*bookstackFiles.tar
rm ${dockerDir}${bookstack}*bookstack_db.sql
EXIT
fi
fi

Now, here's the output from archive.log:

2022/09/13 20:54:29 0 ran successfully!
✅ 2022/09/13 20:54:29 \ Bookstack files tarred and bzipped successfully! 202209132054-bookstackArchive.tar.bz2 ✅
2022/09/13 20:54:29 0 ran successfully!
✅ 2022/09/13 20:54:29 \ Bookstack files encrypted successfully! 202209132054-bookstackArchive.tar.bz2.gpg ✅

The issue I'm trying to figure out is why it's not showing me in the log the last command that was run. In the first line, it should show something like "tar ran successfully!" (I'm not positive if it's going to include the command args or not). The third line should show something like "gpg ran successfully!".

Instead I'm just getting "0". I've tried different permutations of this using printf, echo -e, echo. I've tried different ways of referencing the previous command like $?, !:0, referencing a different variable I declare earlier in a trap statement.

Nothing is giving me the expected output. Thoughts?

https://redd.it/xds4z6
@r_bash

File Metadata "Comment"

How to read the metadata "comment" of a file?

https://redd.it/xdw131
@r_bash

[OC] shortbashpwd: prompt for bash with abbreviated directory names like in fishgi
https://github.com/NikitaIvanovV/shortbashpwd

https://redd.it/x60spv
@r_bash

What does this command do in .zsh file

I am using the docker-compose plugin in zsh and want to use docker compose v2 commands. so when exploring the plugins I found this line

(( ${+commands[docker-compose]} )) && dccmd='docker-compose' || dccmd='docker compose'

can someone explain what does this do?

https://redd.it/xdxqvd
@r_bash

Inserting variables under the third column in a textfile.

So i have two columns and would like add text to the Value3 column.

i've tried code below which works for the first two columns. But not for the third.

Help or suggestions is appreciated.

echo -e "" "val1 $numb1" "\n" "val2 $numb2" | paste -d "\t" /files/test.txt -

Columns

Value1 Value2 Value3
val1 34 val1 55
val2 34 val2 55

https://redd.it/x5y5wa
@r_bash

Linux Tech eBook Bundle by No Starch Press
https://www.humblebundle.com/books/linux-no-starch-press-books?partner=indiekings&charity=1542725

https://redd.it/xe09me
@r_bash

can someone what did I overlook

what is the purpose of the noscript: unlocking the dataset using the curl command

the code that works:

curl "`https://$host/api/v2.0/pool/dataset/unlock`" -k -X POST -H "accept: */*" -H "Content-Type: application/json" -H "Authorization: Bearer 1-$API_TOKEN" -d '{"id": "the_dataset_hardcoded","unlock_options": {"key_file": false,"recursive": false,"toggle_attachments": true,"datasets": [{"name" : "the_dataset_hardcoded" , "passphrase" : "the_password_hardcoded"}]}}'

now this one works flawlessly, but the moment i try to replace the "the_dataset_hardcoded" and/or the "the_password_hardcoded" as an environmental variable the noscript stops working

my next try is to create the part as a string that i can echo out for debugging so i create:

curl_builder3="'{\"id\": \"$pool\",\"unlock_options\": {\"key_file\": false,\"recursive\": false,\"toggle_attachments\": true,\"datasets\": [{\"name\" : \"$pool\" , \"passphrase\" : \"$pass\"}]}}'"

and run it like this:

curl "`https://$host/api/v2.0/pool/dataset/unlock`" -k -X POST -H "accept: */*" -H "Content-Type: application/json" -H "Authorization: Bearer 1-$API_TOKEN" -d $curl_builder3

but what i get for this is:

curl: (3) unmatched brace in URL position 1:

{"key_file":

^

bash: line 3: Server: command not found

​

but when I echo the created env variable i get the correct string that if i copy and run in terminal it does what is supposed to do

​

now what did I miss

​

​

edit:

the curl command is now fixed and it looks like this:

curl "`https://$host/api/v2.0/pool/dataset/unlock`" -k -X POST -H "accept: */*" -H "Content-Type: application/json" -H "Authorization: Bearer 1-$API_TOKEN" -d "{\"id\": \"$pool\",\"unlock_options\": {\"key_file\": false,\"recursive\": false,\"toggle_attachments\": true,\"datasets\": [{\"name\" : \"$pool\" , \"passphrase\" : \"$pass\"}]}}"

​

https://redd.it/x5v910
@r_bash

Launch an app from a noscript

Hi all,

​

I'm making a noscript in which i need to check if an app is launched and if it's not, launch it from the noscript.

​

The app is a .sh file, i can use the bash command to launch it but if i do the noscript doesn't stop until the app is shut down which is not what i'm looking for.

​

Basically i would like to run this noscript every 5 minutes or so, that's why i need a command that just launches the app and not runs it in the noscript. Hope i'm making sense here.

​

Any help is appreciated i'm still new to noscripting, thanks for reading !

https://redd.it/xe5egk
@r_bash

Need help with loop in a bash file

I have a bash noscript which needs to do the following:

change to folder /repos
loop through all the subfolders (git repos) and 
    run command "git config --bool core.bare true"

This is what I have so far:
for dir in ~/repos/*; do (cd "$dir" && git config --bool core.bare true); done

https://redd.it/x5g916
@r_bash

Need help improving my first Bash Script

​

What I wanted to do

* Have this scrip start every 10 minutes
* Exit the noscript if it is already running
* Copy new files to a network folder based on file creation date
* Verify the copy was successful
* Email me on completion and either a message of no errors or a list of files that did not copy correctly
* Unmount the cards

What I have so far

* A noscript that when run with the SD cards in it identifies the type of camera by card name
* Copies all the video files from camera 1's memory card to a network share
* Copies all the video files from camera 2's memory card to a network share
* Copies all the GPS data from camera 2's memory card to a network share.
* Unmount the cards

​

I should have 6 hours of time available to copy before I have to take the cards and put them back in the camera.

My code is below.

​

#!/bin/bash
clear
echo "Checking for SD Cards"
sleep 2
echo "checking for Lamnando Card"
sleep 5
if [ -d "/media/dashcam/6310-5AE7/rearVideo1" ]; then
echo "SD Card for Lanmando is inserted"
echo " Copying Files Front Camera Files"
rsync -a -P /media/dashcam/6310-5AE7/frontVideo0/* /synology-nas/shared/DashCam/
echo "Complete"
echo " Copying Rear Camera Files"
rsync -a -P /media/dashcam/6310-5AE7/rearVideo1/* /synology-nas/shared/DashCam/
echo "Complete"



sleep 2
else
echo "Lanmando Card Not Found"
fi
echo "Checking for VaVa Card"
if [ -d "/media/dashcam/VA-VD009/VAVA" ]; then
echo "SD Card for VaVa is inserted"
echo "Copying Video Files"
rsync -a -P /media/dashcam/VA-VD009/VAVA/MOVIE/* /synology-nas/shared/DashCam
echo "Complete"
echo "Copying GPS data"
rsync -a -P /media/dashcam/VA-VD009/VAVA/GPSDATA/* /synology-nas/shared/DashCam
else
echo "VaVa Card Not Found"
fi
echo "unmounting cards"
sleep 3
if [ -d "/media/dashcam/6310-5AE7/rearVideo1" ]; then
umount /media/dashcam/6310-5AE7
fi

if [ -d "/media/dashcam/VA-VD009/VAVA" ]; then
umount /media/dashcam/VA-VD009
fi
sleep 3
echo "All cards unmounted you can now safely remove them"

https://redd.it/x444uu
@r_bash

https://github.com/dzove855/Bash-Timer - Simple Bash timer

A simple Bash Timer which allows the show the execution and return code of each executed command.

https://redd.it/x3wc19
@r_bash

Script to unpack rar archives in multiple sub folders using bsdtar or p7zip?

I'm using a filezilla docker to download folders that each contain a set of rar files. I had a noscript that used unrar that filezilla would run when the download finished.

Unfortunately the docker is based on alpine linux and no longer has unrar in its repos. So my two options are bsdtar or p7zip and I haven't quite figured out how to get them to replace my previous noscript.

With unrar the line I used was unrar x -r -o- *.rar and it would go through all the folder and unpack all the archives.

7zip has a recurse option, but i can't get it to work. 7z x -r -aou *.rar is what I've tried.

Any suggestions?

https://redd.it/xeg082
@r_bash

How can I write a noscript that replaces all occurrences that match a pattern in all files within a directory and sub directory?

I am a frontend developer and we were using react-bootstrap 3 which had export a component Clearfix.
We have been importing that using

import {Clearfix} from "react-bootstrap" or import {Clearfix, (some other components separated by comma)} from "react-bootstrap"

We upgraded to react-bootstrap 4 but it doesn't export that component anymore and we have built a custom component for that. but because we have used the one from bootstrap in hundreds of files I don't wanna go and replace all one by one.
Is there a way to write a noscript that it will replace

import {Clearfix} from "react-bootstrap"

with

import {Clearfix} from "my/custom/path"

and if it finds

import {Clearfix, (some other components separated by comma)} from "react-bootstrap"

it just removes Clearfix from here and adds a new line below it with

import {Clearfix} from "my/custom/path"

https://redd.it/x3wrm3
@r_bash

"&&" operator not working with "curl"

I have the following commands I want to run in a noscript called example.sh:

./noscript1.sh && ./noscript2.sh && "curl -X POST http://<myServer>:xxxx/yyy/zzz" && "shutdown -h now"

Now the problem is that the && operator somehow breaks the curl command, it exits with the following message:

./example.sh: 3: curl -X POST http://<myServer>:xxxx/yyy/zzz: not found

Is there a working alternative to &&? Semikolon works but it breaks the && operator condition.

Thanks!

https://redd.it/x3w112
@r_bash

Authenticate using external noscript

I have a bunch of small noscripts which are interacting with a rest api. An example of such is the below list.sh noscript.

#!/usr/bin/env bash

set -euo pipefail

$HTTP "$INSTANCE_HOST/instances" "Authorization: Bearer $ACCESS_TOKEN"

Note that HTTP="http --verify=no --check-status"

All of these small noscripts have the need for a valid access token ($ACCESS_TOKEN) in common.

What I'd like to have is another noscript which I can source and have the ACCESS_TOKEN variable updated as needed. I've tried to do that using the following noscript. Named auth.sh.

#!/usr/bin/env bash

exp=$(echo "$ACCESS_TOKEN" | jq -R 'split(".")? | .[1] | @base64d | fromjson | .exp')

NOW=$(date +%s)

if [[ -z "$exp" ]] || (( $exp < $NOW )); then

echo "Update token..."

# shellcheck disable=SC2155

export ACCESS_TOKEN=$($HTTP --auth "$USER_EMAIL:$PASSWORD" post "$INSTANCE_HOST/tokens" | jq -r '.access_token')

else

echo "Token still valid!"

fi

But the above noscript always update the access token.

Using the above I'd like to have my initial list.sh noscript defined as below

#!/usr/bin/env bash

set -euo pipefail

source ./auth.sh

$HTTP "$INSTANCE_HOST/instances" "Authorization: Bearer $ACCESS_TOKEN"

Is this possible with bash? As mentioned before the auth.sh noscript always update the access token.

https://redd.it/x3r0af
@r_bash

Stupid (but documented) bash behavior

This is actually documented (and reading the documentation was what made me write this), but ... extremely surprising. I'm so horrified that I had to share. Try to figure out the output without running it.

Some of the code is not necessary to demonstrate the behavior in the version of bash I tested (5.1). But I left it in because maybe other versions do something else (since the documentation doesn't completely specify the behavior, and it surprised me).

#!/bin/bash

# Blame tilde expansion

alias foo=alias
foo=global

outer()
{
local foo=outer
inner
echo -n outer\ ; declare -p foo
}

inner()
{
#local foo=inner
alias foo=(lol wut)
local foo=inner
echo -n inner\ ; declare -p foo
}

outer
echo -n global\ ; declare -p foo
alias foo

https://redd.it/xeohfi
@r_bash

basic text editor help probably

I have 10000 plus lines , that must be made into a single line

Replacing \n with "" does not seem to work. Any idea how to do it?

https://redd.it/x38hpl
@r_bash