grep and force header (show filename) behaviour. How to format the output?

I have very basic noscript along the lines

for i in $(run some command that gives a list of files)
do grep -H $i ^pattern
done

which is giving me the output

/home/user1/textfile1.txt:pattern is matched in this line
/home/user1/textfile2.txt:pattern is a match here in this line
/home/user1/textfile33.txt:pattern is a word in this file

That is, the filename and the matched grep are concatenated altogether on one line with a colon field separator in between.

What I'd like, without too much more effort, is to put the output on separate lines more like

/home/user1/textfile1.txt
pattern is matched in this line
/home/user1/textfile2.txt
pattern is matched here in this line
/home/user1/textfile33.txt
pattern is matched

Can grep do this? Or do I need to get complicated with the noscript and do something stupid like run grep twice or use an if to print file name if grep returns a result?

Also, using sed to change that colon into a line break isn't going to work where either the filename/path or the matched line include a colon. Grep doesn't appear to have an option to change the field separator from colon to anything else, either (very interesting design choice, I think).

https://redd.it/12i4iqz
@r_bash

Well, experts, show me the way. Not able to get the construction right off if loop

Here is a short program, which restricts the search within the directory(how trivial it may sound to ya)

#!/usr/bin/env bash

1

2 check_git="git -C $PWD rev-parse"

3

4 usage()

5 {

6 echo Provide name with regex delimeters.

-- 7 echo Example : $(basename $0) \"*bash*\"

8 }

9

10 if [ $# -eq 0 ];then

11 usage

12 exit 1

13 fi

14

15 local_search()

16 {

17 if [[ "$(eval "$check_git")" == "" ]];then

18

19 git grep -n "$1"

20

21 elif [[ ! -d .git ]];then

22

23 find "$PWD" -type f,d -name "$1" -ls

24

25 else

26 :

27 fi

28 }

29

30 local_search "$@"

​

Alright, now running this inside a git repo produced the right output , like this ,

~/LaTeX_Workouts [master|✔]

08:15 $ local_search "tex"

Bhaskar_Chowdhury.log:1:This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020 Gentoo Linux) (preloaded format=pdflatex 2020.8.23) 23 AUG 2020 10:59

​

Now, running this on a normal directory i.e. not a git repo, produces this:

tp_x250_08:17:40_Tue Apr 11: :~/Music>local_search "Mark"

fatal: not a git repository (or any of the parent directories): .git

fatal: not a git repository (or any of the parent directories): .git

​

In short, it is not working as "expected" due to the logical flaw in the loop construct, I am not able to fix it.

So, shed some light. ...and please be precise.

https://redd.it/12i6gv7
@r_bash

When using bash variable operators such as${x[@]##y}, can you "anchor" what matches y to the start/end of the variable / array field?

TL;DR: Im looking for functionality in pure-bash matching like what ^ and $ provide in regex matching.



Say I have an array

a=('hi' '-hi' 'hihi' 'hiya' 'hi how are you' 'hi' 'hi'$'\n''there')

Now say I want to clear (replace with `''`) the two fields that contain exactly the string 'hi' and nothing else. Can this be done in an `a=("${a<someOperator>}")` sort of way?

Now I mean I know a bunch of ways to do this. For example, one can do this in a single-line command that uses external commands, pipes, and bash substitution. e.g.,

mapfile -t a < <(printf '%s\n' "${a[@]//$'\n'/$'\034'}" | grep -vE '^hi$' | tr $'\034' $'\n')

I also know a "pure-bash" way that one can do this that uses a temporary non-array variable. e.g.,

a0="$(printf '%s\n' '' "${a[@]//$'\n'/$'\034'}")"
mapfile -t a <<< "${a0//$'\n'hi$'\n'/}"
a=("${a[@]//$'\034'/$'\n'}")

And of course the good old fashioned loop

for kk in "${!a[@]}"; do [[ "${a[$kk]}" == 'hi' ]] && unset a[$kk]; done

but I dont know how to do this in a single command without pipes / subshells / external commands / copying data into a temporary variable / loops (which `a=("${a<someOperator>}")` would seemingly provide). Is this possible?

Thanks in advance.



P.S. Note that the first two methods above methods temporarily replace newlines that are in the array field data with the ASCII field separator (FS) control code (octal 34). In the unlikely event that the array field data already has any of these FS control characters naturally, a different temporary newline replacement (that isnt already present in the data) would need to be used.

https://redd.it/12i7sxy
@r_bash

How to create log of all file modifications with timestamps?

Hi,

Bash newbie here. I'm trying to create a noscript that will create a log of all file modification operations done across my mac for a set time period (year ish). So it'll look something like

[01/02/23\] - fileA.txt OPENED

[01/02/23\] - fileA.txt MODIFIED

[01/06/23\] - fileB.txt DELETED

[01/15/23\] - fileA.txt DELETED

etc...

Not looking to be handed the answers. Just some guidance on where to look. An idea I had was to use the bash noscript to access the mac logs and filter by file operations, and then output those. But I can't find logs that have that file modification info.

https://redd.it/12i5v16
@r_bash

How to add progress bar and log to this noscript?

https://raw.githubusercontent.com/mfn77/Post-Install/main/Post-Install.sh

I wrote a bash noscript for my post install configuration. I am definetely not an expert and this code probably is garbage but it works for me, at least it looks like it 's working. But I am wondering that can I add progress bar between user interactions and write a log file instead for normal terminal output? If I can how?

https://redd.it/12iizna
@r_bash

rclone copy (or sync) with fzf

I'm not sure if I've ever posted any of my code like this; it's usually only in some comment responding to someone that I do that

but this little bash noscript, barely 35 lines if you leave out the comments and the "help" text, seems interesting enough that someone might care to try it

the use case is when you want to run rclone copy (or sync) but you'd like to (a) select what to copy using fzf, and (b) just as importantly, you want to copy multiple directories (or multiple files)

this ability to select is especially useful if you're copying from a remote to a local directory

code is at https://github.com/xkcd386at/noscripts/blob/master/fclone if you want to take a look

https://redd.it/12ijnqh
@r_bash

Adding portions of strings in tab delimited file to another column

Hi,

I have a tab separated file (it's an .ALE file exported from Avid Media Composer). I am trying to add portions of one column's data into other columns on the same row. The second column in this file is a list of filenames. Each filename has specific information about the file which I'd like to add in the same row but under another column.

For example, I have the following columns: Name, Scene, Camera, Take

Name Scene Camera Take
302012_0017_2-33f_1b_b026.mov
302012_0016_2-33f_1a_a026.mov
302012_0019_2-33f_2a_a026.mov
302012_0024_2-33f_3c_c021.mov
302012_0020_2-33f_2b_b026.mov
302012_0018_2-33f_1c_c021.mov
302012_0022_2-33f_3a_a026.mov
302012_0023_2-33f_3b_b026.mov
302012_0021_2-33f_2c_c021.mov

In the first filename, 33f would be the Scene, 1b would be the Camera, and b026 would be the Take. On the command line, I can use the following to get each section one-by-one, changing the field of the last cut command for each section, but I'm also not sure how to iterate through the filename to get each part.

echo {302012_0017_2-33f_1b_b026.mov%.*} | cut -d - -f2 | cut -d _ -f1

Any help or pointers would be appreciated.

Thank you!

https://redd.it/12jcjug
@r_bash

I can't fathom what's wrong with this noscript. Feel like I'm missing something small but can't see what. Can anyone spot something?

Hello. I have the following noscript that scans recursively within directories for .mkv files and shows me those without subnoscripts. Everything about this seems to me that it *should* work, but when I run it I get sh: line 5: : No such file or directory errors, regardless of which directory I use as $dir. There is also never any ouput file generated. Despite the warnings, I can hear the disk scanning the directories for some time, so it's doing *something*, just not what I wanted :D

I first thought it could be a permissions issue, but even when running as root I get the same error. The noscript is of course executable and mkvmerge is indeed in my PATH. I'm a bit stumped! Thanks

&#x200B;

#!/bin/bash

# Set the directory to search for .mkv files

dir="/path/to/directory"

# Set the output file

output_file="output.txt"

# Find all .mkv files recursively in the directory and check if they have subnoscripts

find "$dir" -type f -name "*.mkv" -exec sh -c '

for file do

# Check if the file has subnoscripts

if ! mkvmerge -i "$file" | grep -q "subnoscripts"; then

# Output the file name to the output file

echo "$file" >> "$output_file"

fi

done

' sh {} +

echo "Done. Output saved to $output_file."

https://redd.it/12jexov
@r_bash

"what" -- A tool to get info about commands. I wrote it after I got fed up with how uninformative the standard tools like "type" and "which" can be, and how much digging you have to do to figure out problems.
https://github.com/wjandrea/what-bash

https://redd.it/12jqk01
@r_bash

Why is trap handler not invoked immediately in this noscript

#!/bin/bash


cleanup() {
echo "Received SIGTERM signal. Cleaning up..."
exit 1
}

echo "Spawning child process..."
trap cleanup SIGTERM
sleep 10
echo $?


When I issue a SIGTERM after 2-3 seconds after invoking this process the cleanup is called AFTER 10 seconds. Shoulnd't the trap handler invoke immediately?

https://redd.it/12jqard
@r_bash

bashelim collates a noscript with its sources (nested), and sends the result to stdout

Sometimes it is nice to collate all the noscripts that is at play,
in order to make it easier to inspect, or to share with someone
maybe after some final editing.

This noscript, is an adaption of the `soelim` noscript for collating
`troff` sources, for bash, it understands `.` and `source`, and also
nested files, and that cycles of nested files are bad!

This version doesn't read the path and looks for sources any other
place than in the current folder, tildes are expanded into the home-
folder however.


#!/bin/awk -f
# McUsr 2023 Mostly stolen from Jon Bently's m1.awk
# Vim licence
# bashelim instead of soelim V.0.0.0
# Collates the bashcript presented on the command line
# with all sourced files, for debugging purposes.
# Tildexpands any paths.but doesn't look through the path
# to find files without pathname besides the current folder.
BEGIN {
RS="\n"
hp=ENVIRON["HOME"]
}
function error(s) {
print "m1 error: " s | "cat 1>&2"; exit 1
}
function dofile(fname, savefile, savebuffer, newstring) {
if (fname in activefiles)
error("recursively reading file: " fname)
activefiles[fname] = 1
savefile = file; file = fname
savebuffer = buffer; buffer = ""
while (readline() != EOF) {

if (/^[ \t]*source[ \t]/) {
if (NF != 2) error("bad source line")
sub("~",hp,$2)
dofile(dosubs($2))
} else if (/^[ \t]*\.[ \t]/) {
if (NF != 2) error("bad source line")
sub("~",hp,$2)
dofile(dosubs($2))
} else
print $0
}
close(fname)
delete activefiles[fname]
file = savefile
buffer = savebuffer
}
# readline
#Put next input line into global string "buffer".
#Return "EOF" or "" (null string).
function readline( i, status) {
status = ""
if (buffer != "") {
i = index(buffer, "\n")
$0 = substr(buffer, 1, i-1)
buffer = substr(buffer, i+1)
} else {
# Hume: special case for non v10: if (file == "/dev/stdin")
if (getline <file <= 0)
status = EOF
}
# Hack: allow @Mname at start of line w/o closing @
if ($0 ~ /^@[A-Z][a-zA-Z0-9]*[ \t]*$/)
sub(/[ \t]*$/, "@")
return status
}
function dosubs(s, l, r, i, m) {
if (index(s, "@") == 0)
return s
l = "" # Left of current pos; ready for output
r = s # Right of current; unexamined at this time
while ((i = index(r, "@")) != 0) {
l = l substr(r, 1, i-1)
r = substr(r, i+1) # Currently scanning @
i = index(r, "@")
if (i == 0) {
l = l "@"
break
}
m = substr(r, 1, i-1)
r = substr(r, i+1)
if (m in symtab) {
r = symtab[m] r
} else {
l = l "@" m
r = "@" r
}
}
return l r
}
BEGIN {
EOF = "EOF"
if (ARGC == 1)
dofile("/dev/stdin")
else if (ARGC >= 2) {
for (i = 1; i < ARGC; i++)
dofile(ARGV[i])
} else
error("usage: m1 [fname...]")
}
# This noscript is excavated out of the M1.awk macro processor by Jon L. Bentley.
#M1 was documented in the 1997 sedawk book by Dale Dougherty & Arnold Robbins (ISBN 1-56592-225-5)
#but may have been written earlier.
#.P
# This noscript was adapted from the 1997 sedawk book by Dale Dougherty & Arnold Robbins (ISBN 1-56592-225-5)
# 131.191.66.141:8181/UNIX_BS/sedawk/examples/ch13/m1.pdf (download from
#<a href="http://lawker.googlecode.com/svn/fridge/share/pdf/m1.pdf">LAWKER</a>).
# Author Jon L. Bentley. (Of "Programming Pearls" fame.)

https://redd.it/12jv83a
@r_bash

Yet another To Do manager written in BASH. Simple and colorful.

https://redd.it/12jvxeu
@r_bash

New release of bkt, a subprocess caching utility

Hi all, I recently cut a new release of `bkt` with some additional functionality. Notably, it's now possible to include a file's last-modified time in the cache key, thereby invalidating the cache if the file changes.

Wait, what is bkt?

bkt is a subprocess caching utility you can use to persist a command's output so that subsequent invocations are fast. As an example, I use bkt heavily in my shell prompt to speed up the information it displays.

Another way I use bkt often is to simplify and speed up iterating on command pipelines that are slow to run. For example, if you're using jq to play around with a JSON response you might do something like this:

$ curl http://some.api/data/a | jq '.foo'
$ curl http://some.api/data/a | jq '.foo.bar'
$ curl http://some.api/data/b | jq '.foo.bar.baz'

Which is obviously wasteful and slow. You could write the output to a file and then pipe that to jq, but you often end up juggling multiple response files and it get's tedious quickly.

Instead, using `bkt` ensures each request is only sent once and all subsequent calls return locally cached results:

$ bkt --ttl=1d -- curl http://some.api/data/a | jq '.foo'
$ bkt --ttl=1d -- curl http://some.api/data/a | jq '.foo.bar'
$ bkt --ttl=1d -- curl http://some.api/data/b | jq '.foo.bar.baz'

If you haven't used it before give it a spin! If you find it useful please share how you're using `bkt` so others can benefit :)

https://redd.it/12ke9i3
@r_bash

bash-hackers.org is now a parking domain

Hi,
i have just noticed bash-hackers.org is now a parking domain, narf. Does anybody have some insights what happened and if there is some new place for this very much appreciated resource?

> whois bash-hackers.org
Domain Name: bash-hackers.org
Registry Domain ID: 660cea3369e54dbe9ca037d2d1925eaa-LROR
Registrar WHOIS Server: http://whois.ionos.com
Registrar URL: https://www.ionos.com
Updated Date: 2023-04-13T05:09:00Z
Creation Date: 2007-04-13T04:46:21Z
Registry Expiry Date: 2024-04-13T04:46:21Z
Registrar: IONOS SE

https://redd.it/12klulf
@r_bash

Passing a command with double quotes to a function

Hi, I'm writing a noscript to create thumbnails and see them as preview for different types of files.

#!/bin/sh

file="$1"
w="$2"
h="$3"
x="$4"
y="$5"

mkdir -p '/tmp/lf'

cache() {
 cache="/tmp/lf/$(echo "$file" | tr '/' '%')"
 [ ! -f "$cache" ] && "$@" && echo '[SUCCESS]'
 kitten icat --silent --transfer-mode file --stdin no --place "${w}x${h}@${x}x${y}" "$cache" < /dev/null > /dev/tty
}

case "$(file -Lb --mime-type "$file")" in
 application/json) jq -C "$file" ;;
 application/octet-stream|video/*)
  cache ffmpegthumbnailer -i "$file" -o "$cache.jpg" -s 0 -q 4 ;;
 application/pdf)
  cache pdftoppm -singlefile -jpeg "$file" "$cache" ;;
 application/x-7z-compressed) 7z l -p "$file" ;;
 application/x-tar) tar rf "$1" ;;
 application/x-rar) unrar lt -p- "$file" ;;
 application/zip) unzip -l "$file" ;;
 image/*) kitten icat --silent --transfer-mode file --stdin no --place "${w}x${h}@${x}x${y}" "$file" < /dev/null > /dev/tty ;;
 text/*) cat "$file" ;;
 *) echo '----- File Type Classification -----' && file -Lb "$file" ;;
esac

exit 1

The problematic part is with passing to cache the command to create a thumbnail like in the case of application/pdf. I see the message [SUCCESS] on screen but the files doesn't get created at all. In the case of video/* I don't even see the message on screen.

Does somebody knows if I'm passing those commands to cache the wrong way? (Also if there is any other suggestion on style please tell me)

https://redd.it/12kwbq8
@r_bash

Brackets in sh noscript.

Hi!

I can't understand the code some noscript added to my rc.local when setting up vpn.

(sleep 15

service ipsec restart

service xl2tpd restart

echo 1 > /proc/sys/net/ipv4/ip_forward)&

what are the "()" brackets for and why adding "&" at the end of the block? Wouldn't sh go line by line anyway? Rest of the code are clear to me. Could you clarify it for me? googling bash/sh brackets is like looking for a needle in the haystack.

https://redd.it/12kyyqt
@r_bash

What happened with wiki.bash-hackers.org?

This site was a great guide for me, does anyone know what happened?, today I tried to check the page and it seems to be dead 😢

https://redd.it/12lmqoy
@r_bash

Yo: yet another AI-powered terminal assistant

https://redd.it/12logjr
@r_bash

Need help as an absolute beginner on directories and files

Hi everyone. I have a class this semester which requires me to work with bash. We have a homework and unfortunately my course does not provide any helpful infos on how bash works. Here is my homework

Consider the following output from /usr/bin/tree -p | sed -E "s/(\\[d?)([rxw-\]+)/\\1/g":

.

|- [d\] bars

| |- [d\] baz

| | |- [\] corge

| |- [\] quuz

|- [d\] corge

| |- [d\] grault

| |- [d\] garply

| | |- [\] version

| |- [\] partitions

|- [d\] foo

| |- [\] quux

|- [\] fstab

|- [\]qux

|- [\] rumo

6 directories, 8 files

Create the same structure consisting of 6 directories and 8 files using only shell commands. Give all the commands to solve this task in the correct order. For the version, partitions, and fstab files, use the appropriate /proc/version, /proc/partitions, and /etc/fstab files. All other files should be empty.

From all the research i have done on the internet i came up with something like

mkdir root_.

mkdir root_./bar

mkdir root_./bar/baz

mkdir root_./corge

mkdir root_./corge/grault

mkdir root_./corge/grault/graply

mkdir root_./foo

touch root_./baz/corge.txt

touch root_./bar/quuz.txt

... and so on

Am I on the right path here? It could be compeletly different than what they are asking. Would be amazing if i could learn what am i supposed to do and maybe what to study.

All helps are appreciated thank you!

https://redd.it/12lrj2o
@r_bash

Keyboard Shortcut won't execute noscripts and commands which are easily executable on terminal.

So, I installed a package called pix2tex. I wrote a noscript to run it and there also was a pre-written noscript which would launch a window as you can see here in this video

However, though the noscripts and commands run perfectly fine on terminal, they won't run when they are called with a custom shortcut that I assigned them in keyboard settings.


I recorded another video to demonstrate this issue. The noscript which is being executed is

#!/bin/bash
xfce4-screenshooter --region --save /home/bob/Pictures/Screenshots/a.png
#only the xfce4-screenshooter command would be executed
pix2tex /home/bob/Pictures/Screenshots/a.png | sed 's/.*: //' > /home/bob/Pictures/Screenshots/a.tex
xclip -selection clipboard /home/bob/Pictures/Screenshots/a.tex
exit 0

Pix2tex, takes the screenshot, a.png and converts into latex. It's saved in a.tex and then it's copied. Unfortunately, when I try it with keyboard shortcut, it won't even be saved in a.tex (but it will be for terminal executed noscript).

~Edit: I do want to know the answer for future purposes, but for now anyway to run latexocr gui without actually having to open the terminal would suffice, is there a way to use a shortcut to do the same job as I am doing in the first video?~

https://redd.it/12lv6hs
@r_bash

Is it possible to make zsh look like GitBash without appealing to OhMyZsh?

Hi everyone!

Nothing to add to the noscript, it speaks by itself, but to give you as much informations as possible I tell you what I did!

So, I recently switched to Mac. I'm studying web development and until now I used GitBash for windows. I immediately realized that zsh style was pretty different so I tried to modify it to make it look as close as possible like GitBash. I created a .zshrc file in my home directory and with vscode I did the following modifications PS1="%F{green}%n@%m%f %F{yellow}%1\~%f $ "
The problem is that there is a lot of stuff that I'd like to modify but I wasn't still able to figure out how yet. These are the main things that I'd like to achieve:
\-When I do 'ls' the directories are not listed in blue with the '/' symbol. I have to 'ls -F' to identify them as a directories (and of course they still miss the color).
\-I didn't do anything regarding git yet but I'd like to have the branch names between brackets and coloured in blue
\-In git bash I had the whole paths listed. For example I had \~/one/two/current-directory. Now I just have the current-directory listed. I have to 'pwd' to see the whole path.
\-Finally, in GitBash the prompt $ started in a new line.

I looked online and some people suggest ohmyzsh. I'd like to achieve these results with out it. I also did 'man ls' to see what it says about colours. It talks about CLICOLOR and LSCOLORS but I wasn't able to understand how to apply them to get the result yet.

Could you guys help me?

Thank you!

https://redd.it/12lz3kb
@r_bash