Author Topic: Simplicity of Linux  (Read 21829 times)

Offline Rotten Johnny

  • ROONed for life.
  • Administrator
  • Trade Count: (+31)
  • *****
  • Posts: 14,999
Re: Simplicity of Linux
« Reply #90 on: January 10, 2017, 09:59:58 PM »
You've no idea just how handy that little string was when I needed it.  Have a program that traverses a directory tree, reads metadata and build a database from it.  You can then make changes in the db and write them back to the underlying files.  Only problem is prog handles all media types so the processing would take hours and build a large db whereas I was only interested in a small subset of files scattered across approx 9TB of files.  Solution - change ownership and deny prog access to files I don't want processed. When done, revert ownership and permissions. Sorted everything inside 20 mins. :2thumbs:
Audiophile: There is almost no other group that prides themselves more on wasting good money on utterly worthless ****, and then trying to furiously blow smoke up their own ass to justify it.

Free your mind...and your ass will follow.

Offline Rotten Johnny

  • ROONed for life.
  • Administrator
  • Trade Count: (+31)
  • *****
  • Posts: 14,999
Re: Simplicity of Linux
« Reply #91 on: March 20, 2017, 05:08:24 PM »
Increment the modified timestamp of all folders and files in a directory tree by a given time interval using their existing timestamp as a base

Code: [Select]
find . -print0 | xargs -0 -I '{}' touch -m -r '{}' -d '+1 second' '{}'
Audiophile: There is almost no other group that prides themselves more on wasting good money on utterly worthless ****, and then trying to furiously blow smoke up their own ass to justify it.

Free your mind...and your ass will follow.

Offline BWS

  • Administrator
  • Trade Count: (+1)
  • *****
  • Posts: 15,784
Re: Simplicity of Linux
« Reply #92 on: March 20, 2017, 05:23:05 PM »
^^ Why did you need to do that ?
In the word : Scent, is it the S or the C that is silent?

Offline Rotten Johnny

  • ROONed for life.
  • Administrator
  • Trade Count: (+31)
  • *****
  • Posts: 14,999
Re: Simplicity of Linux
« Reply #93 on: March 20, 2017, 05:41:22 PM »
^^ Why did you need to do that ?

I've got my tagger set to preserve mod time when I make changes and I've got Roon set to regard a file's mod time as the import date...so as long as I always use touch before moving tunes to my server I can always browse albums in the sequence I got them (as well as many other sort order's Roon provides).  If Roon doesn't see a change in last mod time it will not rescan tag contents in existing files.   This triggers a forced rescan to pull in tag changes.
Audiophile: There is almost no other group that prides themselves more on wasting good money on utterly worthless ****, and then trying to furiously blow smoke up their own ass to justify it.

Free your mind...and your ass will follow.

Online dekardy

  • Trade Count: (+27)
  • AVForums Veteran
  • ***
  • Posts: 371
Re: Simplicity of Linux
« Reply #94 on: March 20, 2017, 06:08:51 PM »
Code: [Select]
-d '+1 second' '{}'

That's very clever.  I never knew about the "-d" option.  Very handy.  Guess I should read man pages more :shh:.

Offline BWS

  • Administrator
  • Trade Count: (+1)
  • *****
  • Posts: 15,784
Re: Simplicity of Linux
« Reply #95 on: March 21, 2017, 09:24:26 AM »
Very simple command, more for maintenance and cleaning up, use it all the time to clean up orphaned log files that take up space

Change the head number to increase the number of files. This will output the biggest files in a directory and its subdirectories.

I realize I may be telling you how to suck eggs, but some may not know it but need it


Code: [Select]
find /path/to/dir/ -printf '%s %p\n'| sort -nr | head -10

find . -printf '%s %p\n'| sort -nr | head -10
In the word : Scent, is it the S or the C that is silent?

Offline Rotten Johnny

  • ROONed for life.
  • Administrator
  • Trade Count: (+31)
  • *****
  • Posts: 14,999
Re: Simplicity of Linux
« Reply #96 on: March 21, 2017, 10:13:11 AM »
^^^ you assume I'm a Linux gun, fact is I'm just a few steps ahead of a newbie...hence this thread :D
« Last Edit: March 21, 2017, 01:05:20 PM by Rotten Johnny »
Audiophile: There is almost no other group that prides themselves more on wasting good money on utterly worthless ****, and then trying to furiously blow smoke up their own ass to justify it.

Free your mind...and your ass will follow.

Offline Rotten Johnny

  • ROONed for life.
  • Administrator
  • Trade Count: (+31)
  • *****
  • Posts: 14,999
Re: Simplicity of Linux
« Reply #97 on: June 11, 2017, 10:29:56 AM »
Invoke concurrent instances of a command leveraging multiple CPU cores

In this case I'm re-encoding all FLAC files in a directory tree, four at a time instead of one at a time:

Code: [Select]
find -type f -name \*.flac -print0 | xargs -0 -n1 -P4 flac -f -8 --preserve-modtime --verify --no-padding
Change -P4 to -Px where x is the number of CPU's in your machine.
Audiophile: There is almost no other group that prides themselves more on wasting good money on utterly worthless ****, and then trying to furiously blow smoke up their own ass to justify it.

Free your mind...and your ass will follow.

Offline Rotten Johnny

  • ROONed for life.
  • Administrator
  • Trade Count: (+31)
  • *****
  • Posts: 14,999
Re: Simplicity of Linux
« Reply #98 on: July 08, 2017, 12:51:20 PM »
because I keep forgetting this...  :fuse:

touch all files in a directory tree changing date and time stamp to now:

Code: [Select]
find . -exec touch {} \;
Audiophile: There is almost no other group that prides themselves more on wasting good money on utterly worthless ****, and then trying to furiously blow smoke up their own ass to justify it.

Free your mind...and your ass will follow.

Offline Rotten Johnny

  • ROONed for life.
  • Administrator
  • Trade Count: (+31)
  • *****
  • Posts: 14,999
Re: Simplicity of Linux
« Reply #99 on: July 24, 2017, 08:11:36 AM »
and because I keep forgetting this also   :wallbreak:

Transcode Monkeys Audio (.ape) files to FLAC format

Firstoff install shntool if not already installed, now for the business
Code: [Select]
shntool conv -o flac *.ape
or

Code: [Select]
shnconv -o flac *.ape
or

Code: [Select]
for i in *.ape; do mac "$i" - -d | flac -o "${i%*.ape}.flac" -; done
Audiophile: There is almost no other group that prides themselves more on wasting good money on utterly worthless ****, and then trying to furiously blow smoke up their own ass to justify it.

Free your mind...and your ass will follow.

Offline Rotten Johnny

  • ROONed for life.
  • Administrator
  • Trade Count: (+31)
  • *****
  • Posts: 14,999
Re: Simplicity of Linux
« Reply #100 on: November 26, 2017, 09:13:35 PM »
Increasing inotify watchers

Software that auto detects changes in your filesystem makes use of ionitify watchers to do its work.  If you've many more files in a directory tree than configured inotify watchers it's likely some changes will
Code: [Select]
cat /proc/sys/fs/inotify/max_user_watchesgo undetected.  Ubuntu and Arch Linux default to 8192 watchers.

Check your current inotify file watch limit:
Code: [Select]
cat /proc/sys/fs/inotify/max_user_watches
Increase it and make the change permanent:
Arch Linux:
Code: [Select]
echo fs.inotify.max_user_watches=524288 | sudo tee /etc/sysctl.d/40-max-user-watches.conf && sudo sysctl --system
sudo sysctl -p

Debian, Ubuntu etc:
Code: [Select]
echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf
sudo sysctl -p


Audiophile: There is almost no other group that prides themselves more on wasting good money on utterly worthless ****, and then trying to furiously blow smoke up their own ass to justify it.

Free your mind...and your ass will follow.

Offline Rotten Johnny

  • ROONed for life.
  • Administrator
  • Trade Count: (+31)
  • *****
  • Posts: 14,999
Re: Simplicity of Linux
« Reply #101 on: January 16, 2018, 11:10:28 PM »
Analyzing user/group UID/GID conflicts.

For the first time I can recall I stumbled on a really annoying permissions problem using NFS to access a 2nd PC/server's file system.  When mounting the target server's filesystem from my desktop the UID and GID of the mount point would become x and 100 respectively.  On the target filesystem the UID and GID of folder tree I was mounting is x and users respectively.  End result, user x on the desktop cannot write to the NFS share as the group ownership doesn't line up.  So, time to change the GID of the group "users" to align with that of the target server.

Firstly, check what the GID for the "users" group is on the desktop and target server respectively:
Run the following on each, noting the GID returned
Code: [Select]
getent group | grep users
On my desktop: users:x:999:
On my server: users:x:100:

So, I'm going to want to change the GID of group "users" from 999 to 100 on my desktop.

Ascertain whether or not GID 100 is already assigned on my desktop:
Code: [Select]
getent group | grep 100
On my desktop it returns: x:x:1000:x   ... so GID 100 isn't already in use.

Login as root user and run the following to change the GID for group users on the desktop machine to 100:
Code: [Select]
groupmod -g 100 users
Change the group of all the files on your system that belong to the old group:
Code: [Select]
find / -gid 999 ! -type l -exec chgrp 100 {} \;
Problem solved.
Audiophile: There is almost no other group that prides themselves more on wasting good money on utterly worthless ****, and then trying to furiously blow smoke up their own ass to justify it.

Free your mind...and your ass will follow.

Online dekardy

  • Trade Count: (+27)
  • AVForums Veteran
  • ***
  • Posts: 371
Re: Simplicity of Linux
« Reply #102 on: January 16, 2018, 11:18:40 PM »
Great and also the correct way of doing it.  However if you have multiple clients all with different UIDs and GIDs changing them gets difficult.  If security is not of big concern then the following can also help on the NFS server.

so in /etc/exports.

Code: [Select]
/$dir_to_share   $ip(rw,sync,{no_root_squash or all_squash},anonuid=$UID,anongid=$GID)
Basically the NFS server will ignore incoming UID/GID and substitute them with what was specified as anonuid and/or anongid.

Offline Rotten Johnny

  • ROONed for life.
  • Administrator
  • Trade Count: (+31)
  • *****
  • Posts: 14,999
Re: Simplicity of Linux
« Reply #103 on: January 16, 2018, 11:24:32 PM »
^^^ see, great example of DIY (me) vs master craftsman (you).  Thx  :2thumbs:
Audiophile: There is almost no other group that prides themselves more on wasting good money on utterly worthless ****, and then trying to furiously blow smoke up their own ass to justify it.

Free your mind...and your ass will follow.

Offline Rotten Johnny

  • ROONed for life.
  • Administrator
  • Trade Count: (+31)
  • *****
  • Posts: 14,999
Re: Simplicity of Linux
« Reply #104 on: January 16, 2018, 11:27:55 PM »
Here's some great reference material and scripting to untangle a mess if you need to: UNIX/Linux: Analyzing user/group UID/GID conflicts.
Audiophile: There is almost no other group that prides themselves more on wasting good money on utterly worthless ****, and then trying to furiously blow smoke up their own ass to justify it.

Free your mind...and your ass will follow.