Author Topic: Bit rot is real...check your tunes  (Read 7205 times)

Offline Hi-Phibian

  • Commercial Member
  • Trade Count: (+40)
  • *****
  • Posts: 11,621
  • I really prefer email, see my banner for address..
    • Croak Audio Exploration
Re: Bit rot is real...check your tunes
« Reply #15 on: December 14, 2015, 10:13:55 PM »
I have had some rot on very early discs but anything post 92 has no issues in my experience. 
Proprietor of Croak Audio Exploration.
Fair, not crazy, cash paid for turntables and tonearms from Rega, Linn and Thorens.          http://www.croak.co.za

Offline BillyBawb

  • Trade Count: (+41)
  • AVForums Veteran
  • ***
  • Posts: 818
Re: Bit rot is real...check your tunes
« Reply #16 on: December 15, 2015, 06:32:50 AM »
They run systems capable of detecting and correcting bitrot and they use enterprise drives, not the **** we're sold as consumers.  I agree with Godfather's view...the larger our datasets become the greater the imperative to use ECC ram coupled with self-healing filesystems.  I feel another purchase coming on. :facepalm:

You and I had this chat when you bought the Yamaha for your folks...

While a proper ZFS build with server grade hardware is costly, it's really the only way to be relatively certain that you're not suffering bit corruption.

As I've said to a few guys that enquired about file server builds (and subsequently had a small meltdown when I started talking pricing) -  It all boils down to how much your data integrity is worth to you.
Main: DHC 80.3| Cinema Grand Sig 5ch SII| N802| NHTM1| N804 | SST4| SCM1| ASW 800| SW380| UA65HU9000
Bedroom: AVR-2312| Diamond 10 HCP| UA55D8000
Mancave: FMJ AVR400| CDM-SNT| NHTM2| M1| mX10 Sub| JVC DLA-HD350 + Mirrodex 2.5M
Plex + 90TB FreeNAS

Offline Rotten Johnny

  • ROONed for life.
  • Administrator
  • Trade Count: (+31)
  • *****
  • Posts: 15,206
Re: Bit rot is real...check your tunes
« Reply #17 on: December 15, 2015, 07:01:08 AM »
^^^ that we did, and server grade hardware coupled with zfs or btrfs is the only way out.  So I'm waiting for a great deal on a server :whistler:
I like beautiful melodies telling me terrible things -- Tom Waits

Free your mind...and your ass will follow.

Offline BillyBawb

  • Trade Count: (+41)
  • AVForums Veteran
  • ***
  • Posts: 818
Re: Bit rot is real...check your tunes
« Reply #18 on: December 15, 2015, 08:13:19 AM »
^^^ that we did, and server grade hardware coupled with zfs or btrfs is the only way out.  So I'm waiting for a great deal on a server :whistler:

I don't want to bump a really old thread, but I still have that Dell T320 standing around.  ;D
Main: DHC 80.3| Cinema Grand Sig 5ch SII| N802| NHTM1| N804 | SST4| SCM1| ASW 800| SW380| UA65HU9000
Bedroom: AVR-2312| Diamond 10 HCP| UA55D8000
Mancave: FMJ AVR400| CDM-SNT| NHTM2| M1| mX10 Sub| JVC DLA-HD350 + Mirrodex 2.5M
Plex + 90TB FreeNAS

Offline oradba69

  • Commercial Member
  • Trade Count: (+5)
  • *****
  • Posts: 1,564
    • Lerange
Re: Bit rot is real...check your tunes
« Reply #19 on: December 15, 2015, 08:27:41 AM »
On mac you can use this.

https://github.com/ambv/bitrot
Manufacturer of Audio Video furniture.
Importer and distributor of Trichord.
http://www.lerange.co.za

Offline Rotten Johnny

  • ROONed for life.
  • Administrator
  • Trade Count: (+31)
  • *****
  • Posts: 15,206
Re: Bit rot is real...check your tunes
« Reply #20 on: December 15, 2015, 08:42:07 AM »
On mac you can use this.

https://github.com/ambv/bitrot
You're probably much better off using this: http://www.snapraid.it/download for any files that rarely change...
I like beautiful melodies telling me terrible things -- Tom Waits

Free your mind...and your ass will follow.

Offline Rotten Johnny

  • ROONed for life.
  • Administrator
  • Trade Count: (+31)
  • *****
  • Posts: 15,206
Re: Bit rot is real...check your tunes
« Reply #21 on: December 15, 2015, 08:42:45 AM »
I don't want to bump a really old thread, but I still have that Dell T320 standing around.  ;D
I could store it for you ;D
I like beautiful melodies telling me terrible things -- Tom Waits

Free your mind...and your ass will follow.

Offline chrisc

  • Trade Count: (+107)
  • AVForums Veteran
  • *****
  • Posts: 9,226
  • Cape Town Hi-Fi Club - listen with your ears
Re: Bit rot is real...check your tunes
« Reply #22 on: December 20, 2015, 03:16:13 PM »
Macupdate allows the installation of Github.  Snapraid might well be better, but I try and avoid command-line programs (being lazy)

Offline Larry

  • Trade Count: (0)
  • AVForums Veteran
  • *****
  • Posts: 1,656
Re: Bit rot is real...check your tunes
« Reply #23 on: December 21, 2015, 08:50:17 AM »
^^^ that we did, and server grade hardware coupled with zfs or btrfs is the only way out.  So I'm waiting for a great deal on a server :whistler:

Only problem with this is noise, heat and power consumption.

Even just removing a few disks (some for raid set and some as spares) and using an old raid card can be noisy.

I toss out old enterprise class machines about twice a year, and gave up using the stuff years ago.  5 X 15k rpm disks spinning are noisy, even in a big house!

Best solution is to run the box out of a garage, although garages often get hot and have very fluctuating temperatures.  That being said, the hardware I procure (mostly IBM, Lenovo now for the Intel based stuff) is definitely more resilient these days in that respect.

Further to that - most true enterprise class disks aren't actually very big. 600 GB and bigger start to get lumped into "near line" class of disks.

2.5 inch 300 GB spindles for the win   :D

Online pwatts

  • Trade Count: (+3)
  • AVForums Veteran
  • *****
  • Posts: 1,266
Re: Bit rot is real...check your tunes
« Reply #24 on: December 21, 2015, 07:59:20 PM »
A friend runs his server in his garage using a standard PC with a large Jeep air filter hooked up to the air intakes. Doesn't look too great but has been working flawless since 2008.

Offline wanda

  • Trade Count: (+1)
  • AVForums Veteran
  • *****
  • Posts: 1,616
  • crescendo
    • avforums south africa
Re: Bit rot is real...check your tunes
« Reply #25 on: December 22, 2015, 03:32:32 PM »
The more things change, the more they stay the same. I am sticking to discs: red book, mp3/4. No files thanks.
the person who has a problem has the solution

Offline chrisc

  • Trade Count: (+107)
  • AVForums Veteran
  • *****
  • Posts: 9,226
  • Cape Town Hi-Fi Club - listen with your ears
Re: Bit rot is real...check your tunes
« Reply #26 on: December 22, 2015, 04:49:18 PM »
In the good old days, the guys at the Hi-Fi Club were using a green pencil (like a highlighter) to wipe onto the side of a CD to prevent CD rot.  I spotted one (a treated CD) the other day for sale at a car boot sale and it looked as good as new.  Must be some truth in that idea after all, we thought at the time it was all BS

Offline Atjan

  • Trade Count: (+10)
  • AVForums Veteran
  • *****
  • Posts: 11,331
Re: Bit rot is real...check your tunes
« Reply #27 on: December 22, 2015, 11:02:59 PM »
The more things change, the more they stay the same. I am sticking to discs: red book, mp3/4. No files thanks.
It's not as if you will wake up one morning and find 5% of your collection unplayable. The convenience of electronically stored music trumps the inconvenience of a dead file (not folder) 100x.
Anyway, it's also not as if what is on your server/hard drive is your only available copy. Surely you have the CD somewhere and I can barely think that a file bought on-line will not be accessible if you loose the down-loaded copy.
CD's that I've chucked because they got unplayable somehow has definitely cost me more money than a bit of data to download music again or time to re-rip the CD when an individual file became unplayable.
It's only hifi people....

Offline Eejit

  • Trade Count: (+2)
  • AVForums Member
  • **
  • Posts: 57
Re: Bit rot is real...check your tunes
« Reply #28 on: August 02, 2016, 07:13:03 PM »
Little confused. (Quite normal for me). Is bit rot referring to the loss of the integrity of a file stored on disk, or the inability to transfer data to disk accurately. I've just transferred my cd collection to disk. Used forward error correction in transfer and raid drives for back up and integrity. Is this not enough?

Offline Katji

  • Trade Count: (+1)
  • AVForums Veteran
  • *****
  • Posts: 1,044
    • soundcloud
Re: Bit rot is real...check your tunes
« Reply #29 on: August 03, 2016, 12:52:25 AM »
Until recently I'd never given much thought to bit rot where my music was concerned, however, a series of issues I had with drives failing etc. led me to rethink how I'm storing and backing up my music.  Having just about completed deduplication and consolidation of all my music I figured it was time to check on the health of the underlying FLAC files.  The results are scary and in many cases where I've abandoned an old backup in favour of a fresh copy of the "production files" I've ended up abandoning good files and backing up their corrupted cousins.

Do yourself a favour, periodically check the integrity of your files.  Windows users can use dbpoweramp or the free multithreaded audiotester.  For Linux users the following script can do the job for you:

Code: [Select]
#!/bin/sh
szDate=`/bin/date`
szMusicRoot='/mnt/media/Music'
szErrFile="$szMusicRoot/flac_errors.txt"

echo Recursively testing flacs in $szMusicRoot
echo Flac decoding errors logged to $szErrFile
echo Flac test of $szMusicRoot started at $szDate >"$szErrFile"
echo Errors were found in the following flac files: >>"$szErrFile"

/bin/find "$szMusicRoot/" -name '*.flac' -type f -not -exec /usr/bin/flac -t --totally-silent '{}' \; -and -print >>"$szErrFile"

szDate=`/bin/date`
echo Flac tests of $szMusicRoot compleated at $szDate >>"$szErrFile"

echo Done!

Bottom line is that as drives have grown in size so they've become less reliable.  Where possible I'd strongly recommend switching to a filesystem that can detect and correct bit rot.
unix/linux file system? ...That would at least reduce the need for the high-end server hardware, sort of.  Which is not possible for me, unless I give up sticking to a laptop.  (All complicated/complexified by the need to dispose of stuff and move out.)  :-\

Anyway, that main line in the script... flac -t is a codec program doing testing? But how does it test without re-encoding from the source WAV files and comparing? (I don't see any source file/dir being specified...?)

So I must try audiotester, otherwise just buy dbPoweramp.  I just don't have the time now, to start learning linux, for dealing with all that.  It is years - like a few years before linux started happening - that I was even beginning to learn it. But already I'm faced with eventually being forced to Windows 10, and then a couple years later, it'll be Windows 11.

Quote
The more things change, the more they stay the same. I am sticking to discs: red book, mp3/4. No files thanks.

S\Well, so far, my music files are less than 50gb, so backups on CD are do-able, but...  :-\ the regular backup has to be just what it is, copy to portable HD - to make error-free backups on CD means testing each time, and the possibility of hardware error is there in the process anyway. :-\ I wonder how worthwhile it would be to get ECC RAM for the laptop.  ...Maybe I just have to live with it, accept it, if I want to be mobile /movable.
#SPOTIFYMUSTFALL